Data, Knowledge & Life

Weiwei Cheng's blog

The Nemenyi test

leave a comment »

The Nemenyi test can be used for comparisons of algorithms on multiple data sets.

To apply a Nemenyi test, one firstly calculate the average ranks of each algorithm based on their individual performance in the experiment. The performance of two algorithms is significantly different if their corresponding average ranks differ by at least the critical difference:

where k is the number of algorithms and N is the number of data sets. The critical values qa for the two-tailed Nemenyi test are showed in the following table:

Before applying the Nemenyi test, it is recommended, however, to reject the null-hypothesis. A Friedman test can be used for that purpose.

Reference: Janez Demsar, Statistical Comparisons of Classifiers over Multiple Data Sets, Journal of Machine Learning Research 7 (2006) 1-30

Advertisements

Written by Weiwei

16/03/2009 在 22:55

发表在 学术

发表评论

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / 更改 )

Twitter picture

You are commenting using your Twitter account. Log Out / 更改 )

Facebook photo

You are commenting using your Facebook account. Log Out / 更改 )

Google+ photo

You are commenting using your Google+ account. Log Out / 更改 )

Connecting to %s

%d 博主赞过: