Data, Knowledge & Life

Weiwei Cheng's blog

Google uses old-fashioned way to measure the search quality

leave a comment »

Anand Rajaraman has just posted a new blog article discussing how Google using “real people” to measure the qualities of different search algorithms. It’s quite unlike what we do in machine learning, that we first define a certain kind of “loss function” and then we optimize the ranking model based on this predefined metric. Google’s reasoning is, according to Anand, these metrics never change much during the tests on new ranking models.

I would not say I am that astonished. Not like in classification, there is no golden standard for the loss functions used in ranking problems. Some of the loss functions are quite local, e.g., one-error; some of them are global, e.g., correlation coefficient. You never know in which circumstance which error function can capture the user’s behavior. So why don’t just do it in an old-fashioned way?

Advertisements

Written by Weiwei

19/02/2009 在 00:37

发表在 杂话

发表评论

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / 更改 )

Twitter picture

You are commenting using your Twitter account. Log Out / 更改 )

Facebook photo

You are commenting using your Facebook account. Log Out / 更改 )

Google+ photo

You are commenting using your Google+ account. Log Out / 更改 )

Connecting to %s

%d 博主赞过: