11.02.2011 1 The Influence of Commercial Intent of Search Results on Their Perceived Relevance Dirk Lewandowski Hamburg University of Applied Sciences, Germany Introduction • Ongoing discussion about results quality – Did Google’s results really get worse? – Do search engine results (and especially, Google’s) get more and more commercial? • Discussion on biased search results / “search neutrality” – Does Google prefer its own services in its rankings? • Objective – To find out whether commercial results have a negative effect on relevance as perceived by the users.
15
Embed
The Influence of Commercial Intent of Search Results on Their Perceived Relevance
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
11.02.2011
1
The Influence of Commercial Intent of Search Results on Their Perceived Relevance
Dirk Lewandowski Hamburg University of Applied Sciences, Germany
Introduction
• Ongoing discussion about results quality – Did Google’s results really get worse?
– Do search engine results (and especially, Google’s) get more and more commercial?
• Discussion on biased search results / “search neutrality” – Does Google prefer its own services in its rankings?
• Objective – To find out whether commercial results have a negative effect on relevance as
perceived by the users.
11.02.2011
2
Search engine market share (Germany)
Previous work
• Retrieval effectiveness tests – Lots of work done
– Main objective is to compare results quality of commercial Web search engines
– Tests are highly standardized
• Commercial intent – Some studies on commercial intent of the searchers
– No studies on the commercial intent of the documents
11.02.2011
3
Research questions
• RQ1: How effective are web search engines in retrieving relevant results?
• RQ2: What ratio of the top 10 search engine results are pages with a commercial intent?
• RQ3: Are results with a commercial intent comparably relevant to non-commercial results?
• RQ4: Do banner or textual ads have a negative effect on the perceived relevance?
Test design
• Retrieval effectiveness test – To collect relevance judgments
• Results classification – Classify each result according to its commercial intent
• Commercial intent of the page/website itself
• Commercial intent as expressed through advertisements on the page
11.02.2011
4
Retrieval effectiveness test design
• 50 queries – “Real life queries” from students asked for their last query entered into a Web
search engine
– Mainly simple queries without an explicit commercial intent