Internet Resources Discovery (IRD) Meta-Search Engines (MSEs)
Mar 18, 2016
Internet Resources Discovery (IRD)
Meta-Search Engines (MSEs)
T.Sharon-A.Frank2
Contents• Meta Search Engine (MSE)• Why use several SEs?• Highlighted MSEs• Hebrew MSEs• MSE comparison• When to use MSE – pros and cons• How to choose MSE?
T.Sharon-A.Frank3
Search Engines Generations
• 1st Generation - Basic SEs:
• 2nd Generation - Meta SEs:
• 3rd Generation - Popularity SEs:
T.Sharon-A.Frank4
2nd Generation SEs - MetaSEs• Using several SEs in parallel.• The results are filtered, ranked and
presented to the user as a uniformed list.• The ranking is a combination of the
number of sources each page appeared in, and the ranking in each source.
T.Sharon-A.Frank5
Meta SE is a Meta-Service• It doesn’t use an Index/database of its
own.• It uses other external search services that
provide the information necessary to fulfill user queries.
T.Sharon-A.Frank6
Meta Search Engine
MetaCrawler
Yahoo Web Crawler Open Text Lycos InfoSeek Inktomi Galaxy Excite
Google · Yahoo · Jeeves Ask About · LookSmart · OvertureFindWhat
T.Sharon-A.Frank7
Premises of a Meta SE
• No single search is sufficient.• Problem in expressing the query.• Low quality references can be detected.
T.Sharon-A.Frank8
Why use Several SEs?• Search Engines differ more than we think!
T.Sharon-A.Frank9
Overlap between Google and Yahoo
Source: Jux2 analysis of 500 top search terms, April 2004http://www.jux2.com/stats.php
T.Sharon-A.Frank10
Who Overlaps Whom?
T.Sharon-A.Frank11
Try it yourself @ jux2
T.Sharon-A.Frank12
MSE - Motivation
1. The number and variety of SEs.2. Each SE provides an incomplete snapshot of Web.3. Users are forced to try and retry their queries across
different SEs.4. Each SE has its own interface.5. Irrelevant, outdated or unavailable responses.6. Each query is independent.7. No individual customization.8. The result is not homogenized.
T.Sharon-A.Frank13
Problems of MSEs• No advanced search options.• Using the lowest common denominator.• Sponsored results from the SEs are not
highlighted.
T.Sharon-A.Frank14
Highlighted MSEs
T.Sharon-A.Frank15
Mamma
T.Sharon-A.Frank16
Dogpile
T.Sharon-A.Frank17
Dogpile Advanced (1)
T.Sharon-A.Frank18
Dogpile Advanced (2)
T.Sharon-A.Frank19
Dogpile Advanced (3)
T.Sharon-A.Frank20
Dogpile Advanced (4)
T.Sharon-A.Frank21
Dogpile Preferences (1)
T.Sharon-A.Frank22
Dogpile Preferences (2)
T.Sharon-A.Frank23
Vivisimo• Vivísimo supports the most advanced features
of the major search engines.• Need to just use Vivísimo syntax, which
follows the most standard conventions. • Vivísimo translates your query into the
corresponding syntax of each underlying search engine.
• Also, Vivísimo only queries the search engines that support your chosen syntax.
T.Sharon-A.Frank24
Vivisimo Advanced (1)
T.Sharon-A.Frank25
Vivisimo Advanced (2)
T.Sharon-A.Frank26
Clusty
T.Sharon-A.Frank27
Ixquick (1)
T.Sharon-A.Frank28
Ixquick (2)
T.Sharon-A.Frank29
Ixquick (3)
T.Sharon-A.Frank30
KartOO – Visual MSE
T.Sharon-A.Frank31
MetaSEs in Hebrew
T.Sharon-A.Frank32
When to use a MSE?• When single Basic-SE fails to provide good
results. • One-stop shopping - prefer to search multiple
SEs/sites at once to get blended ranked results (so as to save effort/time).
• Searching for multi-faceted topics. • Want to get clustered results to focus search
on the relevant keywords. • Looking for current events/news.
T.Sharon-A.Frank33
MSE pros• Useful when you want to retrieve a relatively
small number of relevant results. • An excellent choice for obscure topics. • A good option when you are not having luck
finding what you want when you search. • Appropriate when you want to get an overall
picture of what is available on the Web on your topic.
T.Sharon-A.Frank34
MSE cons• Use is limited primarily to simple queries. • Little or no field searching is available. • Most services return a limited number of results
that do not represent the total results from any source engine.
• Sponsored results are not highlighted (even though probably not first).
T.Sharon-A.Frank35
How to Choose your MetaSE
• Search engines used• Operators supported• Special features• Speed• Presentation
T.Sharon-A.Frank36
Meta-SEs Features Chart
Red – not working
T.Sharon-A.Frank37
Practical RecommendationsUse Ixquick for fast results and maximal
syntax flexibility.Use Vivisimo/Clusty (as a start) for
Clustering and/or Hebrew.Use Dogpile to include Google, date
range, or spelling corrections.Use none for non-MSE tasks
(see MSE cons)…
T.Sharon-A.Frank38
Bibliography• http://www.cs.washington.edu/homes/etzioni/papers/m
etacrawler.pdf• http://www.cs.washington.edu/homes/etzioni/papers/ie
ee-metacrawler.pdf• http://searchenginewatch.com/links/article.php/2156241• http://vivisimo.com/advanced?form=Advanced• http://vivisimo.com/help.html• http://searchenginewatch.com/searchday/article.php/
2226841