A. Frank & T. Sharon היה מנוע? – מגמות וחידושים במנועי חיפוש אריאל פרנק מחלקה למדעי המחשב אוניברסיטת בר-אילן

A. Frank & T. Sharon

היה מנוע? – מגמות וחידושים במנועי חיפוש

אריאל פרנקמחלקה למדעי

המחשבאוניברסיטת בר-אילן

ariel@cs.biu.ac.il

טלי שרוןSharon-IT

taly@sharon-it.com

חיפוש זמן-אמתReal-time Search

2

Search (Engines) Technical Buzzwords

• Real-time (web, search, conversation)

• Multimedia RT (video, visual)

• Mobile (local, modes, diversity)

• Personalized (history/privacy)

• Social (networks, circle, trusted)

• Human (crafted results, Q&A)

• Semantic (NLP, context-aware, personal knowledge)

3 A. Frank & T. Sharon

Contents

• What is “Real-time Search”?

• Real-time Search Engines

• Multimedia Real-time Search Engines

• (Google) Mobile Search

4 A. Frank & T. Sharon

What is “Real-time … ”(?-:

http://www.acm.org/crossroads/xrds2-3/gfx/realtime.gif

5

What is “Real-time Web” (timing)?• Microblogging (several seconds):

– tweet, status update, post, talk, chat, ... – e.g., Twitter, FriendFeed, Facebook, MySpace, …

• Social Media Sites (many seconds):– realize, locate, upload, share, expose, comment, …– e.g., Digg, Delicious, Flickr, YouTube, Yahoo Answers, …

• Blogosphere (several minutes):– realize, check, write, post, break news, …

• News Media Sites (many minutes):– gather, investigate, verify, write, edit, report, …

6

What is “Real-time Search” (timeliness)? Is the emphasis on real-time search?

“…finding the right answer to your question based on what’s available right now, about the subject you care about right now. Real-time search is finding the ‘Right Answer, Right Now.’” (Kimbal Musk)

Or is it on real-time search? “...looking through material that literally is published in

real-time. In other words, material where there’s practically no delay between composition and publishing.” (Danny Sullivan)

“I want to know what’s happening in the world right now, I’m interested in gaining access to data that’s been created (written, on video or in photographs) in the last few seconds, not data that has simply been made available in the last few seconds.” (Phil Bradley)

7

Really “Real-time Browse” (discovery)!?

• Real-time search/browse enables discovery of what's “hot” right now – the trending topics.

• A key quality measure for real-time search results is provision of a meaningful summary of the results.

• Leads to use of hot topics as new search terms.• Enables to swing through the real-time

information jungle, exploring related concepts as they emerge.

8 http://inchoo.net/wp-content/uploads/2009/07/real-time-search.jpg

9

Unique “Real-time Search” Challenges

• Real-time “Information Explosion” –– rapid pace and sheer amount of real-time data being

continuously produced and pushed.– instantaneous filter/index/update of the “firehose feed”.– need delicate balance of weighing relevance with immediacy

and popularity, to prevent “information overload”. • Susceptible Authorities –

– dependence on the fact that sources who say that they’re at the scene of a specific event or on top of things, actually are.

• Signal to Noise Ratio –– Spam: once a subject becomes a hot topic/trend some

piggyback to get across their particular marketing messages.– Duplication: much information is replicated, not really new.– Mining: how to extract meaning of the “real conversation”.

10 A. Frank & T. Sharon

Contents

• What is “Real-time Search”?

• Real-time Search Engines

• Multimedia Real-time Search Engines

• (Google) Mobile Search

11

Any Real-time Search Engines!?

12

Classification of RT SEs

• Microblog-search:– Twitter, FriendFeed, FaceBook, CrowdEye, Twazzup,

Topsy, Sency, TweetMeme, Twitscoop, Tinker, dailyRT, Almost.at, TipTop, TweetGrid, Ellerdale

• Mega-search:– Collecta, OneRoit, Scoopler, Twingly, Yauba,

SocialMention, Itpints, Thoora, IceRocket, Wowd

• Meta-search:– Stinky Teddy, LeapFish, Scour, Faxo

• Part-of-Traditional-SEs: – Google, Bing, Yahoo

13

Twitter Search Interface

14

Twitter Example

15

Twitter “Advanced Search”

16

Twitter Search Operators

17

Twitter Is the Future of News!?• Recent analysis reveals that Twitter is remarkably

effective at spreading "important" information. • A multi-part analysis of Twitter reveals that it's a

surprisingly interconnected network and an effective way to filter quality information.

• “...No matter how many followers a user has, the tweet is likely to reach [an audience of a certain size] once the user's tweet starts spreading via retweets, that is, the mechanism of retweet has given every user the power to spread information broadly [...] Individual users have the power to dictate which information is important and should spread by the form of retweet [...] In a way we are witnessing the emergence of collective intelligence.” (Kwak et al.)

18

Collecta Search Interface

19

Collecta

• Results are continuously updated until the user pauses the search; This allows for a constant stream of results from all over the web.

• A preview pane allows the person searching to look at the article, comment, or news report without leaving the window.

• Collecta allows for more than one search at a time in the same window.

20

Collecta “Search Options”

21

Collecta “Share Search”

22

Stinky Teddy Search Interface

23

Stinky Teddy MSE Sources

• Twitter (microblog)

• Collecta (blogs, comments, news)

• OneRiot (real-time pulse of the web)

• Bing (web, news, image)

• Yahoo! (web, news image)

• VideoSurf (videos with thumbnail previews)

24

Stinky Teddy Example

25

Google Awareness

26

Google’s Real-time Search

• Google’s real-time search results are an extension of Google's Universal search results.

• Triggered algorithmically based on query volume. • When a topic or keyword reaches a predefined

frequency, Google's algorithm automatically begins using the real-time search page. 

• Real-time search features are based on more than a dozen new search technologies that enable monitoring billions of documents and processing hundreds of millions of real-time changes each day.

27

Google’s Real-time Search Sources

• Shows news headlines, blog headlines, and social media comments, along with Google's regular results.

• Blogs and news sites that publish regularly are normally crawled many times per day due to the frequency of new content added. 

• Social media comments currently come from Twitter, Facebook, FriendFeed, Google Buzz, Jaiku, MySpace, TwitArmy, and Identi.ca via API data feeds.

• Timestamps are included so that users can choose the timeliest results.

28

Google’s “News results”

29

Sample Signals for Google RT Search

• A sudden spike in the prevalence of a word or combination of words in a message.

• A message on a commonly discussed topic that includes unusual phrasing, shifts in language and other deviations from predicted behavior.

• A Twitter user who attracts many followers, and whose tweets are often "re-tweeted" by other users, is assumed to have more authority.

• Facebook users gain authority as their friends multiply, particularly if those friends also have many friends.

• The geo-location of someone sending a message from a local event is more valuable than those of someone hundreds of miles away.

30

Google’s “Latest results”

31

Google’s “Show options”

32

Google’s “Twitter Timeline”

33

Google’s Twitter Replay

34 A. Frank & T. Sharon

Googlism

35

The “Israeli journalist gag order!?”

36

Google Suggest!!

• Completes search terms as they are typed into the search box.

• Generates suggestions for popular “automatic search completions”.

• Notices increases in the amount of search for combinations of certain terms.

• Uses data about the overall popularity of various searches to help rank the refinements it offers.

• Can be used as a real-time search radar.

37

Google gets to know “Anat Kamm?”

38

“Change in Search” for Anat Kamm

39

Real-time Search!?

40

7th Eye’s article on Anat Kamm

41

relevant column Aharonoth Yedioth

42

Graffiti: “Google Anat Kamm”

43 A. Frank & T. Sharon

Contents

• What is “Real-time Search”?

• Real-time Search Engines

• Multimedia Real-time Search Engines

• (Google) Mobile Search

44

Multimedia Real-time Search Engines

45

Classification of MM RT SEs

• Image/Photos-search:– NachoFoto, yfrog, PicFog, IceRocket

• Music/Songs-search:– Twittify (Twitter+Spotify), Qloud

• Video/Movies-search:– TwitVid, yfrog, Twitmatic, IceRocket

• Visual-search:– Spezify, Surchur, TwitterVision

46

NachoFoto Search Interface

47

NachoFoto Internals

• Has 4 different types of image search queries:1. Static search terms (e.g., “apple”, “girl with balloon” ) 2. Dynamic keywords whose meaning doesn’t change but whose images

undergo significant change with time (e.g., Olympics)3. Dynamic keywords whose meaning and images change significantly

with time (e.g., “9/11”) 4. New keywords (e.g., “iPad”).

• Employs 4 main factors to influence its image search results: 1. Freshness factor (i.e., how new is the image?)2. Image density of a webpage (websites that interlink their photos are

given higher ranking than those who don’t)3. Inward links (websites that link internally, especially with relevant

anchor text, are given higher ranking)4. Domain authority (domains with fresh, family-friendly images are

given higher priority).

48

NachoFoto Example

49

Twittify

50

TwitVid Search Interface

51

TwitVid

• TwitVid, which offers a popular tool for adding video to Twitter, aggregates all TwitVid and YouTube video links being shared on Twitter.

• Ranks videos based on relevancy to a search term as well as popularity and "buzz" on Twitter.

• The newest, most relevant videos are ranked highest. • Offers is a unique video analytics tool that lets

members track their tweeted videos online by the day, week, month or total number of times viewed.

52

TwitVid Example

53

Spezify Search Interface

54

Spezify Example

55

Spezify

• Presents results from a large number of websites in different visual ways.

• Gives a good overview of a subject so as to be able to find useful information.

• Mixes all media types and make no difference between blogs, videos, microblogs and images.

• Everything communicates and helps building the bigger picture.

56 A. Frank & T. Sharon

Contents

• What is “Real-time Search”?

• Real-time Search Engines

• Multimedia Real-time Search Engines

• (Google) Mobile Search

57

Google Mobile Search

• Sensor-rich smartphones are redefining nowadays what a “query” means.

• Search can be carried out by several new modes: gesture, voice, location and sight.

• Use location of your mobile phone for “Search with My Location” or for showing you “What's Nearby“ on Google Maps.

• Google Goggles is a visual search application that lets you search for objects using images rather than words, using your camera phone.

58

Google’s “Gesture Search”

59

Google’s “Voice Search”

60

Google’s “Search with My Location”

61

"What's Nearby" for Google Maps

62

Google Goggles Search Options

63

Google Goggles on Mobile

64

General References

• Danny Sullivan, What Is Real Time Search? Definitions & Players, Search Engine Land, 9 July 2009, http://searchengineland.com/what-is-real-time-search-definitions-players-22172

• Kimbal Musk, RE: What Is Real Time Search? Definitions & Players, 9 July 2009, http://blog.oneriot.com/content/2009/07/re-what-is-real-time-search-definitions-players/

• Phil Bradley, Search Engines: Real-time Search, Ariadne Issue 61, 30 October 2009, http://www.ariadne.ac.uk/issue61/search-engines

• Ron Jones, Real-time Search 101, ClickZ, 19 April 2010, http://www.clickz.com/3640099

• Mark Drummond, What's the Job of a Real-time Search Engine?, Search Engine Watch, 29 April 2010, http://searchenginewatch.com/3640197

• Nicholas Carr, Real-time Search, Technology Review, May/June 2010, http://www.technologyreview.com/computing/25079/?a=f

65

Specific References

• Christopher Mims, Why Twitter Is the Future of News, Technology Review, 30 April 2010, http://www.technologyreview.com/blog/guest/25128/?nlid=2946&a=f

• Ido Kenan, Google Anat Kamm: Interesting Gag Order Circumvention Methods, Ido Kenan’s blog, 9 April 2010, http://www.room404.net/eng/?p=305

• Amit Singhal, Relevance Meets the Real-time Web, Google Blog, 12 July 2009, http://googleblog.blogspot.com/2009/12/relevance-meets-real-time-web.html

• Vic Gundotra, Mobile Search for a New Era: Voice, Location and Sight, Google Mobile Blog, December 7 2009, http://googlemobile.blogspot.com/2009/12/mobile-search-for-new-era-voice.html

66

General Videos

• Real-time Search Engines Rush to Fill New Need, WPN Videos, 10 July 2009, http://videos.webpronews.com/2009/07/10/real-time-search-engines-rush-to-fill-new-need/

• John Edet, What's the Latest with the Real time Search Engines, November 2009, http://vimeo.com/7602248

• Nicholas Carr, Real-time Search, Technology Review, May/June 2010, http://www.technologyreview.com/video/?vid=556

• The Search for the Real Conversation, WPN Videos, 20 October 2009, http://videos.webpronews.com/2009/10/20/the-search-for-the-real-conversation/

• What Is The Real Time Web?, 9 February 2010, http://www.youtube.com/watch?v=hOlcD506z_Y

67

Specific Videos

• Google Real-time Search, 7 December 2009, http://www.youtube.com/watch?v=WRkYmx4A9Do

• Google Mobile App for iPhone with Voice Search, 10 November 2008, http://www.youtube.com/watch?v=GQ3Glr5Ff28

• Introducing Search with My Location, 11 September 2008, http://www.youtube.com/watch?v=KMT7Deky9iY

• Real Time Search Event, 8 December 2009, http://www.youtube.com/watch?v=oXHHkROejik#t=21m05s

• Google Goggles, 6 December 2009, http://www.youtube.com/watch?v=Hhgfz0zPmH4

• Google Goggles Demo Up Close, 7 December 2009, http://www.youtube.com/watch?v=GgcE_EQRpdA

68 A. Frank & T. Sharon

Any Real-time Questions (?-: