BOSS: Yahoo HackU IIIT Hyderabad

Saurabh Sahni

Developer, hacker, evangelist

Build your Own Search Service

Yahoo! HackU IIIT Hyderabad | 22nd Jan 2010

•  About BOSS API – What?

– Why? – Features

•  What can be built with the BOSS API? •  How to use it

– BOSS API – Code example – BOSS Mashup framework

Outline

About BOSS API

•  Open Yahoo’s core search features via web services to let 3rd parties revolutionize Search

What?

http://developer.yahoo.com/search/boss

Usage

Opening the search technology stack

CRAWL

EXTRACT

SPAM <-> Gold

Analyze

Index

Rank Assist

Index

Web Map

Retrieve

Usage

Opening the search technology stack

CRAWL

EXTRACT

SPAM <-> Gold

Analyze

Index

Rank Assist

Index

Web Map

Retrieve

WEB API

Your App here

Why?

•  Removes entry barriers

•  Asset to Innovate –  Develop new relevance models –  Change presentation style

•  Search anywhere –  Improve Vertical Quality with Web comprehensiveness

• No branding or attribution

• Ability to change presentation style • Ability to re-order results and blend-in additional

content

• Access to multiple verticals (web search, image, news)

• Keyword suggestions, spell checks • Semantic data, in-links, abstracts • Ability to monetize

BOSS API features

What can be built with the BOSS API?

middlespot.com

10 

Inquisitor: Browser Extenstion

11 

Cluuz: http://cluuz.com

Keyword finder - http://keywordfinder.org/

askBOSS: http://ask-boss.appspot.com/

askBOSS: http://ask-boss.appspot.com/

askBOSS: http://ask-boss.appspot.com/

Webmeme: http://www.webmeme.in

17 

How to use it?

Get Started

•  Register for an application id http://developer.yahoo.com/wsregapp/

•  Documentation http://developer.yahoo.com/search/boss/boss_guide/

•  Code samples: Javascript, PHP and Python http://www.saurabhsahni.com/boss-examples.zip

Get Started…

•  {query}: term to look for (url-encoded) •  {vert} := {web, news, images, spelling}

•  @ required – appid

•  @ optional – start, count, lang, region, format, callback,

sites, view

BOSS Search API Interface

http://boss.yahooapis.com/ysearch/{vert}/v1/{query}

(Source: http://en.wikipedia.org/wiki/File:Slumdog_Millionaire_poster.jpg)

BOSS API

•  Search for avatar: –  http://boss.yahooapis.com/ysearch/web/v1/avatar?appid=xyz&format=xml

BOSS API: XML response

h"p://boss.yahooapis.com/ysearch/web/v1/avatar?appid=xyz&format=xml 

Site Restrict Search

•  Search for avatar on selected movie sites – Add param

sites=indiatimes.com,movies.yahoo.com,imdb.com

–  http://boss.yahooapis.com/ysearch/web/v1/avatar?appid=xyz&sites=imdb.com%2Cmovies.yahoo.com&format=xml

Site Restrict Search

Search images

•  http://boss.yahooapis.com/ysearch/images/v1/avatar?dimensions=large

http://boss.yahooapis.com/ysearch/images/v1/ avatar

Search News

•  http://boss.yahooapis.com/ysearch/news/v1/avatar?age=15d

Code Example Movie Search

Movie Search Code Example

http://www.saurabhsahni.com/boss-examples.zip

More with BOSS API

Add parameter view=keyterms –  http://boss.yahooapis.com/ysearch/web/v1/avatar?

appid=xyz&view=keyterms&format=xml

Related keywords

http://boss.yahooapis.com/ysearch/web/v1/avatar &appid=xyz&view=keyterms&format=xml

Semantic Data

•  Access structured data acquired through SearchMonkey

Semantic Data

view=searchmonkey_feed view=searchmonkey_rdf

http://developer.yahoo.com/search/boss/stuctureddata.html

•  Add parameter abstract=long – get up to 300 characters instead of 130

Long abstracts

http://boss.yahooapis.com/ysearch/spelling/v1/avataar?format=xml

Response

Spell Check 

•  Get page inlinks –  http://boss.yahooapis.com/ysearch/se_inlink/v1/

{URL}?appid={APPID}

•  Page data: collection of subpages in a domain –  http://boss.yahooapis.com/ysearch/se_pagedata/v1/

{URL}?appid={APPID}

Site Explorer

•  Python (v2.5+) library

•  BOSS Search SDK plus …

•  SQL for remixing arbitrary XML/JSON sources

BOSS Mashup Framework

http://developer.yahoo.com/search/boss/mashup.html

•  Enhanced version of BMF to GAE platform

•  http://zooie.wordpress.com/2008/08/04/yahoo-boss-google-app-engine-integrated/

•  Enables quick deployment of BOSS applications online

BMF + Google App Engine

•  http://mashable.com/boss/ •  http://delicious.com/tag/bossmashup

•  Add yours by tagging it with “bossmashup” on

Del.icio.us!

More BOSS Implementations

One more thing… One more thing…

•  The biggest dataset available on web •  Very useful for Web-mining research

experiments – Natural language processing – Semantic extraction

– Related keywords – Similarity detection – Clustering algorithms – Spelling corrections

BOSS in Academic Research

Learn more at: http://developer.yahoo.com/search/boss/

Slides at: http://www.slideshare.net/saurabhsahni/

Questions?

Appendix

BOSS Mashup Framework simplifies aggregating and presenting multiple data sources

Search UI Templates in BOSS Mashup Framework

•  select, group, sort, union, joins, udfs, where •  Text normalization and duplicate removal

•  All-in-memory storage and retrieval operations

•  Ability to join lists of tables via an arbitrary predicate function (map-like)

•  Search UI template framework •  Single search function provides total access to

BOSS REST API

BMF Features