Web services made easy with YQL

Web services made easy with YQL

Chris&an Heilmann ‐ Global Radio Brown Bag ‐ London, England, June 2010

I am Chris (@codepo8)

I love the web as a medium.

And I hate taking unnecessary steps.

So if I want to get data off the web, I don’t feel like going through lots of steps.

1. Find data API2. Request key 3. Receive key4. Find data endpoint5. Read docs to learn how to

authenticate6. Read docs to know what

parameters you have.7. Get massive amount of

data in some obscure format

8. Use data after converting and filtering it.

Instead I want this:

1. Find data2. Request, filter and

convert data.3. Use data.

And for that, I can use YQL.

YQL    hAp://developer.yahoo.com/yql/console/

YQL    hAp://developer.yahoo.com/yql/console/

select {what} from {where} where {condi&ons}

YQL turns the web into a massive database.

You can select, filter, sort and limit data and you can even insert, update and delete from it.

YQL in itself is a web service, and you don’t need any Yahoo product to use it.

Using the YQL console makes it very easy to use it.

1) Define your Query

2) Select your output

3) Copy the URL.

Or the manly way...

https://query.yahooapis.com/v1/public/yql?q={uri-encoded-query}&format={xml|json}&diagnostics={true|false}&callback={function}&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys

Using the YQL console makes it very easy to use it.

You can use JSON-P and just copy and paste the URL in JavaScript:

Or create nodes to do it dynamically.

Display with PHP:

So what about some demo queries?

select * from search.web where query="london heart"

select abstract from search.web where query="london heart"

select abstract from search.web(100) where query="london heart"

select * from search.termextract where context in (select abstract from search.web(100) where query="london heart")

select * from search.termextract where context in (select abstract from search.web(100) where query="london heart") | sort(field="Result") | unique(field="Result")

select * from query.multi where queries=' select Title,Description,Url,DisplayUrl from microsoft.bing.web(20) where query="cat"; select title,clickurl,abstract,dispurl from search.web(20) where query="cat"; select titleNoFormatting,url,content,visibleUrl from google.search(20) where q="cat"'

http://icant.co.uk/goohoobi/index.php?research

http://vimeo.com/8075850

select * from flickr.photos.search(50) where text="XFM" and has_geo='true'

select * from geo.places where text="glastonbury"

select * from flickr.photos.search(50) where woe_id in (select woeid from geo.places where text="glastonbury")

select * from flickr.photos.search(50) where woe_id in (select woeid from geo.places where text="glastonbury") and min_taken_date = "2009"

I love using geolocation services.

http://isithackday.com/hacks/geo/yql-geo-library/

http://isithackday.com/hacks/geo/around-you/

hAp://isithackday.com/geoplanet‐explorer/

hAp://isithackday.com/hacks/geo/addmap.html

<script src="addmap.js"></script><script>addmap.config.mapkey = 'YOUR_API_KEY';addmap.analyse('content');</script>

YQL is not limited to Yahoo data - at all.

Articles of the New York Times:

select * from nyt.article.search where query="football"

Headlines of elpais.com:

select content from html where url="http://elpais.com" and xpath="//h2/a"

Keywords extracted from elpais.com headlines.

select * from search.termextract where context in( select content from html where url="http://elpais.com" and xpath="//h2/a")

elpais.com headlines in English:

select * from google.translate where q in(select content from html where url="http://elpais.com" and xpath="//h2/a") and target="en"

Updating Twitter:use 'http://www.yqlblog.net/samples/twitter.status.xml';insert into twitter.status (status,username,password)values ( "In your redinnova, blowing ur mindz", "codepo8", "didyoureallythinkIshowit?")

Updating Wordpress:

insert into wordpress.post (title, description, blogurl, username, password) values ("Test Title", "This is a test body", "http://ajaxian.com", "codepo8", "iedoesitright")

YQL is open - if you want your data in this system, all we need is an address where to get it :)

Join us! http://github.com/yql/yql-tables

You can even write server-side JavaScript to build a webservice that way.

One last thing to access through YQL is the Yahoo Firehose.

http://developer.yahoo.net/blog/archives/2010/04/yahoo_updates_firehose.html

Enough talk, let me show you YQL.

Christian Heilmann http://wait-till-i.com http://developer-evangelism.com http://twitter.com/codepo8

Thanks!