Idea2app

Idea 2 app in 1 month

A birds eye view of rails app development to cut corners, save time, save money and

have fun

Before

After

An idea

Ideas are worth nothing

Development & Marketing take time

Original, simple ideas in 1 Month

The problem

“Its hard to follow and join inconversation round a given

topic with social media. Many people don’t get twitter”

The solution

“Use advanced search to curate

simple, useful conversationsaround popular topics.”

Find the interesting stuff easily

Design first

- I’m not a designer- Easy to build- Simple to understand- Min input for max useful output

Mock it up

The design

Also radio, confs, news, brands…

Chat room style2 pages, 1 click.

Twitter Search – Poll vs Stream

- Twitter to do clever stuff- Fast flow of relevant tweets- Quick development- Cheap to host

Polling

+ Super simple REST api+ Powerful query language+ No authentication- Variable unpublished rate limits- Result speed can vary

Polling Code

iprug OR from:iprug OR “ipswich ruby” OR ruby near:ipswich -wedding

http://search.twitter.com/search.json?q=QUERY&result_type=recent&rpp=100&lang=en

POWERFUL QUERIES

SIMPLE API

Streaming

+ Near real-time access+ Simple REST api+ Keyword filters+ AND and OR- No phrases or NOT- Hogs workers/processes- Not well suited to Heroku architecture

Streaming Code

track=iprug, ipswich ruby

http://stream.twitter.com/1/statuses/filter.json?track=iprug,ipswich ruby-uYOUR_TWITTER_USERNAME:YOUR_PASSWORD

TRACKING KEYWORDS

SIMPLE API

Streaming In Ruby

gem install tweetstream

TweetStream::Daemon.new(’usr','pwd') .track('term1’, 'term2') do |status| puts "#{status.text}" end

THERE’S A GEM FOR THAT

BUILT IN DAEMONIZATION

Lazy Twitter Solution

status = Timeout::timeout(timeout) do r = Curl::Easy.perform(url) do |c| c.headers["User-Agent"] = ”TB/1.1” end response = r.body_strend

SIMPLEST: CURB THE BASIC REST API

gem install curb

Lots of DB Writes

tweets = []raw_tweet = grab_tweets()raw_tweet.times do |t| tweets << Tweet.new(to_fields(raw_tweet))endTweet.import tweets

OPTIMIZE IMPORT

gem install ’activerecord-import’

Twitter API Quirks

- JSON and XML have different data- With Heroku must set a user-

agent- Speed Varies- Query Complexity limits- Unpredictable rate limits

Twitter Sentiment Analysis

Bayesian Classification with Classifier gem+ Interesting for age, mood, personality type- Not so good for sentiment on object

require 'classifier’b = Classifier::Bayes.new ’positive', ’negative’b.train_positive ”I <3 my iPhone”b.train_negative ”iPhone can’t make calls #fail”b.classify ”iPhone is a rip off #fail" # returns ’negative'

Twitter Sentiment Analysis

Natural Language Processing with Ruby Linguistics

Find the direct object of the sentence => “dog”

"he is a big dog".en.sentence.object.to_s

"he is a big dog".en.sentence.verb.infinitive

Find the infinitive verb form of the sentence => “be”

Twitter Sentiment Analysis- Combination of the 2 seems to be

successful for some people.- Seems complicated- No I’m too lazy- Cheat: Tweetfeel API

http://svc.webservius.com/v1/tweetFeel/tfapi?type=all&wsvKey=mykey&keyword=iprug

Extracting Interesting Stuff

- I <3 rubular & regex- Detect mentions

- Detect links

- Resolve links

f.match(/@[a-zA-Z0-9_]+/)

starts_with?('http:')

http://api.longurl.org/v2/expand?url=http%3A%2F%2Fis.gd%2Fw&title=1

Extracting Video & Images

- Known video sites youtube, vimeo, livestream

- Known image sites yfrog, twitpic, intagram etc

- Get thumbnails don’t make your own #lazyclass TwitpicResolver < ImageResolver SEARCH_SNIPPET = "twitpic" URL = "http://twitpic.com/show/[size]/[id]" THUMB = "thumb" LARGE = "large" ID_EXTRACTOR = /twitpic\.com\/(.*)/end

UI Stuff- SASS & Blueprint = Compass- SASS is awesome, nesting, variables,

mixins- SCSS default SASS syntax now- HAML- CSS cropping = No imagemagik faff- Highcharts.js (not free)

Lazy Admin- http://ruby-toolbox.com/- activescaffold- Typus is the laziest

- search - authentication - paperclip - configure forms via yaml file

Lazy Hosting = Heroku

git push heroku master

Periodic Workers = Heroku

namespace :tweet do

desc "kick off background jobs” task :eat => :environment do

require 'workers' Delayed::Job.enqueue(TwitterJob.new() Delayed::Job.enqueue(TwitteratiJob.new()) Delayed::Job.enqueue(ExtracterJob.new()) Delayed::Job.enqueue(Sentiment.new())

endend

heroku rake tweet:eat

Workers Cont…

Delayed::Job.enqueue TwitterJob(), :run_at => 10.seconds.from_now

gem install delayed_job_admin

Heroku rake jobs:clear

Heroku not that cheap - Shared database 20GB $15- 1 Worker $36- 1 Dyno = Free- Minimum of $56- Will need at least 2 workers and 2

Dynos and soon I’ll need more storage too

The Pragmatic/Lazy Way- Reuse, reuse, reuse- 3rd party libraries (highchart, typus,

nokogiri)- 3rd party api’s (tweetfeel, longurl)- Keep looking for an easier way- Simple design- Let someone else worry about

hosting

The Missing Twitter API- Case sensitive search- Term extraction (Zemanta, YQL)- User age, gender, interests- Potty mouth- Traveller?

Did I do it in 1 month?- Well kind of…- Live on Heroku eating tweets- http://falling-galaxy-55.heroku.com/- Time consuming adding data- Some conversations work great, some

don’t- Need a bit more filtering cleverness- Target is a great way of following SXSW- Then onto rating brands, products,

mobile…

Adverts- Go to talkbackhq.com (next week) - Interested in getting involved?- Suffolk Twestival @twestuffolk.

Evening of Thu 24th March @ Brewery Tap