Visual Interactivity

Mobile

… or mobile interactive services for normal people

Geoff Ballingerhttp://www.mobileacuity.comhttp://blog.geoffballinger.co.uk

Overview• what is the problem?• some real world motivation• some fun examples• where does AR fit in?• visual search ...• how do developers engage?

Survey …• how many people here:– use a mobile phone for voice calls?– have sent a message from their phone?– have a camera on their phone?– have used the camera on their phone?– have visited a website from their phone?– have downloaded an app or a game?

• what do you think the answers would be for the typical “normob” on the street?

x x x x x x x

Brand Marketing

• mobile becoming a serious option• often combined with traditional

media• needs to be inclusive• mustn’t be off-putting

• actually all of this can be said in general about good mobile services …

x

The Camera• … has now arrived:

Some dull stats

• in 2008, 166.6m handsets (Western Europe) are registered as MMS-enabled. That is a 95% penetration!

• over 90% of all handsets currently being sold have a cameraphone.

• in the 11-19 demographic 53% regularly use picture messaging, with 85% having used it at least once.*

• in the 16-34 demographic 78% have used picture messaging with 48% using it regularly.* Source TGI Youth Q2 2008

x x x x

2D bar codes

Patent No. 6,999,573 31st Jan 2006

x

x

Recognition• the simplest form of interactivity• replaces SMS and keywords etc.

• but have to ask why bother …• … large number of media which can’t

necessarily be adapted?x x x x x

Who’s who

PixTo

Image Zoning

• activation of Vodafone’s Champions League sponsorship

• take penalties by taking a picture of where you want to aim the ball

• send in by MMS to see whether you score or not

• true user interaction

Aim with your camera phone See the outcome in your web browser

x

Colour• Nike PhotoiD by AKQA is a great

example

Videos• Shoot and Score• Darts• Nike PhotoID

Blue-screen• run as part of the

Vodafone Live! Music Awards

• think blackpool beach – users stick their heads through the display

• picture sent in by MMS, heads cut out, and incorporated into a music video

Snap your mates on a camera phone See them in a personalised video

x

Facefinder• think bluescreen without the screen• automatic detection and extraction

of the faces in any image• much more suitable for viral apps

• currently all face-likefeatures:

• in future will distinguishbetween these and real faces

x

Videos• You Rockstar!• Facefinder

AR• What actually is Augmented Reality?• Extra digital information overlaid on

the real world.• AR Toolkit – 3D shapes triggered by

2D codes in the environment.• Recently GPS and compass based

apps such as , Wikitude, and Bionic Eye.

AR Mobile Gaming

• Also “magic monstervision” game mentioned by Paul Coulton earlier!

Layar

Visual AR• Specific and accurate placement of

digital info into the video stream.

Videos• Arf video• snap2Win Live

Visual Search

• General Visual Search?• Recognise “all” images?• Make the world clickable?• Science Fiction?• Product Search is happening

now!

Product Search

• Snaptell Explorer for Amazon• Barnes and Noble store app• Mobile Acuity Visual Shopper

Visual Shopper

Web Services

• How does the developer get involved with this?

• Some related code out there – Haar classifiers and AR Toolkit etc

• Web service API prototypes from Kooaba and Mobile Acuity.

• Build into your apps.

Summary