Why my iPhone sucks: Screen Scraping the O2 Website Simon Lewis - [email protected]@lewis.li.

Why my iPhone sucks:

Screen Scraping the

O2 Website

Simon Lewis http://lewis.li - [email protected]

Overview

• Observations about the iPhone

• Why Screen Scrape?

• MMS solution

• Conclusions

Good Points(Before the Bashing)

• Web Browser

• SSL IMAP Email Client

• Alarm Clock

• Timer

Bad Points

• EDGE

• Bluetooth

• NO 3G

Wet piece of string is better connected

(Camera, Phone Interface, Text Message Interface)

MMS?(maybe it’s not that popular)

ERROR

Mocking from O2

Message List Window

Beautiful Interface?

• No web service available

• http://o2.co.uk/m not optimised for iPhone

There must be a better Solution...

• HTTP interface can be exploited

• iPhone Mail app is very nice: SSL IMAP

Problems:

Opportunities:

Screen Scraping?• Programmatically accessing webpage

content

• Parsing the contents of the page to extract the information you want

• Need a language suited to text parsing

Laziness - one of the principle virtues of a programmer

What tools are available?

Perl

CPAN(Comprehensive Perl Archive Network)

use Config::IniFiles;use WWW::Mechanize;use Getopt::Long;use MIME::Lite;use YAML qw(LoadFile DumpFile);use Data::Dumper;use CGI qw(:standard);

System Overview

lewis.li

o2.co.uk

iPhone1

2

3

4

5

Authentication

my $mech = WWW::Mechanize->new( autocheck => 1 );$mech->get( $site_url );$mech->submit_form( form_name => 'fm', fields => { msisdn => $user_name, pin => $user_password });

Inbox Listing

$mech->content =~ /From:.+?<p>(.+?)<\/p>.+?Subject:.+?<p>(.+?)<\/p>/s; my @mms_message_links = $mech->find_all_links( url_regex => qr/showMessage/ );my %unique_mms_links = map { $_, 1 } map { $_->url_abs } @mms_message_links;

Iterating through MMS

for my $option (@other_options){ $mech->current_form->value('selectedItem', $option); $mech->submit(); $filename = "/tmp/mms$message_part"; open $fh, "> $filename" or die "$filename: $!"; binmode $fh; print $fh $mech->content; close $fh; $mms_details->{$filename} = $mech->ct(); $mech->back();}

Creating the Email

my $msg = MIME::Lite->new( From =>"$sender", To =>"$mail_recipient", Subject =>"$subject", Type =>'multipart/mixed'); for my $part (keys %$mms_details) { ( my $nice_name = $part) =~ s!^/tmp/!!; if ($mms_details->{$part} eq 'image/jpeg') { $msg->attach(Type =>'image/jpeg', Path => $part, Filename =>"$nice_name.jpeg", Disposition => 'attachment'); }} $msg->send;

Receiving MMS via Email

Conclusions

• CPAN had all the modules needed to automate interaction

• Messages on the O2 website expire. Email provides a good way to archive content

• WWW::Mechanize is very useful

• Radio buttons were the biggest challenge