Perl in the Internet of Things

Perl in the Perl in the Internet of Internet of

ThingsThings

Dave CrossMagnum Solutions Ltd

[email protected]

LPW8th November 2014 2

Schedule

9:10 (ish) – Part 1 11:00 – Coffee

Possibly cupcakes 11:30 – Part 2 11:50 – End


What We Will Cover

Perl and the IoT Web Client Primer Web APIs with Dancer Introduction to REST REST APIs in Perl

Perl and the Perl and the Internet of Internet of

ThingsThings


The Internet of Things

Things On the Internet Providing useful services


The Internet of Things


The Hype

Cutting edge interactions Latest technologies Modern languages

Scala Erlang


The Reality

It's just HTTP Usually

Some event triggers action Thing makes an HTTP request Server sends response Thing does something


The Reality

Any language works Perl just as effective as other languages

Of course

Perl has a long history of writing HTTP servers

And HTTP clients Many useful modules


Hardware

Your “thing” is an HTTP client But its interface will be unusual Limited input/output channels Hardware sensors Device::SerialPort Device::BCM2835 Arduino workshop


HTTP Response

Your “thing” won't be displaying web pages So returning HTML is probably unhelpful Data-rich representation

JSON

Web Client Web Client PrimerPrimer


Web Clients

We're used to writing web server applications in Perl And we'll cover more about that later

But we can write web clients too Programs that act like a browser Make an HTTP request Parse the HTTP reponse Take some action


LWP

Most people would reach for LWP LWP is “libwww-perl” Library for writing web clients in Perl Powerful and flexible HTTP client Install from CPAN


HTTP::Tiny

Small web client library for Perl Part of standard Perl installation

Since Perl 5.14 Tiny is good for IoT


Using HTTP::Tiny use HTTP::Tiny;

my $response = HTTP::Tiny->new->get('http://example.com/');die "Failed!\n" unless $response->{success};

print "$response->{status} $response->{reason}\n";

while (my ($k, $v) = each %{$response->{headers}}) { for (ref $v eq 'ARRAY' ? @$v : $v) { print "$k: $_\n"; }} print $response->{content} if length $response->{content};


HTTP::Tiny->new

Create a new user agent object (“browser”) with new()

Various configuration options my $ua = HTTP::Tiny->new(%options);


Options

agent – user agent string cookie_jar – HTTP::CookieJar object

Or equivalent default_headers – hashref


Low-Level Options

local_address keep_alive max_redirect max_size

Of response

timeout


Proxy Options

http_proxy https_proxy proxy no_proxy Environment variables


SSL Options

verify_SSL – default is false SSL_options – passed to IO::Socket::SSL


Making Requests

Use the request() method

$resp = $ua->request( $method, $url, \%options);

Useful options headers content


Responses

Response is a hash Not an object

success – true if status is 2xx url – URL that returned response status reason content headers


Easier Requests

Higher level functions for HTTP requests get head post put delete


Easier Requests Each is a shorthand way to call request() $resp = $ua->get($url, \%options)

$resp = $ua->request( 'get', $url, \%options)

Same options Same response hash


POSTing Data

$ua->request('post', $url, \%options)

$ua->post($url, \%options)

Where do post parameters go? In %options

content key Build it yourself www_form_urlencode()


POSTing Data

$ua->post_form($url, $form_data)

$form_data can be hash ref { key1 => 'value1', key2 => 'value2' }

Or array ref [ key1 => 'value1', key2 => 'value2' ]

Sort order


See Also

HTTP::Tiny::UA Higher level UA features

HTTP::Thin Wrapper adds HTTP::Request/HTTP::Response

compatibility

HTTP::Tiny::Mech WWW::Mechanize wrapper for HTTP::Tiny

Web APIs With Web APIs With DancerDancer

LPW8th November 2014

Dancer

Dancer is a simple route-based web framework for Perl

Easy to get web application up and running See Andrew Solomon's class at 12:00 We're actually going to be using Dancer2


Simple API Return information about MP3s GET /mp3

List MP3s GET /mp3/1

Info about a single MP3


Database

CREATE TABLE mp3 ( id integer primary key, title varchar(200), artist varchar(200), filename varchar(200));


DBIx::Class $ dbicdump -o dump_directory=./lib MP3::Schema dbi:SQLite:mp3.dbDumping manual schema for MP3::Schema to directory ./lib ...Schema dump completed.

$ find lib/lib/lib/MP3lib/MP3/Schema.pmlib/MP3/Schemalib/MP3/Schema/Resultlib/MP3/Schema/Result/Mp3.pm


Data Insert some sample data sqlite> select * from mp3;

1|Royals|Lorde|music/lorde/pure-heroine/royals.mp32|The Mother We Share|Chvrches|music/chvrches/the-bones-of-what-we-believe/the-mother-we-share.mp33|Falling|Haim|music/haim/days-are-gone/falling.mp3


Create Application $ dancer2 gen -a MP3

+ MP3[ ... ]+ MP3/config.yml[ ... ]+ MP3/lib+ MP3/lib/MP3.pm+ MP3/bin+ MP3/bin/app.pl


Run Application

$ MP3/bin/app.pl

>> Dancer2 v0.153002 server 6219 listening on http://0.0.0.0:3000


Run Application


Implement Routes

Our application doesn't do anything Need to implement routes Routes are defined in MP3/lib/MP3.pm get '/' => sub { template 'index';};


Implement Routes



Implement Routes



Implement Routes



/mp3 Route

Display a list of MP3s Get them from the database Use Dancer2::Plugin::DBIC In config.yml Plugins: DBIC: default: dsn: dbi:SQLite:dbname=mp3.db


/mp3 Route

In MP3/lib/MP3.pm get '/mp3' => sub { my @mp3s = schema->resultset('Mp3')->all; content_type 'text/plain'; return join "\n", map { $_->title . ' / ' . $_->artist } @mp3s;};


/mp3 Route



/mp3 Route



/mp3 Route



/mp3 Route



/mp3 Route


/mp3 Route


/mp3/:id Route

Display details of one MP3 Dancer gives us parameters from path $value = param($name)


/mp3/:id Route get '/mp3/:id' => sub { my $mp3 = schema->resultset('Mp3') ->find(param('id'));

unless ($mp3) { status 404; return 'Not found'; } content_type 'text/plain'; return $mp3->title . "\n" . $mp3->artist . "\n" . $mp3->filename;};











/mp3/:id Route


/mp3/:id Route


Plain Text?

Yes, plain text is bad Easy fix Serializer: JSON

In config.yml Rewrite routes to return data structures Dancer serialises them as JSON


Return Data get '/mp3' => sub { my @mp3s = schema-> resultset('Mp3')->all;

return { mp3s => [ map { { title => $_->title, artist => $_->artist } } @mp3s ] };};














Return Data get '/mp3/:id' => sub { my $mp3 = schema-> resultset('Mp3')->find(param('id')); unless ($mp3) { status 404; return 'Not found'; }

return { title => $mp3->title, artist => $mp3->artist, filename => $mp3->filename, };};


Return Data get '/mp3/:id' => sub { my $mp3 = schema-> resultset('Mp3')->find(param('id')); unless ($mp3) { status 404; return 'Not found'; }

return { $mp3->get_columns };};


JSON


JSON


JSON


JSON


URLs

It's good practice to return URLs when you can

Easier for clients to browse our data We can do that for our list


URLs get '/mp3' => sub { my @mp3s = schema->resultset('Mp3')->all;

my $url = uri_for('/mp3') . '/'; return { mp3s => [ map { { title => $_->title, artist => $_->artist, url => $url . $_->id, } } @mp3s ] };};


URLs get '/mp3' => sub { my @mp3s = schema->resultset('Mp3')->all;

my $url = uri_for('/mp3') . '/'; return { mp3s => [ map { { title => $_->title, artist => $_->artist, url => $url . $_->id, } } @mp3s ] };};


URLs $ GET http://localhost:3000/mp3

{"mp3s":[{"url":"http://localhost:3000/mp3/1","title":"Royals","artist":"Lorde"},{"url":"http://localhost:3000/mp3/2","artist":"Chvrches","title":"The Mother We Share"},{"url":"http://localhost:3000/mp3/3","artist":"Haim","title":"Falling"}]}


More on URLs

Currently our system has only one resource mp3

It's usual to have links to other resources MP3s have artists Link to other resources using URLs Make it easier for clients to walk our data

model


More on URLs

In our MP3 JSON we have this “artist”:”Lorde”

It would be better to have “artist_name”:”Lorde”

“artist_url”:”http://localhost:3000/artist/1”

Perhaps add a url() method to all of our objects


Other GET Actions

Getting lists of objects is easy Other things to consider Searching Sorting Paging Filtering CGI Parameters to DBIC to SQL to JSON


Other Actions

We will want to to other things to our data Add objects Update objects Delete objects CRUD operations


Other Actions

Use HTTP methods POST /mp3

Create GET /mp3/:id

Read

PUT /mp3/:id Update

DELETE /mp3/:id Delete


Other Actions

Use HTTP methods POST /mp3

Create GET /mp3/:id

Read

PUT /mp3/:id Update

DELETE /mp3/:id Delete


Other Actions

Easy to write Dancer handlers for these delete '/mp3/:id' => sub { schema->resultset('Mp3')-> find(param('id'))-> delete; }

But it can be hard to get right What should we return here? Is there a better way?

RESTREST


REST

Representational State Transfer Abstraction of web architecture Dissertation by Roy Fielding, 2000 Particularly applicable to web services


RESTful Web Services

Base URI for service Defined media type HTTP methods for interaction Hypertext links for resources Hypertext links for related resources


RESTful vs Non-RESTful

Good test Which HTTP methods does it use? Web services often use only GET and POST GET /delete/mp3/1 GET /mp3/1/delete Not RESTful DELETE /mp3/1 Might be RESTful


RESTful Dancer

Dancer has a REST plugin Dancer2::Plugin::REST Makes our live much easier


Dancer2::Plugin::REST

Does three things for us Creates routes Utility functions for return values Returns data in different formats


Creates Routes

resource mp3 => get => sub { ... }, create => sub { ... }, delete => sub { ... }, update => sub { ... };


Creates Routes

resource mp3 => get => sub { ... }, create => sub { ... }, delete => sub { ... }, update => sub { ... };


CRUD Routes post '/mp3'

Create

get '/mp3/:id'

Read

put '/mp3/:id'

Update

delete '/mp3/:id'

Delete


CRUD Routes post '/mp3'

Create

get '/mp3/:id'

Read

put '/mp3/:id'

Update

delete '/mp3/:id'

Delete


Creates Routes resource mp3 => get => sub { my $mp3 = schema->resultset('Mp3')-> find(params->{id});

if ($mp3) { status_ok( { $mp3->get_columns } ); } else { status_not_found('MP3 Not Found'); } };


Creates Routes

Note: Still have to create main listing route /mp3


Utility Functions

Simple status_* functions for return values

status_ok(\%resource) status_not_found($message) status_created(\%new_resource)


Format Options

Allow user to choose data format By changing the URL get '/mp3/:id'

get '/mp3:id.:format'

YAML, JSON, Data::Dumper support built-in


Format Options - JSON $ GET http://localhost:3000/mp3/1.json

{"id":1,"filename":"music/lorde/pure-heroine/royals.mp3","title":"Royals","artist":"Lorde"}


Format Options -YAML $ GET http://localhost:3000/mp3/1.yml

---artist: Lordefilename: music/lorde/pure-heroine/royals.mp3id: 1title: Royals


Format Options -YAML $ GET http://localhost:3000/mp3/1.dump

$VAR1 = { 'filename' => 'music/lorde/pure-heroine/royals.mp3', 'title' => 'Royals', 'artist' => 'Lorde', 'id' => 1};


More Complex REST Dancer and Dancer2::Plugin::REST make

simple REST easy Other frameworks do the same

But full REST support is more complex Here's a REST state machine





Read a Good Book


CPAN to the Rescue

There's a lot to think about when getting REST right

Can CPAN help? Web::Machine WebAPI::DBIC


Web::Machine

Perl port of Erlang webmachine With bits stolen from Ruby and Javascript

versions too

You write subclasses of Web::Machine::Resource

Override methods where necessary See Stevan Little's YAPC::NA 2012 talk


Example use Web::Machine;

{ package WasteOfTime::Resource;

use parent 'Web::Machine::Resource'; use JSON::XS qw(encode_json);

sub content_types_provided { [{ 'application/json' => 'to_json' }] }

sub to_json { encode_json({ time => scalar localtime }) }}

Web::Machine->new( resource => 'WasteOfTime::Resource')->to_app;


Example $ plackup time.psgi

HTTP::Server::PSGI: Accepting connections at http://0:5000/

$ curl -v http://0:5000 [ ... ] < HTTP/1.0 200 OK < Date: Sat, 08 Nov 2014 11:34:02 GMT < Server: HTTP::Server::PSGI < Content-Length: 35 < Content-Type: application/json < * Closing connection #0 {"time":"Sat Nov 8 11:34:02 2014"}


Example $ curl -v http://0:5000 -H'Accept: text/html'

[ ... ]

< HTTP/1.0 406 Not Acceptable < Date: Sat, 08 Nov 2014 11:34:02 GMT < Server: HTTP::Server::PSGI < Content-Length: 14 < * Closing connection #0 Not Acceptable


Example sub content_types_provided { [

{ 'application/json' => 'to_json' }, { 'text/html' => 'to_html' },] }







sub to_html { my $time = localtime; return “<html> <head><title>The Time Now Is:</title></head> <body> <h1>$time</h1> </body></html>”;}


Example $ curl -v http://0:5000 -H'Accept: text/html'

[ ... ] < HTTP/1.0 200 OK < Date: Sun, 09 Dec 2012 02:26:39 GMT < Server: HTTP::Server::PSGI < Vary: Accept < Content-Length: 103 < Content-Type: text/html < * Closing connection #0 <html><head><title>The Time Now Is:</title></head><body><h1>Sat Nov 8 11:34:02 2014</h1></body></html>


WebAPI::DBIC

“WebAPI::DBIC provides the parts you need to build a feature-rich RESTful JSON web service API backed by DBIx::Class schemas.”

REST API in a box


WebAPI::DBIC

Built on top of Web::Machine And Path::Router And Plack Uses JSON+HAL

Hypertext Application Language


Try It Out $ git clone https://github.com/timbunce/WebAPI-

DBIC.git

$ cd WebAPI-DBIC

$ cpanm Module::CPANfile

$ cpanm --installdeps . # wait ...

$ export WEBAPI_DBIC_SCHEMA=DummyLoadedSchema

$ plackup -Ilib -It/lib webapi-dbic-any.psgi

... open a web browser on port 5000 to browse the API


Try It Out (Your Schema) $ export WEBAPI_DBIC_SCHEMA=Foo::Bar

$ export WEBAPI_DBIC_HTTP_AUTH_TYPE=none

$ export DBI_DSN=dbi:Driver:...

$ export DBI_USER=...

$ export DBI_PASS=...

$ plackup -Ilib webapi-dbic-any.psgi

... open a web browser on port 5000 to browse the API

ConclusionConclusion


Conclusion

Perl is great for the Internet of Things Perl is great for text processing Perl is great for network processing IoT apps are often mainly HTTP transactions

And Perl is good at those


Sponsor's Message

Perl Training Central London Next week Intermediate Perl

11/12 Nov Advanced Perl

13/14 Nov See advert in brochure


Sponsor's Message

Perl Training Central London Next week Intermediate Perl

11/12 Nov Advanced Perl

13/14 Nov See advert in brochure

That's All FolksThat's All Folks• Any Questions?

Perl in the Internet of Things

Technology