WildHeart.HTTPListener WildHeart.HTTPListener an open source Web Server an open source Web Server for Dyalog APL v11.0 for Dyalog APL v11.0 Stefano “WildHeart” Lanzavecchia
Jan 13, 2016
WildHeart.HTTPListeneWildHeart.HTTPListenerr
an open source Web Server an open source Web Server for Dyalog APL v11.0for Dyalog APL v11.0
Stefano“WildHeart”
Lanzavecchia
2006-10-17 Dyalog User Group Conference 2
AgendaAgenda
APL should be a popular language for developing web pages, because:– APL has all you need to generating static and
dynamic web content
It is not used as much as it deserves, because:– Even the best tools need good frameworks to
level the learning curve a bit– No frameworks are readily available
2006-10-17 Dyalog User Group Conference 3
GoalGoal
Explore the problem which needs to be solved and demonstrate a proposed solution... (obviously )
So, what’s the problem?The easy part is generating HTML Hopefully nobody will disagree that APL is
an EXCELLENT language for generating character vectors
2006-10-17 Dyalog User Group Conference 4
A ”web”
Some very simple HTMLSome very simple HTML
2006-10-17 Dyalog User Group Conference 5
The hard partThe hard part
If we have a form like this and the user hits the submit button…
2006-10-17 Dyalog User Group Conference 6
Before we look at the response…Before we look at the response…
<form action="RUN" method="post" name="FrontPage_Form1"><div> <table border="0" cellpadding="3"> <tr> <td>Purchase Amount</td> <td><input type="text" size="9" name="LoanAmt"/></td> </tr> <tr> <td>Percent Down</td> <td><input type="text" size="6" maxlength="6" name="PercentDown"/></td> </tr> <tr> <td>Maximum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="LenMax"/></td> </tr> <tr> <td>Minimum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="LenMin"/></td> </tr>
… continued on the right …
<tr>
<td>Maximum Interest Rate (%)</td>
<td><select name="IntrMax" size="1">
<option>15.0</option>
<option>14.5</option>
..etc down to…
<option>1.0</option></select></td> </tr><tr> <td>Minimum Interest Rate (%)</td> <td><select name="IntrMin" size="1"> <option>15.0</option> <option>14.5</option> <option>14.0</option> <option>13.5</option> <option selected="selected">13.0</option> <option>12.5</option> <option>12.0</option>
…etc… <option>1.5</option>
<option>1.0</option> </select></td> </tr> </table> <p><input type="submit" value="Calculate
Repayments"/></p></div></form>
Some of you will want to knowhow the HTML looks for the form?
2006-10-17 Dyalog User Group Conference 7
When the user hits the button…When the user hits the button…
POST /loan/RUN HTTP/1.1Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel,application/vnd.ms-powerpoint, application/msword, */*Referer: http://localhost:8080/loan/loan.htmAccept-Language: itContent-Type: application/x-www-form-urlencodedAccept-Encoding: gzip, deflateUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; WinFX RunTime 3.0.50727)Host: localhost:8080Content-Length: 74Connection: Keep-AliveCache-Control: no-cache
LoanAmt=10000&PercentDown=50&LenMax=10&LenMin=15&IntrMax=15.0&IntrMin=13.0
2006-10-17 Dyalog User Group Conference 8
The hard part…The hard part…
APL is not a bad tool for taking character vectors apart, either
But: Some knowledge about the HTTP protocol is required to decipher it
A rule for calling the code to deal with the request (in this case #.loan.RUN) is needed
Some way to decode the body of the request
This is what Web Server frameworks like Microsoft IIS coupled with ASP.NET do…
2006-10-17 Dyalog User Group Conference 9
What is a web application?What is a web application?
“A collection of functions that take an HTTP Request as input and produce
an HTTP response as output”
2006-10-17 Dyalog User Group Conference 10
SolutionsSolutions
Dyalog APL integrates quite well with Microsoft IIS+ASP.NET, and can be called from other web servers, but:– These solutions require a lot of heavy machinery from
other vendors– The learning curve is still quite high
Dyalog provide a sample SERVER workspace, but:– It leaves almost all the work of deciphering HTTP to you– It’s based on old standards
2006-10-17 Dyalog User Group Conference 11
SolutionsSolutions
The author has:A keen (geek) interest in OO, web
and other software development methodologies
Written numerous tools on top of the Dyalog TCPSocket object
2006-10-17 Dyalog User Group Conference 12
Enter WildHeart.HTTPListenerEnter WildHeart.HTTPListener
The advent of Object Orientation in Dyalog 11.0 makes it attractive to start again
My goal is to create a framework which:– Requires a little knowledge of HTML and hardly any
knowledge of HTTP– Allows ”ordinary” APL developers to easily write Web
applications, yet:– Is built from reusable, replaceable components which
can be extended by APL developers who learn a little more...
– Is portable across all platforms which support TCP– Is run as an open source project, to be shared and
extended by anyone who wishes to contribute
2006-10-17 Dyalog User Group Conference 13
To write or To write or Not to writeNot to write
Why NOT TO WRITE a web server in APL:– This is the third web server I develop:
• I was paid to develop the first (and it was never used)
• I was almost paid to the develop/steal the second (and it’s heavily used)
• I was NOT paid to develop the third and I had to find the time to do it at night and during weekends
A wise man once said:
“only a fool would do that”
2006-10-17 Dyalog User Group Conference 14
Not to write, seriouslyNot to write, seriously
HTTP is easy but trickyReliabilitySecuritySpeedSo many web servers out there
2006-10-17 Dyalog User Group Conference 15
To writeTo write or not to write or not to write
Why write a web server in APL?(I wrote an editorial about it in Vector a few years ago)
– Easily embeddable in an APL application
– Easily understood by APLers– Can stay near the core of the
application
2006-10-17 Dyalog User Group Conference 16
To writeTo write or not to write or not to write
"One of the reasons to use Lisp in writing Web-based application is that
you can use Lisp. When you're writing software that is only going to
run on your own servers, you can use whatever language you want.“
Paul Graham
2006-10-17 Dyalog User Group Conference 17
To write!To write!
“Morten needs a demo. I need an HTTP/1.1 web server, written in APL. APL Italiana, the company I work for, also needs one, even if they haven’t
realised this yet.”
Stefano Lanzavecchia
2006-10-17 Dyalog User Group Conference 18
Components of… WildServer’06Components of… WildServer’06
HTTPListener HTTPRequest
– Headers– Cookies– Request Variables
HTTPResponse– Cookies
HTML Utils Event pipeline
– User Validation– Sessions
Static file server Namespace as
filesystem…
File Handlers:– aplx files– apls files– …
2006-10-17 Dyalog User Group Conference 19
Dyalog APL: an OO languageDyalog APL: an OO language
Web server: a good test bed for the OO extensions– Abstractions: divide et impera– Inheritance: this looks a bit like that– Keyed properties: syntax tricks
What is missing in the language?Is the IDE good enough?
2006-10-17 Dyalog User Group Conference 20
HTTPListenerHTTPListener
fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve')listener←⎕NEW web.HTTPListener(8080 fs)listener.Start
Then point your web browser at http://localhost:8080/
“It’s a kind of magic”
2006-10-17 Dyalog User Group Conference 21
The trip of an HTTP requestThe trip of an HTTP request
TCPSocket gets TCPRecv eventsHTTPRequest objects collects them,
and parses the request according to the HTTP specs
When the request is complete, it gets shipped to a dispatcher…
2006-10-17 Dyalog User Group Conference 22
The dispatcher: FileSystemThe dispatcher: FileSystem
To me a FileSystem looks like a namespace:– Its folders are sub-namespaces– Idea: we can inject virtual folders:
• A virtual folder can be a folder on the HD coming from a different sub-folder
• A virtual folder can be… an APL namespace! Its functions would be files…
2006-10-17 Dyalog User Group Conference 23
APLNamespaceAPLNamespace
http://localhost:8080/loan/RUNexecutes #.loan.RUN
buthttp://localhost:8080/loan/bullet.gif
will transmit the file c:\aplserve\loan\bullet.gif
fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve')fs['loan']←⎕NEW web.APLNamespace(#.loan'c:\
aplserve\loan')listener←⎕NEW web.HTTPListener(8080 fs)listener.Start
2006-10-17 Dyalog User Group Conference 24
ZipFileSystemZipFileSystem
Now the folders and files come from a zip file. Neat, uh? In fact this presentation…
fs←⎕NEW web.ZipFileSystem('c:\aplserve.zip?aplserve')
fs['loan']←⎕NEW web.APLNamespace(#.loan'c:\aplserve\loan')
listener←⎕NEW web.HTTPListener(8080 fs)listener.Start
2006-10-17 Dyalog User Group Conference 25
FileSystemFileSystemBasically one method: ProcessRequest∇ ProcessRequest ctx;html;ns;raw;aplo :Access Public Instance Override ctx←#.WildHeart.HTTPContext.Current html←_NS⍎MapFileName ctx.URI r←ctx._Response r._IsBodyRaw←raw←82≠⎕DR html r._Body←html :If ''≡⊃r._Headers[⊂'Content-type'] r._Headers[⊂'Content-type']←u.MimeMappings['htm'
'bin'[1+raw]] :EndIf ctx.SendResponse ∇
2006-10-17 Dyalog User Group Conference 26
Tricks: thread local storageTricks: thread local storage
[…] ctx←#.WildHeart.HTTPContext.Current[…]
Thread context: wouldn’t it be nice to have an object local to a thread?
2006-10-17 Dyalog User Group Conference 27
HTTPHandlerHTTPHandler
A FileSystem delegates the handling of the requested resource to the appropriate handler:– StaticFileHandler by default– AplxHandler– AplsHandler– … more to come fs._FileHandlers[⊂'aplx']←#.WildHeart.AplxFileHandler
2006-10-17 Dyalog User Group Conference 28
HTTPHandlerHTTPHandler
One method: ProcessRequest ∇ ProcessRequest ctx;fname;tn;ext;etag :Access Public Instance Overridable fname←ctx._FileSystem.MapFileName ctx.URI tn←fname ⎕NTIE 0 ctx._Response._IsBodyRaw←1 ctx._Response._Body←⎕NREAD tn,83,⎕NSIZE tn ⎕NUNTIE tn ctx.SendResponse ∇
2006-10-17 Dyalog User Group Conference 29
AplxHandlerAplxHandler
“This is only a test”<?@ master="../newindex.maplx" execute="" ?>
<apl:content id="title"><title>Rain Graphics: Climate</title></apl:content><apl:content id="cssstyle"><link rel="stylesheet" type="text/css"
href="../styles/spiffy.css"/></apl:content>
<apl:content id="main" execute="#.rain.Climate"></apl:content>
2006-10-17 Dyalog User Group Conference 30
A master pageA master page<!DOCTYPE html PUBLIC "-//W3C//DTD
XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns=“http://www.w3.org/1999/xhtml” xmlns:apl="http://wildheart.com/2006/10/"
><head>
<apl:placeholder id="cssstyle"><link rel="stylesheet"
type="text/css" href="styles/spiffy.css"/>
</apl:placeholder><apl:placeholder id="title"><title/></apl:placeholder></head>
<body><div id="header"><h1
class="banner">WildHeart.HTTPListener</h1></div>
<div id="subheader"> <apl:placeholder id="bcrumb" execute="#.WildHeart.WebDemos.BreadCrumb" />
</div><div id="toc">
<apl:placeholder id="apltoc"><div class="toc"><h2>APL Web
Server</h2><ul><li><a
href="/intro.aplx">Introduction</a></li>
<li><a href="/cv/Index.aplx">View Code</a></li>
…
2006-10-17 Dyalog User Group Conference 31
The event pipelineThe event pipeline
As a request goes through the FileSystem hierarchy, events are raised:– OnEnter– OnAuthenticate– OnAuthorize– …
2006-10-17 Dyalog User Group Conference 32
Tricks: the observer patternTricks: the observer pattern
:Class Event :field _Observers ∇ make name :Access Public Instance :Implements Constructor _Observers←⍬ ⎕DF'Event: ',name ∇ ∇ Raise(sender arg);obj;fnname :Access Public Instance :For obj fnname :In
_Observers (obj⍎fnname)(sender arg) :EndFor ∇
∇ Add(obj fnname) :Access Public Instance _Observers,←⊂obj fnname ∇ ∇ Del(obj fnname) :Access Public Instance _Observers← _Observers~⊂obj fnname ∇ :EndClass
2006-10-17 Dyalog User Group Conference 33
The observed shows what it’s gotThe observed shows what it’s got
onAuthenticate←⎕NEW #.WildHeart.Event('Authenticate')
auth←⎕NEW web.WebDemos.FormsAuth fs.onAuthenticate.Add auth'OnAuthenticate'
onAuthenticate.Raise ⎕THIS lurl
2006-10-17 Dyalog User Group Conference 34
Two simple authentication modulesTwo simple authentication modules
SimpleAuth: based on HTTP challenge response:– Quick and dirty
FormsAuth: based on cookies– More flexible– More work
2006-10-17 Dyalog User Group Conference 35
SessionsSessions
HTTP is a stateless protocolPeople want to have conversations
with peers who don’t forget instantly what they just said
Have a cookie!
2006-10-17 Dyalog User Group Conference 36
SessionsSessions
As easy as: fs.EnableSession ⎕NULL fs.onSessionStart.Add #’DoSomething' fs.onSessionEnd.Add #’UndoSomething'
In the application: :If ⎕NULL≡ns←ctx._Session[⊂'footer'] ns←⎕NS'' ns.count←0 ctx._Session[⊂'footer']←ns :EndIf ns.count+←1
2006-10-17 Dyalog User Group Conference 37
ExtensibilityExtensibility
Some of the features (ZipFileSystem, session module, authentication, aplx handler) were hacked together to prove that the server could support them
Even before its completion, the WildServer has:– Had a file handler written (by Nic & Morten)– Been used to run a course in writing Web
Applications using Dyalog APL
2006-10-17 Dyalog User Group Conference 38
Open SourceOpen Source
Open Source means: “free as in free beer” (except: is there such a thing as free beer?)
No strings attached:– You get the source code and whatever
documentations exists with it at the time– You get the right to do whatever you want with
it (even sell it! But only a bad boy would do that…)– You get from me all the support I can give you when I
can if I can (alas, I am very busy and maybe one day I’ll get a life)
2006-10-17 Dyalog User Group Conference 39
Why Open Source?Why Open Source?
Have you looked at Ruby on Rails?Tons of contributors do a lot of workTons of clever contributors do a lot
of good work!
OK, maybe that’s a good product.
2006-10-17 Dyalog User Group Conference 40
TODO listTODO list
It’s long, it’s scary, but…APL Prototypes seem to acquire a
life of their ownThe “ducks” have started
contributing
2006-10-17 Dyalog User Group Conference 41
ThanksThanks
Peter Michael Haager: I have adapted his XML parser to produce trees of objects instead of nested arrays
2006-10-17 Dyalog User Group Conference 42
ConclusionConclusion
Initial feedback from users (Nic, Morten and Mondays course participants) is promising
The WildServer is still very much ”work in progress”, and the author wants to keep control of it until about Christmas
However, you are all welcome to try it now and suggest or code enhancements
2006-10-17 Dyalog User Group Conference 43
Questions?Questions?
If not… now I need some sleep.
輝く空と君の声を感じたい
2006-10-17 Dyalog User Group Conference 44
What is missing in the languageWhat is missing in the language
No public operators! C# has them!No protected members! C# has them!No object serialization! C# has it!
C# is a statically typed language. OK, substitute in the slide “C#” with “Ruby”