Camping: Going off the Rails with Ruby

Camping: Going off the Rails with Ruby

Adventures in creative codingfor people who should know better

The weasel words

This presentation contains code

That code is probably broken

If that bothers you - fix it

That’s called a learning experience

Who are these lunatics?

Romek Szczesniakromek@spikyblackcat.co.uk

He does security

Eleanor McHugheleanor@games-with-brains.com

She does real-time systems

Alright, but what are they doing here?

Ruby

Pcap & BitStruct

WEBrick

Camping

but no Rails...

No Rails?

That’s right, we don’t use Rails

But we do use Ruby

And we do write web applications

So how is that possible?

Camping!!!

That’s right, we use Camping

It’s by Why The Lucky Stiff

It’s cool

It’s really cool

It’s so damn cool you’d have to be mad not to use it!!!

It’s this simple!%w[rubygems active_record markaby metaid ostruct].each {|lib| require lib} module Camping;C=self;module Models;end;Models::Base=ActiveRecord::Base module Helpers;def R c,*args;p=/\(.+?\)/;args.inject(c.urls.detect{|x|x. scan(p).size==args.size}.dup){|str,a|str.gsub(p,(a.method(a.class.primary_key )[]rescue a).to_s)};end;def / p;File.join(@root,p) end;end;module Controllers module Base;include Helpers;attr_accessor :input,:cookies,:headers,:body, :status,:root;def method_missing(m,*args,&blk);str=m==:render ? markaview( *args,&blk):eval("markaby.#{m}(*args,&blk)");str=markaview(:layout){str }rescue nil;r(200,str.to_s);end;def r(s,b,h={});@status=s;@headers.merge!(h) @body=b;end;def redirect(c,*args);c=R(c,*args)if c.respond_to?:urls;r(302,'', 'Location'=>self/c);end;def service(r,e,m,a);@status,@headers,@root=200,{},e[ 'SCRIPT_NAME'];@cookies=C.cookie_parse(e['HTTP_COOKIE']||e['COOKIE']);cook= @cookies.marshal_dump.dup;if ("POST"==e['REQUEST_METHOD'])and %r|\Amultipart\ /form-data.*boundary=\"?([^\";,]+)\"?|n.match(e['CONTENT_TYPE']);return r(500, "No multipart/form-data supported.")else;@input=C.qs_parse(e['REQUEST_METHOD' ]=="POST"?r.read(e['CONTENT_LENGTH'].to_i):e['QUERY_STRING']);end;@body= method(m.downcase).call(*a);@headers["Set-Cookie"]=@cookies.marshal_dump.map{ |k,v|"#{k}=#{C.escape(v)}; path=/"if v != cook[k]}.compact;self;end;def to_s "Status: #{@status}\n#{{'Content-Type'=>'text/html'}.merge(@headers).map{|k,v| v.to_a.map{|v2|"#{k}: #{v2}"}}.flatten.join("\n")}\n\n#{@body}";end;def \ markaby;Class.new(Markaby::Builder){@root=@root;include Views;def tag!(*g,&b) [:href,:action].each{|a|(g.last[a]=self./(g.last[a]))rescue 0};super end}.new( instance_variables.map{|iv|[iv[1..-1].intern,instance_variable_get(iv)]},{}) end;def markaview(m,*args,&blk);markaby.instance_eval{Views.instance_method(m ).bind(self).call(*args, &blk);self}.to_s;end;end;class R;include Base end class NotFound<R;def get(p);r(404,div{h1("#{C} Problem!")+h2("#{p} not found") });end end;class ServerError<R;def get(k,m,e);r(500,markaby.div{h1 "#{C} Prob\ lem!";h2 "#{k}.#{m}";h3 "#{e.class} #{e.message}:";ul{e.backtrace.each{|bt|li( bt)}}})end end;class<<self;def R(*urls);Class.new(R){meta_def(:inherited){|c| c.meta_def(:urls){urls}}};end;def D(path);constants.each{|c|k=const_get(c) return k,$~[1..-1] if (k.urls rescue "/#{c.downcase}").find {|x|path=~/^#{x}\ \/?$/}};[NotFound,[path]];end end end;class<<self;def escape(s);s.to_s.gsub( /([^ a-zA-Z0-9_.-]+)/n){'%'+$1.unpack('H2'*$1.size).join('%').upcase}.tr(' ', '+') end;def unescape(s);s.tr('+', ' ').gsub(/((?:%[0-9a-fA-F]{2})+)/n){[$1. delete('%')].pack('H*')} end;def qs_parse(qs,d='&;');OpenStruct.new((qs||''). split(/[#{d}] */n).inject({}){|hsh,p|k,v=p.split('=',2).map{|v|unescape(v)} hsh[k]=v unless v.empty?;hsh}) end;def cookie_parse(s);c=qs_parse(s,';,') end def run(r=$stdin,w=$stdout);w<<begin;k,a=Controllers.D "/#{ENV['PATH_INFO']}". gsub(%r!/+!,'/');m=ENV['REQUEST_METHOD']||"GET";k.class_eval{include C include Controllers::Base;include Models};o=k.new;o.service(r,ENV,m,a);rescue\ =>e;Controllers::ServerError.new.service(r,ENV,"GET",[k,m,e]);end;end;end module Views; include Controllers; include Helpers end;end

Why?

For fun

For profit

For the satisfaction of knowing exactly how your application works

For the look on your boss’s face when he reads the documentation

Earlier...

But let’s not get ahead of ourselves

First we want to take you on a journey

A journey back in time

A journey back to...

January 3rd 2006

Location: The Secret Basement Lairtm of Captain IP and The DNS avengers

Their task: to launch a new Top Level Domain which DOESN’T RESOLVE MACHINE ADDRESSES?!?!?!

Their resources? Hands to wave with and hit keyboards with!

Ruby to the Rescue

It’s easy to learn

It’s quick to code in

It’s pleasing to the eye

It’s fun!

You keep saying that

Yes!!!

Fun makes for better coders

Better coders write good code

Good code stands the test of time

If coding isn’t fun YOU’RE USING THE WRONG TOOLS!!!!

The console jockeys

let’s write a menu driven calculator

output: puts(), print()

input: gets(), termios library

old-fashioned and unattractive

termios is fiddly

A simple calculator#!/usr/bin/env ruby -wrequire 'termios'

$total = 0$menu_entries = [['+', "Add"], ['-', "Subtract"], ['*', "Multiply"], ['/', "Divide"], ['c', 'Clear'], ['q',"Quit"]]

$commands = $entries.inject([]) { | commands, entry | commands << entry[0] }

$captions = $entries.inject([]) { | captions, entry | captions << entry[1] }

loop do puts "\nSimple Calculator\n" entries.each { | entry | puts "#{entry[0]}. #{entry[1]}\n" }

t = Termios.tcgetattr(STDIN) t.lflag &= ~Termios::ICANON Termios.tcsetattr(STDIN,0,t)

begin action = STDIN.getc.chr end until $commands.member?(action)

exit() if action == $commands.last action = $commands.index(action) puts "\n#{$captions[action]}\n\n"

case action when 0 : $total += gets() when 1 : $total -= gets() when 2 : $total *= gets() when 3 : $total /= gets() when 4 : $total = 0 end puts "Total = #{$total}"end

A Ruby packet readerThe 7 layer IP model

What the heck?

We want to look at UDP and DNS traffic

Our first implementation is console-based, so hold on to your hats...

We’re exploring the UDP layer

UDP header in Rubyrequire 'bit-struct'

class IP < BitStruct unsigned :ip_v, 4, "Version" unsigned :ip_hl, 4, "Header length" unsigned :ip_tos, 8, "TOS" unsigned :ip_len, 16, "Length" unsigned :ip_id, 16, "ID" unsigned :ip_off, 16, "Frag offset" unsigned :ip_ttl, 8, "TTL" unsigned :ip_p, 8, "Protocol" unsigned :ip_sum, 16, "Checksum" octets :ip_src, 32, "Source addr" octets :ip_dst, 32, "Dest addr" rest :body, "Body of message" note "rest is application defined message body" initial_value.ip_v = 4 initial_value.ip_hl = 5end

class UDP < BitStruct unsigned :udp_srcport, 16, "Source Port" unsigned :udp_dstport, 16, "Dest Port" unsigned :udp_len, 16, "UDP Length" unsigned :udp_chksum, 16, "UDP Checksum" rest :body, "Body of message" note "rest is application defined message body"end

class DNSQueryHeader < BitStruct unsigned :dns_id, 16, "ID" unsigned :dns_qr, 1, "QR" unsigned :dns_opcode, 4, "OpCode" unsigned :dns_aa, 1, "AA" unsigned :dns_tc, 1, "TC" unsigned :dns_rd, 1, "RD" unsigned :dns_ra, 1, "RA" unsigned :dns_z, 3, "Z" unsigned :dns_rcode, 4, "RCODE" unsigned :dns_qdcount, 16, "QDCount" unsigned :dns_ancount, 16, "ANCount" unsigned :dns_arcount, 16, "ARCount" rest :data, "Data"end

class Time # tcpdump style format def to_s sprintf "%0.2d:%0.2d:%0.2d.%0.6d", hour, min, sec, tv_usec endend

udpip.rb

Capturing UDP packets#!/usr/local/bin/rubyrequire 'pcaplet'include Pcaprequire 'udpip'

DIVIDER = "-" * 50def print_details(section) puts DIVIDER, section, DIVIDERend

pcaplet = Pcaplet.new('-s 1500')pcaplet.each_packet { |pkt| if pkt.udp? puts "Packet: #{pkt.time} #{pkt}" if (pkt.sport == 53) udp = UDP.new udp.udp_srcport = pkt.sport udp.udp_dstport = pkt.dport udp.udp_len = pkt.udp_len udp.udp_chksum = pkt.udp_sum udp.body = pkt.udp_data print_details udp.inspect_detailed

# look for DNS request only dns = DNSQueryHeader.new(pkt.udp_data) bytearray = Array.new udp.body.each_byte { |c| bytearray.concat(c.to_s.to_a) print c.to_s(16), ' ' } print_details dns.inspect_detailed end end }pcaplet.close

tcpdump.rb

A live UDP packet

A live DNS packet

Can we have that on Windows?

A GUI? You gotta be joking!!

Why do you think we use Macs?

How about we just turn it into a web application instead?

Sure, we can do that with Ruby

[What have we let ourselves in for...]

The NDA kicks in

Here’s where we hit the brick wall on what we can talk about

You might imagine a DNS-sniffing web application, but we couldn’t possibly comment

So lets get down to some web app basics

And yes, we will be kicking it old-skool...

Introducing WEBrick

WEBrick is an HTTP server library

It’s part of the Ruby 1.8 release

It can serve static documents

It can serve HTTPS using Ruby/OpenSSL

It can serve arbitrary code blocks

It can serve servlets

Static content#!/usr/local/bin/rubyrequire 'webrick'

server = WEBrick::HTTPServer.new(:Port => 8080, :DocumentRoot => Dir::pwd + "/htdocs")

# mount personal directory, generating directory indexesserver.mount("/~eleanor", WEBrick::HTTPServlet::FileHandler, "/Users/eleanor/Sites", true)

# catch keyboard interrupt signal to terminate servertrap("INT"){ server.shutdown }server.start

#!/usr/local/bin/ruby# This requires Ruby/OpenSSLrequire 'webrick'require 'webrick/https'

certificate_name = [ ["C","UK"], ["O","games-with-brains.org"], ["CN", "WWW"] ]server = WEBrick::HTTPServer.new( :DocumentRoot => Dir::pwd + "/htdocs", :SSLEnable => true, :SSLVerifyClient => ::OpenSSL::SSL::VERIFY_NONE, :SSLCertName => certificate_name )trap("INT"){ s.shutdown }s.start

A standard HTTP server

An HTTPS server

Servlets

#!/usr/local/bin/rubyrequire 'webrick'

server = WEBrick::GenericServer.new()trap("INT"){ server.shutdown }server.start{|socket| socket.puts("This is a code block\r") }

#!/usr/local/bin/rubyrequire 'webrick'

server = WEBrick::HTTPServer.new()trap("INT"){ server.shutdown }

def generate_response(response) response.body = "<HTML>hello, world.</HTML>" response['Content-Type'] = "text/html"end

class HelloServlet < WEBrick::HTTPServlet::AbstractServlet def do_GET(request, response) generate_response(response) endend

server.mount_proc("/hello/simple"){ | request, response | generate_response(response) }server.mount("/hello/advanced", HelloServlet)server.start

A Ruby code block

A WEBrick servlet

It’s that simple?

Yes, it’s that simple

Of course these are trivial examples...

...so let’s build an application server

An application server

Still wondering when we get to the really good stuff?

Soon, we promise

But first to show you how NOT to do it!

Wrap the requestclass RequestContext attr_reader :request, :response, :servlets, :creation_time

def initialize(request, response) @request, @response, = request, response @creation_time = Time.now() end

def page_not_found @response.status = WEBrick::HTTPStatus::NotFound.new() end

def response_page(page) @response['Content-Type'] = page.content_type @response.body = CGI::pretty(page.to_str()) end

def <<(item) @response.body << CGI::pretty(item) endend

A basic request context

Serve the pagesIP_ADDRESS_PATTERN = /^\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}/

class ApplicationServer attr_reader :web_server, :server_address, :servlets, :pages

def initialize(parameters = {}) @server_address = parameters[:my_address] or raise “Please supply a server address” raise “Invalid IP address for server” unless IP_ADDRESS_PATTERN.match(@server_address) @web_server = WEBrick::HTTPServer.new({:BindAddress => @server_address}) @servlets = {} @pages = {} end

def start trap("INT") { @web_server.shutdown } @web_server.start end

def register_page(path, page) @pages[path] = page @web_server.mount_proc(path) { | request, response | context = RequestContext.new(request, response) @pages[request.path] ? context.response_page(@pages[request.path]) : context.page_not_found() } end

def register_method(path, handler) @servlets[path] = self.method(handler).to_proc @web_server.mount_proc(path) { | request, response | context = RequestContext.new(request, response) @servlets[request.path] ? (context << @servlets[request.path].call(context).to_str()) : context.page_not_found() } endend

The application server

Write the application#!/usr/local/bin/ruby

require 'appserver.rb'

class SimpleServer < ApplicationServer def initialize(parameters = {}) super register_page("/hello/simple", "<HTML>Hello, world</HTML>") register_method("/hello/advanced", :hello_world) end

def hello_world(context) "<HTML>Hello, world</HTML>" endend

begin SimpleServer.new({:my_address => ARGV.shift()}).start()

rescue RuntimeError => e $stderr.puts "Usage: simpleserver host-address" $stderr.puts " address must be provided in dotted-quad format (i.e. xxx.xxx.xxx.xxx)"end

Revisiting “hello, world”

What have we done?!?

On the surface this is elegant

But underneath it sucks

There’s no support for HTML

Only methods can be used as servlets

We’re tied to WEBrick - which is slow

The road to perdition

So we added an HTML 4 library

And a server pages container

And ActiveRecord

We meta’d the code to death

But it still lacked va-va-voom...

The case for Rails

So perhaps we should have just used Rails in the first place

We’d be another of those “Rails saved my career” success stories!

Hindsight’s always 20/20

But we’re old-school coders and it’s far too user friendly for our comfort

The pressure against

Working at a very low level

Simple code required

Can Rails talk nicely to low-level code?

Strong management resistance - too high a learning curve?

So why Camping?

Camping is beauty incarnate

It’s less than 4K of code

It uses Markaby and ActiveRecord

It runs on JRuby!!!

Oh, and it’s great fun to abuse...

Gratuitous diagram

lifted fromhttp://redhanded.hobix.com/bits/campingAMicroframework.html

How Why? The Lucky Stiff teaches it

Markaby

An XHTML Domain Specific Language

Allows you to embed XHTML code in Ruby code without building a complex object hierarchy

Can be used with Rails

But that’s so simple!require 'markaby'

page = Markaby::Builder.newpage.xhtml_strict do head { title "Camping Presentation" } body do h1.page_heading "Camping: Going off the Rails with Ruby" ul.page_index do li.page_index { a “introduction”, :href => ‘#introduction’ } li.page_index { a “the presentation”, :href => ‘/presentation’ } li.page_index { a “comments”, :href => ‘#comments’ } end div.introduction! { “Everything will be alright!!!” } div.comments! { “Have your say” } endendputs page.to_s

Markaby embedded in Ruby

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "DTD/xhtml1-strict.dtd"><html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"/> <title>Camping Presentation</title> </head> <body> <h1 class="page_heading">Camping: Going off the Rails with Ruby</h1> <ul class="page_index"> <li class="page_index"><a href="#introduction">introduction</a></li> <li class="page_index"><a href="/presentation">the presentation</a></li> <li class="page_index"><a href="#comments">comments</a></li> </ul>

<div id="introduction">Just breathe deeply...</div> <div id="comments">Have your say</div> </body></html>

Creates this

ActiveRecord

An Object-Relational Mapper

Implements the Active Record pattern

Supports many popular databases

A key component of Rails

ORMtasticUsing Active Recordrequire 'rubygems'require_gem ‘activerecord’

ActiveRecord::Base.establish_connection(:adapter => “sqlite3”, :host => “localhost”, :database => “test.db”)

class User < ActiveRecord::Baseend

user = User.new()user.id = “ellie”user.name = “Eleanor McHugh”user.password = “somerandomtext”user.save

user = User.find(“ellie”)user.destroy()

Totally RAD

Camping builds small applications

Why’s guideline? One file per application

If that’s how you prefer it...

A simple exampleBasic setup#!/usr/bin/env ruby

$:.unshift File.dirname(__FILE__) + "/../../lib"require 'camping'require 'camping/session' Camping.goes :Jotter

module Blog include Camping::Sessionend

Load the camping libraries

Define a namespace for the application

Include session support (if required)

The data modelmodule Jotter::Models class Note < Base; end

class Database < V 1.0 def self.up create_table :jotter_notes, :force => true do |t| t.column :id, :integer, :null => false t.column :created_at, :interger, :null => false t.column :title, :string, :limit => 255 t.column :body, :text end end

def self.down drop_table :jotter_notes end endend

def Jotter.create Jotter::Models.create_schemaend

Defining the data model

We mark our database as version 1.0

A create method builds the database

The controllersAdding controllersmodule Jotter::Controllers class Static < R '/static/(.+)' MIME_TYPES = {'.css' => 'text/css', '.js' => 'text/javascript', '.jpg' => 'image/jpeg'} PATH = __FILE__[/(.*)\//, 1]

def get(path) @headers['Content-Type'] = MIME_TYPES[path[/\.\w+$/, 0]] || "text/plain" @headers['X-Sendfile'] = "#{PATH}/static/#{path}" end end

class Index < R '/' def get @notes = Note.find :all render :index end end

class View < R '/view/(\d+)' def get note_id @note = Note.find post_id render :view end end

class Add < R ‘/add/’ def get @note = Note.new render :add end

def post note = Note.create :title => input.post_title, :body => input.post_body redirect View, post end end

The controllers class Edit < R '/edit/(\d+)', '/edit' def get note_id @note = Note.find note_id render :edit end

def post @note = Note.find input.note_id @note.update_attributes :title => input.post_title, :body => input.post_body redirect View, @note end end

class Delete < R '/delete/(d+)' def get note_id @note = Note.find note_id @note.destroy redirect Index end endend

Adding controllers

Respond to HTTP GET and POST requests

Perform database operations

The viewsApplication viewsmodule Jotter::Views def layout xhtml_strict do head do title 'blog' link :rel => 'stylesheet', :type => 'text/css', :href => '/static/styles.css', :media => 'screen' end

body do h1.header { a 'jotter', :href => R(Index) } div.body do self << yield end end end end

def index @notes.empty? (p 'No posts found.') : (ol.row! { _list_notes(@notes) }) p { a 'new note', :href => R(Add) } end

def edit _form(@note, :action => R(Edit)) end

def view h1 @note.title h2 @note.created_at p @note.body p do [ a("View", :href => R(View, @note)), a("Edit", :href => R(Edit, @note)), a("Delete", :href => R(View, @note)) ].join " | " end end

The views def _list_notes(notes) @notes.each do | note | li do ul do li { a note.title, :href => R(View, note) } li note.created_at li { a "Edit", :href => R(Edit, note) } li { a "Delete", :href => R(Delete, note) } end end end end

def _form(note, opts) form({:method => 'post'}.merge(opts)) do label 'Title', :for => 'note_title'; br input :name => 'note_title', :type => 'text', :value => note.title; br label 'Body', :for => 'note_body'; br textarea note.body, :name => 'note_body'; br input :type => 'hidden', :name => 'note_id', :value => note.id input :type => 'submit' endend

Application views

Views incorporate Markaby for XHTML

Have access to controller data

The post-ambleA basic CGI post-ambleif __FILE__ == $0 Jotter::Models::Base.establish_connection :adapter => 'sqlite3', :database => 'notes.db' Jotter::Models::Base.logger = Logger.new('camping.log') Jotter.create if Jotter.respond_to? :create puts Jotter.runend

if __FILE__ == $0 Jotter::Models::Base.establish_connection :adapter => 'sqlite3', :database => 'notes.db' Jotter::Models::Base.logger = Logger.new('camping.log') Jotter::Models::Base.threaded_connections = false Jotter.create if Jotter.respond_to? :create server = Mongrel::Camping::start(“0.0.0.0”, 3000, “/jotter”, Jotter) puts “Jotter application running at http://localhost:3000/jotter” server.run.joinend

A Mongrel post-amble

Allows an application to self-execute

Can be customised to suit your platform

The style-sheetA simple style sheetbody { font-family: Utopia, Georga, serif; }

h1.header { background-color: #fef; margin: 0; padding: 10px; }

div.body { padding: 10px; }

#row ul { list-style: none; margin: 0; padding: 0; padding-top: 4px; }

#row li { display: inline; }

#row a:link, #row a:visited { padding: 3px 10px 2px 10px; color: #FFFFFF; background-color: #B51032; text-decoration: none; border: 1px solid #711515; }

Larger applications

One application per file is a nice idea

But what about large applications?

Each can be broken down into discrete micro-applications

Each micro-application has its own file and mount points

Sharing a database

Camping apps keep their database tables in separate namespaces

Larger applications will want to share state between micro-applications

We could do some ActiveRecord voodoo

Or we could cheat... guess which?

Camping in the wildsrequire 'rubygems'require_gem 'camping', '>=1.4'require 'camping/session'

module Camping module Models def self.schema(&block) @@schema = block if block_given? @@schema end

class User < Base validates_uniqueness_of :name, :scope => :id validates_presence_of :password end end

def self.create Camping::Models::Session.create_schema ActiveRecord::Schema.define(&Models.schema) end

Models.schema do unless Models::User.table_exists? create_table :users, :force => true do | t | t.column :id, :integer, :null => false t.column :created_on, :integer, :null => false t.column :name, :string, :null => false t.column :password, :string, :null => false t.column :comment, :string, :null => false end

execute "INSERT INTO users (created_on, name, password, comment) VALUES ('#{Time.now}', 'admin', 'admin', 'system administrator')" end endend

Installing a database in the framework

Camping server

The camping server ties together a series of web applications

A simple implementation ships with the framework

The server rules

Monitor a directory

load/reload all camping apps that appear in it or a subdirectory

Mount apps according to the filenames (i.e. jotter.rb mounts as /jotter)

Run create method on app startup

Support the X-Sendfile header

Summing up

Web applications are useful outside the usual web app environment

Cross platform is easy when you only need an XHTML browser

These tasks need a lightweight design

Camping is a good way to solve them

And as you can see, Ruby rocks!!!

http://code.whytheluckystiff.net/camping/wiki

http://www.goto.info.waseda.ac.jp/~fukusima/ruby/pcap-e.html

http://raa.ruby-lang.org/project/bit-struct/

http://raa.ruby-lang.org/project/ruby-termios/

Where to next?