Who makes the best Asado?
RubyConf Argentina 2011
Jun 29, 2015
RubyConf Argentina 2011
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Who makes the best Asado?
What country has the best footballers?
Argentina!
river
Nacional B???
river
NOOOOOOOO!!!!
Aaron Patterson
@tenderlove
Ruby core team
Rails core team
! WARNING !
ZOMG!
We are in Argentina!
YoJosé
WWFMD?
me gusta
¿Por qué Maria?
THANKS!!
ResourceManagement
Path Management
PatternMatching
Move between theory and practice
worldhello greeting
digraph nfa { rankdir=LR;
world [shape = doublecircle]; hello [shape = circle]; hello -> world [label="greeting"];}
graph.dot
digraph nfa { rankdir=LR;
world [shape = doublecircle]; hello [shape = circle]; goodbye [shape = circle];
hello -> world [label="greeting"]; goodbye -> world}
graph.dot
world
hellogreeting
goodbye
2 3.
4
/
/articles(.:format)
5
6 /articles/new(.:format)
0 1/ articles (?-mix:[^./?]+)
new
Journey
JourneyYes, it's named after the '70s rock sensation
What is a router?
Provides1: URL generation2: Path Recognition3: Route parsing
Why a new router?
Maintenance
0
750
1500
2250
3000
Journey Journey-2 Rack-Mount
LOC
Known algorithms
References
•Compilers: Principles, Techniques, & Tools (Aho, Lam, Sethi, Ullman)
• Intro to Formal Languages and Automata (Linz)
Predictability
CPU Time
Memory Usage
Current Performance
url generation path recognition route parsing
rack-mount Journey
me gusta
Patterns
Why patterns matter
Regular Expressions/om(g)!/
Rails Routes
resource :articles
/articles(.:format) /articles/new(.:format) /articles/:id/edit(.:format) /articles/:id(.:format)
Parens don't capture
:whatever => /([^./?]+)/
/articles(.:format)
/\/articles(?:\.([^.\/?]+))?$/
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /articles/new
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /articles/new
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /articles/new
200 OK!
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /foos/new
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /foos/new
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /foos/new
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /foos/new
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /foos/new
/\/articles(?:.([^.\/?]+))?$/
/\/articles\/new(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/
/\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/
GET /foos/new
404 Not Found
How long does this take?
r = routes.length
x = regexp compare
O(r ⨉ x)
Can we do better?
Finite State Machines
Parse Trees
Automata
Parse Trees
Parsing regexp
(a|b)*abb
○○ b
○ b
a *
a b
()
|
Describe node types
Parsing rails routes
Journey::Parserparser = Journey::Parser.newast = parser.parse '/articles(.:format)'
puts ast.to_dot
○
○ ()
/ articles ○
. :format
To String!
parser = Journey::Parser.newast = parser.parse '/articles(.:format)'puts ast.to_sputs ast.to_regexp
OR ASTparser = Journey::Parser.new
trees = [ '/articles(.:format)', '/articles/new(.:format)',].map { |s| parser.parse s }
ast = Journey::Nodes::Or.new trees
puts ast.to_dot
|
○ ○
○ ()
/ articles ○
. :format
○ ()
○ new
○ /
/ articles
○
. :format
SECRET FEATUREparser = Journey::Parser.newast = parser.parse '/articles|books(.:format)'
SECRET FEATUREparser = Journey::Parser.newast = parser.parse '/articles|books(.:format)'
SUPER SECRET!
Automata
(a|b)*abb
3
0b2a
b
1aa
b
b
a
Double circle is acceptance state
baabb
3
0b2a
b
1aa
b
b
a
3
0b2a
b
1aa
b
b
a
b
3
0b2a
b
1aa
b
b
a
ba
3
0b2a
b
1aa
b
b
a
baa
3
0b2a
b
1aa
b
b
a
baabb
aab
3
0b2a
b
1aa
b
b
a
Deterministic Finite Automaton
Only one path
Storageclass DFA attr_reader :table
def initialize @table = Hash.new { |h, from| h[from] = {} } @table[0]['a'] = 1 @table[0]['b'] = 0 @table[1]['a'] = 1 @table[1]['b'] = 2 @table[2]['a'] = 1 @table[2]['b'] = 3 @table[3]['b'] = 0 @table[3]['a'] = 2 endend
FSM Simulation
class Simulator def initialize(dfa) @dfa = dfa end
def simulate(symbols) state = 0 until symbols.empty? state = @dfa.move(state, symbols.shift) end state endend
class DFA ...
def move(from, symbol) @table[from][symbol] endend
move function
irb> sim = Simulator.new(DFA.new)=> #<Simulator:0x007f95929a82e0 ...>irb> sim.simulate %w{ b a a b b }=> 3irb> sim.simulate %w{ a a b }=> 2
Time: O(n)n = string.length
me gusta
Space: S + Tstates.length + transitions.length
Nondeterministic Finite Automaton
Has nil edges
Can't tell direction
Simulation of NFA
602ε
4ε
3a
5b
εε
a|b
602ε
4ε
3a
5b
εε
nil-closure
602ε
4ε
3a ε
5b ε
a
Storageclass NFA def initialize @table = Hash.new { |h, from| h[from] = Hash.new { |i,sym| i[sym] = [] } }
@table[0][nil] << 2 @table[0][nil] << 4 @table[2]['a'] << 3 @table[4]['b'] << 5 @table[3][nil] << 6 @table[5][nil] << 6 end
def nil_closure(states) states.map { |s| @table[s][nil] }.flatten endend
FSM Simulationclass Simulator def initialize(nfa) @nfa = nfa end
def simulate(symbols) states = @nfa.nil_closure([0])
until symbols.empty? next_s = @nfa.move(states, symbols.shift) states = @nfa.nil_closure(next_s) end
states endend
Move function
class NFA ...
def move(states, symbol) states.map { |s| @table[s][symbol] }.flatten endend
irb> sim = Simulator.new(NFA.new)=> #<Simulator:0x007faa188a5f88 ...>irb> sim.simulate %w{ a }=> [6]irb> sim.simulate %w{ b }=> [6]irb> sim.simulate %w{ b b }=> []irb> sim.simulate %w{ c }=> []
Time: O(r ⨉ x)r = operators.length, x = string.length
Who cares about NFA?
NFA Construction
○
○ ()
/ articles ○
. :format
/articles(.:format)
cat nodes
10 /
/articles
20 1/ articles
Optional
30ε
1ε
2????? ε
60 1/ 2articlesε
3ε
4. 5:format ε
parser = Journey::Parser.newast = parser.parse '/articles(.:format)'nfa = Journey::NFA::Builder.new ast
tt = nfa.transition_tableputs tt.to_dot
Journey::NFA
ConvertingNFA to DFA
Eliminate nil transitions
Collapse duplicate edges
60 1/ 2articlesε
3ε
4. 5:format ε
/articles(.:format)
2 3. 40 1/ articles :format
/articles(.:format)
CODES!
parser = Journey::Parser.newast = parser.parse '/articles(.:format)'nfa = Journey::NFA::Builder.new ast
tt = nfa.transition_table.generalized_tableputs tt.to_dot
SHORTER CODES!
parser = Journey::Parser.newast = parser.parse '/articles(.:format)'dfa = Journey::GTG::Builder.new ast
tt = dfa.transition_tableputs tt.to_dot
ANY NFAconverts to DFA
O(r ⨉ x) => O(x)r = operations.length, x = string.length
me gusta
Converting Automata to Regexp
2 3. 40 1/ articles :format
/articles(.:format)
2 3. 40 /articles :format
/articles(.:format)
/articles(.:format)
2 4.:format0 /articles
/articles(.:format)
40 /articles(.:format)?
Generalized Transition Graph
/users(.:format) /users/new(.:format) /users/:id/edit(.:format) /users/:id(.:format)
resource :users
2
3.
4/
5
6 8.
7 9/
10
.
11
12 14.
13
15
0 1/ users
(?-mix:[^./?]+)
new(?-mix:[^./?]+)
(?-mix:[^./?]+)
edit
(?-mix:[^./?]+)
(?-mix:[^./?]+)
resource :users
The Plan?
Combine All Routes
Produce DFA
Simulate in O(n) Time
Routing To The Future
JS Simulation
Table => JSON
parser = Journey::Parser.newast = parser.parse '/articles(.:format)'dfa = Journey::GTG::Builder.new ast
tt = dfa.transition_tableputs tt.to_json
Table => SVG
parser = Journey::Parser.newast = parser.parse '/articles(.:format)'dfa = Journey::GTG::Builder.new ast
tt = dfa.transition_tableputs tt.to_svg
JS Tokenizer
function tokenize(input, callback) { while(input.length > 0) { callback(input.match(/^[\/\.\?]|[^\/\.\?]+/)[0]); input = input.replace(/^[\/\.\?]|[^\/\.\?]+/, ''); }}
JS Simulator tokenize(input, function(token) { var new_states = []; for(var key in states) { var state = states[key];
if(string_states[state] && string_states[state][token]) { var new_state = string_states[state][token]; highlight_edge(state, new_state); highlight_state(new_state); new_states.push(new_state); } }
if(new_states.length == 0) { return; } states = new_states; });
d3.js
Rails Console
irb> File.open('out.html', 'wb') { |f|irb* f.write(irb* Wot::Application.routes.router.visualizerirb> )}=> 69074
routes.rb marshalling
Test suggestions
Test coverage
Usage Heat Maps
REGEXP
AST
NFA
DFA
ROFLSCALE!!!
DFA => YACC
DFA => RACC
DFA => Ragel
Open Questions
Is our GTG deterministic?
/users/new|/users/:id
4
5
0 1/ 2users 3/new
(?-mix:[^./?]+)
"new" =~ /new|[^.\/?]+/
Can we make it deterministic?
L1 = {new}L2 = {[^./?]+}
/users/new|/users/:id
4
5
0 1/ 2users 3/L1
L2 - L1
Is it worth our effort?
REGEXP
AST
NFA
DFA
Is it worth our effort?
Thank You!!
<3<3<3<3<3
Related Documents
