Algorithms

Algorithms

Homework

• None• Next Week: Mid-Term– Open book / Open laptop

Old Business

CFGs: Context Free Grammars

(Ch8)

Ambiguity

• The Chomsky Hierarchy– Type 0 > Type 1 > Type 2 > Type 3– Recursively Enumerable > CS > CF > Regular

• Examples– Type 3: Regular (Finite State):

• Grep & Regular Expressions• Right-Branching: A a A• Left-Branching: B B b

– Type 2: Context-Free (CF): • Center-Embedding: C … x C y• Parenthesis Grammars: <expr> ( <expr> )• w wR

– Type 1: Context-Sensitive (CS): w w– Type 0: Recursively Enumerable– Beyond Type 0: Halting Problem

New Business

http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-006Spring-2008/LectureNotes/index.htm

Hash Tables

n steps

n steps

Linear Time

Constant Time

n2 time

n time

Profiling

O(n) << O(n2)

0 50,000 100,000 150,000 200,000 250,0000

102030405060708090

100

f(x) = 1.8375555701746E-05 x − 0.331345186366877

f(x) = 8.31375E-10 x² + 0.000149303 x − 0.7692457131

slow Polynomial (slow) fast Linear (fast)

Tim

e

n = |words|

Excel

n slow fast1,000 0.032 0.006

10,000 0.065 0.062

100,000 22.580 0.664

260,819 94.713 4.775

0 50,000 100,000 150,000 200,000 250,0000

102030405060708090

100

Group Problems by Time Bounds• log(n)

– Binary Search, Fibonacci• n

– tr, cat, cut, uniq, egrep• n log n

– Sort• n2

– Edit Distance (http://www.csse.monash.edu.au/~lloyd/tildeAlgDS/Dynamic/Edit/)

• n3

– Matrix Multiplication n2.376

– Context-Free Parsing• 2n

– Factoring– Satisfiability

http://ocw.mit.edu/NR/rdonlyres/Electrical-Engineering-and-Computer-Science/6-046JFall-2005/LectureNotes/lec3.pdf

Summary (for Mid-Term)

• Symbolic Processing v. Stat Packages– Practical• Symbolic: LISP, Wolfram Alpha• Stats: Newton’s Method, R

– Fundamental• Stuff you can’t do with stat packages• LISP: Recursion, Eval, Symbolic Differentiation• Lambda Calculus

– Small is Beautiful (beyond reason)– Church’s Thesis & Computability Theory

Summary (continued)

• Unix for Poets– Pipes: tr | sort | uniq –c

• Python & NLTK– Unix for Poets (without Unix)– Objects– Polymorphism– Equivalence Classes– Partial Orders

• Algorithms– Group problems by time & space bounds

Introduction to ProgrammingTraditional(Start with Definitions)• Constants: 1• Variables: x• Objects:

– lists, strings, arrays, matrices• Expressions: x == 1• Statements: Side Effects

– x = 1 (assignment)• Conditionals:

– If (x<=1) return 1;• Iteration: for loops• Functions• Recursion • Streams

Non-Traditional(Start with Examples)

• Recursiondef fact(x):

if(x <= 1): return 1else: return x * fact(x-1)

• Streams:– Unix Pipes

• Briefly mentioned– Everything else

Examples

• Factorial• Fibonacci• Counting Words in a Text

Python

def fact(x):if(x <= 1): return 1else: return x * fact(x-1)

def fact2(x):result=1for i in range(x):

result *=(i+1);return result

Recursion

Iteration

Out-takes

Cosine Distance

Inner Products

Inner Product in Python

def bag_of_words(words):D = {}for w in words:

if(D.has_key(w)):D[w] = D[w]

+ 1;else: D[w]=1;

return D

def inner_product(D1,D2):sum = 0.0

for key in D1: if key in D2: sum += D1[key] *

D2[key] return sum