Clojure: Towards The Essence of Programming

© 2010 Howard Lewis Ship

Clojure: Functional Concurrency for the JVM

Howard M. Lewis ShipTWD [email protected]

1


Clojure: Why Functional Programming Matters


2


Clojure: Towards the Essence of Programming


3


essence |ˈesəns|nounthe intrinsic nature or indispensable quality of something, esp. something abstract, that determines its character : conflict is the essence of drama.

4


Mainstream Programming

5

Operating System

Language

Libraries

Frameworks

Applications


Ceremony vs.

Essence6


Is your language ...

7

© 2010 Howard Lewis Ship8

… holding you back?


Java: Data Encapsulated in Objects

9

Stockticker: AAPL

lastTrade: 203.25open: 204.50shares: 100

Stockticker: MSFT


Stockticker: ORCL


public static void sortByLastTrade(List<Stock> portfolio) { Comparator<Stock> c = new Comparator<Stock>() { public int compare(Stock o1, Stock o2) { return o1.getLastTrade() - o2.getLastTrade(); } };

Collections.sort(portfolio, c); }

public static void sortByOpen(List<Stock> portfolio) { Comparator<Stock> c = new Comparator<Stock>() { public int compare(Stock o1, Stock o2) { return o1.getOpen() - o2.getOpen(); } };

Collections.sort(portfolio, c); }


Clojure: Data in Maps and Lists

10

user=> portfolio[{:ticker "AAPL", :last-trade 203.25M, :open 204.50M, :shares 100} {:ticker "MSFT", :last-trade 29.12M, :open 29.08M, :shares 50} {:ticker "ORCL", :last-trade 21.90M, :open 21.83M, :shares 200}]user=> (sort-by :last-trade portfolio)({:ticker "ORCL", :last-trade 21.90M, :open 21.83M, :shares 200} {:ticker "MSFT", :last-trade 29.12M, :open 29.08M, :shares 50} {:ticker "AAPL", :last-trade 203.25M, :open 204.50M, :shares 100})user=> (sort-by :shares portfolio)({:ticker "MSFT", :last-trade 29.12M, :open 29.08M, :shares 50} {:ticker "AAPL", :last-trade 203.25M, :open 204.50M, :shares 100} {:ticker "ORCL", :last-trade 21.90M, :open 21.83M, :shares 200})user=>

{ }:ticker AAPL

:last-trade 203.25

:open 204.50 { }:ticker MSFT

:last-trade 29.12

:open 29.08 { }:ticker ORCL

:last-trade 21.90

:open 21.83

:shares 100 :shares 50 :shares 200


Functional JavaScript

11

var portfolio = [ { ticker: "AAPL", lastTrade: 203.25, open: 204.50}, { ticker: "MSFT", lastTrade: 29.12, open: 29.08 }, { ticker: "ORCL", lastTrade: 21.90, open: 21.83 } ]

portfolio.sortBy(function (stock) { return stock.lastTrade }).toJSON()[{"ticker": "ORCL", "lastTrade": 21.9, "open": 21.83}, {"ticker": "MSFT", "lastTrade": 29.12, "open": 29.08}, {"ticker": "AAPL", "lastTrade": 203.25, "open": 204.5}]


Clojure: The Language

12

(defn render-json "Renders JSON content (typically, a map or a seq) as the response. The response content type is set to \"application/json\". Returns true." [env json-value] (let [response (-> env :servlet-api :response)] (.setContentType response "application/json") (with-open [writer (.getWriter response)] (binding [*out* writer] (print-json json-value)))) true)


Reading Lisp

(defn render-json "Renders JSON content (typically, a map or a seq) as the response. The response content type is set to \"application/json\". Returns true." [env json-value] (let [response (-> env :servlet-api :response)] (.setContentType response "application/json") (with-open [writer (.getWriter response)] (binding [*out* writer] (print-json json-value)))) true)

Aargh!


Structural View

14

defn render-json env json-value

letresponse -> env :servlet-api :response

.setContentType response "application/json"

with-open writer .getWriter response

binding *out* writer

print-json json-value


Java: Complex Structure

15

int f = 9 * c / 5 + 32;

Higher precendence, then left to right: ((9 * c) / 5) + 32

*

9 c

/

+

5

32(+ (/ (* 9 c) 5) 32)


Clojure: Form Is

Structure16


Code is Data

'(1 2 3)

(biggest 5 42)

(defn biggest "Find the maximum of two numbers" [x y] (if (> x y) x y))

Quoted list of numbers

Function call

Function definition

17


Read Eval Print Loop

user=> (defn biggest "Find the maximum of two numbers" [x y] (if (> x y) x y))#=(var user/biggest)user=> (biggest 5 42)42user=> (doc biggest)-------------------------user/biggest([x y]) Find the maximum of two numbersniluser=> '(1 2 3)(1 2 3)user=> '(biggest 5 42)(biggest 5 42)user=> (first '(biggest 5 42))biggestuser=> (eval '(biggest 5 42))42

18

Operating System

JVM

Java Libraries

User Classes

ClojureRepl Input

Clojure Source Files

Java Compiler

Source Code

Evaluator


REPL and Compilation

19


Clojure Literals

user=> "A Clojure String""A Clojure String"user=> \space\spaceuser=> \A\Auser=> nilniluser=> :balance:balanceuser=> truetrueuser=> falsefalse

20

Java null


Numeric Literals

user=> 55user=> 5.0015.001user=> 22/722/7user=> (* 2 22/7)44/7user=> (* 100000 100000 100000)1000000000000000user=> (+ 5. 0.000000000000000001)5.0user=> (+ 5.0M 0.000000000000000001M)5.000000000000000001M

21

BigInteger

BigDecimal

Ratio


Java Interop: Function Calls

22

(.method-name receiver arguments…)

.setNamespaceAware(factory true)

(.. factory newSaxParser (parse src handler))

newSaxParser. ().parse(src,handler)

(( ))..

factory

1

2

3

lst


Clojure Collections: Lists

user=> (def lst '(1 2 3))#=(var user/lst)user=> lst(1 2 3)user=> (first lst)1user=> (rest lst)(2 3)user=> (conj lst 4)(4 1 2 3)user=> (cons 4 lst)(4 1 2 3)

4

23


Clojure Collections: Vectors

user=> (def v [:moe :larry :curly])#=(var user/v)user=> v[:moe :larry :curly]user=> (first v):moeuser=> (rest v)(:larry :curly)user=> (conj v :shemp)[:moe :larry :curly :shemp]user=> (cons :shemp v)(:shemp :moe :larry :curly)user=> v[:moe :larry :curly]

24


Clojure Collections: Maps

user=> (def m {:first-name "Howard" :last-name "Lewis Ship"})#=(var user/m)user=> m{:last-name "Lewis Ship", :first-name "Howard"}user=> (get m :last-name)"Lewis Ship"user=> (assoc m :company "TWD"){:company "TWD", :last-name "Lewis Ship", :first-name "Howard"}user=> m{:last-name "Lewis Ship", :first-name "Howard"}user=> (get m:ssn)nil

25


Clojure Collections: Sets

user=> (def s #{"Howard" "Suzanne" "Molly" "Jim"})#=(var user/s)user=> s#{"Howard" "Jim" "Molly" "Suzanne"}user=> (contains? s "Howard")trueuser=> (contains? s "howard")falseuser=> (conj s "Howard")#{"Howard" "Jim" "Molly" "Suzanne"}user=> (conj s "Scott")#{"Howard" "Jim" "Molly" "Suzanne" "Scott"}

26


Functional Programming

27


10 X = 120 PRINT X30 X = X + 140 GOTO 20

My First Program

No it doesn't!


No Mutable

State29


No Side Effects

30


Purity31


Simplicity Predictability

Testability32


Functional Java Collections

return CollectionUtils.filter(new Predicate<String>(){ public boolean accept(String value) { return !value.startsWith("."); }}, names);

public static <T> Collection<T> filter(Predicate<T> pred, Collection<T> coll){ Collection<T> out = new ArrayList<T>();

for (T item : coll) { if (pred.accept(item)) out.add(item); }

return out;}

public interface Predicate<T>{ boolean accept(T value);}

33

Essence


Functional Clojure Collections

(filter #(not (.startsWith % ".")) names)

Member access form

user=> (def names ["fred" "barney" ".hidden" "wilma"])#=(var user/names)user=> (filter #(not (.startsWith % ".")) names)("fred" "barney" "wilma")user=> (remove #(.startsWith % ".") names)("fred" "barney" "wilma")user=>

Anonymous function

Function parameter

34

(defn require-extension [ext] (fn [file-name] (= ext (last (split-string file-name ".")))))


Composing Functions (filter #(not (.startsWith % ".")) names)

function as parameter to function

(filter (require-extension "gz") names)

function as return value

composing functions

35

❝Closure Oriented Programming❞


Java: Iterative Steps

36


Java: Data Encapsulated in Objects

public static List<Double> getOpens(List<Stock> portfolio){ List<Double> result = new ArrayList<Double>();

for (Stock stock : portfolio) { result.add(stock.getOpen()); }

return result;}

37

Stockticker: AAPL


Stockticker: MSFT


Stockticker: ORCL



Clojure: Flow of Transformations

38


Clojure: Data in Transformable Collections

user=> portfolio[{:ticker "AAPL", :last-trade 203.25M, :open 204.50M, :shares 100} {:ticker "MSFT", :last-trade 29.12M, :open 29.08M, :shares 50} {:ticker "ORCL", :last-trade 21.90M, :open 21.83M, :shares 200}]user=> (map #(get % :open) portfolio)(204.50M 29.08M 21.83M)user=>

39

{ }:ticker AAPL

:last-trade 203.25


:last-trade 29.12


:last-trade 21.90

:open 21.83



Clojure: Data in Transformable Collections

user=> portfolio[{:ticker "AAPL", :last-trade 203.25M, :open 204.50M, :shares 100} {:ticker "MSFT", :last-trade 29.12M, :open 29.08M, :shares 50} {:ticker "ORCL", :last-trade 21.90M, :open 21.83M, :shares 200}]user=> (apply + (map #(* (:last-trade %) (:shares %)) portfolio))26161.00Muser=> (map #(assoc % :delta (- (% :last-trade) (% :open))) portfolio)({:delta -1.25M, :ticker "AAPL", :last-trade 203.25M, :open 204.50M, :shares 100} {:delta 0.04M, :ticker "MSFT", :last-trade 29.12M, :open 29.08M, :shares 50} {:delta 0.07M, :ticker "ORCL", :last-trade 21.90M, :open 21.83M, :shares 200})user=>

40

{ }:ticker AAPL

:last-trade 203.25


:last-trade 29.12


:last-trade 21.90

:open 21.83



Lazymap

41

user=> (map :open portfolio)(204.5 29.08 21.83)user=> (defn last-trade-value [stock] (* (:last-trade stock) (:shares stock)))#'user/last-trade-valueuser=> (map last-trade-value portfolio)(20325.00M 1456.00M 4380.00M)user=>

f

keywords acts as functions

(:keyword map) == (get map :keyword)


Laziness is a Virtue

user=> (take 20 (iterate inc 1))(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20)user=> (take 20 (map * (iterate inc 1) (iterate inc 1)))(1 4 9 16 25 36 49 64 81 100 121 144 169 196 225 256 289 324 361 400)

42



Lazymap

44

user=> (map #(assoc %1 :sort-index %2) (reverse (sort-by :last-trade portfolio)) (iterate inc 0))({:sort-index 0, :ticker "AAPL", :last-trade 203.25M, :open 204.50M, :shares 100} {:sort-index 1, :ticker "MSFT", :last-trade 29.12M, :open 29.08M, :shares 50} {:sort-index 2, :ticker "ORCL", :last-trade 21.90M, :open 21.83M, :shares 200})user=>

N seqs ➠ N parameters

f

f


reduce

user=> (map #(* (% :last-trade) (% :shares)) portfolio)(20325.00M 1456.00M 4380.00M)user=> (reduce + (map #(* (% :last-trade) (% :shares)) portfolio))26161.00Muser=>

(reduce + (map #(* (% :last-trade) (% :shares)) portfolio))

(reduce + '(20325.00M 1456.00M 4380.00M))

(+ (+ 20325.00M 1456.00M) 4380.00M)

(+ 21781.00M 4380.00M)

26161.00M

45


reduce

user=> (def input-string "Clojure is a fascinating language with unique capabilities and total integration with Java.")#'user/input-stringuser=> (seq input-string) (\C \l \o \j \u \r \e \space \i \s \space \a \space \f \a \s \c \i \n \a \t \i \n \g \space \l \a \n \g \u \a \g \e \space \w \i \t \h \space \u \n \i \q \u \e \space \c \a \p \a \b \i \l \i \t \i \e \s \space \a \n \d \space \t \o \t \a \l \space \i \n \t \e \g \r \a \t \i \o \n \space \w \i \t \h \space \J \a \v \a \.)user=> (reduce (fn [m k] (update-in m [k] #(inc (or % 0)))) {} (seq input-string)) {\space 12, \a 12, \b 1, \C 1, \c 2, \d 1, \e 5, \f 1, \g 4, \h 2, \i 11, \J 1, \j 1, \l 4, \. 1, \n 7, \o 3, \p 1, \q 1, \r 2, \s 3, \t 8, \u 4, \v 1, \w 2}user=>

Optional initial value

f

46


Lazyfilter / remove

47

user=> (remove #(< 100 (% :shares)) portfolio)({:ticker "AAPL", :last-trade 203.25, :open 204.5, :shares 100} {:ticker "ORCL", :last-trade 21.9, :open 21.83, :shares 57})user=> (filter #(< 100 (% :shares)) portfolio)({:ticker "MSFT", :last-trade 29.12, :open 29.08, :shares 125})user=>

f?


Lazyfor: list comprehension

48

user=> (for [suit [:hearts :clubs :spades :diamonds] value (range 1 4)] [suit value])([:hearts 1] [:hearts 2] [:hearts 3] [:clubs 1] [:clubs 2] [:clubs 3] [:spades 1] [:spades 2] [:spades 3] [:diamonds 1] [:diamonds 2] [:diamonds 3])user=> (for [x (range 0 4) y (range 0 (inc x))] [x y])([0 0] [1 0] [1 1] [2 0] [2 1] [2 2] [3 0] [3 1] [3 2] [3 3])user=> (for [x (range 0 9) :when (odd? x) y (range 1 (inc x))] [x y])([1 1] [3 1] [3 2] [3 3] [5 1] [5 2] [5 3] [5 4] [5 5] [7 1] [7 2] [7 3] [7 4] [7 5] [7 6] [7 7])user=>


❝Somehow the idea of reusability got attached to object-oriented programming in the 1980s, and no amount of evidence to the contrary seems to be able to shake it free.❞

Paul Graham


Language Ownership

50


Who Owns the Java Language?

51

James Gosling Mark Reinhold


Not You52


Control of the Compiler

53

Operating System

JVM

Java Libraries

User Classes

ClojureRepl Input

Clojure Source Files

Java Compiler

Source Code

Evaluator


Who Owns Clojure?

54

Rich Hickey


Clojure: written in Clojure

55


Short Circuits

56

if (person.isPharaoh() && person.isDead() && constructPyramid()){ person.bury();}

(if (all-true (.isPharaoh person) (.isDead person) (construct-pyramid)) (.bury person))

&& stops with first false

all parameters to function all-true evaluated first

public static boolean allTrue(boolean... inputs){ for (boolean input : inputs) { if (!input) return false; }

return true;}


Caution: Head Exploding Zone

57


Clojure Macros

58

Reader

Evaluator

Bytecode Generation

macro expansion

(and a b c d)

short circuit at first false/nil

and macro

(let [and__4422__auto__ a] (if and__4422__auto__ (and b c d) and__4422__auto__))

Recursively expanded


Macros ➠ Special Forms

59

Code Forms

Macro Expansion

Bytecode Generation

def if let fn ….

(defmacro and "Evaluates exprs one at a time, from left to right. If a form returns logical false (nil or false), and returns that value and doesn't evaluate any of the other expressions, otherwise it returns the value of the last expr. (and) returns true." ([] true) ([x] x) ([x & next] `(let [and# ~x] (if and# (and ~@next) and#))))


Macros are Special Functions

60

• `(…) — Syntax Quote (allowing replacements)

• and# — Generate a new unique symbol

• ~x — Unquote x

• ~@next — Unquote next and splice in multiple values

(defmacro and ([] true) ([x] x) ([x & next] `(let [and# ~x] (if and# (and ~@next) and#))))


Macros are Special Functions

61

(and a b c d)

• x ➠ a

• next ➠ '(b c d)

• Inside syntax quote:

• and# ➠ and__4422__auto__

• ~x ➠ a

• ~next ➠ '(b c d)

• (and ~@next) ➠ (and b c d)


Simplifying Boilerplate Code

62

(deftest test-link (with-mocks [request HttpServletRequest response HttpServletResponse] (:train (expect .getContextPath request "/ctx") (expect .encodeURL response "/ctx/accounts/list" "*encoded*")) (:test (is (= (link request response list-accounts-with-loop) "*encoded*")))))

public void testLink(){ IMocksControl control = EasyMock.createControl(); HttpServletRequest request = control.newMock(HttpServletRequest.class); HttpServletResponse response = control.newMock(HttpServletResponse.class); EasyMock.expect(request.getContextPath()).andReturn("/ctx"); EasyMock.expect(response.encodeURL("/ctx/accounts/list")).andReturn("*encoded*"); control.replay();

assertEquals(…, "*encoded*");

control.verify();}

(defview root-index [env] :html [ :head [ :title [ "Cascade Blog" ] ] :body [ :h1 [ "Cascade Blog" ]

:ul { :class "recent-postings" } [ (template-for [posting (recent-postings env)]

:li [ (render-link env show-posting (posting :id) (posting :title)) ])

] ] ])


Embedded DSLs

63

:head

:html

:title

"Cascade Blog"

:h1

"Cascade Blog"

:ul

(template-for …)

:body


Macro Expansions

64

(defn list-items [coll] (template (format "%d items" (count coll)) :ul {:class :item-list} [ (template-for [item coll] :li [item])))

(defn list-items [coll] (cascade.internal.viewbuilder/combine (format "%d items" (count coll)) (cascade.dom/element-node :ul {:class :item-list} (cascade.internal.viewbuilder/combine (for [item coll] (cascade.dom/element-node :li nil (cascade.internal.viewbuilder/combine item)))))))

Expand simple placeholder to

executable code

Extend Clojure language from within Clojure


❝More than anything else, I think it is the ability of Lisp programs to manipulate Lisp expressions that sets Lisp apart … when I hear people complain about Lisp's parentheses, it sounds to my ears like someone saying: "I tried one of those bananas, which you say are so delicious. The white part was ok, but the yellow part was very tough and tasted awful."❞

Paul Graham


Wrap Up

66


essence |ˈesəns|nounthe intrinsic nature or indispensable quality of something, esp. something abstract, that determines its character : conflict is the essence of drama.

67


Control is the Essence of

Programming68


Evaluation69


IfWhenHow

70


Clojure

• 1.1 release: Dec 31 2009

• Simple, regular syntax

• Improves on Lisp: vectors, maps, sets

• Fully integrates with Java

• Impressive functional & concurrency support

• Many features not covered here

http://www.clojure.org

71


Stuart Halloway

Pragmatic Bookshelf

http://pragprog.com/titles/shcloj/programming-clojure

72


http://java.ociweb.com/mark/clojure/article.html

73


http://tapestryjava.blogspot.com


Object Oriented

75


Functional

76


Image Credits© 2007 Jon Fifehttp://flickr.com/photos/good-karma/577632972/

© 2003 A. Lipsonhttp://www.andrewlipson.com/escher/relativity.html

© 2007 Alan Chiahttp://flickr.com/photos/seven13avenue/2080281038/

© 2007 Woodley Wonderworkshttp://flickr.com/photos/wwworks/2222523486/

77

© Randall Munroehttp://xkcd.com/297/

© 2008 Manu Gómezhttp://www.flickr.com/photos/manugomi/2884678938/

© 2008 Marcin Wicharyhttp://www.flickr.com/photos/mwichary/2827326852/

© 2006 Marvin (PA)http://www.flickr.com/photos/mscolly/145052885/

© 2007 Casey Marshallhttp://www.flickr.com/photos/rsdio/497112391/

© 2009 Howard M. Lewis Shiphttp://www.flickr.com/photos/hlship/3603090614/

© 2009 Andrew Bairdhttp://www.flickr.com/photos/scruffy/3708615414/

© 2008 Miles Sabinhttp://www.flickr.com/photos/montpelier/2915114545/


Image Credits© 2007 John Kannenberghttp://www.flickr.com/photos/jkannenberg/541057337/

© 2006 scott ogilviehttp://www.flickr.com/photos/scottog/100582274/

78

© 2008 Ariel H.http://www.flickr.com/photos/fotosrotas/2730733412/

© 2006 John Ryan Brubakerhttp://www.flickr.com/photos/subconscience/297682093/

Clojure: Towards The Essence of Programming

Technology

00m usergt

shares200

usergt

usergt

json json

lewis ship

intrinsic

indispensable