Time for Functions

Time for Functions@simontcousins

let rec qsort = function | [] -> [] | hd :: tl -> let lesser, greater = List.partition ((>=) hd) tl List.concat [qsort lesser; [hd]; qsort greater]

http://rosettacode.org/wiki/Sorting_algorithms/Quicksort#F.23

private static List<int> quicksort(List<int> arr) { List<int> loe = new List<int>(), gt = new List<int>(); if (arr.Count < 2) return arr; int pivot = arr.Count / 2; int pivot_val = arr[pivot]; arr.RemoveAt(pivot); foreach (int i in arr) { if (i <= pivot_val) loe.Add(i); else if (i > pivot_val) gt.Add(i); } List<int> resultSet = new List<int>(); resultSet.AddRange(quicksort(loe)); if (loe.Count == 0){ loe.Add(pivot_val); }else{ gt.Add(pivot_val); } resultSet.AddRange(quicksort(gt)); return resultSet; }

• Clear

• closer to a statement of the algorithm

• Concise

• less noise and accidental complexity

• Correct

• the type system works with the developer

Functional code is…

but…and this is a really big but

http://xkcd.com/

Some new things to learn…

let rec qsort = function | [] -> [] | hd :: tl -> let lesser, greater = List.partition ((>=) hd) tl List.concat [qsort lesser; [hd]; qsort greater]

recursion pure functionsimmutable data

pattern matching

partial application

generics by defaulttype inference

higher-order functions

'a list -> 'a list when 'a : comparison

let rec qsort = function | [] -> [] | hd :: tl -> let lesser, greater = List.partition ((>=) hd) tl List.concat [qsort lesser; [hd]; qsort greater]

gotcha!

• Good for demos but what about large programs?

• Good for academics but what about us?

• Elegant code but what about performance?

• Does it work with legacy software?

• Where do I find functional programmers?

Some real-world concerns…

• lots of data

• forecasts

• metered data

• market data

• lots of types

• units of measure

• rates

• station parameters

Bespoke Enterprise Applications for the Energy Sector

• lots of computations

• schedules

• contracts

• analysis

… all changing over time

ww

w.statnett.no

stay at 50Hz

by adjusting

these

THE ENERGY SECTOR

{to make this zero

Project: Balancing Services• Blackstart

• BMSU

• Faststart

• Frequency Response

• Reactive Power

• STOR

contracted services provided by energy

companies to ensure the security and stability of

supply

http://www2.nationalgrid.com/uk/services/balancing-services/

Old System • C#

• OO / Imperative

• Relational Database

• Untestable

• Slow

• Contracts not fully implemented

New System • F#

• Functional / OO / Imperative

• Document Store

• Highly testable

• Fast

• Contracts fully implemented

defeated by complexity tinyurl.com/stor-contract

Dynamic API

Market API

Asset API

Contract Evaluation

APIContract Evaluation

Job API

View Model API

Document Store

Scheduler

Web UI

Test Console

Elegant

beautiful

simple

efficient

functional

Not Elegant

“… but hey, it’s object-oriented!”

• Struggles to be elegant

• abstraction event horizon

• top down designs

• coarse abstractions

• high ceremony

• data and behaviour tightly coupled

• Mutating state is a powerful and dangerous technique

• hard to reason about

• requires synchronised access

Real-world OO• Lots of accidental complexity

• ORMs, IoCs, Mocks, Design Patterns, UML …

• Hard to find developers who have mastered all of this

me preparing

to mutate some state

Not-Only SQL• most applications do not require the

flexibility a relational schema affords

• separate reporting concerns from application concerns

• applications are written in terms of aggregates not relational schemas

• persist aggregates

• making aggregates immutable affords

• as-of

• store inputs and outputs

• what-if, easy test and debug

fits well with functional programs

avoid accidental complexity: ORM,

normal form

Contract Evaluation

API

Contract Evaluation

Job API

Document Store

Scheduler

JobRequest • RunID • Contract • Interval

Input • RunID • Contract Parameters • Asset Parameters • Dynamic Parameters • Market Parameters

JSON Documents

Output • RunID • Revenue • Additional Information

Output

Input

Pure Function

“Pure I/0”

Adoption: F#• Low risk

• Runs on CLR and mono

• Open source

• Inter-op with legacy software and libraries

• Back-out to C#

Adoption: Developers• Self taught

• Hire good .NET developers, not language x developers

• .NET developer cutting F# production code in a week

• Functional programmer in a month

Adoption: Managers

?

Approachexploratory REPL driven

DRYer repeatedly re-factor

test driven documented development

let config = new HttpSelfHostConfiguration(baseAddress) config.MapHttpAttributeRoutes() config.Formatters.JsonFormatter.SerializerSettings <- JsonSerializerSettings( PreserveReferencesHandling = PreserveReferencesHandling.None, Converters = [| Json.TupleConverter() Json.OptionConverter() Json.ArrayConverter() Json.ListConverter() Json.MapTypeConverter() Json.UnionTypeConverter() |]) config.DependencyResolver <- new UnityResolver(container)

Self-host Web API

F# type JSON converters

HostFactory.Run(fun hc -> hc.UseLog4Net("log4net.config") hc.SetServiceName("Job.Api.Host") hc.SetDisplayName("E.ON Ancillary Services Job API Host") hc.SetDescription("An API service for Ancillary Services Jobs.") hc.RunAsNetworkService() |> ignore hc.Service<ApiService>(fun (s: ServiceConfigurator<ApiService>) -> s.ConstructUsing(fun (name: string) -> new ApiService(config)) |> ignore s.WhenStarted(fun (svc: ApiService) -> jobRequestQueue.Start() svc.Start()) |> ignore s.WhenStopped(fun (svc: ApiService) -> svc.Stop() jobRequestQueue.Stop()) |> ignore) |> ignore)

Topshelf Windows ServiceF# working with an

existing OO framework

type ApiService(config: HttpSelfHostConfiguration) =! member val Server = new HttpSelfHostServer(config) with get, set! member this.Start() = this.Server.OpenAsync().Wait() member this.Stop() = if this.Server <> null then this.Server.CloseAsync().Wait() this.Server.Dispose()

Web API Servicean F# class!!!

type JobController(log: ILog, jobRequestQueue: JobRequestQueue) = inherit ApiController()! [<Route("job/ping")>] member x.Get() = log.Debug("ping!!!") "pong" [<Route("job")>] member x.Post(request:JobRequest) = jobRequestQueue.Add(request)

Web API Controlleranother F# class!!!

let requests = BlockingQueueAgent<JobRequest>(config.JobRequestQueueLength)! let workerName (i: int) = String.Format("worker[{0}]", i)! let worker (workerName: string) = async { while true do log.DebugFormat("{0} free", workerName) let! request = requests.AsyncGet() log.DebugFormat("{0} busy: job {1}", workerName, request.JobId) run request }! for i in 1 .. config.JobRequestWorkers do Async.Start(workerName i |> worker, CancellationToken.Token)! requests.Add(request)

Job Queue

github.com/fsprojects/fsharpx/blob/master/src/FSharpx.Core/Agents/BlockingQueueAgent.fs

agents: the safe way to manage state

async, efficient use of threads

scale workers

async { let! input = buildRequest dataProvider let! output = sendToCompute input let result = buildModel input output do! store result }

Execute Jobcomposition of async computations

{

async { use! response = httpClient.PostAsync(uri, toContent request) |> Async.AwaitTask return! response.EnsureSuccessStatusCode().Content.ReadAsStringAsync() |> Async.AwaitTask }

Post

F# async works with TPL Tasks

dispose of resource when done

Async.RunSynchronously( post client config.JobUri request |> Async.Catch, config.Timeout)|> Choice.choice (fun _ -> log.InfoFormat("Executed Job [{0}]", request.JobId)) (fun exn -> log.Error(String.Format("Failed Job [{0}]", request.JobId), exn))

API Callcatch exceptions as Choice2Of2

FSharpxgithub.com/fsprojects/fsharpx/blob/master/src/FSharpx.Core/

ComputationExpressions/Monad.fs

type FrequencyResponseCalculationRequest = { Interval: Interval.Time.T InitialState: FRUnitState ContractParameters: Line.Time.T<ContractParameters> Instruction: Line.Time.T<Instruction> Mel: Line.Time.T<float<MW>> Sel: Line.Time.T<float<MW>> AdjustedPN: Line.Time.T<float<MW>> ActualFrequencies: Line.Time.T<float<Hz>> TargetFrequencies: Line.Time.T<float<Hz>> MarketPrices: Line.Time.T<float<``£``/(MW h)>> }

FR Calculation Request

ubiquitous language

[<Measure>] type min[<Measure>] type hh[<Measure>] type h[<Measure>] type MW!type Interval<'t> = 't * 'ttype Point<'x,'y> = 'x * 'ytype Segment<'x,'y> = Point<'x,'y> * Point<'x,'y>type Line<'x,'y> = Segment<'x,'y> list

Ubiquitous Language

all missing concepts from C# solution

units of measure

module Segment =! type T<'x,'y> = | Instantaneous of Point.T<'x,'y> | Discrete of IntervalType.T * Interval.T<'x> * 'y | Continuous of Point.T<'x,'y> * Point.T<'x,'y>

Ubiquitous Language (Revised)

segment between two data points

segment is an event

segment holds a value over an interval

module Units =! [<AutoOpen>] module UnitNames = /// a unit of time [<Measure>] type minute /// a unit of time [<Measure>] type halfhour /// a unit of time [<Measure>] type hour /// a unit of active power [<Measure>] type megawatt /// a unit of energy [<Measure>] type poundssterling /// a unit of frequency [<Measure>] type hertz [<AutoOpen>] module UnitSymbols = /// a synonym for halfhour, a unit of time [<Measure>] type min = minute /// a synonym for halfhour, a unit of time [<Measure>] type hh = halfhour /// a synonym for hour, a unit of time [<Measure>] type h = hour /// a synonym for megawatt, a unit of power [<Measure>] type MW = megawatt /// a synonym for pounds sterling, a unit of currency [<Measure>] type ``£`` = poundssterling /// a synonym for hertz, a unit of frequency [<Measure>] type Hz = hertz

Units of Measure

https://github.com/fsharp/fsharp/blob/master/src/fsharp/FSharp.Core/SI.fs

// Conversion constants let minutePerHalfhour = 30.0<min>/1.0<hh> let minutePerHour = 60.0<min>/1.0<h> let halfhourPerMinute = 1.0<hh>/30.0<min> let halfhourPerHour = 2.0<hh>/1.0<h> let hourPerMinute = 1.0<h>/60.0<min> let hourPerHalfhour = 1.0<h>/2.0<hh> module Minute = let toHalfhour (a:float<min>) = a * halfhourPerMinute let toHour (a:float<min>) = a * hourPerMinute let inline lift a = LanguagePrimitives.FloatWithMeasure<min>(float a) let liftTimeSpan (t:TimeSpan) = lift t.TotalMinutes

Units of Measure

let run interval initialState parameters actualFrequencies targetFrequencies marketPrices pdtmLine = let deloadLine = DeloadLineCalculation.run … let holdingPayments = holdingPayments … let referencePrices = ReferencePriceCalculation.run … responseEnergyPayments …

Contract Evaluationtop secret

… but it involves a fold

Testing// Straight forward implementation!let rec reverse = function | [] -> [] | x::xs -> reverse xs @ [x] !// Efficient implementation!let rec revAcc xs acc = match xs with | [] -> acc | h::t -> revAcc t (h::acc)!let rev xs = match xs with | [] -> xs | [_] -> xs | h1::h2::t -> revAcc t [h2;h1]!// Generate random tests to see if they behave the same!Check.Quick(fun (xs:int list) -> reverse xs = rev xs)

github.com/fsharp/FsCheck

Testing

open NUnit.Frameworkopen FsUnit![<TestFixture; Category("Unit")>]type ``When I run the deload line calculation`` () =! [<Test>] member x.``with empty MEL line and empty PN line then the deload line is correct`` () = let melLine = Line.empty let pnLine = Line.empty let actual = DeloadLineCalculation.run melLine pnLine let expected : Line.Time.T<float<MW>> = Line.empty actual |> should equal expected

github.com/fsharp/FsUnit

structural equality for free

nice namesopen NUnit.Frameworkopen FsUnit![<TestFixture; Category("Unit")>]type ``When I run the deload line calculation`` () =! [<Test>] member x.``with empty MEL line and empty PN line then the deload line is correct`` () = let melLine = Line.empty let pnLine = Line.empty let actual = DeloadLineCalculation.run melLine pnLine let expected : Line.Time.T<float<MW>> = Line.empty actual |> should equal expected

Two Implementations of the Same ApplicationLi

nes

of C

ode

0

100000

200000

300000

400000

Braces Blanks Null Checks Comments Useful Code App Code Test Code Total Code

30,801

9,359

21,442

16,667

487

15

3,630

643

348,430

42,864

305,566

163,276

53,270

3,01129,080

56,929

C# F#

… things aren’t looking good for the old way of doing things

Logging LOC

Exception Handling LOC

Test Code Ratio

Performance

… and finally say yes to NOOO

Manifesto for Not Only Object-Oriented Development!We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: !• Functions and Types over classes • Purity over mutability • Composition over inheritance • Higher-order functions over method dispatch • Options over nulls !That is, while there is value in the items on the right (except for nulls), we value the items on the left more.

notonlyoo.org

over

0b100,000,000

signatories!

@simontcousins

simontylercousins.net

www.slideshare.net/simontcousins/time-for-functions