Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ClojureReactiveProgramming
TableofContents
ClojureReactiveProgramming
Credits
AbouttheAuthor
Acknowledgments
AbouttheReviewers
www.PacktPub.com
Supportfiles,eBooks,discountoffers,andmore
Whysubscribe?
FreeaccessforPacktaccountholders
Preface
Whatthisbookcovers
Whatyouneedforthisbook
Whothisbookisfor
Conventions
Readerfeedback
Customersupport
Downloadingtheexamplecode
Errata
Piracy
Questions
1.WhatisReactiveProgramming?
AtasteofReactiveProgramming
Creatingtime
Morecolors
Makingitreactive
Exercise1.1
Abitofhistory
Dataflowprogramming
Object-orientedReactiveProgramming
Themostwidelyusedreactiveprogram
TheObserverdesignpattern
FunctionalReactiveProgramming
Higher-orderFRP
Signalsandevents
Implementationchallenges
First-orderFRP
Asynchronousdataflow
ArrowizedFRP
ApplicationsofFRP
Asynchronousprogrammingandnetworking
ComplexGUIsandanimations
Summary
2.ALookatReactiveExtensions
TheObserverpatternrevisited
Observer–anIterator’sdual
CreatingObservables
CustomObservables
ManipulatingObservables
Flatmapandfriends
Onemoreflatmapfortheroad
Errorhandling
OnError
Catch
Retry
Backpressure
Sample
Backpressurestrategies
Summary
3.AsynchronousProgrammingandNetworking
Buildingastockmarketmonitoringapplication
Rollingaverages
Identifyingproblemswithourcurrentapproach
RemovingincidentalcomplexitywithRxClojure
Observablerollingaverages
Summary
4.Introductiontocore.async
Asynchronousprogrammingandconcurrency
core.async
Communicatingsequentialprocesses
Rewritingthestockmarketapplicationwithcore.async
Implementingtheapplicationcode
Errorhandling
Backpressure
Fixedbuffer
Droppingbuffer
Slidingbuffer
Transducers
Transducersandcore.async
Summary
5.CreatingYourOwnCESFrameworkwithcore.async
AminimalCESframework
ClojureorClojureScript?
DesigningthepublicAPI
Implementingtokens
Implementingeventstreams
Implementingbehaviors
Exercises
Exercise5.1
Exercise5.2
Arespondentapplication
CESversuscore.async
Summary
6.BuildingaSimpleClojureScriptGamewithReagi
Settinguptheproject
Gameentities
Puttingitalltogether
Modelinguserinputaseventstreams
Workingwiththeactivekeysstream
ReagiandotherCESframeworks
Summary
7.TheUIasaFunction
TheproblemwithcomplexwebUIs
EnterReact.js
Lessonsfromfunctionalprogramming
ClojureScriptandOm
BuildingasimpleContactsapplicationwithOm
TheContactsapplicationstate
SettinguptheContactsproject
Applicationcomponents
Omcursors
Fillingintheblanks
Intercomponentcommunication
CreatinganagileboardwithOm
Theboardstate
Componentsoverview
Lifecycleandcomponentlocalstate
Remainingcomponents
Utilityfunctions
Exercises
Summary
8.Futures
Clojurefutures
Fetchingdatainparallel
Imminent–acomposablefutureslibraryforClojure
Creatingfutures
Combinatorsandeventhandlers
Themoviesexamplerevisited
FuturesandblockingIO
Summary
9.AReactiveAPItoAmazonWebServices
Theproblem
Infrastructureautomation
AWSresourcesdashboard
CloudFormation
ThedescribeStacksendpoint
ThedescribeStackResourcesendpoint
EC2
ThedescribeInstancesendpoint
RDS
ThedescribeDBInstancesendpoint
Designingthesolution
RunningtheAWSstubserver
Settingupthedashboardproject
CreatingAWSObservables
CombiningtheAWSObservables
Puttingitalltogether
Exercises
Summary
A.TheAlgebraofLibraryDesign
Thesemanticsofmap
Functors
TheOptionFunctor
Findingtheaverageofages
ApplicativeFunctors
Gatheringstatsaboutages
Monads
Summary
B.Bibliography
Index
ClojureReactiveProgramming
ClojureReactiveProgrammingCopyright©2015PacktPublishing
Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.
Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.
PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.
Firstpublished:March2015
Productionreference:1160315
PublishedbyPacktPublishingLtd.
LiveryPlace
35LiveryStreet
BirminghamB32PB,UK.
ISBN978-1-78398-666-8
www.packtpub.com
CreditsAuthor
LeonardoBorges
Reviewers
EduardBondarenko
ColinJones
MichaelKohl
FalkoRiemenschneider
AcquisitionEditor
HarshaBharwani
ContentDevelopmentEditor
ArunNadar
TechnicalEditor
TanviBhatt
CopyEditors
VikrantPhadke
SameenSiddiqui
ProjectCoordinator
NehaBhatnagar
Proofreaders
SimranBhogal
MariaGould
Indexer
MariammalChettiyar
Graphics
AbhinashSahu
ProductionCoordinator
ManuJoseph
CoverWork
ManuJoseph
AbouttheAuthorLeonardoBorgesisaprogramminglanguagesenthusiastwholoveswritingcode,contributingtoopensourcesoftware,andspeakingonsubjectshefeelsstronglyabout.Afternearly5yearsofconsultingatThoughtWorks,whereheworkedintwocommercialClojureprojects,amongmanyothers,heisnowasoftwareengineeratAtlassian.HeusesClojureandClojureScripttohelpbuildreal-timecollaborativeeditingtechnology.Thisishisfirstfull-lengthbook,buthecontributedacoupleofchapterstoClojureCookbook,O’Reilly.
LeonardohasfoundedandrunstheSydneyClojureUserGroupinAustralia.Healsowritespostsaboutsoftware,focusingonfunctionalprogramming,onhiswebsite(http://www.leonardoborges.com).Whenheisn’twritingcode,heenjoysridingmotorcycles,weightlifting,andplayingtheguitar.
AcknowledgmentsIwouldliketotakethisopportunityandstartbythankingmyfamily:mygrandparents,AltamirandAlba,fortheirtirelesssupport;mymother,Sônia,forherunconditionalloveandmotivation;andmyuncle,AltamirFilho,forsupportingmewhenIdecidedtogotoschoolatnightsothatIcouldstartworkingasaprogrammer.Withoutthem,Iwouldhaveneverpursuedsoftwareengineering.
Iwouldalsoliketothankmyfiancee,Enif,whoansweredwitharesounding“yes”whenaskedwhetherIshouldtakeupthechallengeofwritingabook.Herpatience,love,support,andwordsofencouragementwereinvaluable.
Duringthewritingprocess,PacktPublishinginvolvedseveralreviewersandtheirfeedbackwasextremelyusefulinmakingthisabetterbook.Tothesereviewers,thankyou.
Iamalsosincerelygratefulformyfriendswhoprovidedcrucialfeedbackonkeychapters,encouragingmeateverystepoftheway:ClaudioNatoli,FábioLessa,FabioPereira,JulianGamble,SteveBuikhuizen,andmanyothers,whowouldtakemultiplepagestolist.
Lastbutnotleast,awarmthankstothestaffatPacktPublishing,whohelpedmealongthewholeprocess,beingfirmandresponsible,yetunderstanding.
Eachofyouhelpedmakethishappen.Thankyou!
AbouttheReviewersEduardBondarenkoisasoftwaredeveloperlivinginKiev,Ukraine.HestartedprogrammingusingBasiconZXSpectrumalongtimeago.Later,heworkedinthewebdevelopmentdomain.
HehasusedRubyonRailsforabout8years.HavingusedRubyforalongtime,hediscoveredClojureinearly2009,andlikedthelanguage.BesidesRubyandClojure,heisinterestedinErlang,Go,Scala,andHaskelldevelopment.
ColinJonesisdirectorofsoftwareservicesat8thLight,wherehebuildsweb,mobile,anddesktopsystemsforclientsofallsizes.He’stheauthorofMasteringClojureMacros:WriteCleaner,Faster,SmarterCode,PragmaticBookshelf.ColinparticipatesactivelyintheClojureopensourcecommunity,includingworkontheClojureKoans,REPLy,leiningen,andmakessmallcontributionstoClojureitself.
MichaelKohlhasbeendevelopingwithRubysince2004andgotacquaintedwithClojurein2009.Hehasworkedasasystemsadministrator,journalist,systemsengineer,Germanteacher,softwaredeveloper,andpenetrationtester.HecurrentlymakeshislivingasaseniorRubyonRailsdeveloper.HepreviouslyworkedwithPacktPublishingasatechnicalreviewerforRubyandMongoDBWebDevelopmentBeginner’sGuide.
FalkoRiemenschneiderstartedprogrammingin1989.Inthelast15years,hehasworkedonnumerousJavaEnterprisesoftwareprojectsforbackendsaswellasfrontends.He’sespeciallyinterestedindesigningcomplexrich-userinterfaces.In2012,henoticedandlearnedClojure.HequicklycameincontactwithideassuchasFRPandCSPthatshowgreatpotentialforaradicallysimplerUIarchitecturefordesktopandin-browserclients.
Falkoworksforitemis,aGermany-basedsoftwareconsultancyfirmwithstrongcompetenceforlanguage-andmodel-basedsoftwaredevelopment.HecofoundedaClojureusergroup,andencouragesotherdeveloperswithinandoutsideitemistolearnfunctionalprogramming.
Falkopostsregularlyonhttp://www.falkoriemenschneider.de.
www.PacktPub.com
Supportfiles,eBooks,discountoffers,andmoreForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.
DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusat<[email protected]>formoredetails.
Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.
https://www2.packtpub.com/books/subscription/packtlib
DoyouneedinstantsolutionstoyourITquestions?PacktLibisPackt’sonlinedigitalbooklibrary.Here,youcansearch,access,andreadPackt’sentirelibraryofbooks.
Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser
FreeaccessforPacktaccountholdersIfyouhaveanaccountwithPacktatwww.PacktPub.com,youcanusethistoaccessPacktLibtodayandview9entirelyfreebooks.Simplyuseyourlogincredentialsforimmediateaccess.
PrefaceHighlyconcurrentapplicationssuchasuserinterfaceshavetraditionallymanagedstatethroughthemutationofglobalvariables.Variousactionsarecoordinatedviaeventhandlers,whichareproceduralinnature.
Overtime,thecomplexityofasystemincreases.Newfeaturerequestscomein,anditbecomesharderandhardertoreasonabouttheapplication.
Functionalprogrammingpresentsitselfasanextremelypowerfulallyinbuildingreliablesystemsbyeliminatingmutablestatesandallowingapplicationstobewritteninadeclarativeandcomposableway.
SuchprinciplesgaverisetoFunctionalReactiveProgrammingandCompositionalEventSystems(CES),programmingparadigmsthatareexceptionallyusefulinbuildingasynchronousandconcurrentapplications.Theyallowyoutomodelmutablestatesinafunctionalstyle.
Thisbookisdevotedtotheseideasandpresentsanumberofdifferenttoolsandtechniquestohelpmanagetheincreasingcomplexityofmodernsystems.
WhatthisbookcoversChapter1,WhatisReactiveProgramming?,startsbyguidingyouthroughacompellingexampleofareactiveapplicationwritteninClojureScript.ItthentakesyouonajourneythroughthehistoryofReactiveProgramming,duringwhichsomeimportantterminologyisintroduced,settingthetoneforthefollowingchapters.
Chapter2,ALookatReactiveExtensions,exploresthebasicsofthisReactiveProgrammingframework.Itsabstractionsareintroducedandimportantsubjectssuchaserrorhandlingandbackpressurearediscussed.
Chapter3,AsynchronousProgrammingandNetworking,walksyouthroughbuildingastockmarketapplication.ItstartsbyusingamoretraditionalapproachandthenswitchestoanimplementationbasedonReactiveExtensions,examiningthetrade-offsbetweenthetwo.
Chapter4,Introductiontocore.async,describescore.async,alibraryforasynchronousprogramminginClojure.Here,youlearnaboutthebuildingblocksofCommunicatingSequentialProcessesandhowReactiveApplicationsarebuiltwithcore.async.
Chapter5,CreatingYourOwnCESFrameworkwithcore.async,embarksontheambitiousendeavorofbuildingaCESframework.Itleveragesknowledgegainedinthepreviouschapterandusescore.asyncasthefoundationfortheframework.
Chapter6,BuildingaSimpleClojureScriptGamewithReagi,showcasesadomainwhereReactiveframeworkshavebeenusedforgreateffectsingamesdevelopment.
Chapter7,TheUIasaFunction,shiftsgearsandshowshowtheprinciplesoffunctionalprogrammingcanbeappliedtowebUIdevelopmentthroughthelensofOm,aClojureScriptbindingforFacebook’sReact.
Chapter8,Futures,presentsfuturesasaviablealternativetosomeclasses’reactiveapplications.ItexaminesthelimitationsofClojurefuturesandpresentsanalternative:imminent,alibraryofcomposablefuturesforClojure.
Chapter9,AReactiveAPItoAmazonWebServices,describesacasestudytakenfromarealproject,wherealotoftheconceptsintroducedthroughoutthisbookhavebeenputtogethertointeractwithathird-partyservice.
AppendixA,TheAlgebraofLibraryDesign,introducesconceptsfromCategoryTheorythatarehelpfulinbuildingreusableabstractions.Theappendixisoptionalandwon’thinderlearninginthepreviouschapters.ItpresentstheprinciplesusedindesigningthefutureslibraryseeninChapter8,Futures.
AppendixB,Bibliography,providesallthereferencesusedthroughoutthebook.
WhatyouneedforthisbookThisbookassumesthatyouhaveareasonablymoderndesktoporlaptopcomputeraswellasaworkingClojureenvironmentwithleiningen(seehttp://leiningen.org/)properlyconfigured.
Installationinstructionsdependonyourplatformandcanbefoundontheleiningenwebsite(seehttp://leiningen.org/#install).
Youarefreetouseanytexteditorofyourchoice,butpopularchoicesincludeEclipse(seehttps://eclipse.org/downloads/)withtheCounterclockwiseplugin(seehttps://github.com/laurentpetit/ccw),IntelliJ(https://www.jetbrains.com/idea/)withtheCursiveplugin(seehttps://cursiveclojure.com/),LightTable(seehttp://lighttable.com/),Emacs,andVim.
WhothisbookisforThisbookisforClojuredeveloperswhoarecurrentlybuildingorplanningtobuildasynchronousandconcurrentapplicationsandwhoareinterestedinhowtheycanapplytheprinciplesandtoolsofReactiveProgrammingtotheirdailyjobs.
KnowledgeofClojureandleiningen—apopularClojurebuildtool—isrequired.
ThebookalsofeaturesseveralClojureScriptexamples,andassuch,familiaritywithClojureScriptandwebdevelopmentingeneralwillbehelpful.
Notwithstanding,thechaptershavebeencarefullywritteninsuchawaythataslongasyoupossessknowledgeofClojure,followingtheseexamplesshouldonlyrequirealittleextraeffort.
Asthisbookprogresses,itlaysoutthebuildingblocksrequiredbylaterchapters,andassuchmyrecommendationisthatyoustartwithChapter1,WhatisReactiveProgramming?,andworkyourwaythroughsubsequentchaptersinorder.
AclearexceptiontothisisAppendixA,TheAlgebraofLibraryDesign,whichisoptionalandcanbereadindependentoftheothers—althoughreadingChapter8,Futures,mightprovideausefulbackground.
ConventionsInthisbook,youwillfindanumberofstylesoftextthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestyles,andanexplanationoftheirmeaning.
Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:“Wecanincludeothercontextsthroughtheuseoftheincludedirective.”
Ablockofcodeissetasfollows:
(defnumbers(atom[]))
(defnadder[keyrefold-statenew-state]
(print"Currentsumis"(reduce+new-state)))
(add-watchnumbers:adderadder)
Whenwewishtodrawyourattentiontoaparticularpartofacodeblock,therelevantlinesoritemsaresetinbold:
(->(repeat-obs5)
(rx/subscribeprn-to-repl))
;;5
;;5
Anycommand-lineinputoroutputiswrittenasfollows:
leinrun-msin-wave.server
Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,inmenus,ordialogboxes,forexample,appearinthetextlikethis:“IfthiswasawebapplicationouruserswouldbepresentedwithawebservererrorsuchastheHTTPcode500–InternalServerError.”
NoteWarningsorimportantnotesappearinaboxlikethis.
TipTipsandtricksappearlikethis.
ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyoulikedormayhavedisliked.Readerfeedbackisimportantforustodeveloptitlesthatyoureallygetthemostoutof.
Tosendusgeneralfeedback,simplysendane-mailto<[email protected]>,andmentionthebooktitleviathesubjectofyourmessage.
Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideatwww.packtpub.com/authors.
CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.
DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesforallPacktbooksyouhavepurchasedfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.
ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks—maybeamistakeinthetextorthecode—wewouldbegratefulifyouwouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheerratasubmissionformlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedonourwebsite,oraddedtoanylistofexistingerrata,undertheErratasectionofthattitle.Anyexistingerratacanbeviewedbyselectingyourtitlefromhttp://www.packtpub.com/support.
PiracyPiracyofcopyrightmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.Ifyoucomeacrossanyillegalcopiesofourworks,inanyform,ontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.
Pleasecontactusat<[email protected]>withalinktothesuspectedpiratedmaterial.
Weappreciateyourhelpinprotectingourauthors,andourabilitytobringyouvaluablecontent.
QuestionsYoucancontactusat<[email protected]>ifyouarehavingaproblemwithanyaspectofthisbook,andwewilldoourbesttoaddressit.
Chapter1.WhatisReactiveProgramming?ReactiveProgrammingisbothanoverloadedtermandabroadtopic.Assuch,thisbookwillfocusonaspecificformulationofReactiveProgrammingcalledCompositionalEventSystems(CES).
BeforecoveringsomehistoryandbackgroundbehindReactiveProgrammingandCES,Iwouldliketoopenwithaworkingandhopefullycompellingexample:ananimationinwhichwedrawasinewaveontoawebpage.
Thesinewaveissimplythegraphrepresentationofthesinefunction.Itisasmooth,repetitiveoscillation,andattheendofouranimationitwilllooklikethefollowingscreenshot:
ThisexamplewillhighlighthowCES:
UrgesustothinkaboutwhatwewouldliketodoasopposedtohowEncouragessmall,specificabstractionsthatcanbecomposedtogetherProducesterseandmaintainablecodethatiseasytochange
ThecoreofthisprogramboilsdowntofourlinesofClojureScript:
(->sine-wave
(.take600)
(.subscribe(fn[{:keys[xy]}]
(fill-rectxy"orange"))))
Simplybylookingatthiscodeitisimpossibletodeterminepreciselywhatitdoes.However,dotakethetimetoreadandimaginewhatitcoulddo.
First,wehaveavariablecalledsine-wave,whichrepresentsthe2Dcoordinateswewilldrawontothewebpage.Thenextlinegivesustheintuitionthatsine-waveissomesortofcollection-likeabstraction:weuse.taketoretrieve600coordinatesfromit.
Finally,we.subscribetothis“collection”bypassingitacallback.Thiscallbackwillbecalledforeachiteminthesine-wave,finallydrawingatthegivenxandycoordinatesusingthefill-rectfunction.
Thisisquiteabittotakeinfornowaswehaven’tseenanyothercodeyet—butthatwas
thepointofthislittleexercise:eventhoughweknownothingaboutthespecificsofthisexample,weareabletodevelopanintuitionofhowitmightwork.
Let’sseewhatelseisnecessarytomakethissnippetanimateasinewaveonourscreen.
AtasteofReactiveProgrammingThisexampleisbuiltinClojureScriptandusesHTML5CanvasforrenderingandRxJS(seehttps://github.com/Reactive-Extensions/RxJS)—aframeworkforReactiveProgramminginJavaScript.
Beforewestart,keepinmindthatwewillnotgointothedetailsoftheseframeworksyet—thatwillhappenlaterinthisbook.ThismeansI’llbeaskingyoutotakequiteafewthingsatfacevalue,sodon’tworryifyoudon’timmediatelygrasphowthingswork.ThepurposeofthisexampleistosimplygetusstartedintheworldofReactiveProgramming.
Forthisproject,wewillbeusingChestnut(seehttps://github.com/plexus/chestnut)—aleiningentemplateforClojureScriptthatgivesusasampleworkingapplicationwecanuseasaskeleton.
Tocreateournewproject,headovertothecommandlineandinvokeleiningenasfollows:
leinnewchestnutsin-wave
cdsin-wave
Next,weneedtomodifyacoupleofthingsinthegeneratedproject.Openupsin-wave/resources/index.htmlandupdateittolooklikethefollowing:
<!DOCTYPEhtml>
<html>
<head>
<linkhref="css/style.css"rel="stylesheet"type="text/css">
</head>
<body>
<divid="app"></div>
<scriptsrc="/js/rx.all.js"type="text/javascript"></script>
<scriptsrc="/js/app.js"type="text/javascript"></script>
<canvasid="myCanvas"width="650"height="200"style="border:1pxsolid
#d3d3d3;">
</body>
</html>
ThissimplyensuresthatweimportbothourapplicationcodeandRxJS.Wehaven’tdownloadedRxJSyetsolet’sdothisnow.Browsetohttps://github.com/Reactive-Extensions/RxJS/blob/master/dist/rx.all.jsandsavethisfiletosin-wave/resources/public.TheprevioussnippetsalsoaddanHTML5Canvaselementontowhichwewillbedrawing.
Now,open/src/cljs/sin_wave/core.cljs.Thisiswhereourapplicationcodewilllive.Youcanignorewhatiscurrentlythere.Makesureyouhaveacleanslatelikethefollowingone:
(nssin-wave.core)
(defnmain[])
Finally,gobacktothecommandline—underthesin-wavefolder—andstartupthefollowingapplication:
leinrun-msin-wave.server
2015-01-0219:52:34.116:INFO:oejs.Server:jetty-7.6.13.v20130916
2015-01-0219:52:34.158:INFO:oejs.AbstractConnector:Started
[email protected]:10555
Startingfigwheel.
Startingwebserveronport10555.
CompilingClojureScript.
Figwheel:Startingserverathttp://localhost:3449
Figwheel:Servingfilesfrom'(dev-resources|resources)/public'
Oncethepreviouscommandfinishes,theapplicationwillbeavailableathttp://localhost:10555,whereyouwillfindablank,rectangularcanvas.Wearenowreadytobegin.
ThemainreasonweareusingtheChestnuttemplateforthisexampleisthatitperformshot-reloadingofourapplicationcodeviawebsockets.Thismeanswecanhavethebrowserandtheeditorsidebyside,andasweupdateourcode,wewillseetheresultsimmediatelyinthebrowserwithouthavingtoreloadthepage.
Tovalidatethatthisisworking,openyourwebbrowser’sconsolesothatyoucanseetheoutputofthescriptsinthepage.Thenaddthisto/src/cljs/sin_wave/core.cljsasfollows:
(.logjs/console"helloclojurescript")
Youshouldhaveseenthehelloclojurescriptmessageprintedtoyourbrowser’sconsole.Makesureyouhaveaworkingenvironmentuptothispointaswewillberelyingonthisworkflowtointeractivelybuildourapplication.
ItisalsoagoodideatomakesureweclearthecanvaseverytimeChestnutreloadsourfile.Thisissimpleenoughtodobyaddingthefollowingsnippettoourcorenamespace:
(defcanvas(.getElementByIdjs/document"myCanvas"))
(defctx(.getContextcanvas"2d"))
;;Clearcanvasbeforedoinganythingelse
(.clearRectctx00(.-widthcanvas)(.-heightcanvas))
CreatingtimeNowthatwehaveaworkingenvironment,wecanprogresswithouranimation.Itisprobablyagoodideatospecifyhowoftenwewouldliketohaveanewanimationframe.
Thiseffectivelymeansaddingtheconceptoftimetoourapplication.You’refreetoplaywithdifferentvalues,butlet’sstartwithanewframeevery10milliseconds:
(defintervaljs/Rx.Observable.interval)
(deftime(interval10))
AsRxJSisaJavaScriptlibrary,weneedtouseClojureScript’sinteroperabilitytocallitsfunctions.Forconvenience,webindtheintervalfunctionofRxJStoalocalvar.Wewillusethisapproachthroughoutthisbookwhenappropriate.
Next,wecreateaninfinitestreamofnumbers—startingat0—thatwillhaveanewelementevery10milliseconds.Let’smakesurethisisworkingasexpected:
(->time
(.take5)
(.subscribe(fn[n]
(.logjs/consolen))))
;;0
;;1
;;2
;;3
;;4
TipIusethetermstreamverylooselyhere.Itwillbedefinedmorepreciselylaterinthisbook.
Remembertimeisinfinite,soweuse.takeinordertoavoidindefinitelyprintingoutnumberstotheconsole.
Ournextstepistocalculatethe2Dcoordinaterepresentingasegmentofthesinewavewecandraw.Thiswillbegivenbythefollowingfunctions:
(defndeg-to-rad[n]
(*(/Math/PI180)n))
(defnsine-coord[x]
(let[sin(Math/sin(deg-to-radx))
y(-100(*sin90))]
{:xx
:yy
:sinsin}))
Thesine-coordfunctiontakesanxpointofour2DCanvasandcalculatestheypointbasedonthesineofx.Theconstants100and90simplycontrolhowtallandsharptheslopeshouldbe.Asanexample,trycalculatingthesinecoordinatewhenxis50:
(.logjs/console(str(sine-coord50)))
;;{:x50,:y31.05600011929198,:sin0.766044443118978}
Wewillbeusingtimeasthesourceforthevaluesofx.Creatingthesinewavenowisonlyamatterofcombiningbothtimeandsine-coord:
(defsine-wave
(.maptimesine-coord))
Justliketime,sine-waveisaninfinitestream.Thedifferenceisthatinsteadofjustintegers,wewillnowhavethexandycoordinatesofoursinewave,asdemonstratedinthefollowing:
(->sine-wave
(.take5)
(.subscribe(fn[xysin]
(.logjs/console(strxysin)))))
;;{:x0,:y100,:sin0}
;;{:x1,:y98.42928342064448,:sin0.01745240643728351}
;;{:x2,:y96.85904529677491,:sin0.03489949670250097}
;;{:x3,:y95.28976393813505,:sin0.052335956242943835}
;;{:x4,:y93.72191736302872,:sin0.0697564737441253}
Thisbringsustotheoriginalcodesnippetwhichpiquedourinterest,alongsideafunctiontoperformtheactualdrawing:
(defnfill-rect[xycolour]
(set!(.-fillStylectx)colour)
(.fillRectctxxy22))
(->sine-wave
(.take600)
(.subscribe(fn[{:keys[xy]}]
(fill-rectxy"orange"))))
Asthispoint,wecansavethefileagainandwatchasthesinewavewehavejustcreatedgracefullyappearsonthescreen.
MorecolorsOneofthepointsthisexamplesetsouttoillustrateishowthinkingintermsofverysimpleabstractionsandthenbuildingmorecomplexonesontopofthemmakeforcodethatissimplertomaintainandeasiertomodify.
Assuch,wewillnowupdateouranimationtodrawthesinewaveindifferentcolors.Inthiscase,wewouldliketodrawthewaveinredifthesineofxisnegativeandblueotherwise.
Wealreadyhavethesinevaluecomingthroughthesine-wavestream,soallweneedtodoistotransformthisstreamintoonethatwillgiveusthecolorsaccordingtotheprecedingcriteria:
(defcolour(.mapsine-wave
(fn[{:keys[sin]}]
(if(<sin0)
"red"
"blue"))))
Thenextstepistoaddthenewstreamintothemaindrawingloop—remembertocommentthepreviousonesothatwedon’tendupwithmultiplewavesbeingdrawnatthesametime:
(->(.zipsine-wavecolour#(vector%%2))
(.take600)
(.subscribe(fn[[{:keys[xy]}colour]]
(fill-rectxycolour))))
Oncewesavethefile,weshouldseeanewsinewavealternatingbetweenredandblueasthesineofxoscillatesfrom–1to1.
MakingitreactiveAsfunasthishasbeensofar,theanimationwehavecreatedisn’treallyreactive.Sure,itdoesreacttotimeitself,butthatistheverynatureofanimation.Aswewilllatersee,ReactiveProgrammingissocalledbecauseprogramsreacttoexternalinputssuchasmouseornetworkevents.
Wewill,therefore,updatetheanimationsothattheuserisincontrolofwhenthecolorswitchoccurs:thewavewillstartredandswitchtobluewhentheuserclicksanywherewithinthecanvasarea.Furtherclickswillsimplyalternatebetweenredandblue.
Westartbycreatinginfinite—asperthedefinitionoftime—streamsforourcolorprimitivesasfollows:
(defred(.maptime(fn[_]"red")))
(defblue(.maptime(fn[_]"blue")))
Ontheirown,redandbluearen’tthatinterestingastheirvaluesdon’tchange.Wecanthinkofthemasconstantstreams.Theybecomealotmoreinterestingwhencombinedwithanotherinfinitestreamthatcyclesbetweenthembasedonuserinput:
(defconcatjs/Rx.Observable.concat)
(defdeferjs/Rx.Observable.defer)
(deffrom-eventjs/Rx.Observable.fromEvent)
(defmouse-click(from-eventcanvas"click"))
(defcycle-colour
(concat(.takeUntilredmouse-click)
(defer#(concat(.takeUntilbluemouse-click)
cycle-colour))))
Thisisourmostcomplexupdatesofar.Ifyoulookclosely,youwillalsonoticethatcycle-colourisarecursivestream;thatis,itisdefinedintermsofitself.
Whenwefirstsawcodeofthisnature,wetookaleapoffaithintryingtounderstandwhatitdoes.Afteraquickread,however,werealizedthatcycle-colourfollowscloselyhowwemighthavetalkedabouttheproblem:wewillusereduntilamouseclickoccurs,afterwhichwewilluseblueuntilanothermouseclickoccurs.Then,westarttherecursion.
Thechangetoouranimationloopisminimal:
(->(.zipsine-wavecycle-colour#(vector%%2))
(.take600)
(.subscribe(fn[[{:keys[xy]}colour]]
(fill-rectxycolour))))
Thepurposeofthisbookistohelpyoudeveloptheinstinctrequiredtomodelproblemsinthewaydemonstratedhere.Aftereachchapter,moreandmoreofthisexamplewillmakesense.Additionally,anumberofframeworkswillbeusedbothinClojureScriptandClojuretogiveyouawiderangeoftoolstochoosefrom.
Beforewemoveontothat,wemusttakealittledetourandunderstandhowwegothere.
Exercise1.1Modifythepreviousexampleinsuchawaythatthesinewaveisdrawnusingallrainbowcolors.Thedrawingloopshouldlooklikethefollowing:
(->(.zipsine-waverainbow-colours#(vector%%2))
(.take600)
(.subscribe(fn[[{:keys[xy]}colour]]
(fill-rectxycolour))))
Yourtaskistoimplementtherainbow-coloursstream.Aseverythingupuntilnowhasbeenverylightonexplanations,youmightchoosetocomebacktothisexerciselater,oncewehavecoveredmoreaboutCES.
Therepeat,scan,andflatMapfunctionsmaybeusefulinsolvingthisexercise.BesuretoconsultRxJs’APIathttps://github.com/Reactive-Extensions/RxJS/blob/master/doc/libraries/rx.complete.md.
AbitofhistoryBeforewetalkaboutwhatReactiveProgrammingis,itisimportanttounderstandhowotherrelevantprogrammingparadigmsinfluencedhowwedevelopsoftware.Thiswillalsohelpusunderstandthemotivationsbehindreactiveprogramming.
Withfewexceptionsmostofushavebeentaught—eitherself-taughtoratschool/university—imperativeprogramminglanguagessuchasCandPascalorobject-orientedlanguagessuchasJavaandC++.
Inbothcases,theimperativeprogrammingparadigm—ofwhichobject-orientedlanguagesarepart—dictateswewriteprogramsasaseriesofstatementsthatmodifyprogramstate.
Inordertounderstandwhatthismeans,let’slookatashortprogramwritteninpseudo-codethatcalculatesthesumandthemeanvalueofalistofnumbers:
numbers:=[1,2,3,4,5,6]
sum:=0
foreachnumberinnumbers
sum:=sum+number
end
mean:=sum/count(numbers)
TipThemeanvalueistheaverageofthenumbersinthelist,obtainedbydividingthesumbythenumberofelements.
First,wecreateanewarrayofintegers,callednumbers,withnumbersfrom1to6,inclusive.Then,weinitializesumtozero.Next,weiterateoverthearrayofintegers,oneatatime,addingtosumthevalueofeachnumber.
Lastly,wecalculateandassigntheaverageofthenumbersinthelisttothemeanlocalvariable.Thisconcludestheprogramlogic.
Thisprogramwouldprint21forthesumand3forthemean,ifexecuted.
Thoughasimpleexample,ithighlightsitsimperativestyle:wesetupanapplicationstate—sum—andthenexplicitlytellthecomputerhowtomodifythatstateinordertocalculatetheresult.
DataflowprogrammingThepreviousexamplehasaninterestingproperty:thevalueofmeanclearlyhasadependencyonthecontentsofsum.
Dataflowprogrammingmakesthisrelationshipexplicit.Itmodelsapplicationsasadependencygraphthroughwhichdataflows—fromoperationtooperation—andasvalueschange,thesechangesarepropagatedtoitsdependencies.
Historically,dataflowprogramminghasbeensupportedbycustom-builtlanguagessuchasLucidandBLODI,assuch,leavingothergeneralpurposeprogramminglanguagesout.
Let’sseehowthisnewinsightwouldimpactourpreviousexample.Weknowthatoncethelastlinegetsexecuted,thevalueofmeanisassignedandwon’tchangeunlessweexplicitlyreassignthevariable.
However,let’simagineforasecondthatthepseudo-languageweusedearlierdoessupportdataflowprogramming.Inthatcase,assigningmeantoanexpressionthatreferstobothsumandcount,suchassum/count(numbers),wouldbeenoughtocreatethedirecteddependencygraphinthefollowingdiagram:
Notethatadirectsideeffectofthisrelationshipisthatanimplicitdependencyfromsumtonumbersisalsocreated.Thismeansthatifnumberschange,thechangeispropagatedthroughthegraph,firstupdatingsumandthenfinallyupdatingmean.
ThisiswhereReactiveProgrammingcomesin.Thisparadigmbuildsondataflowprogrammingandchangepropagationtobringthisstyleofprogrammingtolanguagesthatdon’thavenativesupportforit.
Forimperativeprogramminglanguages,ReactiveProgrammingcanbemadeavailablevialibrariesorlanguageextensions.Wedon’tcoverthisapproachinthisbook,butshouldthereaderwantmoreinformationonthesubject,pleaserefertodc-lib(seehttps://code.google.com/p/dc-lib/)foranexample.ItisaframeworkthataddsReactiveProgrammingsupporttoC++viadataflowconstraints.
Object-orientedReactiveProgrammingWhendesigninginteractiveapplicationssuchasdesktopGraphicalUserInterfaces(GUIs),weareessentiallyusinganobject-orientedapproachtoReactiveProgramming.Wewillbuildasimplecalculatorapplicationtodemonstratethisstyle.
TipClojureisn’tanobject-orientedlanguage,butwewillbeinteractingwithpartsoftheJavaAPItobuilduserinterfacesthatweredevelopedinanOOparadigm,hencethetitleofthissection.
Let’sstartbycreatinganewleiningenprojectfromthecommandline:
leinnewcalculator
Thiswillcreateadirectorycalledcalculatorinthecurrentfolder.Next,opentheproject.cljfileinyourfavoritetexteditorandaddadependencyonSeesaw,aClojurelibraryforworkingwithJavaSwing:
(defprojectcalculator"0.1.0-SNAPSHOT"
:description"FIXME:writedescription"
:url"http://example.com/FIXME"
:license{:name"EclipsePublicLicense"
:url"http://www.eclipse.org/legal/epl-v10.html"}
:dependencies[[org.clojure/clojure"1.5.1"]
[seesaw"1.4.4"]])
Atthetimeofthiswriting,thelatestSeesawversionavailableis1.4.4.
Next,inthesrc/calculator/core.cljfile,we’llstartbyrequiringtheSeesawlibraryandcreatingthevisualcomponentswe’llbeusing:
(nscalculator.core
(:require[seesaw.core:refer:all]))
(native!)
(defmain-frame(frame:title"Calculator":on-close:exit))
(deffield-x(text"1"))
(deffield-y(text"2"))
(defresult-label(label"Typenumbersintheboxestoaddthemup!"))
TheprecedingsnippetcreatesawindowwiththetitleCalculatorthatendstheprogramwhenclosed.Wealsocreatetwotextinputfields,field-xandfield-y,aswellasalabelthatwillbeusedtodisplaytheresults,aptlynamedresult-label.
Wewouldlikethelabeltobeupdatedautomaticallyassoonasausertypesanewnumberinanyoftheinputfields.Thefollowingcodedoesexactlythat:
(defnupdate-sum[e]
(try
(text!result-label
(str"Sumis"(+(Integer/parseInt(textfield-x))
(Integer/parseInt(textfield-y)))))
(catchExceptione
(println"Errorparsinginput."))))
(listenfield-x:key-releasedupdate-sum)
(listenfield-y:key-releasedupdate-sum)
Thefirstfunction,update-sum,isoureventhandler.Itsetsthetextofresult-labeltothesumofthevaluesinfield-xandfield-y.Weusetry/catchhereasareallybasicwaytohandleerrorssincethekeypressedmightnothavebeenanumber.Wethenaddtheeventhandlertothe:key-releasedeventofbothinputfields.
TipInrealapplications,weneverwantacatchblocksuchasthepreviousone.Thisisconsideredbadstyle,andthecatchblockshoulddosomethingmoreusefulsuchasloggingtheexception,firinganotification,orresumingtheapplicationifpossible.
Wearealmostdone.Allweneedtodonowisaddthecomponentswehavecreatedsofartoourmain-frameandfinallydisplayitasfollows:
(config!main-frame:content
(border-panel
:north(horizontal-panel:items[field-xfield-y])
:centerresult-label
:border5))
(defn-main[&args]
(->main-framepack!show!))
Nowwecansavethefileandruntheprogramfromthecommandlineintheproject’srootdirectory:
leinrun-mcalculator.core
Youshouldseesomethinglikethefollowingscreenshot:
Experimentbytypingsomenumbersineitherorbothtextinputfieldsandwatchhowthevalueofthelabelchangesautomatically,displayingthesumofbothnumbers.
Congratulations!Youhavejustcreatedyourfirstreactiveapplication!
Asalludedtopreviously,thisapplicationisreactivebecausethevalueoftheresultlabelreactstouserinputandisupdatedautomatically.However,thisisn’tthewholestory—itlacksincomposabilityandrequiresustospecifythehow,notthewhatofwhatwe’retryingtoachieve.
Asfamiliarasthisstyleofprogrammingmaybe,makingapplicationsreactivethiswayisn’talwaysideal.
Givenpreviousdiscussions,wenoticewestillhadtobefairlyexplicitinsettinguptherelationshipsbetweenthevariouscomponentsasevidencedbyhavingtowriteacustomhandlerandbindittobothinputfields.
Aswewillseethroughouttherestofthisbook,thereisamuchbetterwaytohandlesimilarscenarios.
ThemostwidelyusedreactiveprogramBothexamplesintheprevioussectionwillfeelfamiliartosomereaders.Ifwecalltheinputtextfields“cells”andtheresultlabel’shandlera“formula”,wenowhavethenomenclatureusedinmodernspreadsheetapplicationssuchasMicrosoftExcel.
ThetermReactiveProgramminghasonlybeeninuseinrecentyears,buttheideaofareactiveapplicationisn’tnew.Thefirstelectronicspreadsheetdatesbackto1969whenRenePardoandRemyLandau,thenrecentgraduatesfromHarvardUniversity,createdLANPAR(LANguageforProgrammingArraysatRandom)[1].
ItwasinventedtosolveaproblemthatBellCanadaandAT&Thadatthetime:theirbudgetingformshad2000cellsthat,whenmodified,forcedasoftwarere-writetakinganywherefromsixmonthstotwoyears.
Tothisday,electronicspreadsheetsremainapowerfulandusefultoolforprofessionalsofvariousfields.
TheObserverdesignpatternAnothersimilaritythekeenreadermayhavenoticediswiththeObserverdesignpattern.Itismainlyusedinobject-orientedapplicationsasawayforobjectstocommunicatewitheachotherwithouthavinganyknowledgeofwhodependsonitschanges.
InClojure,asimpleversionoftheObserverpatterncanbeimplementedusingwatches:
(defnumbers(atom[]))
(defnadder[keyrefold-statenew-state]
(print"Currentsumis"(reduce+new-state)))
(add-watchnumbers:adderadder)
Westartbycreatingourprogramstate,inthiscaseanatomholdinganemptyvector.Next,wecreateawatchfunctionthatknowshowtosumallnumbersinnumbers.Finally,weaddourwatchfunctiontothenumbersatomunderthe:adderkey(usefulforremovingwatches).
TheadderkeyconformswiththeAPIcontractrequiredbyadd-watchandreceivesfourarguments.Inthisexample,weonlycareaboutnew-state.
Now,wheneverweupdatethevalueofnumbers,itswatchwillbeexecuted,asdemonstratedinthefollowing:
(swap!numbersconj1)
;;Currentsumis1
(swap!numbersconj2)
;;Currentsumis3
(swap!numbersconj7)
;;Currentsumis10
Thehighlightedlinesaboveindicatetheresultthatisprintedonthescreeneachtimeweupdatetheatom.
Thoughuseful,theObserverpatternstillrequiressomeamountofworkinsettingupthedependenciesandtherequiredprogramstateinadditiontobeinghardtocompose.
Thatbeingsaid,thispatternhasbeenextendedandisatthecoreofoneoftheReactiveProgrammingframeworkswewilllookatlaterinthisbook,Microsoft’sReactiveExtensions(Rx).
FunctionalReactiveProgrammingJustlikeReactiveProgramming,FunctionalReactiveProgramming—FRPforshort—hasunfortunatelybecomeanoverloadedterm.
FrameworkssuchasRxJava(seehttps://github.com/ReactiveX/RxJava),ReactiveCocoa(seehttps://github.com/ReactiveCocoa/ReactiveCocoa),andBacon.js(seehttps://baconjs.github.io/)becameextremelypopularinrecentyearsandhadpositionedthemselvesincorrectlyasFRPlibraries.Thisledtotheconfusionsurroundingtheterminology.
Aswewillsee,theseframeworksdonotimplementFRPbutratherareinspiredbyit.
Intheinterestofusingthecorrectterminologyaswellasunderstandingwhat“inspiredbyFRP”means,wewillhaveabrieflookatthedifferentformulationsofFRP.
Higher-orderFRPHigher-orderFRPreferstotheoriginalresearchonFRPdevelopedbyConalElliottandPaulHudakintheirpaperFunctionalReactiveAnimation[2]from1997.ThispaperpresentsFran,adomain-specificlanguageembeddedinHaskellforcreatingreactiveanimations.Ithassincebeenimplementedinseverallanguagesasalibraryaswellaspurposebuiltreactivelanguages.
Ifyourecallthecalculatorexamplewecreatedafewpagesago,wecanseehowthatstyleofReactiveProgrammingrequiresustomanagestateexplicitlybydirectlyreadingandwritingfrom/totheinputfields.AsClojuredevelopers,weknowthatavoidingstateandmutabledataisagoodprincipletokeepinmindwhenbuildingsoftware.ThisprincipleisatthecoreofFunctionalProgramming:
(->>[123456]
(mapinc)
(filtereven?)
(reduce+))
;;12
Thisshortprogramincrementsbyoneallelementsintheoriginallist,filtersallevennumbers,andaddsthemupusingreduce.
Notehowwedidn’thavetoexplicitlymanagelocalstatethroughateachstepofthecomputation.
Differentlyfromimperativeprogramming,wefocusonwhatwewanttodo,forexampleiteration,andnothowwewantittobedone,forexampleusingaforloop.Thisiswhytheimplementationmatchesourdescriptionoftheprogramclosely.Thisisknownasdeclarativeprogramming.
FRPbringsthesamephilosophytoReactiveProgramming.AstheHaskellprogramminglanguagewikionthesubjecthaswiselyputit:
FRPisabouthandlingtime-varyingvaluesliketheywereregularvalues.
Putanotherway,FRPisadeclarativewayofmodelingsystemsthatrespondtoinputovertime.
Bothstatementstouchontheconceptoftime.We’llbeexploringthatinthenextsection,whereweintroducethekeyabstractionsprovidedbyFRP:signals(orbehaviors)andevents.
SignalsandeventsSofarwehavebeendealingwiththeideaofprogramsthatreacttouserinput.Thisisofcourseonlyasmallsubsetofreactivesystemsbutisenoughforthepurposesofthisdiscussion.
Userinputhappensseveraltimesthroughtheexecutionofaprogram:keypresses,mousedrags,andclicksarebutafewexamplesofhowausermightinteractwithoursystem.Alltheseinteractionshappenoveraperiodoftime.FRPrecognizesthattimeisanimportantaspectofreactiveprogramsandmakesitafirst-classcitizenthroughitsabstractions.
Bothsignals(alsocalledbehaviors)andeventsarerelatedtotime.Signalsrepresentcontinuous,time-varyingvalues.Events,ontheotherhand,representdiscreteoccurrencesatagivenpointintime.
Forexample,timeisitselfasignal.Itvariescontinuouslyandindefinitely.Ontheotherhand,akeypressbyauserisanevent,adiscreteoccurrence.
Itisimportanttonote,however,thatthesemanticsofhowasignalchangesneednotbecontinuous.Imagineasignalthatrepresentsthecurrent(x,y)coordinatesofyourmousepointer.
Thissignalissaidtochangediscretelyasitdependsontheusermovingthemousepointer—anevent—whichisn’tacontinuousaction.
ImplementationchallengesPerhapsthemostdefiningcharacteristicofclassicalFRPistheuseofcontinuoustime.
ThismeansFRPassumesthatsignalsarechangingallthetime,eveniftheirvalueisstillthesame,leadingtoneedlessrecomputation.Forexample,themousepositionsignalwilltriggerupdatestotheapplicationdependencygraph—liketheonewesawpreviouslyforthemeanprogram—evenwhenthemouseisstationary.
AnotherproblemisthatclassicalFRPissynchronousbydefault:eventsareprocessedinorder,oneatatime.Harmlessatfirst,thiscancausedelays,whichwouldrenderanapplicationunresponsiveshouldaneventtakesubstantiallylongertoprocess.
PaulHudakandothersfurtheredresearchonhigher-orderFRP[7][8]toaddresstheseissues,butthatcameatthecostofexpressivity.
TheotherformulationsofFRPaimtoovercometheseimplementationchallenges.
Throughouttherestofthechapter,I’llbeusingsignalsandbehaviorsinterchangeably.
First-orderFRPThemostwell-knownreactivelanguageinthiscategoryisElm(seehttp://elm-lang.org/),anFRPlanguagethatcompilestoJavaScript.ItwascreatedbyEvanCzaplickiandpresentedinhispaperElm:ConcurrentFRPforFunctionalGUIs[3].
Elmmakessomesignificantchangestohigher-orderFRP.
Itabandonstheideaofcontinuoustimeandisentirelyevent-driven.Asaresult,itsolvestheproblemofneedlessrecomputationhighlightedearlier.First-orderFRPcombinesbothbehaviorsandeventsintosignalswhich,incontrasttohigher-orderFRP,arediscrete.
Additionally,first-orderFRPallowstheprogrammertospecifywhensynchronousprocessingofeventsisn’tnecessary,preventingunnecessaryprocessingdelays.
Finally,Elmisastrictprogramminglanguage—meaningargumentstofunctionsareevaluatedeagerly—andthatisaconsciousdecisionasitpreventsspaceandtimeleaks,whicharepossibleinalazylanguagesuchasHaskell.
TipInanFRPlibrarysuchasFran,implementedinalazylanguage,memoryusagecangrowunwieldyascomputationsaredeferredtotheabsolutelylastpossiblemoment,thereforecausingaspaceleak.Theselargercomputations,accumulatedovertimeduetolaziness,canthencauseunexpecteddelayswhenfinallyexecuted,causingtimeleaks.
AsynchronousdataflowAsynchronousDataFlowgenerallyreferstoframeworkssuchasReactiveExtensions(Rx),ReactiveCocoa,andBacon.js.Itiscalledassuchasitcompletelyeliminatessynchronousupdates.
TheseframeworksintroducetheconceptofObservableSequences[4],sometimescalledEventStreams.
ThisformulationofFRPhastheadvantageofnotbeingconfinedtofunctionallanguages.Therefore,evenimperativelanguageslikeJavacantakeadvantageofthisstyleofprogramming.
Arguably,theseframeworkswereresponsiblefortheconfusionaroundFRPterminology.ConalElliottatsomepointsuggestedthetermCES(seehttps://twitter.com/conal/status/468875014461468677).
Ihavesinceadoptedthisterminology(seehttp://vimeo.com/100688924)asIbelieveithighlightstwoimportantfactors:
AfundamentaldifferencebetweenCESandFRP:CESisentirelyevent-drivenCESishighlycomposableviacombinators,takinginspirationfromFRP
CESisthemainfocusofthisbook.
ArrowizedFRPThisisthelastformulationwewilllookat.ArrowizedFRP[5]introducestwomaindifferencesoverhigher-orderFRP:itusessignalfunctionsinsteadofsignalsandisbuiltontopofJohnHughes’Arrowcombinators[6].
Itismostlyaboutadifferentwayofstructuringcodeandcanbeimplementedasalibrary.Asanexample,ElmsupportsArrowizedFRPviaitsAutomaton(seehttps://github.com/evancz/automaton)library.
TipThefirstdraftofthischaptergroupedthedifferentformulationsofFRPunderthebroadcategoriesofContinuousandDiscreteFRP.ThankstoEvanCzaplicki’sexcellenttalkControllingTimeandSpace:understandingthemanyformulationsofFRP(seehttps://www.youtube.com/watch?v=Agu6jipKfYw),Iwasabletoborrowthemorespecificcategoriesusedhere.ThesecomeinhandywhendiscussingthedifferentapproachestoFRP.
ApplicationsofFRPThedifferentFRPformulationsarebeingusedtodayinseveralproblemspacesbyprofessionalsandbigorganizationsalike.Throughoutthisbook,we’lllookatseveralexamplesofhowCEScanbeapplied.Someoftheseareinterrelatedasmostmodernprogramshaveseveralcross-cuttingconcerns,butwewillhighlighttwomainareas.
AsynchronousprogrammingandnetworkingGUIsareagreatexampleofasynchronousprogramming.Onceyouopenaweboradesktopapplication,itsimplysitsthere,idle,waitingforuserinput.
Thisstateisoftencalledtheeventormaineventloop.Itissimplywaitingforexternalstimuli,suchasakeypress,amousebuttonclick,newdatafromthenetwork,orevenasimpletimer.
Eachofthesestimuliisassociatedwithaneventhandlerthatgetscalledwhenoneoftheseeventshappen,hencetheasynchronousnatureofGUIsystems.
Thisisastyleofprogrammingwehavebeenusedtoformanyyears,butasbusinessanduserneedsgrow,theseapplicationsgrowincomplexityaswell,andbetterabstractionsareneededtohandlethedependenciesbetweenallthecomponentsofanapplication.
AnothergreatexamplethatdealswithmanagingcomplexityaroundnetworktrafficisNetflix,whichusesCEStoprovideareactiveAPItotheirbackendservices.
ComplexGUIsandanimationsGamesare,perhaps,thebestexampleofcomplexuserinterfacesastheyhaveintricaterequirementsarounduserinputandanimations.
TheElmlanguagewementionedbeforeisoneofthemostexcitingeffortsinbuildingcomplexGUIs.AnotherexampleisFlapjax,alsotargetedatwebapplications,butisprovidedasaJavaScriptlibrarythatcanbeintegratedwithexistingJavaScriptcodebases.
SummaryReactiveProgrammingisallaboutbuildingresponsiveapplications.Thereareseveralwaysinwhichwecanmakeourapplicationsreactive.Someareoldideas:dataflowprogramming,electronicspreadsheets,andtheObserverpatternareallexamples.ButCESinparticularhasbecomepopularinrecentyears.
CESaimstobringtoReactiveProgrammingthedeclarativewayofmodelingproblemsthatisatthecoreofFunctionalProgramming.Weshouldworryaboutwhatandnotabouthow.
Innextchapters,wewilllearnhowwecanapplyCEStoourownprograms.
Chapter2.ALookatReactiveExtensionsReactiveExtensions—orRx—isaReactiveProgramminglibraryfromMicrosofttobuildcomplexasynchronousprograms.Itmodelstime-varyingvaluesandeventsasobservablesequencesandisimplementedbyextendingtheObserverdesignpattern.
Itsfirsttargetplatformwas.NET,butNetflixhasportedRxtotheJVMunderthenameRxJava.MicrosoftalsodevelopsandmaintainsaportofRxtoJavaScriptcalledRxJS,whichisthetoolweusedtobuildthesine-waveapplication.ThetwoportsworkatreatforussinceClojurerunsontheJVMandClojureScriptinJavaScriptenvironments.
AswesawinChapter1,WhatisReactiveProgramming?,RxisinspiredbyFunctionalReactiveProgrammingbutusesdifferentterminology.InFRP,thetwomainabstractionsarebehaviorsandevents.Althoughtheimplementationdetailsaredifferent,observablesequencesrepresentevents.Rxalsoprovidesabehavior-likeabstractionthroughanotherdatatypecalledBehaviorSubject.
Inthischapter,wewill:
ExploreRx’smainabstraction:observablesLearnaboutthedualitybetweeniteratorsandobservablesCreateandmanipulateobservablesequences
TheObserverpatternrevisitedInChapter1,WhatisReactiveProgramming?,wesawabriefoverviewoftheObserverdesignpatternandasimpleimplementationofitinClojureusingwatches.Here’showwedidit:
(defnumbers(atom[]))
(defnadder[keyrefold-statenew-state]
(print"Currentsumis"(reduce+new-state)))
(add-watchnumbers:adderadder)
Intheprecedingexample,ourobservablesubjectisthevar,numbers.Theobserveristheadderwatch.Whentheobservablechanges,itpushesitschangestotheobserversynchronously.
Now,contrastthistoworkingwithsequences:
(->>[123456]
(mapinc)
(filtereven?)
(reduce+))
Thistimearound,thevectoristhesubjectbeingobservedandthefunctionsprocessingitcanbethoughtofastheobservers.However,thisworksinapull-basedmodel.Thevectordoesn’tpushanyelementsdownthesequence.Instead,mapandfriendsaskthesequenceformoreelements.Thisisasynchronousoperation.
Rxmakessequences—andmore—behavelikeobservablessothatyoucanstillmap,filter,andcomposethemjustasyouwouldcomposefunctionsovernormalsequences.
Observer–anIterator’sdualClojure’ssequenceoperatorssuchasmap,filter,reduce,andsoonsupportJavaIterables.Asthenameimplies,anIterableisanobjectthatcanbeiteratedover.Atalowlevel,thisissupportedbyretrievinganIteratorreferencefromsuchobject.Java’sIteratorinterfacelookslikethefollowing:
publicinterfaceIterator<E>{
booleanhasNext();
Enext();
voidremove();
}
Whenpassedinanobjectthatimplementsthisinterface,Clojure’ssequenceoperatorspulldatafromitbyusingthenextmethod,whileusingthehasNextmethodtoknowwhentostop.
TipTheremovemethodisrequiredtoremoveitslastelementfromtheunderlyingcollection.Thisin-placemutationisclearlyunsafeinamultithreadedenvironment.WheneverClojureimplementsthisinterfaceforthepurposesofinteroperability,theremovemethodsimplythrowsUnsupportedOperationException.
Anobservable,ontheotherhand,hasobserverssubscribedtoit.Observershavethefollowinginterface:
publicinterfaceObserver<T>{
voidonCompleted();
voidonError(Throwablee);
voidonNext(Tt);
}
Aswecansee,anObserverimplementingthisinterfacewillhaveitsonNextmethodcalledwiththenextvalueavailablefromwhateverobservableit’ssubscribedto.Hence,itbeingapush-basednotificationmodel.
Thisduality[4]becomesclearerifwelookatboththeinterfacessidebyside:
Iterator<E>{Observer<T>{
booleanhasNext();voidonCompleted();
Enext();voidonError(Throwablee);
voidremove();voidonNext(Tt);
}}
Observablesprovidetheabilitytohaveproducerspushitemsasynchronouslytoconsumers.Afewexampleswillhelpsolidifyourunderstanding.
CreatingObservablesThischapterisallaboutReactiveExtensions,solet’sgoaheadandcreateaprojectcalledrx-playgroundthatwewillbeusinginourexploratorytour.WewilluseRxClojure(seehttps://github.com/ReactiveX/RxClojure),alibrarythatprovidesClojurebindingsforRxJava()(seehttps://github.com/ReactiveX/RxJava):
$leinnewrx-playground
OpentheprojectfileandaddadependencyonRxJava’sClojurebindings:
(defprojectrx-playground"0.1.0-SNAPSHOT"
:description"FIXME:writedescription"
:url"http://example.com/FIXME"
:license{:name"EclipsePublicLicense"
:url"http://www.eclipse.org/legal/epl-v10.html"}
:dependencies[[org.clojure/clojure"1.5.1"]
[io.reactivex/rxclojure"1.0.0"]])"]])
Now,fireupaREPLintheproject’srootdirectorysothatwecanstartcreatingsomeobservables:
$leinrepl
ThefirstthingweneedtodoisimportRxClojure,solet’sgetthisoutofthewaybytypingthefollowingintheREPL:
(require'[rx.lang.clojure.core:asrx])
(import'(rxObservable))
Thesimplestwaytocreateanewobservableisbycallingthejustreturnfunction:
(defobs(rx/return10))
Now,wecansubscribetoit:
(rx/subscribeobs
(fn[value]
(prn(str"Gotvalue:"value))))
Thiswillprintthestring"Gotvalue:10"totheREPL.
Thesubscribefunctionofanobservableallowsustoregisterhandlersforthreemainthingsthathappenthroughoutitslifecycle:newvalues,errors,oranotificationthattheobservableisdoneemittingvalues.ThiscorrespondstotheonNext,onError,andonCompletedmethodsoftheObserverinterface,respectively.
Intheprecedingexample,wearesimplysubscribingtoonNext,whichiswhywegetnotifiedabouttheobservable’sonlyvalue,10.
Asingle-valueObservableisn’tterriblyinterestingthough.Let’screateandinteractwithonethatemitsmultiplevalues:
(->(rx/seq->o[12345678910])
(rx/subscribeprn))
Thiswillprintthenumbersfrom1to10,inclusive,totheREPL.seq->oisawaytocreateobservablesfromClojuresequences.ItjustsohappensthattheprecedingsnippetcanberewrittenusingRx’sownrangeoperator:
(->(rx/range110)
(rx/subscribeprn))
Ofcourse,thisdoesn’tyetpresentanyadvantagestoworkingwithrawvaluesorsequencesinClojure.
Butwhatifweneedanobservablethatemitsanundefinednumberofintegersatagiveninterval?ThisbecomeschallengingtorepresentasasequenceinClojure,butRxmakesittrivial:
(import'(java.util.concurrentTimeUnit))
(rx/subscribe(Observable/interval100TimeUnit/MILLISECONDS)
prn-to-repl)
TipRxClojuredoesn’tyetprovidebindingstoallofRxJava’sAPI.Theintervalmethodisonesuchexample.We’rerequiredtouseinteroperabilityandcallthemethoddirectlyontheObservableclassfromRxJava.
Observable/intervaltakesasargumentsanumberandatimeunit.Inthiscase,wearetellingittoemitaninteger—startingfromzero—every100milliseconds.IfwetypethisinanREPL-connectededitor,however,twothingswillhappen:
Wewillnotseeanyoutput(dependingonyourREPL;thisistrueforEmacs)Wewillhavearoguethreademittingnumbersindefinitely
BothissuesarisefromthefactthatObservable/intervalisthefirstfactorymethodwehaveusedthatdoesn’temitvaluessynchronously.Instead,itreturnsanObservablethatdeferstheworktoaseparatethread.
Thefirstissueissimpleenoughtofix.Functionssuchasprnwillprinttowhateverthedynamicvar*out*isboundto.WhenworkingincertainREPLenvironmentssuchasEmacs’,thisisboundtotheREPLstream,whichiswhywecangenerallyseeeverythingweprint.
However,sinceRxisdeferringtheworktoaseparatethread,*out*isn’tboundtotheREPLstreamanymoresowedon’tseetheoutput.Inordertofixthis,weneedtocapturethecurrentvalueof*out*andbinditinoursubscription.ThiswillbeincrediblyusefulasweexperimentwithRxintheREPL.Assuch,let’screateahelperfunctionforit:
(defrepl-out*out*)
(defnprn-to-repl[&args]
(binding[*out*repl-out]
(applyprnargs)))
Thefirstthingwedoiscreateavarrepl-outthatcontainsthecurrentREPLstream.Next,wecreateafunctionprn-to-replthatworksjustlikeprn,exceptitusesthebindingmacrotocreateanewbindingfor*out*thatisvalidwithinthatscope.
Thisstillleavesuswiththeroguethreadproblem.NowistheappropriatetimetomentionthatthesubscribemethodfromanObservablereturnsasubscriptionobject.Byholdingontoareferencetoit,wecancallitsunsubscribemethodtoindicatethatwearenolongerinterestedinthevaluesproducedbythatobservable.
Puttingitalltogether,ourintervalexamplecanberewrittenlikethefollowing:
(defsubscription(rx/subscribe(Observable/interval100
TimeUnit/MILLISECONDS)
prn-to-repl))
(Thread/sleep1000)
(rx/unsubscribesubscription)
Wecreateanewintervalobservableandimmediatelysubscribetoit,justaswedidbefore.Thistime,however,weassigntheresultingsubscriptiontoalocalvar.Notethatitnowusesourhelperfunctionprn-to-repl,sowewillstartseeingvaluesbeingprintedtotheREPLstraightaway.
Next,wesleepthecurrent—theREPL—threadforasecond.ThisisenoughtimefortheObservabletoproducenumbersfrom0to9.That’sroughlywhentheREPLthreadwakesupandunsubscribesfromthatobservable,causingittostopemittingvalues.
CustomObservablesRxprovidesmanymorefactorymethodstocreateObservables(seehttps://github.com/ReactiveX/RxJava/wiki/Creating-Observables),butitisbeyondthescopeofthisbooktocoverthemall.
Nevertheless,sometimes,noneofthebuilt-infactoriesiswhatyouwant.Forsuchcases,Rxprovidesthecreatemethod.Wecanuseittocreateacustomobservablefromscratch.
Asanexample,we’llcreateourownversionofthejustobservableweusedearlierinthischapter:
(defnjust-obs[v]
(rx/observable*
(fn[observer]
(rx/on-nextobserverv)
(rx/on-completedobserver))))
(rx/subscribe(just-obs20)prn)
First,wecreateafunction,just-obs,whichimplementsourobservablebycallingtheobservable*function.
Whencreatinganobservablethisway,thefunctionpassedtoobservable*willgetcalledwithanobserverassoonasonesubscribestous.Whenthishappens,wearefreetodowhatevercomputation—andevenI/O—weneedinordertoproducevaluesandpushthemtotheobserver.
Weshouldremembertocalltheobserver’sonCompletedmethodwheneverwe’redoneproducingvalues.Theprecedingsnippetwillprint20totheREPL.
TipWhilecreatingcustomobservablesisfairlystraightforward,weshouldmakesureweexhaustthebuilt-infactoryfunctionsfirst,onlythenresortingtocreatingourown.
ManipulatingObservablesNowthatweknowhowtocreateobservables,weshouldlookatwhatkindsofinterestingthingswecandowiththem.Inthissection,wewillseewhatitmeanstotreatObservablesassequences.
We’llstartwithsomethingsimple.Let’sprintthesumofthefirstfivepositiveevenintegersfromanobservableofallintegers:
(rx/subscribe(->>(Observable/interval1TimeUnit/MICROSECONDS)
(rx/filtereven?)
(rx/take5)
(rx/reduce+))
prn-to-repl)
Thisisstartingtolookawfullyfamiliartous.Wecreateanintervalthatwillemitallpositiveintegersstartingatzeroevery1microsecond.Then,wefilterallevennumbersinthisobservable.Obviously,thisistoobigalisttohandle,sowesimplytakethefirstfiveelementsfromit.Finally,wereducethevalueusing+.Theresultis20.
Todrivehomethepointthatprogrammingwithobservablesreallyisjustlikeoperatingonsequences,wewilllookatonemoreexamplewherewewillcombinetwodifferentObservablesequences.OnecontainsthenamesofmusiciansI’mafanofandtheotherthenamesoftheirrespectivebands:
(defnmusicians[]
(rx/seq->o["JamesHetfield""DaveMustaine""KerryKing"]))
(defnbands[]
(rx/seq->o["Metallica""Megadeth""Slayer"]))
WewouldliketoprinttotheREPLastringoftheformatMusicianname–from:bandname.Anaddedrequirementisthatthebandnamesshouldbeprintedinuppercaseforimpact.
We’llstartbycreatinganotherobservablethatcontainstheuppercasedbandnames:
(defnuppercased-obs[]
(rx/map(fn[s](.toUpperCases))(bands)))
Whilenotstrictlynecessary,thismakesareusablepieceofcodethatcanbehandyinseveralplacesoftheprogram,thusavoidingduplication.Subscribersinterestedintheoriginalbandnamescankeepsubscribingtothebandsobservable.
Withthetwoobservablesinhand,wecanproceedtocombinethem:
(->(rx/mapvector
(musicians)
(uppercased-obs))
(rx/subscribe(fn[[musicianband]]
(prn-to-repl(strmusician"-from:"band)))))
Oncemore,thisexampleshouldfeelfamiliar.Thesolutionwewereafterwasawaytozip
thetwoobservablestogether.RxClojureprovideszipbehaviorthroughmap,muchlikeClojure’scoremapfunctiondoes.Wecallitwiththreearguments:thetwoobservablestozipandafunctionthatwillbecalledwithbothelements,onefromeachobservable,andshouldreturnanappropriaterepresentation.Inthiscase,wesimplyturnthemintoavector.
Next,inoursubscriber,wesimplydestructurethevectorinordertoaccessthemusicianandbandnames.WecanfinallyprintthefinalresulttotheREPL:
"JamesHetfield-from:METALLICA"
"DaveMustaine-from:MEGADETH"
"KerryKing-from:SLAYER"
FlatmapandfriendsIntheprevioussection,welearnedhowtotransformandcombineobservableswithoperationssuchasmap,reduce,andzip.However,thetwoobservablesabove—musiciansandbands—wereperfectlycapableofproducingvaluesontheirown.Theydidnotneedanyextrainput.
Inthissection,weexamineadifferentscenario:we’lllearnhowwecancombineobservables,wheretheoutputofoneistheinputofanother.WeencounteredflatmapbeforeinChapter1,WhatisReactiveProgramming?Ifyouhavebeenwonderingwhatitsroleis,thissectionaddressesexactlythat.
Here’swhatwearegoingtodo:givenanobservablerepresentingalistofallpositiveintegers,we’llcalculatethefactorialforallevennumbersinthatlist.Sincethelististoobig,we’lltakefiveitemsfromit.Theendresultshouldbethefactorialsof0,2,4,6,and8,respectively.
Thefirstthingweneedisafunctiontocalculatethefactorialofanumbern,aswellasourobservable:
(defnfactorial[n]
(reduce*(range1(incn))))
(defnall-positive-integers[]
(Observable/interval1TimeUnit/MICROSECONDS))
Usingsometypeofvisualaidwillbehelpfulinthissection,sowe’llstartwithamarblediagramrepresentingthepreviousobservable:
Themiddlearrowrepresentstimeanditflowsfromlefttoright.ThisdiagramrepresentsaninfiniteObservablesequence,asindicatedbytheuseofellipsisattheendofit.
Sincewe’recombiningalltheobservablesnow,we’llcreateonethat,givenanumber,emitsitsfactorialusingthehelperfunctiondefinedearlier.We’lluseRx’screatemethodforthispurpose:
(defnfact-obs[n]
(rx/observable*
(fn[observer]
(rx/on-nextobserver(factorialn))
(rx/on-completedobserver))))
Thisisverysimilartothejust-obsobservablewecreatedearlierinthischapter,exceptthatitcalculatesthefactorialofitsargumentandemitstheresult/factorialinstead,endingthesequenceimmediatelythereafter.Thefollowingdiagramillustrateshowitworks:
Wefeedthenumber5totheobservable,whichinturnemitsitsfactorial,120.Theverticalbarattheendofthetimelineindicatesthesequenceterminatesthen.
Runningthecodeconfirmsthatourfunctioniscorrect:
(rx/subscribe(fact-obs5)prn-to-repl)
;;120
Sofarsogood.Now,weneedtocombinebothobservablesinordertoachieveourgoal.ThisiswhereflatmapofRxcomesin.We’llfirstseeitinactionandthengetintotheexplanation:
(rx/subscribe(->>(all-positive-integers)
(rx/filtereven?)
(rx/flatmapfact-obs)
(rx/take5))
prn-to-repl)
Ifweruntheprecedingcode,itwillprintthefactorialsfor0,2,4,6,and8,justlikewewanted:
1
2
24
720
40320
Mostoftheprecedingcodesnippetshouldlookfamiliar.Thefirstthingwedoisfilterallevennumbersfromall-positive-numbers.Thisleavesuswiththefollowingobservablesequence:
Muchlikeall-positive-integers,this,too,isaninfiniteobservable.
However,thenextlineofourcodelooksalittleodd.Wecallflatmapandgiveitthefact-obsfunction.Afunctionweknowitselfreturnsanotherobservable.flatmapwillcallfact-obswitheachvalueitemits.fact-obswill,inturn,returnasingle-valueobservableforeachnumber.However,oursubscriberdoesn’tknowhowtodealwithobservables!It’ssimplyinterestedinthefactorials!
Thisiswhy,aftercallingfact-obstoobtainanobservable,flatmapflattensallofthemintoasingleobservablewecansubscribeto.Thisisquiteamouthful,solet’svisualizewhatthismeans:
Asyoucanseeintheprecedingdiagram,throughouttheexecutionofflatmap,weendupwithalistofobservables.However,wedon’tcareabouteachobservablebutratheraboutthevaluestheyemit.Flatmap,then,istheperfecttoolasitcombines—flattens—allofthemintotheobservablesequenceshownatthebottomofthefigure.
Youcanthinkofflatmapasmapcatforobservablesequences.
Therestofthecodeisstraightforward.Wesimplytakethefirstfiveelementsfromthisobservableandsubscribetoit,aswehavebeendoingsofar.
OnemoreflatmapfortheroadYoumightbewonderingwhatwouldhappeniftheobservablesequencewe’reflatmappingemittedmorethanonevalue.Whatthen?
We’llseeonelastexamplebeforewebeginthenextsectioninordertoillustratethebehaviorofflatMapinsuchcases.
Here’sanobservablethatemitsitsargumenttwice:
(defnrepeat-obs[n]
(rx/seq->o(repeat2n)))
Usingitisstraightforward:
(->(repeat-obs5)
(rx/subscribeprn-to-repl))
;;5
;;5
Aspreviously,we’llnowcombinethisobservablewiththeonewecreatedearlier,all-positive-integers.Beforereadingon,thinkaboutwhatyouexpecttheoutputtobefor,say,thefirstthreepositiveintegers.
Thecodeisasfollows:
(rx/subscribe(->>(all-positive-integers)
(rx/flatmaprepeat-obs)
(rx/take6))
prn-to-repl)
Andtheoutputisasfollows:
0
0
1
1
2
2
Theresultmightbeunexpectedforsomereaders.Let’shavealookatthemarblediagramforthisexampleandmakesureweunderstandhowitworks:
Eachtimerepeat-obsgetscalled,itemitstwovaluesandterminates.flatmapthencombinesthemallinasingleobservable,makingthepreviousoutputclearer.
Onelastthingworthmentioningaboutflatmap—andthetitleofthissection—isthatits“friends”refertotheseveralnamesbywhichflatmapisknown.
Forinstance,Rx.NETcallsitselectMany.RxJavaandScalacallitflatMap—thoughRxJavahasanaliasforitcalledmapMany.TheHaskellcommunitycallsitbind.Thoughtheyhavedifferentnames,thesefunctionssemanticsarethesameandarepartofahigher-orderabstractioncalledaMonad.Wedon’tneedtoknowanythingaboutMonadstoproceed.
Theimportantthingtokeepinmindisthatwhenyou’resittingatthebartalkingtoyourfriendsaboutCompositionalEventSystems,allthesenamesmeanthesamething.
ErrorhandlingAveryimportantaspectofbuildingreliableapplicationsisknowingwhattodowhenthingsgowrong.Itisnaivetoassumethatthenetworkisreliable,thathardwarewon’tfail,orthatwe,asdevelopers,won’tmakemistakes.
RxJavaembracesthisfactandprovidesarichsetofcombinatorstodealwithfailure,afewofwhichweexaminehere.
OnErrorLet’sgetstartedbycreatingabadlybehavedobservablethatalwaysthrowsanexception:
(defnexceptional-obs[]
(rx/observable*
(fn[observer]
(rx/on-nextobserver(throw(Exception."Oops.Somethingwent
wrong")))
(rx/on-completedobserver))))
Nowlet’swatchwhathappensifwesubscribetoit:
(rx/subscribe(->>(exceptional-obs)
(rx/mapinc))
(fn[v](prn-to-repl"resultis"v)))
;;ExceptionOops.Somethingwentwrongrx-playground.core/exceptional-
obs/fn--1505
Theexceptionthrownbyexceptional-obsisn’tcaughtanywheresoitsimplybubblesuptotheREPL.IfthiswasawebapplicationouruserswouldbepresentedwithawebservererrorsuchastheHTTPcode500–InternalServerError.Thoseuserswouldprobablynotuseoursystemagain.
Ideally,wewouldliketogetachancetohandlethisexceptiongracefully,possiblyrenderingafriendlyerrormessagethatwillletoursusersknowwecareaboutthem.
Aswehaveseenearlierinthechapter,thesubscribefunctioncantakeupto3functionsasarguments:
Thefirst,ortheonNexthandler,iscalledwhentheobservableemitsanewvalueThesecond,oronError,iscalledwhenevertheobservablethrowsanexceptionThethirdandlastfunction,oronComplete,iscalledwhentheobservablehascompletedandwillnotemitanynewitems
ForourpurposesweareinterestedintheonErrorhandler,andusingitisstraightforward:
(rx/subscribe(->>(exceptional-obs)
(rx/mapinc))
(fn[v](prn-to-repl"resultis"v))
(fn[e](prn-to-repl"erroris"e)))
;;"erroris"#<Exceptionjava.lang.Exception:Oops.Somethingwentwrong>
Thistime,insteadofthrowingtheexception,ourerrorhandlergetscalledwithit.Thisgivesustheopportunitytodisplayanappropriatemessagetoourusers.
CatchTheuseofonErrorgivesusamuchbetterexperienceoverallbutitisn’tveryflexible.
Let’simagineadifferentscenariowherewehaveanobservableretrievingdatafromthenetwork.Whatif,whenthisobserverfails,wewouldliketopresenttheuserwithacachedvalueinsteadofanerrormessage?
Thisiswherethecatchcombinatorcomesin.Itallowsustospecifyafunctiontobeinvokedwhentheobservablethrowsanexception,muchlikeOnErrordoes.
DifferentlyfromOnError,however,catchhastoreturnanewObservablethatwillbethenewsourceofitemsfromthemomenttheexceptionwasthrown:
(rx/subscribe(->>(exceptional-obs)
(rx/catchExceptione
(rx/return10))
(rx/mapinc))
(fn[v](prn-to-repl"resultis"v)))
;;"resultis"11
Inthepreviousexample,weareessentiallyspecifyingthat,wheneverexceptional-obsthrows,weshouldreturnthevalue10.Wearenotlimitedtosinglevalues,however.Infact,wecanuseanyObservablewelikeasthenewsource:
(rx/subscribe(->>(exceptional-obs)
(rx/catchExceptione
(rx/seq->o(range5)))
(rx/mapinc))
(fn[v](prn-to-repl"resultis"v)))
;;"resultis"1
;;"resultis"2
;;"resultis"3
;;"resultis"4
;;"resultis"5
RetryThelasterrorhandlingcombinatorwe’llexamineisretry.ThiscombinatorisusefulwhenweknowanerrororexceptionisonlytransientsoweshouldprobablygiveitanothershotbyresubscribingtotheObservable.
First,we’llcreateanobservablethatfailswhenitissubscribedtoforthefirsttime.However,thenexttimeitissubscribedto,itsucceedsandemitsanewitem:
(defnretry-obs[]
(let[errored(atomfalse)]
(rx/observable*
(fn[observer]
(if@errored
(rx/on-nextobserver20)
(do(reset!erroredtrue)
(throw(Exception."Oops.Somethingwentwrong"))))))))
Let’sseewhathappensifwesimplysubscribetoit:
(rx/subscribe(retry-obs)
(fn[v](prn-to-repl"resultis"v)))
;;ExceptionOops.Somethingwentwrongrx-playground.core/retry-obs/fn-
-1476
Asexpected,theexceptionsimplybubblesupasinourfirstexample.Howeverweknow—forthepurposesofthisexample—thatthisisatransientfailure.Let’sseewhatchangesifweuseretry:
(rx/subscribe(->>(retry-obs)
(.retry))
(fn[v](prn-to-repl"resultis"v)))
;;"resultis"20
Now,ourcodeisresponsibleforretryingtheObservableandasexpectedwegetthecorrectoutput.
It’simportanttonotethatretrywillattempttoresubscribeindefinitelyuntilitsucceeds.ThismightnotbewhatyouwantsoRxprovidesavariation,calledretryWith,whichallowsustospecifyapredicatefunctionthatcontrolswhenandifretryingshouldstop.
Alltheseoperatorsgiveusthetoolsweneedtobuildreliablereactiveapplicationsandweshouldalwayskeeptheminmindastheyare,withoutadoubt,agreatadditiontoourtoolbox.TheRxJavawikionthesubjectshouldbereferredtoformoreinformation:https://github.com/ReactiveX/RxJava/wiki/Error-Handling-Operators.
BackpressureAnotherissuewemightbefacedwithistheoneofobservablesthatproduceitemsfasterthanwecanconsume.Theproblemthatarisesinthisscenarioiswhattodowiththisever-growingbacklogofitems.
Asanexample,thinkaboutzippingtwoobservablestogether.Thezipoperator(ormapinRxClojure)willonlyemitanewvaluewhenallobservableshaveemittedanitem.
Soifoneoftheseobservablesisalotfasteratproducingitemsthantheothers,mapwillneedtobuffertheseitemsandwaitfortheothers,whichwillmostlikelycauseanerror,asshownhere:
(defnfast-producing-obs[]
(rx/mapinc(Observable/interval1TimeUnit/MILLISECONDS)))
(defnslow-producing-obs[]
(rx/mapinc(Observable/interval500TimeUnit/MILLISECONDS)))
(rx/subscribe(->>(rx/mapvector
(fast-producing-obs)
(slow-producing-obs))
(rx/map(fn[[xy]]
(+xy)))
(rx/take10))
prn-to-repl
(fn[e](prn-to-repl"erroris"e)))
;;"erroris"#<MissingBackpressureException
rx.exceptions.MissingBackpressureException>
Asseenintheprecedingcode,wehaveafastproducingobservablethatemitsitems500timesfasterthantheslowerObservable.Clearly,wecan’tkeepupwithitandsurelyenough,RxthrowsMissingBackpressureException.
Whatthisexceptionistellingusisthatthefastproducingobservabledoesn’tsupportanytypeofbackpressure—whatRxcallsReactivepullbackpressure—thatis,consumerscan’ttellittogoslower.ThankfullyRxprovidesuswithcombinatorsthatarehelpfulinthesescenarios.
SampleOnesuchcombinatorissample,whichallowsustosampleanobservableatagiveninterval,thusthrottlingthesourceobservable’soutput.Let’sapplyittoourpreviousexample:
(rx/subscribe(->>(rx/mapvector
(.sample(fast-producing-obs)200
TimeUnit/MILLISECONDS)
(slow-producing-obs))
(rx/map(fn[[xy]]
(+xy)))
(rx/take10))
prn-to-repl
(fn[e](prn-to-repl"erroris"e)))
;;204
;;404
;;604
;;807
;;1010
;;1206
;;1407
;;1613
;;1813
;;2012
TheonlychangeisthatwecallsampleonourfastproducingObservablebeforecallingmap.Wewillsampleitevery200milliseconds.
Byignoringallotheritemsemittedinthistimeslice,wehavemitigatedourinitialproblem,eventhoughtheoriginalObservabledoesn’tsupportanyformofbackpressure.
Thesamplecombinatorisonlyoneofthecombinatorsusefulinsuchcases.OthersincludethrottleFirst,debounce,buffer,andwindow.Onedrawbackofthisapproach,however,isthatalotoftheitemsgeneratedendupbeingignored.
Dependingonthetypeofapplicationwearebuilding,thismightbeanacceptablecompromise.Butwhatifweareinterestedinallitems?
BackpressurestrategiesIfanObservabledoesn’tsupportbackpressurebutwearestillinterestedinallitemsitemits,wecanuseoneofthebuilt-inbackpressurecombinatorsprovidedbyRx.
Asanexamplewewilllookatonesuchcombinator,onBackpressureBuffer:
(rx/subscribe(->>(rx/mapvector
(.onBackpressureBuffer(fast-producing-obs))
(slow-producing-obs))
(rx/map(fn[[xy]]
(+xy)))
(rx/take10))
prn-to-repl
(fn[e](prn-to-repl"erroris"e)))
;;2
;;4
;;6
;;8
;;10
;;12
;;14
;;16
;;18
;;20
Theexampleisverysimilartotheonewhereweusedsample,buttheoutputisfairlydifferent.Thistimewegetallitemsemittedbybothobservables.
TheonBackpressureBufferstrategyimplementsastrategythatsimplybuffersallitemsemittedbytheslowerObservable,emittingthemwhenevertheconsumerisready.Inourcase,thathappensevery500milliseconds.
OtherstrategiesincludeonBackpressureDropandonBackpressureBlock.
It’sworthnotingthatReactivepullbackpressureisstillworkinprogressandthebestwaytokeepuptodatewithprogressisontheRxJavawikionthesubject:https://github.com/ReactiveX/RxJava/wiki/Backpressure.
SummaryInthischapter,wetookadeepdiveintoRxJava,aportformMicrosoft’sReactiveExtensionsfrom.NET.Welearnedaboutitsmainabstraction,theobservable,andhowitrelatestoiterables.
Wealsolearnedhowtocreate,manipulate,andcombineobservablesinseveralways.Theexamplesshownherewerecontrivedtokeepthingssimple.Nevertheless,allconceptspresentedareextremelyusefulinrealapplicationsandwillcomeinhandyforournextchapter,whereweputthemtouseinamoresubstantialexample.
Finally,wefinishedbylookingaterrorhandlingandbackpressure,bothofwhichareimportantcharacteristicsofreliableapplicationsthatshouldalwaysbekeptinmind.
Chapter3.AsynchronousProgrammingandNetworkingSeveralbusinessapplicationsneedtoreacttoexternalstimuli—suchasnetworktraffic—asynchronously.Anexampleofsuchsoftwaremightbeadesktopapplicationthatallowsustotrackacompany’ssharepricesinthestockmarket.
Wewillbuildthisapplicationfirstusingamoretraditionalapproach.Indoingso,wewill:
BeabletoidentifyandunderstandthedrawbacksofthefirstdesignLearnhowtouseRxClojuretodealwithstatefulcomputationssuchasrollingaveragesRewritetheexampleinadeclarativefashionusingobservablesequences,thusreducingthecomplexityfoundinourfirstapproach
BuildingastockmarketmonitoringapplicationOurstockmarketprogramwillconsistofthreemaincomponents:
Afunctionsimulatinganexternalservicefromwhichwecanquerythecurrentprice—thiswouldlikelybeanetworkcallinarealsettingAschedulerthatpollstheprecedingfunctionatapredefinedintervalAdisplayfunctionresponsibleforupdatingthescreen
We’llstartbycreatinganewleiningenproject,wherethesourcecodeforourapplicationwilllive.Typethefollowingonthecommandlineandthenswitchintothenewlycreateddirectory:
leinnewstock-market-monitor
cdstock-market-monitor
Aswe’llbebuildingaGUIforthisapplication,goaheadandaddadependencyonSeesawtothedependenciessectionofyourproject.clj:
[seesaw"1.4.4"]
Next,createasrc/stock_market_monitor/core.cljfileinyourfavoriteeditor.Let’screateandconfigureourapplication’sUIcomponents:
(nsstock-market-monitor.core
(:require[seesaw.core:refer:all])
(:import(java.util.concurrentScheduledThreadPoolExecutor
TimeUnit)))
(native!)
(defmain-frame(frame:title"Stockpricemonitor"
:width200:height100
:on-close:exit))
(defprice-label(label"Price:-"))
(config!main-frame:contentprice-label)
Asyoucansee,theUIisfairlysimple.Itconsistsofasinglelabelthatwilldisplayacompany’sshareprice.WealsoimportedtwoJavaclasses,ScheduledThreadPoolExecutorandTimeUnit,whichwewilluseshortly.
Thenextthingweneedisourpollingmachinerysothatwecaninvokethepricingserviceonagivenschedule.We’llimplementthisviaathreadpoolsoasnottoblockthemainthread:
TipUserinterfaceSDKssuchasswinghavetheconceptofamain—orUI—thread.ThisisthethreadusedbytheSDKtorendertheUIcomponentstothescreen.Assuch,ifwe
haveblocking—orevensimplyslowrunning—operationsexecuteinthisthread,theuserexperiencewillbeseverelyaffected,hencetheuseofathreadpooltooffloadexpensivefunctioncalls.
(defpool(atomnil))
(defninit-scheduler[num-threads]
(reset!pool(ScheduledThreadPoolExecutor.num-threads)))
(defnrun-every[poolmillisf]
(.scheduleWithFixedDelaypool
f
0millisTimeUnit/MILLISECONDS))
(defnshutdown[pool]
(println"Shuttingdownscheduler…")
(.shutdownpool))
Theinit-schedulerfunctioncreatesScheduledThreadPoolExecutorwiththegivennumberofthreads.That’sthethreadpoolinwhichourperiodicfunctionwillrun.Therun-everyfunctionschedulesafunctionfinthegivenpooltorunattheintervalspecifiedbymillis.Finally,shutdownisafunctionthatwillbecalledonprogramterminationandshutdownthethreadpoolgracefully.
Therestoftheprogramputsallthesepartstogether:
(defnshare-price[company-code]
(Thread/sleep200)
(rand-int1000))
(defn-main[&args]
(show!main-frame)
(.addShutdownHook(Runtime/getRuntime)
(Thread.#(shutdown@pool)))
(init-scheduler1)
(run-every@pool500
#(->>(str"Price:"(share-price"XYZ"))
(text!price-label)
invoke-now)))
Theshare-pricefunctionsleepsfor200millisecondstosimulatenetworklatencyandreturnsarandomintegerbetween0and1,000representingthestock’sprice.
Thefirstlineofour-mainfunctionaddsashutdownhooktotheruntime.Thisallowsourprogramtointercepttermination—suchaspressingCtrl+Cinaterminalwindow—andgivesustheopportunitytoshutdownthethreadpool.
TipTheScheduledThreadPoolExecutorpoolcreatesnon-daemonthreadsbydefault.Aprogramcannotterminateifthereareanynon-daemonthreadsaliveinadditiontotheprogram’smainthread.Thisiswhytheshutdownhookisnecessary.
Next,weinitializetheschedulerwithasinglethreadandscheduleafunctiontobe
executedevery500milliseconds.Thisfunctionaskstheshare-pricefunctionforXYZ’scurrentpriceandupdatesthelabel.
TipDesktopapplicationsrequireallrenderingtobedoneintheUIthread.However,ourperiodicfunctionrunsonaseparatethreadandneedstoupdatethepricelabel.Thisiswhyweuseinvoke-now,whichisaSeesawfunctionthatschedulesitsbodytobeexecutedintheUIthreadassoonaspossible.
Let’sruntheprogrambytypingthefollowingcommandintheproject’srootdirectory:
leintrampolinerun-mstock-market-monitor.core
TipTrampoliningtellsleiningennottonestourprogram’sJVMwithinitsown,thusfreeingustohandleusesofCtrl+Courselvesthroughshutdownhooks.
Awindowliketheoneshowninthefollowingscreenshotwillbedisplayed,withthevaluesonitbeingupdatedasperthescheduleimplementedearlier:
Thisisafinesolution.Thecodeisrelativelystraightforwardandsatisfiesouroriginalrequirements.However,ifwelookatthebigpicture,thereisafairbitofnoiseinourprogram.Mostofitslinesofcodearedealingwithcreatingandmanagingathreadpool,which,whilenecessary,isn’tcentraltotheproblemwe’resolving—it’sanimplementationdetail.
We’llkeepthingsastheyareforthemomentandaddanewrequirement:rollingaverages.
RollingaveragesNowthatwecanseetheup-to-datestockpriceforagivencompany,itmakessensetodisplayarollingaverageofthepast,say,fivestockprices.Inarealscenario,thiswouldprovideanobjectiveviewofacompany’ssharetrendinthestockmarket.
Let’sextendourprogramtoaccommodatethisnewrequirement.
First,we’llneedtomodifyournamespacedefinition:
(nsstock-market-monitor.core
(:require[seesaw.core:refer:all])
(:import(java.util.concurrentScheduledThreadPoolExecutor
TimeUnit)
(clojure.langPersistentQueue)))
Theonlychangeisanewimportclause,forClojure’sPersistentQueueclass.Wewillbeusingthatlater.
We’llalsoneedanewlabeltodisplaythecurrentrunningaverage:
(defrunning-avg-label(label"Runningaverage:-"))
(config!main-frame:content
(border-panel
:northprice-label
:centerrunning-avg-label
:border5))
Next,weneedafunctiontocalculaterollingaverages.Arolling—ormoving—averageisacalculationinstatistics,whereyoutaketheaverageofasubsetofitemsinadataset.Thissubsethasafixedsizeanditshiftsforwardasdatacomesin.Thiswillbecomeclearwithanexample.
Supposeyouhavealistwithnumbersfrom1to10,inclusive.Ifweuse3asthesubsetsize,therollingaveragesareasfollows:
[12345678910]=>2.0
[12345678910]=>3.0
[12345678910]=>4.0
Thehighlightedpartsintheprecedingcodeshowthecurrentwindowbeingusedtocalculatethesubsetaverage.
Nowthatweknowwhatrollingaveragesare,wecanmoveontoimplementitinourprogram:
(defnroll-buffer[buffernumbuffer-size]
(let[buffer(conjbuffernum)]
(if(>(countbuffer)buffer-size)
(popbuffer)
buffer)))
(defnavg[numbers]
(float(/(reduce+numbers)
(countnumbers))))
(defnmake-running-avg[buffer-size]
(let[buffer(atomclojure.lang.PersistentQueue/EMPTY)]
(fn[n]
(swap!bufferroll-buffernbuffer-size)
(avg@buffer))))
(defrunning-avg(running-avg5))
Theroll-bufferfunctionisautilityfunctionthattakesaqueue,anumber,andabuffersizeasarguments.Itaddsthatnumbertothequeue,poppingtheoldestelementifthequeuegoesoverthebufferlimit,thuscausingitscontentstorollover.
Next,wehaveafunctionforcalculatingtheaverageofacollectionofnumbers.Wecasttheresulttofloatifthere’sanunevendivision.
Finally,thehigher-ordermake-running-avgfunctionreturnsastateful,singleargumentfunctionthatclosesoveranemptypersistentqueue.Thisqueueisusedtokeeptrackofthecurrentsubsetofdata.
Wethencreateaninstanceofthisfunctionbycallingitwithabuffersizeof5andsaveittotherunning-avgvar.Eachtimewecallthisnewfunctionwithanumber,itwilladdittothequeueusingtheroll-bufferfunctionandthenfinallyreturntheaverageoftheitemsinthequeue.
Thecodewehavewrittentomanagethethreadpoolwillbereusedasissoallthatislefttodoisupdateourperiodicfunction:
(defnworker[]
(let[price(share-price"XYZ")]
(->>(str"Price:"price)(text!price-label))
(->>(str"Runningaverage:"(running-avgprice))
(text!running-avg-label))))
(defn-main[&args]
(show!main-frame)
(.addShutdownHook(Runtime/getRuntime)
(Thread.#(shutdown@pool)))
(init-scheduler1)
(run-every@pool500
#(invoke-now(worker))))
Sinceourfunctionisn’taone-lineranymore,weabstractitawayinitsownfunctioncalledworker.Asbefore,itupdatesthepricelabel,butwehavealsoextendedittousetherunning-avgfunctioncreatedearlier.
We’rereadytoruntheprogramoncemore:
leintrampolinerun-mstock-market-monitor.core
Youshouldseeawindowliketheoneshowninthefollowingscreenshot:
YoushouldseethatinadditiontodisplayingthecurrentsharepriceforXYZ,theprogramalsokeepstrackandrefreshestherunningaverageofthestreamofprices.
IdentifyingproblemswithourcurrentapproachAsidefromthelinesofcoderesponsibleforbuildingtheuserinterface,ourprogramisroughly48lineslong.
Thecoreoftheprogramresidesintheshare-priceandavgfunctions,whichareresponsibleforqueryingthepriceserviceandcalculatingtheaverageofalistofnnumbers,respectively.Theyrepresentonlysixlinesofcode.Thereisalotofincidentalcomplexityinthissmallprogram.
Incidentalcomplexityiscomplexitycausedbycodethatisnotessentialtotheproblemathand.Inthisexample,wehavetwosourcesofsuchcomplexity—wearedisregardingUIspecificcodeforthisdiscussion:thethreadpoolandtherollingbufferfunction.Theyaddagreatdealofcognitiveloadtosomeonereadingandmaintainingthecode.
Thethreadpoolisexternaltoourproblem.Itisonlyconcernedwiththesemanticsofhowtoruntasksasynchronously.Therollingbufferfunctionspecifiesadetailedimplementationofaqueueandhowtouseittorepresenttheconcept.
Ideally,weshouldbeabletoabstractoverthesedetailsandfocusonthecoreofourproblem;CompositionalEventSystems(CES)allowsustodojustthat.
RemovingincidentalcomplexitywithRxClojureInChapter2,ALookatReactiveExtensions,welearnedaboutthebasicbuildingblocksofRxClojure,anopen-sourceCESframework.Inthissection,we’llusethisknowledgeinordertoremovetheincidentalcomplexityfromourprogram.Thiswillgiveusaclear,declarativewaytodisplaybothpricesandrollingaverages.
TheUIcodewe’vewrittensofarremainsunchanged,butweneedtomakesureRxClojureisdeclaredinthedependenciessectionofourproject.cljfile:
[io.reactivex/rxclojure"1.0.0"]
Then,ensurewerequirethefollowinglibrary:
(nsstock-market-monitor.core
(:require[rx.lang.clojure.core:asrx]
[seesaw.core:refer:all])
(:import(java.util.concurrentTimeUnit)
(rxObservable)))
Thewayweapproachtheproblemthistimeisalsodifferent.Let’stakealookatthefirstrequirement:itrequireswedisplaythecurrentpriceofacompany’sshareinthestockmarket.
Everytimewequerythepriceservice,wegeta—possiblydifferent—priceforthecompanyinquestion.AswesawinChapter2,ALookatReactiveExtensions,modelingthisasanobservablesequenceiseasy,sowe’llstartwiththat.We’llcreateafunctionthatgivesusbackastockpriceobservableforthegivencompany:
(defnmake-price-obs[company-code]
(rx/return(share-pricecompany-code)))
Thisisanobservablethatyieldsasinglevalueandterminates.It’sequivalenttothefollowingmarblediagram:
Partofthefirstrequirementisthatwequerytheserviceonapredefinedtimeinterval—every500millisecondsinthiscase.Thishintsatanobservablewehaveencounteredbefore,aptlynamedinterval.Inordertogetthepollingbehaviorwewant,weneedto
combinetheintervalandthepriceobservables.
Asyouprobablyrecall,flatmapisthetoolforthejobhere:
(rx/flatmap(fn[_](make-price-obs"XYZ"))
(Observable/interval500
TimeUnit/MILLISECONDS))
TheprecedingsnippetcreatesanobservablethatwillyieldthelateststockpriceforXYZevery500millisecondsindefinitely.Itcorrespondstothefollowingdiagram:
Infact,wecansimplysubscribetothisnewobservableandtestitout.Modifyyourmainfunctiontothefollowingsnippetandruntheprogram:
(defn-main[&args]
(show!main-frame)
(let[price-obs(rx/flatmap(fn[_](make-price-obs"XYZ"))
(Observable/interval500
TimeUnit/MILLISECONDS))]
(rx/subscribeprice-obs
(fn[price]
(text!price-label(str"Price:"price))))))
Thisisverycool!Wereplicatedthebehaviorofourfirstprogramwithonlyafewlinesofcode.Thebestpartisthatwedidnothavetoworryaboutthreadpoolsorschedulingactions.Bythinkingabouttheproblemintermsofobservablesequences,aswellascombiningexistingandnewobservables,wewereabletodeclarativelyexpresswhatwe
wanttheprogramtodo.
Thisalreadyprovidesgreatbenefitsinmaintainabilityandreadability.However,wearestillmissingtheotherhalfofourprogram:rollingaverages.
ObservablerollingaveragesItmightnotbeimmediatelyobvioushowwecanmodelrollingaveragesasobservables.Whatweneedtokeepinmindisthatprettymuchanythingwecanthinkofasasequenceofvalues,wecanprobablymodelasanobservablesequence.
Rollingaveragesarenodifferent.Let’sforgetforamomentthatthepricesarecomingfromanetworkcallwrappedinanobservable.Let’simaginewehaveallvalueswecareaboutinaClojurevector:
(defvalues(range10))
Whatweneedisawaytoprocessthesevaluesinpartitions—orbuffers—ofsize5insuchawaythatonlyasinglevalueisdroppedateachinteraction.InClojure,wecanusethepartitionfunctionforthispurpose:
(doseq[buffer(partition51values)]
(prnbuffer))
(01234)
(12345)
(23456)
(34567)
(45678)
...
Thesecondargumenttothepartitionfunctioniscalledastepanditistheoffsetofhowmanyitemsshouldbeskippedbeforestartinganewpartition.Here,wesetitto1inordertocreatetheslidingwindoweffectweneed.
Thebigquestionthenis:canwesomehowleveragepartitionwhenworkingwithobservablesequences?
ItturnsoutthatRxJavahasatransformercalledbufferjustforthispurpose.Thepreviousexamplecanberewrittenasfollows:
(->(rx/seq->o(vec(range10)))
(.buffer51)
(rx/subscribe
(fn[price]
(prn(str"Value:"price)))))
TipAsmentionedpreviously,notallRxJava’sAPIisexposedthroughRxClojure,sohereweneedtouseinteroptoaccessthebuffermethodfromtheobservablesequence.
Asbefore,thesecondargumenttobufferistheoffset,butit’scalledskipintheRxJavadocumentation.IfyourunthisattheREPLyou’llseethefollowingoutput:
"Value:[0,1,2,3,4]"
"Value:[1,2,3,4,5]"
"Value:[2,3,4,5,6]"
"Value:[3,4,5,6,7]"
"Value:[4,5,6,7,8]"
...
Thisisexactlywhatwewant.Theonlydifferenceisthatthebuffermethodwaitsuntilithasenoughelements—fiveinthiscase—beforeproceeding.
Now,wecangobacktoourprogramandincorporatethisideawithourmainfunction.Hereiswhatitlookslike:
(defn-main[&args]
(show!main-frame)
(let[price-obs(->(rx/flatmapmake-price-obs
(Observable/interval500
TimeUnit/MILLISECONDS))
(.publish))
sliding-buffer-obs(.bufferprice-obs51)]
(rx/subscribeprice-obs
(fn[price]
(text!price-label(str"Price:"price))))
(rx/subscribesliding-buffer-obs
(fn[buffer]
(text!running-avg-label(str"Runningaverage:"(avg
buffer)))))
(.connectprice-obs)))
Theprecedingsnippetworksbycreatingtwoobservables.Thefirstone,price-obs,wehadcreatedbefore.Thenewslidingbufferobservableiscreatedusingthebuffertransformeronprice-obs.
Wecan,then,independentlysubscribetoeachoneinordertoupdatethepriceandrollingaveragelabels.Runningtheprogramwilldisplaythesamescreenwe’veseenpreviously:
Youmighthavenoticedtwomethodcallswehadn’tseenbefore:publishandconnect.
Thepublishmethodreturnsaconnectableobservable.Thismeansthattheobservablewon’tstartemittingvaluesuntilitsconnectmethodhasbeencalled.Wedothisherebecausewewanttomakesurethatallthesubscribersreceiveallthevaluesemittedbytheoriginalobservable.
Inconclusion,withoutmuchadditionalcode,weimplementedallrequirementsinaconcise,declarativemannerthatiseasytomaintainandfollow.Wehavealsomadethepreviousroll-bufferfunctioncompletelyunnecessary.
ThefullsourcecodefortheCESversionoftheprogramisgivenhereforreference:
(nsstock-market-monitor.05frp-price-monitor-rolling-avg
(:require[rx.lang.clojure.core:asrx]
[seesaw.core:refer:all])
(:import(java.util.concurrentTimeUnit)
(rxObservable)))
(native!)
(defmain-frame(frame:title"Stockpricemonitor"
:width200:height100
:on-close:exit))
(defprice-label(label"Price:-"))
(defrunning-avg-label(label"Runningaverage:-"))
(config!main-frame:content
(border-panel
:northprice-label
:centerrunning-avg-label
:border5))
(defnshare-price[company-code]
(Thread/sleep200)
(rand-int1000))
(defnavg[numbers]
(float(/(reduce+numbers)
(countnumbers))))
(defnmake-price-obs[_]
(rx/return(share-price"XYZ")))
(defn-main[&args]
(show!main-frame)
(let[price-obs(->(rx/flatmapmake-price-obs
(Observable/interval500
TimeUnit/MILLISECONDS))
(.publish))
sliding-buffer-obs(.bufferprice-obs51)]
(rx/subscribeprice-obs
(fn[price]
(text!price-label(str"Price:"price))))
(rx/subscribesliding-buffer-obs
(fn[buffer]
(text!running-avg-label(str"Runningaverage:"(avg
buffer)))))
(.connectprice-obs)))
Notehowinthisversionoftheprogram,wedidn’thavetouseashutdownhook.ThisisbecauseRxClojurecreatesdaemonthreads,whichareautomaticallyterminatedoncetheapplicationexits.
SummaryInthischapter,wesimulatedareal-worldapplicationwithourstockmarketprogram.We’vewrittenitinasomewhattraditionalwayusingthreadpoolsandacustomqueueimplementation.WethenrefactoredittoaCESstyleusingRxClojure’sobservablesequences.
Theresultingprogramisshorter,simpler,andeasiertoreadonceyougetfamiliarwiththecoreconceptsofRxClojureandRxJava.
InthenextChapterwewillbeintroducedtocore.asyncinpreparationforimplementingourownbasicCESframework.
Chapter4.Introductiontocore.asyncLonggonearethedayswhenprogramswererequiredtodoonlyonethingatatime.Beingabletoperformseveraltasksconcurrentlyisatthecoreofthevastmajorityofmodernbusinessapplications.Thisiswhereasynchronousprogrammingcomesin.
Asynchronousprogramming—and,moregenerally,concurrency—isaboutdoingmorewithyourhardwareresourcesthanyoupreviouslycould.Itmeansfetchingdatafromthenetworkoradatabaseconnectionwithouthavingtowaitfortheresult.Or,perhaps,readinganExcelspreadsheetintomemorywhiletheusercanstilloperatethegraphicalinterface.Ingeneral,itimprovesasystem’sresponsiveness.
Inthischapter,wewilllookathowdifferentplatformshandlethisstyleofprogramming.Morespecifically,wewill:
Beintroducedtocore.async’sbackgroundandAPISolidifyourunderstandingofcore.asyncbyre-implementingthestockmarketapplicationintermsofitsabstractionsUnderstandhowcore.asyncdealswitherrorhandlingandbackpressureTakeabrieftourontransducers
AsynchronousprogrammingandconcurrencyDifferentplatformshavedifferentprogrammingmodels.Forinstance,JavaScriptapplicationsaresingle-threadedandhaveaneventloop.Whenmakinganetworkcall,itiscommontoregisteracallbackthatwillbeinvokedatalaterstage,whenthatnetworkcallcompleteseithersuccessfullyorwithanerror.
Incontrast,whenwe’reontheJVM,wecantakefulladvantageofmultithreadingtoachieveconcurrency.ItissimpletospawnnewthreadsviaoneofthemanyconcurrencyprimitivesprovidedbyClojure,suchasfutures.
However,asynchronousprogrammingbecomescumbersome.Clojurefuturesdon’tprovideanativewayforustobenotifiedoftheircompletionatalaterstage.Inaddition,retrievingvaluesfromanot-yet-completedfutureisablockingoperation.Thiscanbeseenclearlyinthefollowingsnippet:
(defndo-something-important[]
(let[f(future(do(prn"Calculating…")
(Thread/sleep10000)))]
(prn"Perhapsthefuturehasdoneitsjob?")
(prn@f)
(prn"Youwillonlyseethisinabout10seconds…")))
(do-something-important)
Thesecondcalltoprintdereferencesthefuture,causingthemainthreadtoblocksinceithasn’tfinishedyet.Thisiswhyyouonlyseethelastprintafterthethreadinwhichthefutureisrunninghasfinished.Callbackscan,ofcourse,besimulatedbyspawningaseparatethreadtomonitorthefirstone,butthissolutionisclunkyatbest.
AnexceptiontothelackofcallbacksisGUIprogramminginClojure.MuchlikeJavaScript,ClojureSwingapplicationsalsopossessaneventloopandcanrespondtouserinputandinvokelisteners(callbacks)tohandlethem.
Anotheroptionisrewritingthepreviousexamplewithacustomcallbackthatispassedintothefuture:
(defndo-something-important[callback]
(let[f(future(let[answer42]
(Thread/sleep10000)
(callbackanswer)))]
(prn"Perhapsthefuturehasdoneitsjob?")
(prn"Youshouldseethisalmostimmediatelyandthenin10secs…")
f))
(do-something-important(fn[answer]
(prn"Futureisdone.Answeris"answer)))
Thistimetheorderoftheoutputsshouldmakemoresense.However,ifwereturnthefuturefromthisfunction,wehavenowaytogiveitanothercallback.Wehavelostthe
abilitytoperformanactionwhenthefutureendsandarebacktohavingtodereferenceit,thusblockingthemainthreadagain—exactlywhatwewantedtoavoid.
TipJava8introducesanewclass,CompletableFuture,thatallowsregisteringacallbacktobeinvokedoncethefuturecompletes.Ifthat’sanoptionforyou,youcanuseinteroptomakeClojureleveragethenewclass.
Asyoumighthaverealized,CESiscloselyrelatedtoasynchronousprogramming:thestockmarketapplicationwebuiltinthepreviouschapterisanexampleofsuchaprogram.Themain—orUI—threadisneverblockedbytheObservablesfetchingdatafromthenetwork.Additionally,wewerealsoabletoregistercallbackswhensubscribingtothem.
Inmanyasynchronousapplications,however,callbacksarenotthebestwaytogo.Heavyuseofcallbackscanleadtowhatisknownascallbackhell.Clojureprovidesamorepowerfulandelegantsolution.
Inthenextfewsections,wewillexplorecore.async,aClojurelibraryforasynchronousprogramming,andhowitrelatestoReactiveProgramming.
core.asyncIfyou’veeverdoneanyamountofJavaScriptprogramming,youhaveprobablyexperiencedcallbackhell.Ifyouhaven’t,thefollowingcodeshouldgiveyouagoodidea:
http.get('api/users/find?name='+name,function(user){
http.get('api/orders?userId='+user.id,function(orders){
orders.forEach(function(order){
container.append(order);
});
});
});
Thisstyleofprogrammingcaneasilygetoutofhand—insteadofwritingmorenatural,sequentialstepstoachievingatask,thatlogicisinsteadscatteredacrossmultiplecallbacks,increasingthedeveloper’scognitiveload.
Inresponsetothisissue,theJavaScriptcommunityreleasedseveralpromiseslibrariesthataremeanttosolvetheissue.Wecanthinkofpromisesasemptyboxeswecanpassintoandreturnfromourfunctions.Atsomepointinthefuture,anotherprocessmightputavalueinsidethisbox.
Asanexample,theprecedingsnippetcanbewrittenwithpromiseslikethefollowing:
http.get('api/users/find?name='+name)
.then(function(user){
returnhttp.get('api/orders?userId='+user.id);
})
.then(function(orders){
orders.forEach(function(order){
container.append(order);
});
});
Theprecedingsnippetshowshowusingpromisescanflattenyourcallbackpyramid,buttheydon’teliminatecallbacks.ThethenfunctionisapublicfunctionofthepromisesAPI.Itisdefinitelyastepintherightdirectionasthecodeiscomposableandeasiertoread.
Aswetendtothinkinsequencesofsteps,however,wewouldliketowritethefollowing:
user=http.get('api/users/find?name='+name);
orders=http.get('api/orders?userId='+user.id);
orders.forEach(function(order){
container.append(order);
});
Eventhoughthecodelookssynchronous,thebehaviorshouldbenodifferentfromthepreviousexamples.Thisisexactlywhatcore.asyncletsusdoinbothClojureandClojureScript.
CommunicatingsequentialprocessesThecore.asynclibraryisbuiltonanoldidea.ThefoundationuponwhichitlieswasfirstdescribedbyTonyHoare—ofQuicksortfame—inhis1978paperCommunicatingSequentialProcesses(CSP;seehttp://www.cs.ucf.edu/courses/cop4020/sum2009/CSP-hoare.pdf).CSPhassincebeenextendedandimplementedinseverallanguages,thelatestofwhichbeingGoogle’sGoprogramminglanguage.
Itisbeyondthescopeofthisbooktogointothedetailsofthisseminalpaper,sowhatfollowsisasimplifieddescriptionofthemainideas.
InCSP,workismodeledusingtwomainabstractions:channelsandprocesses.CSPisalsomessage-drivenand,assuch,itcompletelydecouplestheproducerfromtheconsumerofthemessage.Itisusefultothinkofchannelsasblockingqueues.
Asimplisticapproachdemonstratingthesebasicabstractionsisasfollows:
(import'java.util.concurrent.ArrayBlockingQueue)
(defnproducer[c]
(prn"Takinganap")
(Thread/sleep5000)
(prn"Nowputtinganameinqueue…")
(.putc"Leo"))
(defnconsumer[c]
(prn"Attemptingtotakevaluefromqueuenow…")
(prn(str"Gotit.Hello"(.takec)"!")))
(defchan(ArrayBlockingQueue.10))
(future(consumerchan))
(future(producerchan))
RunningthiscodeintheREPLshouldshowusoutputsimilartothefollowing:
"Attemptingtotakevaluefromqueuenow…"
"Takinganap"
;;then5secondslater
"Nowputtinganameinquequeue…"
"Gotit.HelloLeo!"
Inordernottoblockourprogram,westartboththeconsumerandtheproducerintheirownthreadsusingafuture.Sincetheconsumerwasstartedfirst,wemostlikelywillseeitsoutputimmediately.However,assoonasitattemptstotakeavaluefromthechannel—orqueue—itwillblock.Itwillwaitforavaluetobecomeavailableandwillonlyproceedaftertheproducerisdonetakingitsnap—clearlyaveryimportanttask.
Now,let’scompareitwithasolutionusingcore.async.First,createanewleiningenprojectandaddadependencyonit:
[org.clojure/core.async"0.1.278.0-76b25b-alpha"]
Now,typethisintheREPLorinyourcorenamespace:
(nscore-async-playground.core
(:require[clojure.core.async:refer[gochan<!>!timeout]]))
(defnprn-with-thread-id[s]
(prn(strs"-Threadid:"(.getId(Thread/currentThread)))))
(defnproducer[c]
(go(prn-with-thread-id"Takinganap")
(<!(timeout5000))
(prn-with-thread-id"Nowputtinganameinquequeue…")
(>!c"Leo")))
(defnconsumer[c]
(go(prn-with-thread-id"Attemptingtotakevaluefromqueuenow…")
(prn-with-thread-id(str"Gotit.Hello"(<!c)"!"))))
(defc(chan))
(consumerc)
(producerc)
Thistimeweareusingahelperfunction,prn-with-thread-id,whichappendsthecurrentthreadIDtotheoutputstring.Iwillexplainwhyshortly,butapartfromthat,theoutputwillhavebeenequivalenttothepreviousone:
"Attemptingtotakevaluefromqueuenow…-Threadid:43"
"Takinganap-Threadid:44"
"Nowputtinganameinquequeue…-Threadid:48"
"Gotit.HelloLeo!-Threadid:48"
Structurally,bothsolutionslookfairlysimilar,butsinceweareusingquiteafewnewfunctionshere,let’sbreakitdown:
chanisafunctionthatcreatesacore.asyncchannel.Asmentionedpreviously,itcanbethoughtofasaconcurrentblockingqueueandisthemainabstractioninthelibrary.Bydefaultchancreatesanunboundedchannel,butcore.asyncprovidesmanymoreusefulchannelconstructors,afewofwhichwe’llbeusinglater.timeoutisanothersuchchannelconstructor.Itgivesusachannelthatwillwaitforagivenamountoftimebeforereturningniltothetakingprocess,closingitselfimmediatelyafterward.Thisisthecore.asyncequivalentofThread/sleep.Thefunctions>!and<!areusedtoputandtakevaluesfromachannel,respectively.Thecaveatisthattheyhavetobeusedinsideagoblock,aswewillexplainlater.goisamacrothattakesabodyofexpressions—whichformagoblock—andcreateslightweightprocesses.Thisiswherethemagichappens.Insideagoblock,anycallsto>!and<!thatwouldordinarilyblockwaitingforvaluestobeavailableinchannelsareinsteadparked.Parkingisaspecialtypeofblockingusedinternallyinthestatemachineofcore.async.TheblogpostbyHueyPetersencoversthisstatemachineindepth(seehttp://hueypetersen.com/posts/2013/08/02/the-state-machines-of-core-async/).
GoblocksaretheveryreasonforwhichIchosetoprintthethreadIDsinourexample.Ifwelookclosely,we’llrealizethatthelasttwostatementswereexecutedinthesamethread
—thisisn’ttrue100percentofthetimeasconcurrencyisinherentlynon-deterministic.Thisisafundamentaldifferencebetweencore.asyncandsolutionsusingthreads/futures.
Threadscanbeexpensive.OntheJVM,theirdefaultstacksizeis512kilobytes—configurableviathe-XssJVMstartupoption.Whendevelopingahighlyconcurrentsystem,creatingthousandsofthreadscanquicklydraintheresourcesofthemachinetheapplicationisrunningon.
core.asyncacknowledgesthislimitationandgivesuslightweightprocesses.Internally,theydoshareathreadpool,butinsteadofwastefullycreatingathreadpergoblock,threadsarerecycledandreusedwhenaput/takeoperationiswaitingforavaluetobecomeavailable.
TipAtthetimeofwriting,thethreadpoolusedbycore.asyncdefaultstothenumberofavailableprocessorsx2,+42.So,amachinewitheightprocessorswillhaveapoolwith58threads.
Therefore,itiscommonforcore.asyncapplicationstohavedozensofthousandsoflightweightprocesses.Theyareextremelycheaptocreate.
SincethisisabookonReactiveProgramming,thequestionthatmightbeinyourheadnowis:canwebuildreactiveapplicationsusingcore.async?Theshortanswerisyes,wecan!Toproveit,wewillrevisitourstockmarketapplicationandrewriteitusingcore.async.
Rewritingthestockmarketapplicationwithcore.asyncByusinganexamplewearefamiliarwith,weareabletofocusonthedifferencesbetweenallapproachesdiscussedsofar,withoutgettingsidetrackedwithnew,specificdomainrules.
Beforewediveintotheimplementation,let’squicklydoanoverviewofhowoursolutionshouldwork.
Justlikeinourpreviousimplementations,wehaveaservicefromwhichwecanqueryshareprices.Whereourapproachdiffers,however,isadirectconsequenceofhowcore.asyncchannelswork.
Onagivenschedule,wewouldliketowritethecurrentpricetoacore.asyncchannel.Thismightlooklikeso:
Thisprocesswillcontinuouslyputpricesintheoutchannel.Weneedtodotwothingswitheachprice:displayitanddisplaythecalculatedslidingwindow.Sincewelikeourfunctionsdecoupled,wewillusetwogoblocks,oneforeachtask:
Holdon.Thereseemstobesomethingoffwithourapproach.Oncewetakeapricefrom
theoutputchannel,itisnotavailableanylongertobetakenbyothergoblocks,so,insteadofcalculatingtheslidingwindowstartingwith10,ourfunctionendsupgettingthesecondvalue,20.Withthisapproach,wewillendupwithaslidingwindowthatcalculatesaslidingwindowwithroughlyeveryotheritem,dependingonhowconsistenttheinterleavingbetweenthegoblocksis.
Clearly,thisisnotwhatwewant,butithelpsusthinkabouttheproblemalittlemore.Thesemanticsofcore.asyncpreventusfromreadingavaluefromachannelmorethanonce.Mostofthetime,thisbehaviorisjustfine—especiallyifyouthinkofthemasqueues.Sohowcanweprovidethesamevaluetobothfunctions?
Tosolvethisproblem,wewilltakeadvantageofanotherchannelconstructorprovidedbycore.asynccalledbroadcast.Asthenameimplies,broadcastreturnsachannel,which,whenwrittento,writesitsvalueintothechannelspassedtoitasarguments.Effectively,thischangesourhigh-levelpicturetosomethinglikethefollowing:
Insummary,wewillhaveagoloopwritingpricestothisbroadcastchannel,whichwillthenforwarditsvaluestothetwochannelsfromwhichwewillbeoperating:pricesandtheslidingwindow.
Withthegeneralideainplace,wearereadytodiveintothecode.
ImplementingtheapplicationcodeWealreadyhaveaprojectdependingoncore.asyncthatwecreatedintheprevioussection,sowe’llbeworkingoffthat.Let’sstartbyaddinganextradependencyonseesawtoyourproject.cljfile:
:dependencies[[org.clojure/clojure"1.5.1"]
[org.clojure/core.async"0.1.278.0-76b25b-alpha"]
[seesaw"1.4.4"]]
Next,createafilecalledstock_market.cljinthesrcdirectoryandaddthisnamespacedeclaration:
(nscore-async-playground.stock-market
(:require[clojure.core.async
:refer[gochan<!>!timeoutgo-loopmap>]:asasync])
(:require[clojure.core.async.lab:refer[broadcast]])
(:use[seesaw.core]))
ThismightbeagoodpointtorestartyourREPLifyouhaven’tdoneso.Don’tworryaboutanyfunctionswehaven’tseenyet.We’llgetafeelfortheminthissection.
TheGUIcoderemainslargelyunchanged,sonoexplanationshouldbenecessaryforthenextsnippet:
(native!)
(defmain-frame(frame:title"Stockpricemonitor"
:width200:height100
:on-close:exit))
(defprice-label(label"Price:-"))
(defrunning-avg-label(label"Runningaverage:-"))
(config!main-frame:content
(border-panel
:northprice-label
:centerrunning-avg-label
:border5))
(defnshare-price[company-code]
(Thread/sleep200)
(rand-int1000))
(defnavg[numbers]
(float(/(reduce+numbers)
(countnumbers))))
(defnroll-buffer[buffervalbuffer-size]
(let[buffer(conjbufferval)]
(if(>(countbuffer)buffer-size)
(popbuffer)
buffer)))
(defnmake-sliding-buffer[buffer-size]
(let[buffer(atomclojure.lang.PersistentQueue/EMPTY)]
(fn[n]
(swap!bufferroll-buffernbuffer-size))))
(defsliding-buffer(make-sliding-buffer5))
Theonlydifferenceisthatnowwehaveasliding-bufferfunctionthatreturnsawindowofdata.Thisisincontrastwithouroriginalapplication,wheretherolling-avgfunctionwasresponsibleforbothcreatingthewindowandcalculatingtheaverage.Thisnewdesignismoregeneralasitmakesthisfunctioneasiertoreuse.Theslidinglogicisthesame,however.
Next,wehaveourmainapplicationlogicusingcore.async:
(defnbroadcast-at-interval[msecstask&ports]
(go-loop[out(applybroadcastports)]
(<!(timeoutmsecs))
(>!out(task))
(recurout)))
(defn-main[&args]
(show!main-frame)
(let[prices-ch(chan)
sliding-buffer-ch(map>sliding-buffer(chan))]
(broadcast-at-interval500#(share-price"XYZ")prices-chsliding-
buffer-ch)
(go-loop[]
(when-let[price(<!prices-ch)]
(text!price-label(str"Price:"price))
(recur)))
(go-loop[]
(when-let[buffer(<!sliding-buffer-ch)]
(text!running-avg-label(str"Runningaverage:"(avgbuffer)))
(recur)))))
Let’swalkthroughthecode.
Thefirstfunction,broadcast-at-interval,isresponsibleforcreatingthebroadcastingchannel.Itreceivesavariablenumberofarguments:anumberofmillisecondsdescribingtheinterval,thefunctionrepresentingthetasktobeexecuted,andasequenceofoneofmoreoutputchannels.Thesechannelsareusedtocreatethebroadcastingchanneltowhichthegoloopwillbewritingprices.
Next,wehaveourmainfunction.Theletblockiswheretheinterestingbitsare.Aswediscussedinourhigh-leveldiagrams,weneedtwooutputchannels:oneforpricesandonefortheslidingwindow.Theyarebothcreatedinthefollowing:
...
(let[prices-ch(chan)
sliding-buffer-ch(map>sliding-buffer(chan))]
...
prices-chshouldbeself-explanatory;however,sliding-buffer-chisusingafunctionwehaven’tencounteredbefore:map>.Thisisyetanotherusefulchannelconstructorin
core.async.Ittakestwoarguments:afunctionandatargetchannel.Itreturnsachannelthatappliesthisfunctiontoeachvaluebeforewritingittothetargetchannel.Anexamplewillhelpillustratehowitworks:
(defc(map>sliding-buffer(chan10)))
(go(doseq[n(range10)]
(>!cn)))
(go(doseq[n(range10)]
(prn(vec(<!c)))))
;;[0]
;;[01]
;;[012]
;;[0123]
;;[01234]
;;[12345]
;;[23456]
;;[34567]
;;[45678]
;;[56789]
Thatis,wewriteapricetothechannelandgetaslidingwindowontheotherend.Finally,wecreatethetwogoblockscontainingthesideeffects.Theyloopindefinitely,gettingvaluesfrombothchannelsandupdatingtheuserinterface.
Youcanseeitinactionbyrunningtheprogramfromtheterminal:
$leinrun-mcore-async-playground.stock-market
ErrorhandlingBackinChapter2,ALookatReactiveExtensions,welearnedhowReactiveExtensionstreatserrorsandexceptions.Itprovidesarichsetofcombinatorstodealwithexceptionalcasesandarestraightforwardtouse.
Despitebeingapleasuretoworkwith,core.asyncdoesn’tshipwithmuchsupportforexceptionhandling.Infact,ifwewriteourcodewithonlythehappypathinmindwedon’tevenknowanerroroccurred!
Let’shavealookatanexample:
(defnget-data[]
(throw(Exception."Badthingshappen!")))
(defnprocess[]
(let[result(chan)]
;;dosomeprocessing…
(go(>!result(get-data)))
result))
Intheprecedingsnippet,weintroducedtwofunctions:
get-datasimulatesafunctionthatfetchesdatafromthenetworkoranin-memorycache.Inthiscaseitsimplythrowsanexception.processisafunctionthatdependsonget-datatodosomethinginterestingandputstheresultintoachannel,whichisreturnedattheend.
Let’swatchwhathappenswhenweputthistogether:
(go(let[result(<!(->>(process"data")
(map>#(*%%))
(map>#(prn%))))]
(prn"resultis:"result)))
Nothinghappens.Zero,zip,zilch,nada.
Thisispreciselytheproblemwitherrorhandlingincore.async:bydefault,ourexceptionsareswallowedbythegoblockasitrunsonaseparatethread.Weareleftinthisstatewherewedon’treallyknowwhathappened.
Notallislost,however.DavidNolenoutlinedonhisblogapatternfordealingwithsuchasynchronousexceptions.Itonlyrequiresafewextralinesofcode.
Westartbydefiningahelperfunctionandmacro—thiswouldprobablyliveinautilitynamespacewerequireanywhereweusecore.async:
(defnthrow-err[e]
(when(instance?Throwablee)(throwe))
e)
(defmacro<?[ch]
`(throw-err(async/<!~ch)))
Thethrow-errfunctionreceivesavalueand,ifit’sasubclassofThrowable,itisthrown.Otherwise,itissimplyreturned.
Themacro<?isessentiallyadrop-inreplacementfor<!.Infact,ituses<!togetthevalueoutofthechannelbutpassesittothrow-errfirst.
Withtheseutilitiesinplace,weneedtomakeacoupleofchanges,firsttoourprocessfunction:
(defnprocess[]
(let[result(chan)]
;;dosomeprocessing…
(go(>!result(try(get-data)
(catchExceptione
e))))
result))
Theonlychangeisthatwewrappedget-datainatry/catchblock.Lookcloselyatthecatchblock:itsimplyreturnstheexception.
Thisisimportantasweneedtoensuretheexceptiongetsputintothechannel.
Next,weupdateourconsumercode:
(go(try(let[result(<?(->>(process"data")
(map>#(*%%))
(map>#(prn%))))]
(prn"resultis:"result))
(catchExceptione
(prn"Oops,anerrorhappened!Webetterdosomethingaboutit
here!"))))
;;"Oops,anerrorhappened!Webetterdosomethingaboutithere!"
Thistimeweuse<?inplaceof<!.Thismakessenseasitwillrethrowanyexceptionsfoundinthechannel.Asaresultwecannowuseasimpletry/catchtoregaincontroloverourexceptions.
BackpressureThemainmechanismbywhichcore.asyncallowsforcoordinatingbackpressureisbuffering.core.asyncdoesn’tallowunboundedbuffersasthiscanbeasourceofbugsandaresourcehog.
Instead,wearerequiredtothinkhardaboutourapplication’suniqueneedsandchooseanappropriatebufferingstrategy.
FixedbufferThisisthesimplestformofbuffering.Itisfixedtoachosennumbern,allowingproducerstoputitemsinthechannelwithouthavingtowaitforconsumers:
(defresult(chan(buffer5)))
(go-loop[]
(<!(async/timeout1000))
(when-let[x(<!result)]
(prn"Gotvalue:"x)
(recur)))
(go(doseq[n(range5)]
(>!resultn))
(prn"Doneputtingvalues!")
(close!result))
;;"Doneputtingvalues!"
;;"Gotvalue:"0
;;"Gotvalue:"1
;;"Gotvalue:"2
;;"Gotvalue:"3
;;"Gotvalue:"4
Intheprecedingexample,wecreatedabufferofsize5andstartedagolooptoconsumevaluesfromit.Thegoloopusesatimeoutchanneltodelayitsstart.
Then,westartanothergoblockthatputsnumbersfrom0to4intotheresultchannelandprintstotheconsoleonceit’sdone.
Bythen,thefirsttimeoutwillhaveexpiredandwewillseethevaluesprintedtotheREPL.
Nowlet’swatchwhathappensifthebufferisn’tlargeenough:
(defresult(chan(buffer2)))
(go-loop[]
(<!(async/timeout1000))
(when-let[x(<!result)]
(prn"Gotvalue:"x)
(recur)))
(go(doseq[n(range5)]
(>!resultn))
(prn"Doneputtingvalues!")
(close!Result))
;;"Gotvalue:"0
;;"Gotvalue:"1
;;"Gotvalue:"2
;;"Doneputtingvalues!"
;;"Gotvalue:"3
;;"Gotvalue:"4
Thistimeourbuffersizeis2buteverythingelseisthesame.Asyoucanseethegoloopfinishesmuchlaterasitattemptedtoputanothervalueintheresultchannelandwas
blocked/parkedsinceitsbufferwasfull.
Aswithmostthings,thismightbeOKbutifwearenotwillingtoblockafastproducerjustbecausewecan’tconsumeitsitemsfastenough,wemustlookforanotheroption.
DroppingbufferAdroppingbufferalsohasafixedsize.However,insteadofblockingproducerswhenitisfull,itsimplyignoresanynewitemsasshownhere:
(defresult(chan(dropping-buffer2)))
(go-loop[]
(<!(async/timeout1000))
(when-let[x(<!result)]
(prn"Gotvalue:"x)
(recur)))
(go(doseq[n(range5)]
(>!resultn))
(prn"Doneputtingvalues!")
(close!result))
;;"Doneputtingvalues!"
;;"Gotvalue:"0
;;"Gotvalue:"1
Asbefore,westillhaveabufferofsizetwo,butthistimetheproducerendsquicklywithoutevergettingblocked.Thedropping-buffersimplyignoredallitemsoveritslimit.
SlidingbufferAdrawbackofdroppingbuffersisthatwemightnotbeprocessingthelatestitemsatagiventime.Forthetimeswhereprocessingthelatestinformationisamust,wecanuseaslidingbuffer:
(defresult(chan(sliding-buffer2)))
(go-loop[]
(<!(async/timeout1000))
(when-let[x(<!result)]
(prn"Gotvalue:"x)
(recur)))
(go(doseq[n(range5)]
(>!resultn))
(prn"Doneputtingvalues!")
(close!result))
;;"Doneputtingvalues!"
;;"Gotvalue:"3
;;"Gotvalue:"4
Asbefore,weonlygettwovaluesbuttheyarethelatestonesproducedbythegoloop.
Whenthelimitoftheslidingbufferisoverrun,core.asyncdropstheoldestitemstomakeroomforthenewestones.Iendupusingthisbufferingstrategymostofthetime.
TransducersBeforewefinishupwithourcore.asyncportionofthebook,itwouldbeunwiseofmenottomentionwhatiscomingupinClojure1.7aswellashowthisaffectscore.async.
Atthetimeofthiswriting,Clojure’slatestreleaseis1.7.0-alpha5—andeventhoughitisanalpharelease,alotofpeople—myselfincluded—arealreadyusingitinproduction.
Assuch,afinalversioncouldbejustaroundthecornerandperhapsbythetimeyoureadthis,1.7finalwillbeoutalready.
Oneofthebigchangesinthisupcomingreleaseistheintroductionoftransducers.Wewillnotcoverthenutsandboltsofitherebutratherfocusonwhatitmeansatahigh-levelwithexamplesusingbothClojuresequencesandcore.asyncchannels.
IfyouwouldliketoknowmoreIrecommendCarinMeier’sGreenEggsandTransducersblogpost(http://gigasquidsoftware.com/blog/2014/09/06/green-eggs-and-transducers/).It’sagreatplacetostart.
Additionally,theofficialClojuredocumentationsiteonthesubjectisanotherusefulresource(http://clojure.org/transducers).
Let’sgetstartedbycreatinganewleiningenproject:
$leinnewcore-async-transducers
Now,openyourproject.cljfileandmakesureyouhavetherightdependencies:
...
:dependencies[[org.clojure/clojure"1.7.0-alpha5"]
[org.clojure/core.async"0.1.346.0-17112a-alpha"]]
...
Next,fireupaREPLsessionintheprojectrootandrequirecore.async,whichwewillbeusingshortly:
$leinrepl
user>(require'[clojure.core.async:refer[gochanmap<filter<into>!<!
go-loopclose!pipe]])
Wewillstartwithafamiliarexample:
(->>(range10)
(mapinc);;createsanewsequence
(filtereven?);;createsanewsequence
(prn"resultis"))
;;"resultis"(246810)
TheprecedingsnippetisstraightforwardandhighlightsaninterestingpropertyofwhathappenswhenweapplycombinatorstoClojuresequences:eachcombinatorcreatesanintermediatesequence.
Inthepreviousexample,weendedupwiththreeintotal:theonecreatedbyrange,theonecreatedbymap,andfinallytheonecreatedbyfilter.Mostofthetime,thiswon’t
reallybeanissuebutforlargesequencesthismeansalotofunnecessaryallocation.
StartinginClojure1.7,thepreviousexamplecanbewrittenlikeso:
(defxform
(comp(mapinc)
(filtereven?)));;nointermediatesequencecreated
(->>(range10)
(sequencexform)
(prn"resultis"))
;;"resultis"(246810)
TheClojuredocumentationdescribestransducersascomposablealgorithmictransformations.Let’sseewhythatis.
Inthenewversion,awholerangeofthecoresequencecombinators,suchasmapandfilter,havegainedanextraarity:ifyoudon’tpassitacollection,itinsteadreturnsatransducer.
Inthepreviousexample,(mapinc)returnsatransducerthatknowshowtoapplythefunctioninctoelementsofasequence.Similarly,(filtereven?)returnsatransducerthatwilleventuallyfilterelementsofasequence.Neitherofthemdoanythingyet,theysimplyreturnfunctions.
Thisisinterestingbecausetransducersarecomposable.Webuildlargerandmorecomplextransducersbyusingsimplefunctioncomposition:
(defxform
(comp(mapinc)
(filtereven?)))
Oncewehaveourtransducerready,wecanapplyittoacollectioninafewdifferentways.Forthisexample,wechosesequenceasitwillreturnalazysequenceoftheapplicationsofthegiventransducertotheinputsequence:
(->>(range10)
(sequencexform)
(prn"resultis"))
;;"resultis"(246810)
Aspreviouslyhighlighted,thiscodedoesnotcreateintermediatesequences;transducersextracttheverycoreofthealgorithmictransformationathandandabstractsitawayfromhavingtodealwithsequencesdirectly.
Transducersandcore.asyncWemightnowbeaskingourselves“Whatdotransducershavetodowithcore.async?”
Itturnsoutthatoncewe’reabletoextractthecoreofthesetransformationsandputthemtogetherusingsimplefunctioncomposition,thereisnothingstoppingusfromusingtransducerswithdatastructuresotherthansequences!
Let’srevisitourfirstexampleusingstandardcore.asyncfunctions:
(defresult(chan10))
(deftransformed
(->>result
(map<inc);;createsanewchannel
(filter<even?);;createsanewchannel
(into[])))
(go
(prn"resultis"(<!transformed)))
(go
(doseq[n(range10)]
(>!resultn))
(close!result))
;;"resultis"[246810]
Thiscodeshouldlookfamiliarbynow:it’sthecore.asyncequivalentofthesequence-onlyversionshownearlier.Asbefore,wehaveunnecessaryallocationshereaswell,exceptthatthistimewe’reallocatingchannels.
Withthenewsupportfortransducers,core.asynccantakeadvantageofthesametransformationdefinedearlier:
(defresult(chan10))
(defxform
(comp(mapinc)
(filtereven?)));;nointermediatechannelscreated
(deftransformed(->>(piperesult(chan10xform))
(into[])))
(go
(prn"resultis"(<!transformed)))
(go
(doseq[n(range10)]
(>!resultn))
(close!result))
;;"resultis"[246810]
Thecoderemainslargelyunchangedexceptwenowusethesamexformtransformationdefinedearlierwhencreatinganewchannel.It’simportanttonotethatwedidnothavetousecore.asynccombinators—infactalotofthesecombinatorshavebeendeprecatedandwillberemovedinfutureversionsofcore.async.
Thefunctionsmapandfilterusedtodefinexformarethesameonesweusedpreviously,thatis,theyarecoreClojurefunctions.
Thisisthenextbigadvantageofusingtransducers:byremovingtheunderlyingdatastructurefromtheequationviatransducers,librariessuchascore.asynccanreuseClojure’scorecombinatorstopreventunnecessaryallocationandcodeduplication.
It’snottoofarfetchedtoimagineotherframeworkslikeRxClojurecouldtakeadvantageoftransducersaswell.Allofthemwouldbeabletousethesamecorefunctionacrosssubstantiallydifferentdatastructuresandcontexts:sequences,channels,andObervables.
TipTheconceptofextractingtheessenceofcomputationsdisregardingtheirunderlyingdatastructuresisanexcitingtopicandhasbeenseenbeforeintheHaskellcommunity,althoughtheydealwithlistsspecifically.
TwopapersworthmentioningonthesubjectareStreamFusion[11]byDuncanCoutts,RomanLeshchinskiyandDonStewartandTransformingprogramstoeliminatetrees[12]byPhilipWadler.Therearesomeoverlapssothereadermightfindtheseinteresting.
SummaryBynow,Ihopetohaveprovedthatyoucanwritereactiveapplicationsusingcore.async.It’sanextremelypowerfulandflexibleconcurrencymodelwitharichAPI.Ifyoucandesignyoursolutionintermsofqueues,mostlikelycore.asyncisthetoolyouwanttoreachfor.
ThisversionofthestockmarketapplicationisshorterandsimplerthantheversionusingonlythestandardJavaAPIwedevelopedearlierinthisbook—forinstance,wedidn’thavetoworryaboutthreadpools.Ontheotherhand,itfeelslikeitisalittlemorecomplexthantheversionimplementedusingReactiveExtensionsinChapter3,AsynchronousProgrammingandNetworking.
Thisisbecausecore.asyncoperatesatalowerlevelofabstractionwhencomparedtootherframeworks.Thisbecomesespeciallyobviousinourapplicationaswehadtoworryaboutcreatingbroadcastingchannels,goloops,andsoon—allofwhichcanbeconsideredincidentalcomplexity,notdirectlyrelatedtotheproblemathand.
core.asyncdoes,however,provideanexcellentfoundationforbuildingourownCESabstractions.Thisiswhatwewillbeexploringnext.
Chapter5.CreatingYourOwnCESFrameworkwithcore.asyncInthepreviouschapter,itwasalludedtothatcore.asyncoperatesatalowerlevelofabstractionwhencomparedtootherframeworkssuchasRxClojureorRxJava.
Thisisbecausemostofthetimewehavetothinkcarefullyaboutthechannelswearecreatingaswellaswhattypesandsizesofbufferstouse,whetherweneedpub/subfunctionality,andsoon.
Notallapplicationsrequiresuchlevelofcontrol,however.Nowthatwearefamiliarwiththemotivationsandmainabstractionsofcore.asyncwecanembarkintowritingaminimalCESframeworkusingcore.asyncastheunderlyingfoundation.
Bydoingso,weavoidhavingtothinkaboutthreadpoolmanagementastheframeworktakescareofthatforus.
Inthischapter,wewillcoverthefollowingtopics:
BuildingaCESframeworkusingcore.asyncasitsunderlyingconcurrencystrategyBuildinganapplicationthatusesourCESframeworkUnderstandingthetrade-offsofthedifferentapproachespresentedsofar
AminimalCESframeworkBeforewegetstartonthedetails,weshoulddefineatahighlevelwhatminimalmeans.
Let’sstartwiththetwomainabstractionsourframeworkwillprovide:behaviorsandeventstreams.
IfyoucanrecallfromChapter1,WhatisReactiveProgramming?,behaviorsrepresentcontinuous,time-varyingvaluessuchastimeoramousepositionbehavior.Eventstreams,ontheotherhand,representdiscreteoccurrencesatapointintimeT,suchaskeypress.
Next,weshouldthinkaboutwhatkindsofoperationswewouldliketosupport.Behaviorsarefairlysimplesoattheveryminimumweneedto:
CreatenewbehaviorsRetrievethecurrentvalueofabehaviorConvertabehaviorintoaneventstream
Eventstreamshavemoreinterestinglogicinplayandweshouldatleastsupporttheseoperations:
Push/deliveravaluedownthestreamCreateastreamfromagivenintervalTransformthestreamwiththemapandfilteroperationsCombinestreamswithflatmapSubscribetoastream
ThisisasmallsubsetbutbigenoughtodemonstratetheoverallarchitectureofaCESframework.Oncewe’redone,we’lluseittobuildasimpleexample.
ClojureorClojureScript?Herewe’llshiftgearsandaddanotherrequirementtoourlittlelibrary:itshouldworkbothinClojureandClojureScript.Atfirst,thismightsoundlikeatoughrequirement.However,rememberthatcore.async—thefoundationofourframework—worksbothontheJVMandinJavaScript.Thismeanswehavealotlessworktodotomakeithappen.
Itdoesmean,however,thatweneedtobecapableofproducingtwoartifactsfromourlibrary:ajarfileandaJavaScriptfile.Luckily,thereareleiningenplugins,whichhelpusdothatandwewillbeusingacoupleofthem:
lein-cljsbuild(seehttps://github.com/emezeske/lein-cljsbuild):LeiningenplugintomakebuildingClojureScripteasycljx(seehttps://github.com/lynaghk/cljx):ApreprocessorusedtowriteportableClojurecodebases,thatis,writeasinglefileandoutputboth.cljand.cljsfiles
Youdon’tneedtounderstandtheselibrariesingreatdetail.Weareonlyusingtheirbasicfunctionalityandwillbeexplainingthebitsandpiecesasweencounterthem.
Let’sgetstartedbycreatinganewleiningenproject.We’llcallourframeworkrespondent—oneofthemanysynonymsforthewordreactive:
$leinnewrespondent
Weneedtomakeafewchangestotheproject.cljfiletoincludethedependenciesandconfigurationswe’llbeusing.First,makesuretheprojectdependencieslooklikethefollowing:
:dependencies[[org.clojure/clojure"1.5.1"]
[org.clojure/core.async"0.1.303.0-886421-alpha"]
[org.clojure/clojurescript"0.0-2202"]]
Thereshouldbenosurpriseshere.Stillintheprojectfile,addthenecessaryplugins:
:plugins[[com.keminglabs/cljx"0.3.2"]
[lein-cljsbuild"1.0.3"]]
Thesearethepluginsthatwe’vementionedpreviously.Bythemselvestheydon’tdomuch,however,andneedtobeconfigured.
Forcljx,addthefollowingtotheprojectfile:
:cljx{:builds[{:source-paths["src/cljx"]
:output-path"target/classes"
:rules:clj}
{:source-paths["src/cljx"]
:output-path"target/classes"
:rules:cljs}]}
:hooks[cljx.hooks]
Theprevioussnippetdeservessomeexplanation.cljxallowsustowritecodethatisportablebetweenClojureandClojureScriptbyplacingannotationsitspreprocessorcanunderstand.Wewillseelaterwhattheseannotationslooklike,butthischunkof
configurationtellscljxwheretofindtheannotatedfilesandwheretooutputthemoncethey’reprocessed.
Forexample,basedontheprecedingrules,ifwehaveafilecalledsrc/cljx/core.cljxandwerunthepreprocessorwewillendupwiththetarget/classes/core.cljandtarget/classes/core.cljsoutputfiles.Thehooks,ontheotherhand,aresimplyaconvenientwaytoautomaticallyruncljxwheneverwestartaREPLsession.
Thenextpartoftheconfigurationisforcljsbuild:
:cljsbuild
{:builds[{:source-paths["target/classes"]
:compiler{:output-to"target/main.js"}}]}
cljsbuildprovidesleiningentaskstocompileClojuresriptsourcecodeintoJavaScript.Weknowfromourprecedingcljxconfigurationthatthesource.cljsfileswillbeundertarget/classes,soherewe’resimplytellingcljsbuildtocompileallClojureScriptfilesinthatdirectoryandspitthecontentstotarget/main.js.Thisisthelastpieceneededfortheprojectfile.
Goaheadanddeletethesefilescreatedbytheleiningentemplateaswewon’tbeusingthem:
$rmsrc/respondent/core.clj
$rmtest/respondent/core_test.clj
Then,createanewcore.cljxfileundersrc/cljx/respondent/andaddthefollowingnamespacedeclaration:
(nsrespondent.core
(:refer-clojure:exclude[filtermapdeliver])
#+clj
(:import[clojure.langIDeref])
#+clj
(:require[clojure.core.async:asasync
:refer[gogo-loopchan<!>!timeout
map>filter>close!multtapuntap]])
#+cljs
(:require[cljs.core.async:asasync
:refer[chan<!>!timeoutmap>filter>
close!multtapuntap]])
#+cljs
(:require-macros[respondent.core:refer[behavior]]
[cljs.core.async.macros:refer[gogo-loop]]))
Here,westartseeingcljxannotations.cljxissimplyatextpreprocessor,sowhenitisprocessingafileusingcljrules—asseenintheconfiguration—itwillkeepthes-expressionsprecededbytheannotation#+cljintheoutputfile,whileremovingtheonesprefixedby#+cljs.Thereverseprocesshappenswhenusingcljsrules.
ThisisnecessarybecausemacrosneedtobecompiledontheJVM,sotheyhavetobe
includedseparatelyusingthe:require-macrosnamespaceoptionwhenusingClojureScript.Don’tworryaboutthecore.asyncfunctionswehaven’tencounteredbefore;theywillbeexplainedasweusethemtobuildourframework.
Also,notehowweareexcludingfunctionsfromtheClojurestandardAPIaswewishtousethesamenamesanddon’twantanyundesirednamingcollisions.
Thissectionsetusupwithanewprojectandthepluginsandconfigurationsneededforourframework.We’rereadytostartimplementingit.
DesigningthepublicAPIOneoftherequirementsforbehaviorsweagreedonistheabilitytoturnagivenbehaviorintoaneventstream.Acommonwayofdoingthisisbysamplingabehaviorataspecificinterval.Ifwetakethemousepositionbehaviorasanexample,bysamplingiteveryxsecondswegetaneventstream,whichwillemitthecurrentmousepositionatdiscretepointsintime.
Thisleadstothefollowingprotocol:
(defprotocolIBehavior
(sample[binterval]
"TurnsthisBehaviorintoanEventStreamfromthesampledvaluesatthe
giveninterval"))
Ithasasinglefunction,sample,whichwedescribedintheprecedingcode.Therearemorethingsweneedtodowithabehavior,butfornowthiswillsuffice.
OurnextmainabstractionisEventStream,which—basedontherequirementsseenpreviously—leadstothefollowingprotocol:
(defprotocolIEventStream
(map[sf]
"Returnsanewstreamcontainingtheresultofapplyingf
tothevaluesins")
(filter[spred]
"Returnsanewstreamcontainingtheitemsfroms
forwhichpredreturnstrue")
(flatmap[sf]
"TakesafunctionffromvaluesinstoanewEventStream.
ReturnsanEventStreamcontainingvaluesfromallunderlyingstreams
combined.")
(deliver[svalue]
"Deliversavaluetothestreams")
(completed?[s]
"Returnstrueifthisstreamhasstoppedemittingvalues.False
otherwise."))
Thisgivesusafewbasicfunctionstotransformandqueryaneventstream.Itdoesleaveouttheabilitytosubscribetoastream.Don’tworry,Ididn’tforgetit!
Although,itiscommontosubscribetoaneventstream,theprotocolitselfdoesn’tmandateitandthisisbecausetheoperationfitsbestinitsownprotocol:
(defprotocolIObservable
(subscribe[obsf]"Registeracallbacktobeinvokedwhentheunderlying
sourcechanges.
Returnsatokenthesubscribercanusetocancelthesubscription."))
Asfarassubscriptionsgo,itisusefultohaveawayofunsubscribingfromastream.Thiscanbeimplementedinacoupleofways,butdocstringoftheprecedingfunctionhintsataspecificone:atokenthatcanbeusedtounsubscribefromastream.Thisleadstoourlastprotocol:
(defprotocolIToken
(dispose[tk]
"Calledwhenthesubscriberisn'tinterestedinreceivingmoreitems"))
ImplementingtokensThetokentypeisthesimplestinthewholeframeworkasithasgotasinglefunctionwithastraightforwardimplementation:
(deftypeToken[ch]
IToken
(dispose[_]
(close!ch)))
Itsimplycloseswhateverchannelitisgiven,stoppingeventsfromflowingthroughsubscriptions.
ImplementingeventstreamsTheeventstreamimplementation,ontheotherhand,isthemostcomplexinourframework.We’lltackleitgradually,implementingandexperimentingaswego.
First,let’slookatourmainconstructorfunction,event-stream:
(defnevent-stream
"Createsandreturnsaneweventstream.Youcanoptionallyprovidean
existing
core.asyncchannelasthesourceforthenewstream"
([]
(event-stream(chan)))
([ch]
(let[multiple(multch)
completed(atomfalse)]
(EventStream.chmultiplecompleted))))
ThedocstringshouldbesufficienttounderstandthepublicAPI.Whatmightnotbeclear,however,iswhatalltheconstructorargumentsmean.Fromlefttoright,theargumentstoEventStreamare:
ch:Thisisthecore.asyncchannelbackingthisstream.multiple:Thisisawaytobroadcastinformationfromonechanneltomanyotherchannels.It’sacore.asyncconceptwewillbeexplainingshortly.completed:ABooleanflagindicatingwhetherthiseventstreamhascompletedandwillnotemitanynewvalues.
Fromtheimplementation,youcanseethatthemultipleiscreatedfromthechannelbackingthestream.multipleworkskindoflikeabroadcast.Considerthefollowingexample:
(defin(chan))
(defmultiple(multin))
(defout-1(chan))
(tapmultipleout-1)
(defout-2(chan))
(tapmultipleout-2)
(go(>!in"Singleput!"))
(go(prn"Gotfromout-1"(<!out-1)))
(go(prn"Gotfromout-2"(<!out-2)))
Intheprevioussnippet,wecreateaninputchannel,in,andmultofitcalledmultiple.Then,wecreatetwooutputchannels,out-1andout-2,whicharebothfollowedbyacalltotap.Thisessentiallymeansthatwhatevervaluesarewrittentoinwillbetakenbymultipleandwrittentoanychannelstappedintoitasthefollowingoutputshows:
"Gotfromout-1""Singleput!"
"Gotfromout-2""Singleput!"
ThiswillmakeunderstandingtheEventStreamimplementationeasier.
Next,let’shavealookatwhataminimalimplementationoftheEventStreamlookslikethefollowing—makesuretheimplementationgoesbeforetheconstructorfunctiondescribedearlier:
(declareevent-stream)
(deftypeEventStream[channelmultiplecompleted]
IEventStream
(map[_f]
(let[out(map>f(chan))]
(tapmultipleout)
(event-streamout)))
(deliver[_value]
(if(=value::complete)
(do(reset!completedtrue)
(go(>!channelvalue)
(close!channel)))
(go(>!channelvalue))))
IObservable
(subscribe[thisf]
(let[out(chan)]
(tapmultipleout)
(go-loop[]
(let[value(<!out)]
(when(andvalue(not=value::complete))
(fvalue)
(recur))))
(Token.out))))
Fornow,wehavechosentoimplementonlythemapanddeliverfunctionsfromtheIEventStreamprotocol.Thisallowsustodelivervaluestothestreamaswellastransformthosevalues.
However,thiswouldnotbeveryusefulifwecouldnotretrievethevaluesdelivered.ThisiswhywealsoimplementthesubscribefunctionfromtheIObservableprotocol.
Inanutshell,mapneedstotakeavaluefromtheinputstream,applyafunctiontoit,andsendittothenewlycreatedstream.Wedothisbycreatinganoutputchannelthattapsoncurrentmultiple.Wethenusethischanneltobacktheneweventstream.
Thedeliverfunctionsimplyputsthevalueintothebackingchannel.Ifthevalueisthenamespacedkeyword::complete,weupdatethecompletedatomandclosethebackingchannel.Thisensuresthestreamwillnotemitanyothervalues.
Finally,wehavethesubscribefunction.Thewaysubscribersarenotifiedisbyusinganoutputchanneltappedtobackingmultiple.Weloopindefinitelycallingthesubscribingfunctionwheneveranewvalueisemitted.
Wefinishbyreturningatoken,whichwillclosetheoutputchanneloncedisposed,causingthego-looptostop.
Let’smakesurethatallthismakessensebyexperimentingwithacoupleofexamplesintheREPL:
(defes1(event-stream))
(subscribees1#(prn"firsteventstreamemitted:"%))
(deliveres110)
;;"firsteventstreamemitted:"10
(defes2(mapes1#(*2%)))
(subscribees2#(prn"secondeventstreamemitted:"%))
(deliveres120)
;;"firsteventstreamemitted:"20
;;"secondeventstreamemitted:"40
Excellent!Wehaveaminimal,workingimplementationofourIEventStreamprotocol!
Thenextfunctionwe’llimplementisfilteranditisverysimilartomap:
(filter[_pred]
(let[out(filter>pred(chan))]
(tapmultipleout)
(event-streamout)))
Theonlydifferenceisthatweusethefilter>functionandpredshouldbeaBooleanfunction:
(defes1(event-stream))
(defes2(filteres1even?))
(subscribees1#(prn"firsteventstreamemitted:"%))
(subscribees2#(prn"secondeventstreamemitted:"%))
(deliveres12)
(deliveres13)
(deliveres14)
;;"firsteventstreamemitted:"2
;;"secondeventstreamemitted:"2
;;"firsteventstreamemitted:"3
;;"firsteventstreamemitted:"4
;;"secondeventstreamemitted:"4
Aswewitness,es2onlyemitsanewvalueifandonlyifthatvalueisanevennumber.
TipIfyouarefollowingalong,typingtheexamplesstepbystep,youwillneedtorestartyourREPLwheneverweaddnewfunctionstoanydeftypedefinition.ThisisbecausedeftypegeneratesandcompilesaJavaclasswhenevaluated.Assuch,simplyreloadingthenamespacewon’tbeenough.
Alternatively,youcanuseatoolsuchastools.namespace(seehttps://github.com/clojure/tools.namespace)thataddressessomeoftheseREPLreloadinglimitations.
Movingdownourlist,wenowhaveflatmap:
(flatmap[_f]
(let[es(event-stream)
out(chan)]
(tapmultipleout)
(go-loop[]
(when-let[a(<!out)]
(let[mb(fa)]
(subscribemb(fn[b]
(deliveresb)))
(recur))))
es))
We’veencounteredthisoperatorbeforewhensurveyingReactiveExtensions.Thisiswhatourdocstringsaysaboutit:
TakesafunctionffromvaluesinstoanewEventStream.
ReturnsanEventStreamcontainingvaluesfromallunderlyingstreamscombined.
Thismeansflatmapneedstocombineallthepossibleeventstreamsintoasingleoutputeventstream.Asbefore,wetapanewchanneltothemultiplestream,butthenweloopovertheoutputchannel,applyingftoeachoutputvalue.
However,aswesaw,fitselfreturnsaneweventstream,sowesimplysubscribetoit.Wheneverthefunctionregisteredinthesubscriptiongetscalled,wedeliverthatvaluetotheoutputeventstream,effectivelycombiningallstreamsintoasingleone.
Considerthefollowingexample:
(defnrange-es[n]
(let[es(event-stream(chann))]
(doseq[n(rangen)]
(deliveresn))
es))
(defes1(event-stream))
(defes2(flatmapes1range-es))
(subscribees1#(prn"firsteventstreamemitted:"%))
(subscribees2#(prn"secondeventstreamemitted:"%))
(deliveres12)
;;"firsteventstreamemitted:"2
;;"secondeventstreamemitted:"0
;;"secondeventstreamemitted:"1
(deliveres13)
;;"firsteventstreamemitted:"3
;;"secondeventstreamemitted:"0
;;"secondeventstreamemitted:"1
;;"secondeventstreamemitted:"2
Wehaveafunction,range-es,thatreceivesanumbernandreturnsaneventstreamthatemitsnumbersfrom0ton.Asbefore,wehaveastartingstream,es1,andatransformedstreamcreatedwithflatmap,es2.
Wecanseefromtheprecedingoutputthatthestreamcreatedbyrange-esgetsflattenedintoes2allowingustoreceiveallvaluesbysimplysubscribingtoitonce.
ThisleavesuswithsinglefunctionfromIEventStreamlefttoimplement:
(completed?[_]@completed)
completed?simplyreturnsthecurrentvalueofthecompletedatom.Wearenowreadytoimplementbehaviors.
ImplementingbehaviorsIfyourecall,theIBehaviorprotocolhasasinglefunctioncalledsamplewhosedocstringstates:TurnsthisBehaviorintoanEventStreamfromthesampledvaluesatthegiveninterval.
Inordertoimplementsample,wewillfirstcreateausefulhelperfunctionthatwewillcallfrom-interval:
(defnfrom-interval
"Createsandreturnsaneweventstreamwhichemitsvaluesatthegiven
interval.
Ifnootherargumentsaregiven,thevaluesstartat0andincrementby
oneateachdelivery.
Ifgivenseedandsuccitemitsseedandappliessucctoseedtoget
thenextvalue.Itthenappliessucctothepreviousresultandsoon."
([msecs]
(from-intervalmsecs0inc))
([msecsseedsucc]
(let[es(event-stream)]
(go-loop[timeout-ch(timeoutmsecs)
valueseed]
(when-not(completed?es)
(<!timeout-ch)
(deliveresvalue)
(recur(timeoutmsecs)(succvalue))))
es)))
Thedocstringfunctionshouldbeclearenoughatthisstage,butwewouldliketoensureweunderstanditsbehaviorcorrectlybytryingitattheREPL:
(defes1(from-interval500))
(defes1-token(subscribees1#(prn"Got:"%)))
;;"Got:"0
;;"Got:"1
;;"Got:"2
;;"Got:"3
;;...
(disposees1-token)
Asexpected,es1emitsintegersstartingatzeroat500-millisecondintervals.Bydefault,itwouldemitnumbersindefinitely;therefore,wekeepareferencetothetokenreturnedbycallingsubscribe.
Thiswaywecandisposeitwheneverwe’redone,causinges-1tocompleteandstopemittingitems.
Next,wecanfinallyimplementtheBehaviortype:
(deftypeBehavior[f]
IBehavior
(sample[_interval]
(from-intervalinterval(f)(fn[&args](f))))
IDeref
(#+cljderef#+cljs-deref[_]
(f)))
(defmacrobehavior[&body]
`(Behavior.#(do~@body)))
Abehavioriscreatedbypassingitafunction.Youcanthinkofthisfunctionasageneratorresponsibleforgeneratingthenextvalueinthiseventstream.
Thisgeneratorfunctionwillbecalledwheneverwe(1)dereftheBehavioror(2)attheintervalgiventosample.
ThebehaviormacroisthereforconvenienceandallowsustocreateanewBehaviorwithoutwrappingthebodyinafunctionourselves:
(deftime-behavior(behavior(System/nanoTime)))
@time-behavior
;;201003153977194
@time-behavior
;;201005133457949
Intheprecedingexample,wedefinedtime-behaviorthatalwayscontainsthecurrentsystemtime.Wecanthenturnthisbehaviorintoastreamofdiscreteeventsbyusingthesamplefunction:
(deftime-stream(sampletime-behavior1500))
(deftoken(subscribetime-stream#(prn"Timeis"%)))
;;"Timeis"201668521217402
;;"Timeis"201670030219351
;;...
(disposetoken)
TipAlwaysremembertokeepareferencetothesubscriptiontokenwhendealingwithinfinitestreamssuchastheonescreatedbysampleandfrom-interval,orelseyoumightincurundesiredmemoryleaks.
Congratulations!Wehaveaworking,minimalCESframeworkusingcore.async!
Wedidn’tproveitworkswithClojureScript,however,whichwasoneofthemainrequirementsearlyon.That’sokay.WewillbetacklingthatsoonbydevelopingasimpleClojureScriptapplication,whichmakesuseofournewframework.
Inordertodothis,weneedtodeploytheframeworktoourlocalMavenrepository.Fromtheprojectroot,typethefollowingleincommand:
$leininstall
Rewritingsrc/cljxtotarget/classes(clj)withfeatures#{clj}and0
transformations.
Rewritingsrc/cljxtotarget/classes(cljs)withfeatures#{cljs}and1
transformations.
Createdrespondent/target/respondent-0.1.0-SNAPSHOT.jar
Wroterespondent/pom.xml
ExercisesThefollowingsectionshaveafewexercisesforyou.
Exercise5.1ExtendourcurrentEventStreamimplementationtoincludeafunctioncalledtake.ItworksmuchlikeClojure’scoretakefunctionforsequences:itwilltakenitemsfromtheunderlyingeventstreamafterwhichitwillstopemittingitems.
Asampleinteraction,whichtakesthefirstfiveitemsemittedfromtheoriginaleventstream,isshownhere:
(defes1(from-interval500))
(deftake-es(takees15))
(subscribetake-es#(prn"Takevalues:"%))
;;"Takevalues:"0
;;"Takevalues:"1
;;"Takevalues:"2
;;"Takevalues:"3
;;"Takevalues:"4
TipKeepingsomestatemightbeusefulhere.Atomscanhelp.Additionally,trytothinkofawaytodisposeofanyunwantedsubscriptionsrequiredbythesolution.
Exercise5.2Inthisexercise,wewilladdafunctioncalledzipthatzipstogetheritemsemittedfromtwodifferenteventstreamsintoavector.
Asampleinteractionwiththezipfunctionisasfollows:
(defes1(from-interval500))
(defes2(map(from-interval500)#(*%2)))
(defzipped(zipes1es2))
(deftoken(subscribezipped#(prn"Zippedvalues:"%)))
;;"Zippedvalues:"[00]
;;"Zippedvalues:"[12]
;;"Zippedvalues:"[24]
;;"Zippedvalues:"[36]
;;"Zippedvalues:"[48]
(disposetoken)
TipForthisexercise,weneedawaytoknowwhenwehaveenoughitemstoemitfrombotheventstreams.Managingthisinternalstatecanbetrickyatfirst.Clojure’sreftypesand,inparticular,dosync,canbeofuse.
ArespondentapplicationThischapterwouldnotbecompleteifwedidn’tgothroughthewholedevelopmentlifecycleofdeployingandusingthenewframeworkinanewapplication.Thisisthepurposeofthissection.
Theapplicationwewillbuildisextremelysimple.Allitdoesistrackthepositionofthemouseusingthereactiveprimitiveswebuiltintorespondent.
Tothatend,wewillbeusingtheexcellentleintemplatecljs-start(seehttps://github.com/magomimmo/cljs-start),createdbyMimmoCosenzatohelpdevelopersgetstartedwithClojureScript.
Let’sgetstarted:
leinnewcljs-startrespondent-app
Next,let’smodifytheprojectfiletoincludethefollowingdependencies:
[clojure-reactive-programming/respondent"0.1.0-SNAPSHOT"]
[prismatic/dommy"0.1.2"]
Thefirstdependencyisself-explanatory.It’ssimplyourownframework.dommyisaDOMmanipulationlibraryforClojureScript.We’llbrieflyuseitwhenbuildingourwebpage.
Next,editthedev-resources/public/index.htmlfiletomatchthefollowing:
<!doctypehtml>
<htmllang="en">
<head>
<metacharset="utf-8">
<title>Example:trackingmouseposition</title>
<!--[ifltIE9]>
<scriptsrc="http://html5shiv.googlecode.com/svn/trunk/html5.js">
</script>
<![endif]-->
</head>
<body>
<divid="test">
<h1>Mouse(x,y)coordinates:</h1>
</div>
<divid="mouse-xy">
(0,0)
</div>
<scriptsrc="js/respondent_app.js"></script>
</body>
</html>
Intheprecedingsnippet,wecreatedanewdivelement,whichwillcontainthemouseposition.Itdefaultsto(0,0).
Thelastpieceofthepuzzleismodifyingsrc/cljs/respondent_app/core.cljstomatchthefollowing:
(nsrespondent-app.core
(:require[respondent.core:asr]
[dommy.core:asdommy])
(:use-macros
[dommy.macros:only[sel1]]))
(defmouse-pos-stream(r/event-stream))
(set!(.-onmousemovejs/document)
(fn[e]
(r/delivermouse-pos-stream[(.-pageXe)(.-pageYe)])))
(r/subscribemouse-pos-stream
(fn[[xy]]
(dommy/set-text!(sel1:#mouse-xy)
(str"("x","y")"))))
Thisisourmainapplicationlogic.Itcreatesaneventstreamtowhichwedeliverthecurrentmousepositionfromtheonmousemoveeventofthebrowserwindow.
Later,wesimplysubscribetoitandusedommytoselectandsetthetextofthedivelementweaddedpreviously.
Wearenowreadytousetheapp!Let’sstartbycompilingClojureScript:
$leincompile
Thisshouldtakeafewseconds.Ifalliswell,thenextthingtodoistostartaREPLsessionandstartuptheserver:
$leinrepl
user=>(run)
Now,pointyourbrowsertohttp://localhost:3000/anddragthemousearoundtoseeitscurrentposition.
Congratulationsonsuccessfullydeveloping,deploying,andusingyourownCESframework!
CESversuscore.asyncAtthisstage,youmightbewonderingwhenyoushouldchooseoneapproachovertheother.Afterall,asdemonstratedatthebeginningofthischapter,wecouldusecore.asynctodoeverythingwehavedoneusingrespondent.
Itallcomesdowntousingtherightlevelofabstractionforthetaskathand.
core.asyncgivesusmanylowlevelprimitivesthatareextremelyusefulwhenworkingwithprocesses,whichneedtotalktoeachother.Thecore.asyncchannelsworkasconcurrentblockingqueuesandareanexcellentsynchronizationmechanisminthesescenarios.
However,itmakesothersolutionshardertoimplement:forinstance,channelsaresingle-takebydefault,soifwehavemultipleconsumersinterestedinthevaluesputinsideachannel,wehavetoimplementthedistributionourselvesusingtoolssuchasmultandtap.
CESframeworks,ontheotherhand,operateatahigherlevelofabstractionandworkwithmultiplesubscribersbydefault.
Additionally,core.asyncreliesonsideeffects,ascanbeseenbytheuseoffunctionssuchas>!insidegoblocks.FrameworkssuchasRxClojurepromotestreamtransformationsbytheuseofpurefunctions.
Thisisnottosaytherearen’tsideeffectsinCESframeworks.Theremostdefinitelyare.However,asaconsumerofthelibrary,thisismostlyhiddenfromoureyes,allowingustothinkofmostofourcodeassimplesequencetransformations.
Inconclusion,differentapplicationdomainswillbenefitmoreorlessfromeitherapproach—sometimestheycanbenefitfromboth.Weshouldthinkhardaboutourapplicationdomainandanalyzethetypesofsolutionsandidiomsdevelopersaremostlikelytodesign.Thiswillpointusinthedirectionofbetterabstractionforwhateverapplicationwearedevelopingatagiventime.
SummaryInthischapter,wedevelopedourveryownCESframework.Bydevelopingourownframework,wehavesolidifiedourunderstandingofbothCESandhowtoeffectivelyusecore.async.
Theideathatcore.asynccouldbeusedasthefoundationofaCESframeworkisn’tmine,however.JamesReeves(seehttps://github.com/weavejester)—creatoroftheroutinglibraryCompojure(seehttps://github.com/weavejester/compojure)andmanyotherusefulClojurelibraries—alsosawthesamepotentialandsetofftowriteReagi(seehttps://github.com/weavejester/reagi),aCESlibrarybuiltontopofcore.async,similarinspirittotheonewedevelopedinthischapter.
Hehasputalotmoreeffortintoit,makingitamorerobustoptionforapureClojureframework.We’llbelookingatitinthenextchapter.
Chapter6.BuildingaSimpleClojureScriptGamewithReagiInthepreviouschapter,welearnedhowaframeworkforCompositionalEventSystems(CES)worksbybuildingourownframework,whichwecalledrespondent.Itgaveusagreatinsightintothemainabstractionsinvolvedinsuchapieceofsoftwareaswellasagoodoverviewofcore.async,Clojure’slibraryforasynchronousprogrammingandthefoundationofourframework.
Respondentisbutatoyframework,however.Wepaidlittleattentiontocross-cuttingconcernssuchasmemoryefficiencyandexceptionhandling.Thatisokayasweuseditasavehicleforlearningmoreabouthandlingandcomposingeventsystemswithcore.async.Additionally,itsdesignisintentionallysimilartoReagi’sdesign.
Inthischapter,wewill:
LearnaboutReagi,aCESframeworkbuiltontopofcore.asyncUseReagitobuildtherudimentsofaClojureScriptgamethatwillteachushowtohandleuserinputinacleanandmaintainablewayBrieflycompareReagitootherCESframeworksandgetafeelforwhentouseeachone
SettinguptheprojectHaveyoueverplayedAsteroids?Ifyouhaven’t,AsteroidsisanarcadespaceshooterfirstreleasedbyAtariin1979.InAsteroids,youarethepilotofashipflyingthroughspace.Asyoudoso,yougetsurroundedbyasteroidsandflyingsaucersyouhavetoshootanddestroy.
Developingthewholegameinonechapteristooambitiousandwoulddistractusfromthesubjectofthisbook.Wewilllimitourselvestomakingsurewehaveashiponthescreenwecanflyaroundaswellasshootspacebulletsintothevoid.Bytheendofthischapter,wewillhavesomethingthatlookslikewhatisshowninthefollowingscreenshot:
Togetstarted,wewillcreateanewClojureScriptprojectusingthesameleiningentemplateweusedinthepreviouschapter,cljs-start(seehttps://github.com/magomimmo/cljs-start):
leinnewcljs-startreagi-game
Next,addthefollowingdependenciestoyourprojectfile:
[org.clojure/clojurescript"0.0-2138"]
[reagi"0.10.0"]
[rm-hull/monet"0.1.12"]
Thelastdependency,monet(seehttps://github.com/rm-hull/monet),isaClojureScriptlibraryyoucanusetoworkwithHTML5Canvas.Itisahigh-levelwrapperontopoftheCanvasAPIandmakesinteractingwithitalotsimpler.
Beforewecontinue,it’sprobablyagoodideatomakesureoursetupisworkingproperly.Changeintotheprojectdirectory,startaClojureREPL,andthenstarttheembeddedwebserver:
cdreagi-game/
leinrepl
CompilingClojureScript.
Compiling"dev-resources/public/js/reagi_game.js"from("src/cljs"
"test/cljs""dev-resources/tools/repl")...
user=>(run)
2014-06-1419:21:40.381:INFO:oejs.Server:jetty-7.6.8.v20121106
2014-06-1419:21:40.403:INFO:oejs.AbstractConnector:Started
[email protected]:3000
#<Serverorg.eclipse.jetty.server.Server@51f6292b>
ThiswillcompiletheClojureScriptsourcefilestoJavaScriptandstartthesamplewebserver.Inyourbrowser,navigatetohttp://localhost:3000/.Ifyouseesomethinglikethefollowing,wearegoodtogo:
AswewillbeworkingwithHTML5Canvas,weneedanactualcanvastorenderto.Let’supdateourHTMLdocumenttoincludethat.It’slocatedunderdev-resources/public/index.html:
<!doctypehtml>
<htmllang="en">
<head>
<metacharset="utf-8">
<title>bREPLConnection</title>
<!--[ifltIE9]>
<scriptsrc="http://html5shiv.googlecode.com/svn/trunk/html5.js">
</script>
<![endif]-->
</head>
<body>
<canvasid="canvas"width="800"height="600"></canvas>
<scriptsrc="js/reagi_game.js"></script>
</body>
</html>
WehaveaddedacanvasDOMelementtoourdocument.Allrenderingwillhappeninthiscontext.
GameentitiesOurgamewillhaveonlytwoentities:onerepresentingthespaceshipandtheotherrepresentingbullets.Tobetterorganizethecode,wewillputallentity-relatedcodeinitsownfile,src/cljs/reagi_game/entities.cljs.Thisfilewillalsocontainsomeoftherenderinglogic,sowe’llneedtorequiremonet:
(nsreagi-game.entities
(:require[monet.canvas:ascanvas]
[monet.geometry:asgeom]))
Next,we’lladdafewhelperfunctionstoavoidrepeatingourselvestoomuch:
(defnshape-x[shape]
(->shape:posderef:x))
(defnshape-y[shape]
(->shape:posderef:y))
(defnshape-angle[shape]
@(:angleshape))
(defnshape-data[xyangle]
{:pos(atom{:xx:yy})
:angle(atomangle)})
Thefirstthreefunctionsaresimplyashorterwayofgettingdataoutofourshapedatastructure.Theshape-datafunctioncreatesastructure.Notethatweareusingatoms,oneofClojure’sreferencetypes,torepresentashape’spositionandangle.
Thisway,wecansafelypassourshapedataintomonet’srenderingfunctionsandstillbeabletoupdateitinaconsistentway.
Nextupisourshipconstructorfunction.Thisiswherethebulkoftheinteractionwithmonethappens:
(defnship-entity[ship]
(canvas/entity{:x(shape-xship)
:y(shape-yship)
:angle(shape-angleship)}
(fn[value]
(->value
(assoc:x(shape-xship))
(assoc:y(shape-yship))
(assoc:angle(shape-angleship))))
(fn[ctxval]
(->ctx
canvas/save
(canvas/translate(:xval)(:yval))
(canvas/rotate(:angleval))
(canvas/begin-path)
(canvas/move-to500)
(canvas/line-to0-15)
(canvas/line-to015)
(canvas/fill)
canvas/restore))))
Thereisquiteabitgoingon,solet’sbreakitdown.
canvas/entityisamonetconstructorandexpectsyoutoprovidethreeargumentsthatdescribeourship:itsinitialx,ycoordinatesandangle,anupdatefunctionthatgetscalledinthedrawloop,andadrawfunctionthatisresponsibleforactuallydrawingtheshapeontothescreenaftereachupdate.
Theupdatefunctionisfairlystraightforward:
(fn[value]
(->value
(assoc:x(shape-xship))
(assoc:y(shape-yship))
(assoc:angle(shape-angleship))))
Wesimplyupdateitsattributestothecurrentvaluesfromtheship’satoms.
Thenextfunction,responsiblefordrawing,interactswithmonet’sAPImoreheavily:
(fn[ctxval]
(->ctx
canvas/save
(canvas/translate(:xval)(:yval))
(canvas/rotate(:angleval))
(canvas/begin-path)
(canvas/move-to500)
(canvas/line-to0-15)
(canvas/line-to015)
(canvas/fill)
canvas/restore))
Westartbysavingthecurrentcontextsothatwecanrestorethingssuchasdrawingstyleandcanvaspositioninglater.Next,wetranslatethecanvastotheship’sx,ycoordinatesandrotateitaccordingtoitsangle.Wethenstartdrawingourshape,atriangle,andfinishbyrestoringoursavedcontext.
Thenextfunctionalsocreatesanentity,ourbullet:
(declaremove-forward!)
(defnmake-bullet-entity[monet-canvaskeyshape]
(canvas/entity{:x(shape-xshape)
:y(shape-yshape)
:angle(shape-angleshape)}
(fn[value]
(when(not
(geom/contained?
{:x0:y0
:w(.-width(:canvasmonet-canvas))
:h(.-height(:canvasmonet-canvas))}
{:x(shape-xshape)
:y(shape-yshape)
:r5}))
(canvas/remove-entitymonet-canvaskey))
(move-forward!shape)
(->value
(assoc:x(shape-xshape))
(assoc:y(shape-yshape))
(assoc:angle(shape-angleshape))))
(fn[ctxval]
(->ctx
canvas/save
(canvas/translate(:xval)(:yval))
(canvas/rotate(:angleval))
(canvas/fill-style"red")
(canvas/circle{:x10:y0:r5})
canvas/restore))))
Asbefore,let’sinspecttheupdateanddrawingfunctions.We’llstartwithupdate:
(fn[value]
(when(not
(geom/contained?
{:x0:y0
:w(.-width(:canvasmonet-canvas))
:h(.-height(:canvasmonet-canvas))}
{:x(shape-xshape)
:y(shape-yshape)
:r5}))
(canvas/remove-entitymonet-canvaskey))
(move-forward!shape)
(->value
(assoc:x(shape-xshape))
(assoc:y(shape-yshape))
(assoc:angle(shape-angleshape))))
Bulletshavealittlemorelogicintheirupdatefunction.Asyoufirethemfromtheship,wemightcreatehundredsoftheseentities,soit’sagoodpracticetogetridofthemassoonastheygooffthevisiblecanvasarea.That’sthefirstthingthefunctiondoes:itusesgeom/contained?tocheckwhethertheentityiswithinthedimensionsofthecanvas,removingitwhenitisn’t.
Differentfromtheship,however,bulletsdon’tneeduserinputinordertomove.Oncefired,theymoveontheirown.That’swhythenextthingwedoiscallmove-forward!Wehaven’timplementedthisfunctionyet,sowehadtodeclareitbeforehand.We’llgettoit.
Oncethebullet’scoordinatesandanglehavebeenupdated,wesimplyreturnthenewentity.
Thedrawfunctionisabitsimplerthantheship’sversionmostlyduetoitsshapebeingsimpler;it’sjustaredcircle:
(fn[ctxval]
(->ctx
canvas/save
(canvas/translate(:xval)(:yval))
(canvas/rotate(:angleval))
(canvas/fill-style"red")
(canvas/circle{:x10:y0:r5})
canvas/restore))
Now,we’llmoveontothefunctionsresponsibleforupdatingourshape’scoordinatesandangle,startingwithmove!:
(defspeed200)
(defncalculate-x[angle]
(*speed(/(*(Math/cosangle)
Math/PI)
180)))
(defncalculate-y[angle]
(*speed(/(*(Math/sinangle)
Math/PI)
180)))
(defnmove![shapef]
(let[pos(:posshape)]
(swap!pos(fn[xy]
(->xy
(update-in[:x]
#(f%(calculate-x
(shape-angleshape))))
(update-in[:y]
#(f%(calculate-y
(shape-angleshape)))))))))
Tokeepthingssimple,boththeshipandbulletsusethesamespeedvaluetocalculatetheirpositioning,heredefinedas200.
move!takestwoarguments:theshapemapandafunctionf.Thisfunctionwilleitherbethe+(plus)orthe-(minus)function,dependingonwhetherwe’removingforwardorbackward,respectively.Next,itupdatestheshape’sx,ycoordinatesusingsomebasictrigonometry.
Ifyou’rewonderingwhywearepassingtheplusandminusfunctionsasarguments,it’sallaboutnotrepeatingourselves,asthenexttwofunctionsshow:
(defnmove-forward![shape]
(move!shape+))
(defnmove-backward![shape]
(move!shape-))
Withmovementtakencareof,thenextstepistowritetherotationfunctions:
(defnrotate![shapef]
(swap!(:angleshape)#(f%(/(/Math/PI3)20))))
(defnrotate-right![shape]
(rotate!shape+))
(defnrotate-left![shape]
(rotate!shape-))
Sofar,we’vegotshipmovementcovered!Butwhatgoodisourshipifwecan’tfirebullets?Let’smakesurewehavethatcoveredaswell:
(defnfire![monet-canvasship]
(let[entity-key(keyword(gensym"bullet"))
data(shape-data(shape-xship)
(shape-yship)
(shape-angleship))
bullet(make-bullet-entitymonet-canvas
entity-key
data)]
(canvas/add-entitymonet-canvasentity-keybullet)))
Thefire!functiontakestwoarguments:areferencetothegamecanvasandtheship.Itthencreatesanewbulletbycallingmake-bullet-entityandaddsittothecanvas.
NotehowweuseClojure’sgensymfunctiontocreateauniquekeyforthenewentity.Weusethiskeytoremoveanentityfromthegame.
Thisconcludesthecodefortheentitiesnamespace.
Tipgensymisquiteheavilyusedinwritinghygienicmacrosasyoucanbesurethatthegeneratedsymbolswillnotclashwithanylocalbindingsbelongingtothecodeusingthemacro.Macrosarebeyondthescopeofthisbook,butyoumightfindthisseriesofmacroexercisesusefulinthelearningprocess,athttps://github.com/leonardoborges/clojure-macros-workshop.
PuttingitalltogetherWe’renowreadytoassembleourgame.Goaheadandopenthecorenamespacefile,src/cljs/reagi_game/core.cljs,andaddthefollowing:
(nsreagi-game.core
(:require[monet.canvas:ascanvas]
[reagi.core:asr]
[clojure.set:asset]
[reagi-game.entities:asentities
:refer[move-forward!move-backward!rotate-left!rotate-
right!fire!]]))
Thefollowingsnippetsetsupvariousdatastructuresandreferenceswe’llneedinordertodevelopthegame:
(defcanvas-dom(.getElementByIdjs/document"canvas"))
(defmonet-canvas(canvas/initcanvas-dom"2d"))
(defship
(entities/shape-data(/(.-width(:canvasmonet-canvas))2)
(/(.-height(:canvasmonet-canvas))2)
0))
(defship-entity(entities/ship-entityship))
(canvas/add-entitymonet-canvas:ship-entityship-entity)
(canvas/draw-loopmonet-canvas)
Westartbycreatingmonet-canvasfromareferencetoourcanvasDOMelement.Wethencreateourshipdata,placingitatthecenterofthecanvas,andaddtheentitytomonet-canvas.Finally,westartadraw-loop,whichwillhandleouranimationsusingthebrowser’snativecapabilities—internallyitcallswindow.requestAnimationFrame(),ifavailable,butitfallsbacktowindow.setTimemout()otherwise.
Ifyouweretotrytheapplicationnow,thiswouldbeenoughtodrawtheshiponthemiddleofthescreen,butnothingelsewouldhappenaswehaven’tstartedhandlinguserinputyet.
Asfarasuserinputgoes,we’reconcernedwithafewactions:
Shipmovement:rotation,forward,andbackwardFiringtheship’sgunPausingthegame
Toaccountfortheseactions,we’lldefinesomeconstantsthatrepresenttheASCIIcodesofthekeysinvolved:
(defUP38)
(defRIGHT39)
(defDOWN40)
(defLEFT37)
(defFIRE32);;space
(defPAUSE80);;lower-caseP
Thisshouldlooksensibleasweareusingthekeystraditionallyusedforthesetypesofactions.
ModelinguserinputaseventstreamsOneofthethingsdiscussedintheearlierchaptersisthatifyoucanthinkofeventsasalistofthingsthathaven’thappenedyet;youcanprobablymodelitasaneventstream.Inourcase,thislistiscomposedbythekeystheplayerpressesduringthegameandcanbevisualizedlikeso:
Thereisacatchthough.Mostgamesneedtohandlesimultaneouslypressedkeys.
Sayyou’reflyingthespaceshipforwards.Youdon’twanttohavetostopitinordertorotateittotheleftandthencontinuemovingforwards.Whatyouwantistopressleftatthesametimeyou’repressingupandhavetheshiprespondaccordingly.
Thishintsatthefactthatweneedtobeabletotellwhethertheplayeriscurrentlypressingmultiplekeys.TraditionallythisisdoneinJavaScriptbykeepingtrackofwhichkeysarebeinghelddowninamap-likeobject,usingflags.Somethingsimilartothefollowingsnippet:
varkeysPressed={};
document.addEventListener('keydown',function(e){
keysPressed[e.keyCode]=true;
},false);
document.addEventListener('keyup',function(e){
keysPressed[e.keyCode]=false;
},false);
Then,laterinthegameloop,youwouldcheckwhethertherearemultiplekeysbeingpressed:
functiongameLoop(){
if(keyPressed[UP]&&keyPressed[LEFT]){
//updateshipposition
}
//...
}
Whilethiscodeworks,itreliesonmutatingthekeysPressedobjectwhichisn’tideal.
Additionally,withasetupsimilartotheprecedingone,thekeysPressedobjectisglobaltotheapplicationasitisneededbothinthekeyup/keydowneventhandlersaswellasinthegameloopitself.
Infunctionalprogramming,westrivetoeliminateorreducetheamountofglobalmutable
stateinordertowritereadable,maintainablecodethatislesserror-prone.Wewillapplytheseprincipleshere.
AsseenintheprecedingJavaScriptexample,wecanregistercallbackstobenotifiedwheneverakeyuporkeydowneventhappens.Thisisusefulaswecaneasilyturnthemintoeventstreams:
(defnkeydown-stream[]
(let[out(r/events)]
(set!(.-onkeydownjs/document)
#(r/deliverout[::down(.-keyCode%)]))
out))
(defnkeyup-stream[]
(let[out(r/events)]
(set!(.-onkeyupjs/document)
#(r/deliverout[::up(.-keyCode%)]))
out))
Bothkeydown-streamandkeyup-streamreturnanewstreamtowhichtheydelivereventswhenevertheyhappen.Eacheventistaggedwithakeyword,sowecaneasilyidentifyitstype.
Wewouldliketohandlebothtypesofeventssimultaneouslyandassuchweneedawaytocombinethesetwostreamsintoasingleone.
Therearemanywaysinwhichwecancombinestreams,forexample,usingoperatorssuchaszipandflatmap.Forthisinstance,however,weareinterestedinthemergeoperator.mergecreatesanewstreamthatemitsvaluesfrombothstreamsastheyarrive:
Thisgivesusenoughtostartcreatingourstreamofactivekeys.Basedonwhatwehavediscussedsofar,ourstreamlookssomethinglikethefollowingatthemoment:
(defactive-keys-stream
(->>(r/merge(keydown-stream)(keyup-stream))
...
))
Tokeeptrackofwhichkeysarecurrentlypressed,wewilluseaClojureScriptset.Thiswaywedon’thavetoworryaboutsettingflagstotrueorfalse—wecansimplyperformstandardsetoperationsandadd/removekeysfromthedatastructure.
Thenextthingweneedisawaytoaccumulatethepressedkeysintothissetasneweventsareemittedfromthemergedstream.
Infunctionalprogramming,wheneverwewishtoaccumulateoraggregatesometypeofdataoverasequenceofvalues,weusereduce.
Most—ifnotall—CESframeworkshavethisfunctionbuilt-in.RxJavacallsitscan.Reagi,ontheotherhand,callsitreduce,makingitintuitivetofunctionalprogrammersingeneral.
Thatisthefunctionwewillusetofinishtheimplementationofactive-keys-stream:
(defactive-keys-stream
(->>(r/merge(keydown-stream)(keyup-stream))
(r/reduce(fn[acc[event-typekey-code]]
(condp=event-type
::down(conjacckey-code)
::up(disjacckey-code)
acc))
#{})
(r/sample25)))
r/reducetakesthreearguments:areducingfunction,anoptionalinitial/seedvalue,andthestreamtoreduceover.
Ourseedvalueisanemptysetasinitiallytheuserhasn’tyetpressedanykeys.Then,ourreducingfunctioncheckstheeventtype,removingoraddingthekeyfrom/tothesetasappropriate.
Asaresult,whatwehaveisastreamliketheonerepresentedasfollows:
WorkingwiththeactivekeysstreamThegroundworkwe’vedonesofarwillmakesurewecaneasilyhandlegameeventsinacleanandmaintainableway.Themainideabehindhavingastreamrepresentingthegamekeysisthatnowwecanpartitionitmuchlikewewouldanormallist.
Forinstance,ifwe’reinterestedinalleventswherethekeypressedisUP,wewouldrunthefollowingcode:
(->>active-keys-stream
(r/filter(partialsome#{UP}))
(r/map(fn[_](.logjs/console"Pressedup…"))))
Similarly,foreventsinvolvingtheFIREkey,wecoulddothefollowing:
(->>active-keys-stream
(r/filter(partialsome#{FIRE}))
(r/map(fn[_](.logjs/console"Pressedfire…"))))
ThisworksbecauseinClojure,setscanbeusedaspredicates.WecanquicklyverifythisattheREPL:
user>(defnumbers#{121314})
#'user/numbers
user>(some#{12}numbers)
12
user>(some#{15}numbers)
nil
Byrepresentingtheeventsasastream,wecaneasilyoperateonthemusingfamiliarsequencefunctionssuchasmapandfilter.
Writingcodelikethis,however,isalittlerepetitive.Thetwopreviousexamplesareprettymuchsayingsomethingalongtheselines:filteralleventsmatchingagivenpredicatepredandthenmaptheffunctionoverthem.Wecanabstractthispatterninafunctionwe’llcallfilter-map:
(defnfilter-map[predf&args]
(->>active-keys-stream
(r/filter(partialsomepred))
(r/map(fn[_](applyfargs)))))
Withthishelperfunctioninplace,itbecomeseasytohandleourgameactions:
(filter-map#{FIRE}fire!monet-canvasship)
(filter-map#{UP}move-forward!ship)
(filter-map#{DOWN}move-backward!ship)
(filter-map#{RIGHT}rotate-right!ship)
(filter-map#{LEFT}rotate-left!ship)
TheonlythingmissingnowistakingcareofpausingtheanimationswhentheplayerpressesthePAUSEkey.Wefollowthesamelogicasabove,butwithaslightchange:
(defnpause![_]
(if@(:updating?monet-canvas)
(canvas/stop-updatingmonet-canvas)
(canvas/start-updatingmonet-canvas)))
(->>active-keys-stream
(r/filter(partialsome#{PAUSE}))
(r/throttle100)
(r/mappause!))
Monetmakesaflagavailablethattellsuswhetheritiscurrentlyupdatingtheanimationstate.Weusethatasacheapmechanismto“pause”thegame.
Notethatactive-keys-streampusheseventsastheyhappenso,ifauserisholdingabuttondownforanyamountoftime,wewillgetmultipleeventsforthatkey.Assuch,wewouldprobablygetmultipleoccurrencesofthePAUSEkeyinaveryshortamountoftime.Thiswouldcausethegametofranticallystop/start.Inordertopreventthisfromhappening,wethrottlethefilteredstreamandignoreallPAUSEeventsthathappeninawindowshorterthan100milliseconds.
Tomakesurewedidn’tmissanything,thisiswhatoursrc/cljs/reagi_game/core.cljsfileshouldlooklike,infull:
(nsreagi-game.core
(:require[monet.canvas:ascanvas]
[reagi.core:asr]
[clojure.set:asset]
[reagi-game.entities:asentities
:refer[move-forward!move-backward!rotate-left!rotate-
right!fire!]]))
(defcanvas-dom(.getElementByIdjs/document"canvas"))
(defmonet-canvas(canvas/initcanvas-dom"2d"))
(defship(entities/shape-data(/(.-width(:canvasmonet-canvas))2)
(/(.-height(:canvasmonet-canvas))2)
0))
(defship-entity(entities/ship-entityship))
(canvas/add-entitymonet-canvas:ship-entityship-entity)
(canvas/draw-loopmonet-canvas)
(defUP38)
(defRIGHT39)
(defDOWN40)
(defLEFT37)
(defFIRE32);;space
(defPAUSE80);;lower-caseP
(defnkeydown-stream[]
(let[out(r/events)]
(set!(.-onkeydownjs/document)#(r/deliverout[::down(.-keyCode
%)]))
out))
(defnkeyup-stream[]
(let[out(r/events)]
(set!(.-onkeyupjs/document)#(r/deliverout[::up(.-keyCode%)]))
out))
(defactive-keys-stream
(->>(r/merge(keydown-stream)(keyup-stream))
(r/reduce(fn[acc[event-typekey-code]]
(condp=event-type
::down(conjacckey-code)
::up(disjacckey-code)
acc))
#{})
(r/sample25)))
(defnfilter-map[predf&args]
(->>active-keys-stream
(r/filter(partialsomepred))
(r/map(fn[_](applyfargs)))))
(filter-map#{FIRE}fire!monet-canvasship)
(filter-map#{UP}move-forward!ship)
(filter-map#{DOWN}move-backward!ship)
(filter-map#{RIGHT}rotate-right!ship)
(filter-map#{LEFT}rotate-left!ship)
(defnpause![_]
(if@(:updating?monet-canvas)
(canvas/stop-updatingmonet-canvas)
(canvas/start-updatingmonet-canvas)))
(->>active-keys-stream
(r/filter(partialsome#{PAUSE}))
(r/throttle100)
(r/mappause!))
Thiscompletesthecodeandwe’renowreadytohavealookattheresults.
Ifyoustillhavetheserverrunningfromearlierinthischapter,simplyexittheREPL,startitagain,andstarttheembeddedwebserver:
leinrepl
CompilingClojureScript.
Compiling"dev-resources/public/js/reagi_game.js"from("src/cljs"
"test/cljs""dev-resources/tools/repl")...
user=>(run)
2014-06-1419:21:40.381:INFO:oejs.Server:jetty-7.6.8.v20121106
2014-06-1419:21:40.403:INFO:oejs.AbstractConnector:Started
[email protected]:3000
#<Serverorg.eclipse.jetty.server.Server@51f6292b>
ThiswillcompilethelatestversionofourClojureScriptsourcetoJavaScript.
Alternatively,youcanleavetheREPLrunningandsimplyaskcljsbuildtoauto-compilethesourcecodefromanotherterminalwindow:
leincljsbuildauto
Compiling"dev-resources/public/js/reagi_game.js"from("src/cljs"
"test/cljs""dev-resources/tools/repl")...
Successfullycompiled"dev-resources/public/js/reagi_game.js"in13.23869
seconds.
Nowyoucanpointyourbrowsertohttp://localhost:3000/andflyaroundyourspaceship!Don’tforgettoshootsomebulletsaswell!
ReagiandotherCESframeworksBackinChapter4,Introductiontocore.async,wehadanoverviewofthemaindifferencesbetweencore.asyncandCES.Anotherquestionthatmighthaveariseninthischapteristhis:howdowedecidewhichCESframeworktouse?
Theanswerislessclearthanbeforeandoftendependsonthespecificsofthetoolbeinglookedat.Wehavelearnedabouttwosuchtoolssofar:ReactiveExtensions(encompassingRxJS,RxJava,andRxClojure)andReagi.
ReactiveExtensions(Rx)isamuchmorematureframework.Itsfirstversionforthe.NETplatformwasreleasedin2011andtheideasinithavesinceevolvedsubstantially.
Additionally,portsforotherplatformssuchasRxJavaarebeingheavilyusedinproductionbybignamessuchasNetflix.
AdrawbackofRxisthatifyouwouldliketouseitbothinthebrowserandontheserver,youhavetousetwoseparateframeworks,RxJSandRxJava,respectively.WhiletheydosharethesameAPI,theyaredifferentcodebases,whichcanincurbugsthatmighthavebeensolvedinoneportbutnotyetinanother.
ForClojuredevelopers,italsomeansrelyingmoreoninteroperabilitytointeractwiththefullAPIofRx.
Reagi,ontheotherhand,isanewplayerinthisspacebutbuildsonthesolidfoundationlaidoutbycore.async.ItisfullydevelopedinClojureandsolvesthein-browser/on-serverissuebycompilingtobothClojureandClojureScript.
Reagialsoallowsseamlessintegrationwithcore.asyncviafunctionssuchasportandsubscribe,whichallowchannelstobecreatedfromeventstreams.
Moreover,theuseofcore.asyncinClojureScriptapplicationsisbecomingubiquitous,sochancesareyoualreadyhaveitasadependency.ThismakesReagianattractiveoptionforthetimeswhenweneedahigherlevelofabstractionthantheoneprovidedbycore.async.
SummaryInthischapter,welearnedhowwecanusethetechniquesfromreactiveprogrammingwehavelearnedsofarinordertowritecodethatiscleanerandeasiertomaintain.Todoso,weinsistedonthinkingaboutasynchronouseventssimplyaslistsandsawhowthatwayofthinkinglendsitselfquiteeasilytobeingmodeledasaneventstream.Allourgamehastodo,then,isoperateonthesestreamsusingfamiliarsequenceprocessingfunctions.
WealsolearnedthebasicsofReagi,aframeworkforCESsimilartotheonewecreatedinChapter4,Introductiontocore.async,butthatismorefeaturerichandrobust.
Inthenextchapter,wewilltakeabreakfromCESandseehowamoretraditionalreactiveapproachbasedondataflowscanbeuseful.
Chapter7.TheUIasaFunctionSofarwehavetakenajourneythroughmanagingcomplexitybyefficientlyhandlingandmodelingasynchronousworkflowsintermsofstreamsofdata.Inparticular,Chapter4,Introductiontocore.asyncandChapter5,CreatingYourOwnCESFrameworkwithcore.asyncexploredwhat’sinvolvedinlibrariesthatprovideprimitivesandcombinatorsforCompositionalEventSystems.WealsobuiltasimpleClojureScriptapplicationthatmadeuseofourframework.
Onethingyoumighthavenoticedisthatnoneoftheexamplessofarhavedealtwithwhathappenstothedataoncewearereadytopresentittoourusers.It’sstillanopenquestionthatwe,asapplicationdevelopers,needtoanswer.
Inthischapter,wewilllookatonewaytohandleReactiveUserInterfacesinwebapplicationsusingReact(seehttp://facebook.github.io/react/),amodernJavaScriptframeworkdevelopedbyFacebook,aswellas:
LearnhowReactrendersuserinterfacesefficientlyBeintroducedtoOm,aClojureScriptinterfacetoReactLearnhowOmleveragespersistentdatastructuresforperformanceDeveloptwofullyworkingClojureScriptapplicationswithOm,includingtheuseofcore.asyncforintercomponentcommunication
TheproblemwithcomplexwebUIsWiththeriseofsingle-pagewebapplications,itbecameamusttobeabletomanagethegrowthandcomplexityofaJavaScriptcodebase.ThesameappliestoClojureScript.
Inanefforttomanagethiscomplexity,aplethoraofJavaScriptMVCframeworkshaveemergedsuchasAngularJS,Backbone.js,Ember.js,andKnockoutJStonameafew.
Theyareverydifferent,butshareafewcommonfeatures:
Givesingle-pageapplicationsmorestructurebyprovidingmodels,views,controllers,templates,andsoonProvideclient-sideroutingTwo-waydatabinding
Inthischapter,we’llbefocusingonthelastgoal.
Two-waydatabindingisabsolutelycrucialifwearetodevelopevenamoderatelycomplexsingle-pagewebapplication.Here’showitworks.
Supposewe’redevelopingaphonebookapplication.Morethanlikely,wewillhaveamodel—orentity,map,whathaveyou—thatrepresentsacontact.Thecontactmodelmighthaveattributessuchasname,phonenumber,ande-mailaddress.
Ofcourse,thisapplicationwouldnotbeallthatusefulifuserscouldn’tupdatecontactinformation,sowewillneedaformwhichdisplaysthecurrentdetailsforacontactandletsyouupdatethecontact’sinformation.
ThecontactmodelmighthavebeenloadedviaanAJAXrequestandthenmighthaveusedexplicitDOMmanipulationcodetodisplaytheform.Thiswouldlooksomethinglikethefollowingpseudo-code:
functioneditContact(contactId){
contactService.get(contactId,function(data){
contactForm.setName(data.name);
contactForm.setPhone(data.phone);
contactForm.setEmail(data.email);
})
}
Butwhathappenswhentheuserupdatessomeone’sinformation?Weneedtostoreitsomehow.Onclickingonsave,afunctionsuchasthefollowingwoulddothetrick,assumingyou’reusingjQuery:
$("save-button").click(function(){
contactService.update(contactForm.serialize(),function(){
flashMessage.set("ContactUpdated.")
})
Thisseeminglyharmlesscodeposesabigproblem.Thecontactmodelforthisparticularpersonisnowoutofdate.Ifwewerestilldevelopingwebapplicationstheoldway,wherewereloadthepageateveryupdate,thiswouldn’tbeaproblem.However,thewholepointofsingle-pagewebapplicationsistoberesponsive,soitkeepsalotofstateontheclient,
anditisimportanttokeepourmodelssyncedwithourviews.
Thisiswheretwo-waydatabindingcomesin.AnexamplefromAngularJSwouldlooklikethefollowing:
//JS
//intheController
$scope.contact={
name:'LeonardoBorges',
phone'+61xxxxxxxxx',
email:'[email protected]'
}
<!--HTML-->
<!--intheView-->
<form>
<inputtype="text"name="contactName"ng-model="contact.name"/>
<inputtype="text"name="contactPhone"ng-model="contact.phone"/>
<inputtype="text"name="contactEmail"ng-model="contact.email"/>
</form>
Angularisn’tthetargetofthischapter,soIwon’tdigintothedetails.Allweneedtoknowfromthisexampleisthat$scopeishowwetellAngulartomakeourcontactmodelavailabletoourviews.Intheview,thecustomattributeng-modeltellsAngulartolookupthatpropertyinthescope.Thisestablishesatwo-wayrelationshipinsuchawaythatwhenyourmodeldatachangesinthescope,AngularrefreshestheUI.Similarly,iftheusereditstheform,Angularupdatesthemodel,keepingeverythinginsync.
Thereare,however,twomainproblemswiththisapproach:
Itcanbeslow.ThewayAngularandfriendsimplementtwo-waydatabindingis,roughlyspeaking,byattachingeventhandlersandwatcherstoviewbothcustom-attributesandmodelattributes.Forcomplexenoughuserinterfaces,youwillstartnoticingthattheUIbecomesslowertorender,diminishingtheuserexperience.Itreliesheavilyonmutation.Asfunctionalprogrammers,westrivetolimitsideeffectstoaminimum.
Theslownessthatcomeswiththisandsimilarapproachesistwo-fold:firstly,AngularJSandfriendshaveto“watch”allpropertiesofeverymodelinthescopeinordertotrackupdates.Oncetheframeworkdeterminesthatdatahaschangedinthemodel,itthenlooksuppartsoftheUI,whichdependonthatinformation—suchasthefragmentsusingng-modelabove—andthenitre-rendersthem.
Secondly,theDOMistheslowestpartofmostsingle-pagewebapplications.Ifwethinkaboutitforamoment,theseframeworksaretriggeringdozensorperhapshundredsofDOMeventhandlersinordertokeepthedatainsync,eachofwhichendsupupdatinganode—orseveral—intheDOM.
Don’ttakemywordforitthough.IranasimplebenchmarktocompareapurecalculationversuslocatingaDOMelementandupdatingitsvaluetotheresultofthesaidcalculation.Herearetheresults—I’veusedJSPerftorunthebenchmark,andtheseresultsarefor
Chrome37.0.2062.94onMacOSXMavericks(seehttp://jsperf.com/purefunctions-vs-dom):
document.getElementsByName("sum")[0].value=1+2
//Operationspersecond:2,090,202
1+2
//Operationspersecond:780,538,120
UpdatingtheDOMisordersofmagnitudeslowerthanperformingasimplecalculation.Itseemslogicalthatwewouldwanttodothisinthemostefficientmannerpossible.
However,ifwedon’tkeepourdatainsync,we’rebackatsquareone.Thereshouldbeawaybywhichwecandrasticallyreducetheamountofrenderingbeingdone,whileretainingtheconvenienceoftwo-waydatabinding.Canwehaveourcakeandeatittoo?
EnterReact.jsAswe’llseeinthischapter,theanswertothequestionposedintheprevioussectionisaresoundingyesand,asyoumighthaveguessed,itinvolvesReact.js.
Butwhatmakesitspecial?
It’swisetostartwithwhatReactisnot.ItisnotanMVCframeworkandassuchitisnotareplacementforthelikesofAngularJS,Backbone.js,andsoon.ReactfocusessolelyontheVinMVC,andpresentsarefreshinglydifferentwaytothinkaboutuserinterfaces.Wemusttakeaslightdetourinordertoexplorehowitdoesthat.
LessonsfromfunctionalprogrammingAsfunctionalprogrammers,wedon’tneedtobeconvincedofthebenefitsofimmutability.Weboughtintothepremiselongago.However,shouldwenotbeabletouseimmutabilityefficiently,itwouldnothavebecomecommonplaceinfunctionalprogramminglanguages.
WeoweittothehugeamountofresearchthatwentintoPurelyFunctionalDataStructures—firstbyOkasakiinhisbookofthesametitle(seehttp://www.amazon.com/Purely-Functional-Structures-Chris-Okasaki/dp/0521663504/ref=sr_1_1?ie=UTF8&qid=1409550695&sr=8-1&keywords=purely+functional+data+structures)andthenimprovedbyothers.
Withoutit,ourprogramswouldbeballooning,bothinspaceandruntimecomplexity.
Thegeneralideaisthatgivenadatastructure,theonlywaytoupdateitisbycreatingacopyofitwiththedesireddeltaapplied:
(conj[123]4);;[1234]
Intheprecedingimage,wehaveasimplisticviewofhowconjoperates.Ontheleft,youhavetheunderlyingdatastructurerepresentingthevectorwewishtoupdate.Ontheright,wehavethenewlycreatedvector,which,aswecansee,sharessomestructurewiththepreviousvector,aswellascontainingournewitem.
TipInreality,theunderlyingdatastructureisatreeandtherepresentationwassimplifiedforthepurposesofthisbook.IhighlyrecommendreferringtoOkasaki’sbookshouldthereaderwantmoredetailsonhowpurelyfunctionaldatastructureswork.
Additionally,thesefunctionsareconsideredpure.Thatis,itrelateseveryinputtoasingleoutputanddoesnothingelse.Thisis,infact,remarkablysimilartohowReacthandlesuserinterfaces.
IfwethinkofaUIasavisualrepresentationofadatastructure,whichreflectsthecurrentstateofourapplication,wecan,withouttoomucheffort,thinkofUIupdatesasasimple
functionwhoseinputistheapplicationstateandtheoutputisaDOMrepresentation.
You’llhavenoticedIdidn’tsaytheoutputisrenderingtotheDOM—thatwouldmakethefunctionimpureasrenderingisclearlyasideeffect.Itwouldalsomakeitjustasslowasthealternatives.
ThisDOMrepresentationisessentiallyatreeofDOMnodesthatmodelhowyourUIshouldlook,andnothingelse.
ReactcallsthisrepresentationaVirtualDOM,androughlyspeaking,insteadofwatchingindividualbitsandpiecesofapplicationstatethattriggeraDOMre-renderuponchange,ReactturnsyourUIintoafunctiontowhichyougivethewholeapplicationstate.
Whenyougivethisfunctionthenewupdatedstate,ReactrendersthatstatetotheVirtualDOM.RemembertheVirtualDOMissimplyadatastructure,sotherenderingisextremelyfast.Onceit’sdone,Reactdoesoneoftwothings:
ItcommitstheVirtualDOMtotheactualDOMifthisisthefirstrender.Otherwise,itcomparesthenewVirtualDOMwiththecurrentVirtualDOM,cachedfromthepreviousrenderoftheapplication.ItthenusesanefficientdiffalgorithmtocomputetheminimumsetofchangesrequiredtoupdatetherealDOM.Finally,itcommitsthisdeltatotheDOM.
WithoutdiggingintothenutsandboltsofReact,thisisessentiallyhowitisimplementedandthereasonitisfasterthanthealternatives.Conceptually,Reacthitsthe“refresh”buttonwheneveryourapplicationstatechanges.
AnothergreatbenefitisthatbythinkingofyourUIasafunctionfromapplicationstatetoaVirtualDOM,werecoversomeofthereasoningwe’reabletodowhenworkingwithimmutabledatastructuresinfunctionallanguages.
Intheupcomingsections,wewillunderstandwhythisisabigwinforusClojuredevelopers.
ClojureScriptandOmWhyhaveIspentsixpagestalkingaboutJavaScriptandReactinaClojurebook?IpromiseI’mnottryingtowasteyourprecioustime;wesimplyneededsomecontexttounderstandwhat’stocome.
OmisaClojureScriptinterfacetoReact.jsdevelopedbytheprolificandamazingindividualDavidNolen,fromCognitect.Yes,hehasalsodevelopedcore.logic,core.match,andtheClojureScriptcompiler.That’showprolific.ButIdigress.
WhenFacebookreleasedReact,Davidimmediatelysawthepotentialand,moreimportantly,howtotakeadvantageoftheassumptionsweareabletomakewhenprogramminginClojure,themostimportantofwhichisthatdatastructuresdon’tchange.
Reactprovidesseveralcomponentlife-cyclefunctionsthatallowdeveloperstocontrolvariouspropertiesandbehaviors.Oneinparticular,shouldComponentUpdate,isusedtodecidewhetheracomponentneedstobere-rendered.
Reacthasabigchallengehere.JavaScriptisinherentlymutable,soitisextremelyhard,whencomparingVirtualDOMTrees,toidentifywhichnodeshavechangedinanefficientway.ReactemploysafewheuristicsinordertoavoidO(n3)worst-caseperformanceandisabletodoitinO(n)mostofthetime.Sinceheuristicsaren’tperfect,wecanchoosetoprovideourownimplementationofshouldComponentUpdateandtakeadvantageoftheknowledgewepossesswhenrenderingacomponent.
ClojureScript,ontheotherhand,usesimmutabledatastructures.Assuch,OmprovidesthesimplestandmostefficientimplementationpossibleforshouldComponentUpdate:asimplereferenceequalitycheck.
Becausewe’realwaysdealingwithimmutabledatastructures,inordertoknowwhethertwotreesarethesame,allweneedtodoiscomparewhethertheirrootsarethesame.Iftheyare,we’redone.Otherwise,descendandrepeattheprocess.ThisisguaranteedtoyieldO(logn)runtimecomplexityandallowsOmtoalwaysrendertheUIfromtherootefficiently.
Ofcourse,performanceisn’ttheonlythingthat’sgoodaboutOm—wewillnowexplorewhatmakesanOmapplication.
BuildingasimpleContactsapplicationwithOmThischapterhasbeenverytextheavysofar.It’stimewegetourhandsdirtyandbuildasimpleOmapplication.Sincewetalkedaboutcontactsbefore,that’swhatwewillstartwith.
ThemaindriverbehindReactandOmistheabilitytobuildhighlyreusable,self-containedcomponentsand,assuch,eveninasimpleContactsapplication,wewillhavemultiplecomponentsworkinginconcerttoachieveacommongoal.
Thisiswhatourusersshouldbeabletodointheapplication:
DisplayalistofcontactscurrentlyinstorageDisplaythedetailsofagivencontactEditthedetailsofaspecificcontact
Andoncewe’redone,itwilllooklikethefollowing:
TheContactsapplicationstateAsmentionedpreviously,Om/ReactwilleventuallyrendertheDOMbasedonourapplicationstate.We’llbeusingdatathat’sinmemorytokeeptheexamplesimple.Here’swhatourapplicationstatewilllooklike:
(defapp-state
(atom{:contacts{1{:id1
:name"JamesHetfield"
:email"[email protected]"
:phone"+1XXXXXXXXX"}
2{:id2
:name"AdamDarski"
:email"[email protected]"
:phone"+48XXXXXXXXX"}}
:selected-contact-id[]
:editing[false]}))
ThereasonwekeepthestateinanatomisthatOmusesthattore-rendertheapplicationifweswap!orreset!it,forinstance,ifweloadsomedatafromtheserveraftertheapplicationhasbeenrenderedforthefirsttime.
Thedatainthestateitselfshouldbemostlyself-explanatory.Wehaveamapcontainingallcontacts,akeyrepresentingwhetherthereiscurrentlyacontactselected,andaflagthatindicateswhetherwearecurrentlyeditingtheselectedcontact.Whatmightlookoddisthatboth:selected-contact-idand:editingkeyspointtoavector.Justbearwithmeforamoment;thereasonforthiswillbecomeclearshortly.
Nowthatwehaveadraftofourapplicationstate,it’stimewethinkabouthowthestatewillflowthroughthedifferentcomponentsinourapp.Apictureisworthathousandwords,sothefollowingdiagramshowsthehigh-levelarchitecturethroughwhichourdatawillflow:
Intheprecedingimage,eachfunctioncorrespondstoanOmcomponent.Attheveryleast,theytakesomepieceofdataastheirinitialstate.Whatisinterestinginthisimageisthataswedescendintoourmorespecializedcomponents,theyrequestlessstatethanthemaincomponent,contacts-app.Forinstance,thecontacts-viewcomponentneedsallcontactsaswellastheIDoftheselectedcontact.Thedetails-panel-viewcomponent,ontheotherhand,onlyneedsthecurrentlyselectedcontact,andwhetherit’sbeingeditedornot.ThisisacommonpatterninOmandweusuallywanttoavoidover-sharingtheapplicationstate.
Witharoughunderstandingofourhigh-levelarchitecture,wearereadytostartbuildingourContactsapplication.
SettinguptheContactsprojectOnceagain,wewillusealeiningentemplatetohelpusgetstarted.Thistimewe’llbeusingom-start(seehttps://github.com/magomimmo/om-start-template),alsobyMimmoCosenza(seehttps://github.com/magomimmo).Typethisintheterminaltocreateabaseprojectusingthistemplate:
leinnewom-startcontacts
cdcontacts
Next,let’sopentheproject.cljfileandmakesurewehavethesameversionsforthevariousdifferentdependenciesthetemplatepullsin.Thisisjustsothatwedon’thaveanysurpriseswithincompatibleversions:
...
:dependencies[[org.clojure/clojure"1.6.0"]
[org.clojure/clojurescript"0.0-2277"]
[org.clojure/core.async"0.1.338.0-5c5012-alpha"]
[om"0.7.1"]
[com.facebook/react"0.11.1"]]
...
Tovalidatethenewprojectskeleton,stillintheterminal,typethefollowingtoauto-compileyourClojureScriptsourcefiles:
leincljsbuildauto
CompilingClojureScript.
Compiling"dev-resources/public/js/contacts.js"from("src/cljs""dev-
resources/tools/repl")...
Successfullycompiled"dev-resources/public/js/contacts.js"in9.563
seconds.
Now,weshouldseethetemplatedefault“HelloWorld”pageifweopenthedev-resources/public/index.htmlfileinthebrowser.
ApplicationcomponentsThenextthingwe’lldoisopenthesrc/cljs/contacts/core.cljsfile,whichiswhereourapplicationcodewillgo,andmakesureitlookslikethefollowingsothatwehaveacleanslatewiththeappropriatenamespacedeclaration:
(nscontacts.core
(:require[om.core:asom:include-macrostrue]
[om.dom:asdom:include-macrostrue]))
(enable-console-print!)
(defapp-state
(atom{:contacts{1{:id1
:name"JamesHetfield"
:email"[email protected]"
:phone"+1XXXXXXXXX"}
2{:id2
:name"AdamDarski"
:email"[email protected]"
:phone"+48XXXXXXXXX"}}
:selected-contact-id[]
:editing[false]}))
(om/root
contacts-app
app-state
{:target(.js/document(getElementById"app"))})
EveryOmapplicationstartswitharootcomponentcreatedbytheom/rootfunction.Ittakesasargumentsafunctionrepresentingacomponent—contacts-app—theinitialstateoftheapplication—app-state—andamapofoptionsofwhichtheonlyonewecareaboutis:target,whichtellsOmwheretomountourrootcomponentontheDOM.
Inthisinstance,itwillmountonaDOMelementwhoseIDisapp.Thiselementwasgiventousbytheom-starttemplateandislocatedinthedev-resources/public/index.htmlfile.
Ofcourse,thiscodewon’tcompileyet,aswedon’thavethecontacts-apptemplate.Let’ssolvethatandcreateitabovetheprecedingdeclaration—we’reimplementingthecomponentsbottom-up:
(defncontacts-app[dataowner]
(reify
om/IRender
(render[this]
(let[[selected-id:asselected-id-cursor]
(:selected-contact-iddata)]
(dom/divnil
(om/buildcontacts-view
{:contacts(:contactsdata)
:selected-id-cursorselected-id-cursor})
(om/builddetails-panel-view
{:contact(get-indata[:contacts
selected-id])
:editing-cursor(:editingdata)}))))))
Thissnippetintroducesanumberofnewfeaturesandterminology,soitdeservesafewparagraphs.
Whendescribingom/root,wesawthatitsfirstargumentmustbeanOmcomponent.Thecontact-appfunctioncreatesonebyreifyingtheom/IRenderprotocol.Thisprotocolcontainsasinglefunction—render—whichgetscalledwhentheapplicationstatechanges.
TipClojureusesreifytoimplementprotocolsorJavainterfacesonthefly,withouttheneedtocreateanewtype.YoucanreadmoreaboutthisonthedatatypespageoftheClojuredocumentationathttp://clojure.org/datatypes.
TherenderfunctionmustreturnanOm/ReactcomponentorsomethingReactknowshowtorender—suchasaDOMrepresentationofthecomponent.Theargumentstocontacts-apparestraightforward:dataisthecomponentstateandowneristhebackingReactcomponent.
Movingdownthesourcefile,intheimplementationofrender,wehavethefollowing:
(let[[selected-id:asselected-id-cursor]
(:selected-contact-iddata)]
...)
Ifwerecallfromourapplicationstate,thevalueof:selected-contact-idis,atthisstage,anemptyvector.Here,then,wearedestructuringthisvectorandgivingitaname.Whatyoumightbewonderingnowiswhyweboundthevectortoavariablenamedselected-id-cursor.Thisistoreflectthefactthatatthispointinthelifecycleofacomponent,selected-id-cursorisn’tavectoranylongerbutratheritisacursor.
OmcursorsOnceom/rootcreatesourrootcomponent,sub-componentsdon’thavedirectaccesstothestateatomanylonger.Instead,componentsreceiveacursorcreatedfromtheapplicationstate.
Cursorsaredatastructuresthatrepresentaplaceintheoriginalstateatom.Youcanusecursorstoread,delete,update,orcreateavaluewithnoknowledgeoftheoriginaldatastructure.Let’staketheselected-id-cursorcursorasanexample:
Atthetop,wehaveouroriginalapplicationstate,whichOmturnsintoacursor.Whenwerequestthe:selected-contact-idkeyfromit,Omgivesusanothercursorrepresentingthatparticularplaceinthedatastructure.Itjustsohappensthatitsvalueistheemptyvector.
WhatisinterestingaboutthiscursoristhatifweupdateitsvalueusingoneofOm’sstatetransitionfunctionssuchasom/transact!andom/update!—wewillexplaintheseshortly—itknowshowtopropagatethechangeupthetreeandallthewaybacktotheapplicationstateatom.
Thisisimportantbecauseaswehavebrieflystatedbefore,itiscommonpracticetohaveourmorespecializedcomponentsdependonspecificpartsoftheapplicationstaterequiredforitscorrectoperation.
Byusingcursors,wecaneasilypropagatechangeswithoutknowingwhattheapplicationstatelookslike,thusavoidingtheneedtoaccesstheglobalstate.
TipYoucanthinkofcursorsaszippers.Conceptually,theyserveasimilarpurposebuthavedifferentAPIs.
FillingintheblanksMovingdownthecontacts-appcomponent,wenowhavethefollowing:
(dom/divnil
(om/buildcontacts-view
{:contacts(:contactsdata)
:selected-id-cursorselected-id-cursor})
(om/builddetails-panel-view
{:contact(get-indata[:contacts
selected-id])
:editing-cursor(:editingdata)}))
ThedomnamespacecontainsthinwrappersaroundReact’sDOMclasses.It’sessentiallythedatastructurerepresentingwhattheapplicationwilllooklike.Next,weseetwoexamplesofhowwecancreateOmcomponentsinsideanotherOmcomponent.Weusetheom/buildfunctionforthatandcreatethecontacts-viewanddetails-panel-viewcomponents.Theom/buildfunctiontakesasargumentsthecomponentfunction,thecomponentstate,and,optionally,amapofoptionswhicharen’timportantforthisexample.
Atthispoint,wehavealreadystartedtolimitthestatewewillpassintothesub-componentsbycreatingsub-cursors.
Accordingtothesourcecode,thenextcomponentweshouldlookatiscontacts-view.Hereitisinfull:
(defncontacts-view[{:keys[contactsselected-id-cursor]}owner]
(reify
om/IRender
(render[_]
(dom/div#js{:style#js{:float"left"
:width"50%"}}
(applydom/ulnil
(om/build-allcontact-summary-view(valscontacts)
{:shared{:selected-id-cursorselected-
id-cursor}}))))))
Hopefully,thesourceofthiscomponentlooksalittlemorefamiliarnow.Asbefore,wereifyom/IRendertoprovideaDOMrepresentationofourcomponent.Itcomprisesasingledivelement.Thistimewegiveasthesecondargumenttodom/divahash-maprepresentingHTMLattributes.Weareusingsomeinlinestyles,butideallywewoulduseanexternalstylesheet.
TipIfyouarenotfamiliarwiththe#js{…}syntax,it’ssimplyareadermacrothatexpandsto(clj->js{…})inordertoconvertaClojureScripthash-mapintoaJavaScriptobject.Theonlythingtowatchforisthatitisnotrecursive,asevidencedbythenesteduseof#js.
Thethirdargumenttodom/divisslightlymorecomplexthanwhatwehaveseensofar:
(applydom/ulnil
(om/build-allcontact-summary-view(valscontacts)
{:shared{:selected-id-cursorselected-
id-cursor}}))
Eachcontactwillberepresentedbyali(listitem)HTMLnode,sowestartbywrappingtheresultintoadom/ulelement.Then,weuseom/build-alltobuildalistofcontact-summary-viewcomponents.Omwill,inturn,callom/buildforeachcontactinvalscontacts.
Lastly,weusethethirdargumenttoom/build-all—theoptionsmap—todemonstratehowwecansharestatebetweencomponentswithouttheuseofglobalstate.We’llseehowthat’susedinthenextcomponent,contact-summary-view:
(defncontact-summary-view[{:keys[namephone]:ascontact}owner]
(reify
om/IRender
(render[_]
(dom/li#js{:onClick#(select-contact!@contact
(om/get-sharedowner
:selected-id-cursor))}
(dom/spannilname)
(dom/spannilphone)))))
Ifwethinkofourapplicationasatreeofcomponents,wehavenowreachedoneofitsleaves.Thiscomponentsimplyreturnsadom/linodewiththecontact’snameandphoneinit,wrappedindom/spannodes.
Italsoinstallsahandlertothedom/lionClickevent,whichwecanusetoupdatethestatecursor.
Weuseom/get-sharedtoaccessthesharedstateweinstalledearlierandpasstheresultingcursorintoselect-contact!Wealsopassthecurrentcontact,but,ifyoulookclosely,wehavetoderefitfirst:
@contact
ThereasonforthisisthatOmdoesn’tallowustomanipulatecursorsoutsideoftherenderphase.Byderefingthecursor,wehaveitsmostrecentunderlyingvalue.Nowselect-contact!hasallitneedstoperformtheupdate:
(defnselect-contact![contactselected-id-cursor]
(om/update!selected-id-cursor0(:idcontact)))
Wesimplyuseom/update!tosetthevalueoftheselected-id-cursorcursoratindex0totheidofthecontact.Asmentionedpreviously,thecursortakescareofpropagatingthechange.
TipYoucanthinkofom/update!asthecursorsversionofclojure.core/reset!usedinatoms.Conversely,thesameappliestoom/transact!andclojure.core/swap!,respectively.
Wearemovingatagoodpace.It’stimewelookatthenextcomponent,details-panel-
view:
(defndetails-panel-view[dataowner]
(reify
om/IRender
(render[_]
(dom/div#js{:style#js{:float"right"
:width"50%"}}
(om/buildcontact-details-viewdata)
(om/buildcontact-details-form-viewdata)))))
Thiscomponentshouldnowlookfairlyfamiliar.Allitdoesisbuildtwoothercomponents,contact-details-viewandcontact-details-form-view:
(defncontact-details-view[{{:keys[namephoneemailid]:ascontact}
:contact
editing:editing-cursor}
owner]
(reify
om/IRender
(render[_]
(dom/div#js{:style#js{:display(if(getediting0)"none""")}}
(dom/h2nil"Contactdetails")
(ifcontact
(dom/divnil
(dom/h3#js{:style#js{:margin-bottom"0px"}}
(:namecontact))
(dom/spannil(:phonecontact))(dom/brnil)
(dom/spannil(:emailcontact))(dom/brnil)
(dom/button#js{:onClick#(om/update!editing0
true)}
"Edit"))
(dom/spannil"Nocontactselected"))))))
Thecontact-details-viewcomponentreceivestwopiecesofstate:thecontactandtheeditingflag.Ifwehaveacontact,wesimplyrenderthecomponent.However,weusetheeditingflagtohideit,ifweareeditingit.Thisissothatwecanshowtheeditforminthenextcomponent.WealsoinstallanonClickhandlertotheEditbuttonsothatwecanupdatetheeditingcursor.
Thecontact-details-form-viewcomponentreceivesthesameargumentsbutrendersthefollowingforminstead:
(defncontact-details-form-view[{{:keys[namephoneemailid]:ascontact}
:contact
editing:editing-cursor}
owner]
(reify
om/IRender
(render[_]
(dom/div#js{:style#js{:display(if(getediting0)"""none")}}
(dom/h2nil"Contactdetails")
(ifcontact
(dom/divnil
(dom/input#js{:type"text"
:valuename
:onChange#(update-
contact!%contact:name)})
(dom/input#js{:type"text"
:valuephone
:onChange#(update-
contact!%contact:phone)})
(dom/input#js{:type"text"
:valueemail
:onChange#(update-
contact!%contact:email)})
(dom/button#js{:onClick#(om/update!editing0
false)}
"Save"))
(dom/divnil"Nocontactselected"))))))
Thisisthecomponentresponsibleforactuallyupdatingthecontactinformationbasedontheform.Itdoessobycallingupdate-contact!withtheJavaScriptevent,thecontactcursor,andthekeyrepresentingtheattributetobeupdated:
(defnupdate-contact![econtactkey]
(om/update!contactkey(..e-target-value)))
Asbefore,wesimplyuseom/update!insteadofom/transact!aswearesimplyreplacingthevalueofthecursorattributewiththecurrentvalueoftheformfieldwhichtriggeredtheevente.
NoteIfyou’renotfamiliarwiththe..syntax,it’ssimplyaconveniencemacroforJavaandJavaScriptinteroperability.Thepreviousexampleexpandsto:
(.(.e-target)-value)
ThisandotherinteroperabilityoperatorsaredescribedintheJavaInteroppageoftheClojurewebsite(seehttp://clojure.org/java_interop).
Thisisit.Makesureyourcodeisstillcompiling—orifyouhaven’tyet,starttheauto-compilationbytypingthefollowingintheterminal:
leincljsbuildauto
Then,openupdev-resources/public/index.htmlagaininyourbrowserandtakeourContactsappforaspin!Noteinparticularhowtheapplicationstateisalwaysinsyncwhileyoueditthecontactattributes.
Ifthereareanyissuesatthisstage,makesurethesrc/cljs/contacts/core.cljsfilematchesthecompanioncodeforthisbook.
IntercomponentcommunicationInourpreviousexample,thecomponentswebuiltcommunicatedwitheachotherexclusivelythroughtheapplicationstate,bothforreadingandtransactingdata.Whilethisapproachworks,itisnotalwaysthebestexceptforverysimpleusecases.Inthissection,wewilllearnanalternatewayofperformingthiscommunicationusingcore.asyncchannels.
Theapplicationwewillbuildisasupersimplevirtualagileboard.Ifyou’veheardofit,it’ssimilartoTrello(seehttps://trello.com/).Ifyouhaven’t,fearnot,it’sessentiallyataskmanagementwebapplicationinwhichyouhavecardsthatrepresenttasksandyoumovethembetweencolumnssuchasBacklog,InProgress,andDone.
Bytheendofthissection,theapplicationwilllooklikethefollowing:
We’lllimitourselvestoasinglefeature:movingcardsbetweencolumnsbydragginganddroppingthem.Let’sgetstarted.
CreatinganagileboardwithOmWe’realreadyfamiliarwiththeom-start(seehttps://github.com/magomimmo/om-start-template)leiningentemplate,andsincethereisnoreasontochangeit,that’swhatwewillusetocreateourproject—whichIcalledom-pmforOmProjectManagement:
leinnewom-startom-pm
cdom-pm
Asbefore,weshouldensurewehavetherightdependenciesinourproject.cljfile:
:dependencies[[org.clojure/clojure"1.6.0"]
[org.clojure/clojurescript"0.0-2511"]
[org.om/om"0.8.1"]
[org.clojure/core.async"0.1.346.0-17112a-alpha"]
[com.facebook/react"0.12.2"]]
Nowvalidatethatweareingoodshapebymakingsuretheprojectcompilesproperly:
leincljsbuildauto
CompilingClojureScript.
Compiling"dev-resources/public/js/om_pm.js"from("src/cljs""dev-
resources/tools/repl")...
Successfullycompiled"dev-resources/public/js/om_pm.js"in13.101seconds.
Next,openthesrc/cljs/om_pm/core.cljsfileandaddthenamespacesthatwewillbeusingtobuildtheapplication:
(nsom-pm.core
(:require[om.core:asom:include-macrostrue]
[om.dom:asdom:include-macrostrue]
[cljs.core.async:refer[put!chan<!]]
[om-pm.util:refer[set-transfer-data!get-transfer-data!move-
card!]])
(:require-macros[cljs.core.async.macros:refer[gogo-loop]]))
Themaindifferencethistimeisthatwearerequiringcore.asyncfunctionsandmacros.Wedon’tyethaveanom-pm.utilnamespace,butwe’llgettothatattheend.
TheboardstateIt’stimewethinkwhatourapplicationstatewilllooklike.Ourmainentityinthisapplicationisthecard,whichrepresentsataskandhastheattributesid,title,anddescription.Wewillstartbydefiningacoupleofcards:
(defcards[{:id1
:title"Groceriesshopping"
:description"Almondmilk,mixednuts,eggs…"}
{:id2
:title"Expenses"
:description"Submitlastclient'sexpensereport"}])
Thisisn’tourapplicationstateyet,butratherapartofit.Anotherimportantpieceofstateisawaytotrackwhichcardsareonwhichcolumns.Tokeepthingssimple,wewillworkwithonlythreecolumns:Backlog,InProgress,andDone.Bydefault,allcardsstartoutinthebacklog:
(defapp-state
(atom{:cardscards
:columns[{:title"Backlog"
:cards(mapv:idcards)}
{:title"InProgress"
:cards[]}
{:title"Done"
:cards[]}]}))
Thisisallthestateweneed.Columnshavea:titleanda:cardsattribute,whichcontainstheIDsofallcardsinthatcolumn.
Additionally,wewillhaveahelperfunctiontomakefindingcardsmoreconvenient:
(defncard-by-id[id]
(first(filterv#(=id(:id%))cards)))
TipBewareoflazysequences
YoumighthavenoticedtheuseofmapvinsteadofmapforretrievingthecardsIDs.Thisisasubtlebutimportantdifference:mapislazybydefault,butOmcanonlycreatecursorsformapsandvectors.Usingmapvgivesusavectorback,avoidinglazinessaltogether.
Hadwenotdonethat,OmwouldconsiderthelistofIDsasanormalvalueandwewouldnotbeabletotransactit.
ComponentsoverviewTherearemanywaystosliceupanOmapplicationintocomponents,andinthissection,wewillpresentonewayaswewalkthrougheachcomponent’simplementation.
Theapproachwewillfollowissimilartoourpreviousapplicationinthatfromthispointon,wepresentthecomponentsbottom-up.
Beforeweseeourfirstcomponent,however,weshouldstartwithOm’sownrootcomponent:
(om/rootproject-viewapp-state
{:target(.js/document(getElementById"app"))})
Thisgivesusahintastowhatournextcomponentwillbe,project-view:
(defnproject-view[appowner]
(reify
om/IInitState
(init-state[_]
{:transfer-chan(chan)})
om/IWillMount
(will-mount[_]
(let[transfer-chan(om/get-stateowner:transfer-chan)]
(go-loop[]
(let[transfer-data(<!transfer-chan)]
(om/transact!app:columns
#(move-card!%transfer-data))
(recur)))))
om/IRenderState
(render-state[thisstate]
(dom/divnil
(applydom/ulnil
(om/build-allcolumn-view(:columnsapp)
{:shared{:cards(:cardsapp)}
:init-statestate}))))))
LifecycleandcomponentlocalstateThepreviouscomponentisfairlydifferentfromtheoneswehaveseensofar.Morespecifically,itimplementstwonewprotocols:om/IInitStateandom/IWillMount.Additionally,wedroppedom/IRenderaltogetherinfavorofom/IRenderState.Beforeweexplainwhatthesenewprotocolsaregoodfor,weneedtodiscussourhigh-leveldesign.
Theproject-viewcomponentisourapplication’smainentrypointandreceivesthewholeapplicationstateasitsfirstargument.AsinourearlierContactsapplication,ittheninstantiatestheremainingcomponentswiththedatatheyneed.
DifferentfromtheContactsexample,however,itcreatesacore.asyncchannel—transfer-chan—whichworksasamessagebus.Theideaisthatwhenwedragacardfromonecolumnanddropitonanother,oneofourcomponentswillputatransfereventinthischannelandletsomeoneelse—mostlikelyagoblock—performtheactualmoveoperation.
Thisisdoneinthefollowingsnippettakenfromthecomponentshownearlier:
om/IInitState
(init-state[_]
{:transfer-chan(chan)})
ThiscreateswhatOmcallsthecomponentlocalstate.Itusesadifferentlifecycleprotocol,om/IInitState,whichisguaranteedtobecalledonlyonce.Afterall,weneedasinglechannelforthiscomponent.init-stateshouldreturnamaprepresentingthelocalstate.
Nowthatwehavethechannel,weneedtoinstallago-looptohandlemessagessenttoit.Forthispurpose,weuseadifferentprotocol:
om/IWillMount
(will-mount[_]
(let[transfer-chan(om/get-stateowner:transfer-chan)]
(go-loop[]
(let[transfer-data(<!transfer-chan)]
(om/transact!app:columns#(move-card!%transfer-data))
(recur)))))
Likethepreviousprotocol,om/IWillMountisalsoguaranteedtobecalledonceinthecomponentlifecycle.ItiscalledwhenitisabouttobemountedintotheDOMandistheperfectplacetoinstallthego-loopintoourchannel.
TipWhencreatingcore.asyncchannelsinOmapplications,itisimportanttoavoidcreatingtheminsidelife-cyclefunctionsthatarecalledmultipletimes.Besidesnon-deterministicbehavior,thisisasourceofmemoryleaks.
Wegetholdofitfromthecomponentlocalstateusingtheom/get-statefunction.Oncewegetamessage,wetransactthestate.Wewillseewhattransfer-datalookslikeveryshortly.
Wecompletethecomponentbyimplementingitsrenderfunction:
...
om/IRenderState
(render-state[thisstate]
(dom/divnil
(applydom/ulnil
(om/build-allcolumn-view(:columnsapp)
{:shared{:cards(:cardsapp)}
:init-statestate}))))
...
Theom/IRenderStatefunctionservesthesamepurposeofom/IRender,thatis,itshouldreturntheDOMrepresentationofwhatthecomponentshouldlooklike.However,itdefinesadifferentfunction,render-state,whichreceivesthecomponentlocalstateasitssecondargument.Thisstatecontainsthemapwecreatedduringtheinit-statephase.
RemainingcomponentsNext,wewillbuildmultiplecolumn-viewcomponents,onepercolumn.Eachofthemreceivesthelistofcardsfromtheapplicationstateastheirsharedstate.WewillusethattoretrievethecarddetailsfromtheIDswestoreineachcolumn.
Wealsousethe:init-statekeytoinitializethelocalstateofeachcolumnviewwithourchannel,sinceallcolumnsneedareferencetoit.Here’swhatthecomponentlookslike:
(defncolumn-view[{:keys[titlecards]}owner]
(reify
om/IRenderState
(render-state[this{:keys[transfer-chan]}]
(dom/div#js{:style#js{:border"1pxsolidblack"
:float"left"
:height"100%"
:width"320px"
:padding"10px"}
:onDragOver#(.preventDefault%)
:onDrop#(handle-drop%transfer-chantitle)}
(dom/h2niltitle)
(applydom/ul#js{:style#js{:list-style-type"none"
:padding"0px"}}
(om/build-all(partialcard-viewtitle)
(mapvcard-by-idcards)))))))
Thecodeshouldlookfairlyfamiliaratthispoint.WeusedinlineCSSintheexampletokeepitsimple,butinarealapplication,wewouldprobablyhaveusedanexternalstylesheet.
Weimplementrender-stateoncemoretoretrievethetransferchannel,whichwillbeusedwhenhandlingtheonDropJavaScriptevent.ThiseventisfiredbythebrowserwhenauserdropsadraggableDOMelementontothiscomponent.handle-droptakescareofthatlikeso:
(defnhandle-drop[etransfer-chancolumn-title]
(.preventDefaulte)
(let[data{:card-id
(js/parseInt(get-transfer-data!e"cardId"))
:source-column
(get-transfer-data!e"sourceColumn")
:destination-column
column-title}]
(put!transfer-chandata)))
Thisfunctioncreatesthetransferdata—amapwiththekeys:card-id,:source-column,and:destination-column—whichiseverythingweneedtomovethecardsbetweencolumns.Finally,weput!itintothetransferchannel.
Next,webuildanumberorcard-viewcomponents.Asmentionedpreviously,Omcan’tcreatecursorsfromlazysequences,soweusefiltervtogiveeachcard-viewavectorcontainingtheirrespectivecards.Let’sseeitssource:
(defncard-view[column{:keys[idtitledescription]:ascard}owner]
(reify
om/IRender
(render[this]
(dom/li#js{:style#js{:border"1pxsolidblack"}
:draggabletrue
:onDragStart(fn[e]
(set-transfer-data!e"cardId"id)
(set-transfer-data!e"sourceColumn"
column))}
(dom/spanniltitle)
(dom/pnildescription)))))
Asthiscomponentdoesn’tneedanylocalstate,wegobacktousingtheIRenderprotocol.Additionally,wemakeitdraggableandinstallaneventhandlerontheonDragStartevent,whichwillbetriggeredwhentheuserstartsdraggingthecard.
Thiseventhandlersetsthetransferdata,whichweusefromhandle-drop.
Wehaveglossedoverthefactthatthesecomponentsuseafewutilityfunctions.That’sOK,aswewillnowdefinetheminanewnamespace.
UtilityfunctionsGoaheadandcreateanewfileundersrc/cljs/om_pm/calledutil.cljsandaddthefollowingnamespacedeclaration:
(nsom-pm.util)
Forconsistency,wewilllookatthefunctionsbottom-up,startingwithmove-card!:
(defncolumn-idx[titlecolumns]
(first(keep-indexed(fn[idxcolumn]
(when(=title(:titlecolumn))
idx))
columns)))
(defnmove-card![columns{:keys[card-idsource-columndestination-
column]}]
(let[from(column-idxsource-columncolumns)
to(column-idxdestination-columncolumns)]
(->columns
(update-in[from:cards](fn[cards]
(remove#{card-id}cards)))
(update-in[to:cards](fn[cards]
(conjcardscard-id))))))
Themove-card!functionreceivesacursorforthecolumnsinourapplicationstateandsimplymovescard-idbetweenthesourceanddestination.Youwillnoticewedidn’tneedanyaccesstocore.asyncorOmspecificfunctions,whichmeansthisfunctionispureandthereforeeasytotest.
Next,wehavethefunctionsthathandletransferdata:
(defnset-transfer-data![ekeyvalue]
(.setData(->e.-nativeEvent.-dataTransfer)
keyvalue))
(defnget-transfer-data![ekey]
(->(->e.-nativeEvent.-dataTransfer)
(.getDatakey)))
ThesefunctionsuseJavaScriptinteroperabilitytointeractwithHTML’sDataTransfer(seehttps://developer.mozilla.org/en-US/docs/Web/API/DataTransfer)object.Thisishowbrowserssharedatarelatedtodraganddropevents.
Now,let’ssimplysavethefileandmakesurethecodecompilesproperly.Wecanfinallyopendev-resources/public/index.htmlinthebrowserandplayaroundwiththeproductofourwork!
ExercisesInthisexercise,wewillmodifytheom-pmprojectwecreatedintheprevioussection.Theobjectiveistoaddkeyboardshortcutssothatpoweruserscanoperatetheagileboardmoreefficiently.
Theshortcutstobesupportedare:
Theup,down,left,andrightarrowkeys:Theseallowtheusertonavigatethroughthecards,highlightingthecurrentoneThenandpkeys:Theseareusedtomovethecurrentcardtothenext(right)orprevious(left)column,respectively
Thekeyinsighthereistocreateanewcore.asyncchannel,whichwillcontainkeypressevents.Theseeventswillthentriggertheactionsoutlinedpreviously.WecanusetheGoogleclosurelibrarytolistenforevents.Justaddthefollowingrequiretotheapplicationnamespace:
(:require[goog.events:asevents])
Then,usethisfunctiontocreateachannelfromDOMevents:
(defnlisten[eltype]
(let[c(chan)]
(events/listeneltype#(put!c%))
c))
Theactuallogicofmovingthecardsaroundbasedonkeyboardshortcutscanbeimplementedinanumberofways,sodon’tforgettocompareyoursolutionwiththeanswersprovidedinthisbook’scompanioncode.
SummaryInthischapter,wesawadifferentapproachonhowtohandlereactivewebinterfacesbyOmandReact.Inturn,theseframeworksmakethispossibleandpainlessbyapplyingfunctionalprogrammingprinciplessuchasimmutabilityandpersistentdatastructuresforefficientrendering.
WealsolearnedtothinktheOmwaybystructuringourapplicationsasaseriesoffunctions,whichreceivestateandoutputaDOMrepresentationofstatechanges.
Additionally,wesawthatbystructuringapplicationstatetransitionsthroughcore.asyncchannels,weseparatethepresentationlogicfromthecode,whichwillactuallyperformthework,makingourcomponentseveneasiertoreasonabout.
Inthenextchapter,wewillturntoanoftenoverlookedyetusefultoolforcreatingreactiveapplications:Futures.
Chapter8.FuturesThefirststeptowardsreactiveapplicationsistobreakoutofsynchronousprocessing.Ingeneral,applicationswastealotoftimewaitingforthingstohappen.Maybewearewaitingonanexpensivecomputation—say,calculatingthe1000thFibonaccinumber.Perhapswearewaitingforsomeinformationtobewrittentothedatabase.Wecouldalsobewaitingforanetworkcalltoreturn,bringingusthelatestrecommendationsfromourfavoriteonlinestore.
Regardlessofwhatwe’rewaitingfor,weshouldneverblockclientsofourapplication.Thisiscrucialtoachievetheresponsivenesswedesirewhenbuildingreactivesystems.
Inanagewhereprocessingcoresareabundant—myMacBookProhaseightprocessorcores—blockingAPIsseverelyunderutilizestheresourceswehaveatourdisposal.
Asweapproachtheendofthisbook,itisappropriatetostepbackalittleandappreciatethatnotallclassesofproblemsthatdealwithconcurrent,asynchronouscomputationsrequirethemachineryofframeworkssuchasRxJavaorcore.async.
Inthischapter,wewilllookatanotherabstractionthathelpsusdevelopconcurrent,asynchronousapplications:futures.Wewilllearnabout:
TheproblemsandlimitationswithClojure’simplementationoffuturesAnalternativetoClojure’sfuturesthatprovidesasynchronous,composablesemanticsHowtooptimizeconcurrencyinthefaceofblockingIO
ClojurefuturesThefirststeptowardfixingthisissue—thatis,topreventapotentiallylong-runningtaskfromblockingourapplication—istocreatenewthreads,whichdotheworkandwaitforittocomplete.Thisway,wekeeptheapplication’smainthreadfreetoservemoreclients.
Workingdirectlywiththreads,however,istediousanderror-prone,soClojure’scorelibraryincludesfutures,whichareextremelysimpletouse:
(deff(clojure.core/future
(println"doingsomeexpensivework…")
(Thread/sleep5000)
(println"done")
10))
(println"You'llseemebeforethefuturefinishes")
;;doingsomeexpensivework…
;;You'llseemebeforethefuturefinishes
;;done
Intheprecedingsnippet,weinvoketheclojure.core/futuremacrowithabodysimulatinganexpensivecomputation.Inthisexample,itsimplysleepsfor5secondsbeforereturningthevalue10.Astheoutputdemonstrates,thisdoesnotblockthemainthread,whichisfreetoservemoreclients,pickworkitemsfromaqueue,orwhathaveyou.
Ofcourse,themostinterestingcomputations,suchastheexpensiveone,returnresultswecareabout.ThisiswherethefirstlimitationofClojurefuturesbecomesapparent.Ifweattempttoretrievetheresultofafuture—byderefingit—beforeithascompleted,thecallingthreadwillblockuntilthefuturereturnsavalue.Tryrunningthefollowingslightlymodifiedversionoftheprevioussnippet:
(deff(clojure.core/future
(println"doingsomeexpensivework…")
(Thread/sleep5000)
(println"done")
10))
(println"You'llseemebeforethefuturefinishes")
@f
(println"Icouldbedoingsomethingelse.InsteadIhadtowait")
;;doingsomeexpensivework…
;;You'llseemebeforethefuturefinishes
;;5SECONDSLATER
;;done
;;Icouldbedoingsomethingelse.Instead,Ihadtowait
Theonlydifferencenowisthatweimmediatelytrytoderefthefutureafterwecreateit.Sincethefutureisn’tdone,wesittherewaitingfor5secondsuntilitreturnsitsvalue.Onlythenisourprogramallowedtocontinue.
Ingeneral,thisposesaproblemwhenbuildingmodularsystems.Often,along-running
operationliketheonedescribedearlierwouldbeinitiatedwithinaspecificmoduleorfunction,andhandedovertothenextlogicalstepforfurtherprocessing.
Clojurefuturesdon’tallowustoscheduleafunctiontobeexecutedwhenthefuturefinishesinordertoperformsuchfurtherprocessing.Thisisanimportantfeatureinbuildingreactivesystems.
FetchingdatainparallelTounderstandbettertheissuesoutlinedintheprevioussection,let’sbuildamorecomplexexamplethatfetchesdataaboutoneofmyfavoritemovies,TheLordoftheRings.
Theideaisthatgiventhemovie,wewishtoretrieveitsactorsand,foreachactor,retrievethemoviestheyhavebeenapartof.Wealsowouldliketofindoutmoreinformationabouteachactor,suchastheirspouses.
Additionally,wewillmatcheachactor’smovieagainstthelistoftopfivemoviesinordertohighlightthemassuch.Finally,theresultwillbeprintedtothescreen.
Fromtheproblemstatement,weidentifythefollowingtwomaincharacteristicswewillneedtoaccountfor:
SomeofthesetasksneedtobeperformedinparallelTheyestablishdependenciesoneachother
Togetstarted,let’screateanewleiningenproject:
leinnewclj-futures-playground
Next,openthecorenamespacefileinsrc/clj_futures_playground/core.cljandaddthedatawewillbeworkingwith:
(nsclj-futures-playground.core
(:require[clojure.pprint:refer[pprint]]))
(defmovie
{:name"LordofTheRings:TheFellowshipofTheRing"
:cast["CateBlanchett"
"ElijahWood"
"LivTyler"
"OrlandoBloom"]})
(defactor-movies
[{:name"CateBlanchett"
:movies["LordofTheRings:TheFellowshipofTheRing"
"LordofTheRings:TheReturnofTheKing"
"TheCuriousCaseofBenjaminButton"]}
{:name"ElijahWood"
:movies["EternalSunshineoftheSpotlessMind"
"GreenStreetHooligans"
"TheHobbit:AnUnexpectedJourney"]}
{:name"LivTyler"
:movies["LordofTheRings:TheFellowshipofTheRing"
"LordofTheRings:TheReturnofTheKing"
"Armageddon"]}
{:name"OrlandoBloom"
:movies["LordofTheRings:TheFellowshipofTheRing"
"LordofTheRings:TheReturnofTheKing"
"PiratesoftheCaribbean:TheCurseoftheBlackPearl"]}])
(defactor-spouse
[{:name"CateBlanchett":spouse"AndrewUpton"}
{:name"ElijahWood":spouse"Unknown"}
{:name"LivTyler":spouse"RoystonLangdon"}
{:name"OrlandoBloom":spouse"MirandaKerr"}])
(deftop-5-movies
["LordofTheRings:TheFellowshipofTheRing"
"TheMatrix"
"TheMatrixReloaded"
"PiratesoftheCaribbean:TheCurseoftheBlackPearl"
"Terminator"])
Thenamespacedeclarationissimpleandonlyrequiresthepprintfunction,whichwillhelpusprintourresultinaneasy-to-readformat.Withallthedatainplace,wecancreatethefunctionsthatwillsimulateremoteservicesresponsibleforfetchingtherelevantdata:
(defncast-by-movie[name]
(future(do(Thread/sleep5000)
(:castmovie))))
(defnmovies-by-actor[name]
(do(Thread/sleep2000)
(->>actor-movies
(filter#(=name(:name%)))
first)))
(defnspouse-of[name]
(do(Thread/sleep2000)
(->>actor-spouse
(filter#(=name(:name%)))
first)))
(defntop-5[]
(future(do(Thread/sleep5000)
top-5-movies)))
Eachservicefunctionsleepsthecurrentthreadbyagivenamountoftimetosimulateaslownetwork.Thefunctionscast-by-movieandTop5eachreturnsafuture,indicatingwewishtofetchthisdataonadifferentthread.Theremainingfunctionssimplyreturntheactualdata.Theywillalsobeexecutedinadifferentthread,however,aswewillseeshortly.
Thenextthingweneedisafunctiontoaggregateallfetcheddata,matchspousestoactors,andhighlightmoviesintheTop5list.We’llcallittheaggregate-actor-datafunction:
(defnaggregate-actor-data[spousesmoviestop-5]
(map(fn[{:keys[namespouse]}{:keys[movies]}]
{:namename
:spousespouse
:movies(map(fn[m]
(if(some#{m}top-5)
(strm"-(top5)")
m))
movies)})
spouses
movies))
Theprecedingfunctionisfairlystraightforward.Itsimplyzipsspousesandmoviestogether,buildingamapofkeys:name,:spouse,and:movies.ItfurthertransformsmoviestoappendtheTop5suffixtotheonesinthetop-5list.
Thelastpieceofthepuzzleisthe-mainfunction,whichallowsustoruntheprogramfromthecommandline:
(defn-main[&args]
(time(let[cast(cast-by-movie"LordofTheRings:TheFellowshipof
TheRing")
movies(pmapmovies-by-actor@cast)
spouses(pmapspouse-of@cast)
top-5(top-5)]
(prn"Fetchingdata…")
(pprint(aggregate-actor-dataspousesmovies@top-5))
(shutdown-agents))))
Thereareanumberofthingsworthhighlightingintheprecedingsnippet.
First,wewrapthewholebodyinacalltotime,asimplebenchmarkingfunctionthatcomeswithClojure.Thisisjustsoweknowhowlongtheprogramtooktofetchalldata—thisinformationwillbecomerelevantlater.
Then,wesetupanumberofletbindings.Thefirst,cast,istheresultofcallingcast-by-movie,whichreturnsafuture.
Thenextbinding,movies,usesafunctionwehaven’tseenbefore:pmap.
Thepmapfunctionworkslikemap,exceptthefunctionismappedovertheitemsinthelistinparallel.Thepmapfunctionusesfuturesunderthecoversandthatisthereasonmovies-by-actordoesn’treturnafuture—itleavesthatforpmaptohandle.
TipThepmapfunctionisactuallymeantforCPU-boundoperations,butisusedheretokeepthecodesimple.InthefaceofblockingIO,pmapwouldn’tperformoptimally.WewilltalkmoreaboutblockingIOlaterinthischapter.
Wegetthelistofactorsbyderefingthecastbinding,which,aswesawintheprevioussection,blocksthecurrentthreadwaitingfortheasynchronousfetchtofinish.Onceallresultsareready,wesimplycalltheaggregate-actor-datafunction.
Lastly,wecalltheshutdown-agentsfunction,whichshutsdowntheThreadPoolbackingfuturesinClojure.Thisisnecessaryforourprogramtoterminateproperly,otherwiseitwouldsimplyhangintheterminal.
Toruntheprogram,typethefollowingintheterminal,undertheproject’srootdirectory:
leinrun-mclj-futures-playground.core
"Fetchingdata…"
({:name"CateBlanchett",
:spouse"AndrewUpton",
:movies
("LordofTheRings:TheFellowshipofTheRing-(top5)"
"LordofTheRings:TheReturnofTheKing"
"TheCuriousCaseofBenjaminButton")}
{:name"ElijahWood",
:spouse"Unknown",
:movies
("EternalSunshineoftheSpotlessMind"
"GreenStreetHooligans"
"TheHobbit:AnUnexpectedJourney")}
{:name"LivTyler",
:spouse"RoystonLangdon",
:movies
("LordofTheRings:TheFellowshipofTheRing-(top5)"
"LordofTheRings:TheReturnofTheKing"
"Armageddon")}
{:name"OrlandoBloom",
:spouse"MirandaKerr",
:movies
("LordofTheRings:TheFellowshipofTheRing-(top5)"
"LordofTheRings:TheReturnofTheKing"
"PiratesoftheCaribbean:TheCurseoftheBlackPearl-(top5)")})
"Elapsedtime:10120.267msecs"
Youwillhavenoticedthattheprogramtakesawhiletoprintthefirstmessage.Additionally,becausefuturesblockwhentheyarederefed,theprogramdoesn’tstartfetchingthelistoftopfivemoviesuntilithascompletelyfinishedfetchingthecastofTheLordofTheRings.
Let’shavealookatwhythatisso:
(time(let[cast(cast-by-movie"LordofTheRings:TheFellowshipof
TheRing")
;;thefollowinglineblocks
movies(pmapmovies-by-actor@cast)
spouses(pmapspouse-of@cast)
top-5(top-5)]
Thehighlightedsectionintheprecedingsnippetshowswheretheprogramblockswaitingforcast-by-movietofinish.Asstatedpreviously,Clojurefuturesdon’tgiveusawaytorunsomepieceofcodewhenthefuturefinishes—likeacallback—forcingustoblocktoosoon.
Thispreventstop-5—acompletelyindependentparalleldatafetch—fromrunningbeforeweretrievethemovie’scast.
Ofcourse,thisisacontrivedexample,andwecouldsolvethisparticularannoyancebycallingtop-5beforeanythingelse.Theproblemisthatthesolutionisn’talwayscrystalclearandideallyweshouldnothavetoworryabouttheorderofexecution.
Aswewillseeinthenextsection,thereisabetterway.
Imminent–acomposablefutureslibraryforClojureInthepastfewmonths,IhavebeenworkingonanopensourcelibrarythataimstofixthepreviousissueswithClojurefutures.Theresultofthisworkiscalledimminent(seehttps://github.com/leonardoborges/imminent).
Thefundamentaldifferenceisthatimminentfuturesareasynchronousbydefaultandprovideanumberofcombinatorsthatallowustodeclarativelywriteourprogramswithouthavingtoworryaboutitsorderofexecution.
Thebestwaytodemonstratehowthelibraryworksistorewritethepreviousmoviesexampleinit.Wewilldothisintwosteps.
First,wewillexamineindividuallythebitsofimminent’sAPIthatwillbepartofourfinalsolution.Then,we’llputitalltogetherinaworkingapplication.Let’sstartbycreatinganewproject:
leinnewimminent-playground
Next,addadependencyonimminenttoyourproject.clj:
:dependencies[[org.clojure/clojure"1.6.0"]
[com.leonardoborges/imminent"0.1.0"]]
Then,createanewfile,src/imminent_playground/repl.clj,andaddimminent’scorenamespace:
(nsimminent-playground.repl
(:require[imminent.core:asIi]))
(defrepl-out*out*)
(defnprn-to-repl[&args]
(binding[*out*repl-out]
(applyprnargs)))
Theprecedingsnippetalsocreatesahelperfunctionthatisusefulwhenwe’redealingwithmultiplethreadsintheREPL—thiswillbeexplainedindetaillater,butfornowjusttakethisasbeingareliablewaytoprinttotheREPLacrossmultiplethreads.
FeelfreetotypethisintheREPLaswegoalong.Otherwise,youcanrequirethenamespacefilefromarunningREPLlikeso:
(require'imminent-playground.repl)
Allthefollowingexamplesshouldbeinthisfile.
CreatingfuturesCreatingafutureinimminentisn’tmuchdifferentfromcreatingafutureinClojure.It’sassimpleasthefollowing:
(defage(i/future31))
;;#<Future@2ea0ca7d:#<Success@3e4dec75:31>>
Whatlooksverydifferent,however,isthereturnvalue.Akeydecisioninimminent’sAPIistorepresentthevalueofacomputationaseitheraSuccessoraFailuretype.Success,asintheprecedingexample,wrapstheresultofthecomputation.Failure,asyoumighthaveguessed,willwrapanyexceptionsthathappenedinthefuture:
(deffailed-computation(i/future(throw(Exception."Error"))))
;;#<Future@63cd0d58:#<Failure@2b273f98:#<Exceptionjava.lang.Exception:
Error>>>
(deffailed-computation-1(i/failed-future:invalid-data))
;;#<Future@a03588f:#<Failure@61ab196b::invalid-data>>
Asyoucansee,you’renotlimitedtoexceptionsonly.Wecanusethefailed-futurefunctiontocreateafuturethatcompletesimmediatelywiththegivenreason,which,inthesecondexample,issimplyakeyword.
Thenextquestionwemightaskis“Howdowegettheresultoutofafuture?”.AswithClojurefutures,wecanderefitasfollows:
@age;;#<Success@3e4dec75:31>
(deref@age);;31
(i/dderefage);;31
Theidiomofusingadouble-derefiscommon,soimminentprovidestheconvenienceshown,dderef,whichisequivalenttocallingdereftwice.
However,differentfromClojurefutures,thisisanon-blockingoperation,soifthefuturehasn’tcompletedyet,thefollowingiswhatyou’llget:
@(i/future(do(Thread/sleep500)
"hello"))
;;:imminent.future/unresolved
Theinitialstateofafutureisunresolved,sounlessyouareabsolutelycertainafuturehascompleted,derefingmightnotbethebestwaytoworkwiththeresultofacomputation.Thisiswherecombinatorsbecomeuseful.
CombinatorsandeventhandlersLet’ssaywewouldliketodoublethevalueintheagefuture.Aswewouldwithlists,wecansimplymapafunctionoverthefuturetodojustthis:
(defdouble-age(i/mapage#(*%2)))
;;#<Future@659684cb:#<Success@7ce85f87:62>>
TipWhilei/futureschedulesitsbodyforexecutiononaseparatethread,it’sworthnotingthatfuturecombinatorssuchasmap,filter,andsoon,donotcreateanewthreadimmediately.Instead,theyscheduleafunctiontobeexecutedasynchronouslyinthethreadpooloncetheoriginalfuturecompletes.
Anotherwaytodosomethingwiththevalueofafutureistousetheon-successeventhandlerthatgetscalledwiththewrappedvalueofthefutureincaseitissuccessful:
(i/on-successage#(prn-to-repl(str"Ageis:"%)))
;;"Ageis:31"
Similarly,anon-failurehandlerexists,whichdoesthesameforFailuretypes.Whileonthesubjectoffailures,imminentfuturesunderstandthecontextinwhichtheyarebeingexecutedand,ifthecurrentfutureyieldsaFailure,itsimplyshort-circuitsthecomputation:
(->failed-computation
(i/map#(*%2)))
;;#<Future@7f74297a:#<Failure@2b273f98:#<Exceptionjava.lang.Exception:
Error>>>
Intheprecedingexample,wedon’tgetanewerror,butrathertheoriginalexceptioncontainedinfailed-computation.Thefunctionpassedtomapneverruns.
ThedecisiontowraptheresultofafutureinatypesuchasSuccessorFailuremightseemarbitrarybutisactuallyquitetheopposite.BothtypesimplementtheprotocolIReturn—andacoupleofotherones—whichcomeswithasetofusefulfunctions,oneofwhichismap:
(i/map(i/success"hello")
#(str%"world"))
;;#<Success@714eea92:"helloworld">
(i/map(i/failure"error")
#(str%"world"))
;;#<Failure@6d685b65:"error">
Wegetasimilarbehaviorhereaswedidpreviously:mappingafunctionoverafailuresimplyshort-circuitsthewholecomputation.Ifyoudo,however,wishtomapoverthefailure,youcanusemap’scounterpartmap-failure,whichbehavessimilarlytomapbutisitsinverse:
(i/map-failure(i/success"hello")
#(str%"world"))
;;#<Success@779af3f4:"hello">
(i/map-failure(i/failure"Error")
#(str"Wefailed:"%))
;;#<Failure@52a02597:"Wefailed:Error">
Thisplayswellwiththelasteventhandlersimminentprovides—on-complete:
(i/on-completeage
(fn[result]
(i/mapresult#(prn-to-repl"success:"%))
(i/map-failureresult#(prn-to-repl"error:"%))))
;;"success:"31
Oncontrarytoon-successandon-failure,on-completecallstheprovidedfunctionwiththeresulttypewrapper,soitisaconvenientwaytohandlebothcasesinasinglefunction.
Comingbacktocombinators,sometimeswewillneedtomapafunctionoverafuture,whichitselfreturnsafuture:
(defnrange-future[n]
(i/const-future(rangen)))
(defage-range(i/mapagerange-future))
;;#<Future@3d24069e:#<Success@82e8e6e:#<Future@2888dbf4:#
<Success@312084f6:(012…)>>>>
Therange-futurefunctionreturnsasuccessfulfuturethatyieldsarangeofn.Theconst-futurefunctionisanalogoustofailed-future,exceptitimmediatelycompletesthefuturewithaSuccesstype.
However,weendupwithanestedfuture,whichisalmostneverwhatyouwant.That’sOK.Thisispreciselythescenarioinwhichyouwoulduseanothercombinator,flatmap.
Youcanthinkofitasmapcatforfutures—itflattensthecomputationforus:
(defage-range(i/flatmapagerange-future))
;;#<Future@601c1dfc:#<Success@55f4bcaf:(012…)>>
Anotherveryusefulcombinatorisusedtobringtogethermultiplecomputationstobeusedinasinglefunction—sequence:
(defname(i/future(do(Thread/sleep500)
"Leo")))
(defgenres(i/future(do(Thread/sleep500)
["HeavyMetal""BlackMetal""DeathMetal""Rock
'nRoll"])))
(->(i/sequence[nameagegenres])
(i/on-success
(fn[[nameagegenres]]
(prn-to-repl(format"%sis%syearsoldandenjoys%s"
name
age
(clojure.string/join","genres))))))
;;"Leois31yearsoldandenjoysHeavyMetal,BlackMetal,DeathMetal,Rock
'nRoll"
Essentially,sequencecreatesanewfuture,whichwillcompleteonlywhenallotherfuturesinthevectorhavecompletedoranyoneofthemhavefailed.
Thisisanicesegueintothelastcombinatorwewilllookat—map-future—whichwewoulduseinplaceofpmap,usedinthemoviesexample:
(defncalculate-double[n]
(i/const-future(*n2)))
(->(i/map-futurecalculate-double[1234])
i/await
i/dderef)
;;[2468]
Intheprecedingexample,calculate-doubleisafunctionthatreturnsafuturewiththevaluendoubled.Themap-futurefunctionthenmapscalculate-doubleoverthelist,effectivelyperformingthecalculationsinparallel.Finally,map-futuresequencesallfuturestogether,returningasinglefuture,whichyieldstheresultofallcomputations.
Becauseweareperforminganumberofparallelcomputationsanddon’treallyknowwhentheywillfinish,wecallawaitonthefuture,whichisawaytoblockthecurrentthreaduntilitsresultisready.Ingeneral,youwouldusethecombinatorsandeventhandlersinstead,butforthisexample,usingawaitisacceptable.
Imminent’sAPIprovidesmanymorecombinators,whichhelpuswriteasynchronousprogramsinadeclarativeway.ThissectiongaveusatasteofwhatispossiblewiththeAPIandisenoughtoallowustowritethemoviesexampleusingimminentfutures.
ThemoviesexamplerevisitedStillwithinourimminent-playgroundproject,openthesrc/imminent_playground/core.cljfileandaddtheappropriatedefinitions:
(nsimminent-playground.core
(:require[clojure.pprint:refer[pprint]]
[imminent.core:asi]))
(defmovie…)
(defactor-movies…)
(defactor-spouse…)
(deftop-5-movies…)
Wewillbeusingthesamedataasinthepreviousprogram,representedintheprecedingsnippetbytheuseofellipses.Simplycopytherelevantdeclarationsover.
Theservicefunctionswillneedsmalltweaksinthisnewversion:
(defncast-by-movie[name]
(i/future(do(Thread/sleep5000)
(:castmovie))))
(defnmovies-by-actor[name]
(i/future(do(Thread/sleep2000)
(->>actor-movies
(filter#(=name(:name%)))
first))))
(defnspouse-of[name]
(i/future(do(Thread/sleep2000)
(->>actor-spouse
(filter#(=name(:name%)))
first))))
(defntop-5[]
(i/future(do(Thread/sleep5000)
top-5-movies)))
(defnaggregate-actor-data[spousesmoviestop-5]
...)
Themaindifferenceisthatallofthemnowreturnanimminentfuture.Theaggregate-actor-datafunctionisalsothesameasbefore.
Thisbringsustothe-mainfunction,whichwasrewrittentouseimminentcombinators:
(defn-main[&args]
(time(let[cast(cast-by-movie"LordofTheRings:TheFellowshipof
TheRing")
movies(i/flatmapcast#(i/map-futuremovies-by-actor%))
spouses(i/flatmapcast#(i/map-futurespouse-of%))
result(i/sequence[spousesmovies(top-5)])]
(prn"Fetchingdata…")
(pprint(applyaggregate-actor-data
(i/dderef(i/awaitresult)))))))
Thefunctionstartsmuchlikeitspreviousversion,andeventhefirstbinding,cast,looksfamiliar.Nextwehavemovies,whichisobtainedbyfetchinganactor’smoviesinparallel.Thisinitselfreturnsafuture,soweflatmapitoverthecastfuturetoobtainourfinalresult:
movies(i/flatmapcast#(i/map-futuremovies-by-actor%))
spousesworksinexactlythesamewayasmovies,whichbringsustoresult.Thisiswherewewouldliketobringallasynchronouscomputationstogether.Therefore,weusethesequencecombinator:
result(i/sequence[spousesmovies(top-5)])
Finally,wedecidetoblockontheresultfuture—byusingawait—sowecanprintthefinalresult:
(pprint(applyaggregate-actor-data
(i/dderef(i/awaitresult)))
Weruntheprograminthesamewayasbefore,sosimplytypethefollowinginthecommandline,undertheproject’srootdirectory:
leinrun-mimminent-playground.core
"Fetchingdata…"
({:name"CateBlanchett",
:spouse"AndrewUpton",
:movies
("LordofTheRings:TheFellowshipofTheRing-(top5)"
"LordofTheRings:TheReturnofTheKing"
"TheCuriousCaseofBenjaminButton")}
...
"Elapsedtime:7088.398msecs"
Theresultoutputwastrimmedasitisexactlythesameasbefore,buttwothingsaredifferentanddeserveattention:
Thefirstoutput,Fetchingdata…,isprintedtothescreenalotfasterthanintheexampleusingClojurefuturesTheoveralltimeittooktofetchallthatisshorter,clockinginatjustover7seconds
Thishighlightstheasynchronousnatureofimminentfuturesandcombinators.Theonlytimewehadtowaitiswhenweexplicitlycalledawaitattheendoftheprogram.
Morespecifically,theperformanceboostcomesfromthefollowingsectioninthecode:
(let[...
result(i/sequence[spousesmovies(top-5)])]
...)
Becausenoneofthepreviousbindingsblockthecurrentthread,weneverhavetowaitto
kickofftop-5inparallel,shavingoffroughly3secondsfromtheoverallexecutiontime.Wedidn’thavetoexplicitlythinkabouttheorderofexecution—thecombinatorssimplydidtherightthing.
Finally,onelastdifferenceisthatwedidn’thavetoexplicitlycallshutdown-agentsasbefore.Thereasonforthisisthatimminentusesadifferenttypeofthreadpool:aForkJoinPool(seehttp://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ForkJoinPool.html).
Thispoolhasanumberofadvantages—eachwithitsowntrade-off—overtheotherthreadpools,andonecharacteristicisthatwedon’tneedtoexplicitlyshutitdown—allthreadsitcreatesdaemonthreads.
WhentheJVMshutsdown,ithangswaitingforallnon-daemonthreadstofinish.Onlythendoesitexit.That’swhyusingClojurefutureswouldcausetheJVMtohang,ifwehadnotcalledshutdown-agents.
AllthreadscreatedbytheForkJoinPoolaresetasdaemonthreadsbydefault:whentheJVMattemptstoshutdown,andiftheonlythreadsrunningaredaemonones,theyareabandonedandtheJVMexitsgracefully.
Combinatorssuchasmapandflatmap,aswellasthefunctionssequenceandmap-future,aren’texclusivetofutures.Theyhavemanymorefundamentalprinciplesbywhichtheyabide,makingthemusefulinarangeofdomains.Understandingtheseprinciplesisn’tnecessaryforfollowingthecontentsofthisbook.Shouldyouwanttoknowmoreabouttheseprinciples,pleaserefertotheAppendix,TheAlgebraofLibraryDesign.
FuturesandblockingIOThechoiceofusingForkJoinPoolforimminentisdeliberate.TheForkJoinPool—addedonJava7—isextremelysmart.Whencreated,yougiveitadesiredlevelofparallelism,whichdefaultstothenumberofavailableprocessors.
ForkJoinPoolthenattemptstohonorthedesiredparallelismbydynamicallyshrinkingandexpandingthepoolasrequired.Whenataskissubmittedtothispool,itdoesn’tnecessarilycreateanewthreadifitdoesn’thaveto.Thisallowsthepooltoserveanextremelylargenumberoftaskswithamuchsmallernumberofactualthreads.
However,itcannotguaranteesuchoptimizationsinthefaceofblockingIO,asitcan’tknowwhetherthethreadisblockingwaitingforanexternalresource.Nevertheless,ForkJoinPoolprovidesamechanismbywhichthreadscannotifythepoolwhentheymightblock.
ImminenttakesadvantageofthismechanismbyimplementingtheManagedBlocker(seehttp://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ForkJoinPool.ManagedBlocker.htmlinterface—andprovidesanotherwaytocreatefutures,asdemonstratedhere:
(->(immi/blocking-future
(Thread/sleep100)
10)
(immi/await))
;;#<Future@4c8ac77a:#<Success@45525276:10>>
(->(immi/blocking-future-call
(fn[]
(Thread/sleep100)
10))
(immi/await))
;;#<Future@37162438:#<Success@5a13697f:10>>
Theblocking-futureandblocking-future-callhavethesamesemanticsastheircounterparts,futureandfuture-call,butshouldbeusedwhenthetasktobeperformedisofablockingnature(thatis,notCPU-bound).ThisallowstheForkJoinPooltobetterutilizeitsresources,makingitapowerfulandflexiblesolution.
SummaryInthischapter,welearnedthatClojurefuturesleavealottobedesired.Morespecifically,Clojurefuturesdon’tprovideawaytoexpressdependenciesbetweenresults.Itdoesn’tmean,however,thatweshoulddismissfuturesaltogether.
Theyarestillausefulabstractionandwiththerightsemanticsforasynchronouscomputationsandarichsetofcombinators—suchastheonesprovidedbyimminent—theycanbeabigallyinbuildingreactiveapplicationsthatareperformantandresponsive.Sometimes,thisisallweneed.
Forthetimeswhereweneedtomodeldatathatvariesovertime,weturntoricherframeworksinspiredbyFunctionalReactiveProgramming(FRP)andCompositionalEventSystems(CES)—suchasRxJava—orCommunicatingSequentialProcesses(CSP)—suchascore.async.Astheyhavealotmoretooffer,muchofthisbookhasbeendedicatedtothoseapproaches.
Inthenextchapter,wewillgobacktodiscussingFRP/CESbywayofacasestudy.
Chapter9.AReactiveAPItoAmazonWebServicesThroughoutthisbook,wehavelearnedanumberoftoolsandtechniquestoaidusinbuildingreactiveapplications—futureswithimminent,ObservableswithRxClojure/RxJava,channelswithcore.async—andeveninbuildingreactiveuserinterfacesusingOmandReact.
Intheprocess,wealsobecameacquaintedwiththeconceptofFunctionalReactiveProgrammingandCompositionalEventSystems,aswellaswhatmakesthemdifferent.
Inthislastchapter,wewillbringafewofthesedifferenttoolsandconceptstogetherbydevelopinganapplicationbasedonareal-worldusecasefromaclientIworkedwithinSydney,Australia.Wewill:
DescribetheproblemofinfrastructureautomationweweretryingtosolveHaveabrieflookatsomeofAmazon’sAWSservicesBuildanAWSdashboardusingtheconceptswehavelearnedsofar
TheproblemThisclient—whichwewillcallBubbleCorpfromnowon—hadabigproblemthatisalltoocommonandwellknowntobigenterprises:onemassivemonolithicapplication.
Besidesmakingthemmoveslow,asindividualcomponentscan’tbeevolvedindependently,thisapplicationmakesdeploymentincrediblyhardduetoitsenvironmentconstraints:allinfrastructureneedstobeavailableinorderfortheapplicationtoworkatall.
Asaresult,developingnewfeaturesandbugfixesinvolveshavingonlyahandfulofdevelopmentenvironmentssharedacrossdozensofdeveloperseach.Thisrequiresawastefulamountofcoordinationbetweenteamsjustsothattheywon’tsteponeachother’stoes,contributingtoslowthewholelife-cyclefurther.
Thelong-termsolutiontothisproblemistobreakdownthisbigapplicationintosmallercomponents,whichcanbedeployedandworkedonindependently,butasgoodasthissounds,it’salaboriousandlengthyprocess.
Asafirststep,BubbleCorpdecidedthebestthingtheycouldimproveintheshorttermistogivedeveloperstheabilitytoworkintheapplicationindependentlyfromeachother,whichimpliesbeingabletocreateanewenvironmentaswell.
Giventheinfrastructureconstraints,runningtheapplicationonasingledevelopermachineisprohibitive.
Instead,theyturnedtoinfrastructureautomation:theywantedatoolthat,withthepressofabutton,wouldspinupacompletelynewenvironment.
Thisnewenvironmentwouldbealreadypreconfiguredwiththeproperapplicationservers,databaseinstances,DNSentries,andeverythingelseneededtoruntheapplication.
Thisway,developerswouldonlyneedtodeploytheircodeandtesttheirchanges,withouthavingtoworryabouttheapplicationsetup.
InfrastructureautomationAmazonWebServices(AWS)isthemostmatureandcomprehensivecloudcomputingplatformavailabletoday,andassuchitwasanaturalchoiceforBubbleCorptohostitsinfrastructurein.
Ifyouhaven’tusedAWSbefore,don’tworry,we’llfocusonlyonafewofitsservices:
ElasticComputeCloud(EC2):Aservicethatprovidesuserswiththeabilitytorentvirtualcomputersinwhichtoruntheirapplications.RelationalDatabaseService(RDS):ThiscanbethoughtofasaspecializedversionofEC2thatprovidesmanageddatabaseservices.CloudFormation:WithCloudFormation,usershavetheabilitytospecifyinfrastructuretemplates,calledstacks,ofseveraldifferentAWSresources—suchasEC2,AWS,andmanyothers—aswellashowtheyinteractwitheachother.Oncewritten,theinfrastructuretemplatecanbesenttoAWStobeexecuted.
ForBubbleCorp,theideawastowritetheseinfrastructuretemplates,whichoncesubmittedwouldresultintoacompletelynew,isolatedenvironmentcontainingalldataandcomponentsrequiredtorunitsapp.Atanygiventime,therewouldbedozensoftheseenvironmentsrunningwithdevelopersworkingonthem.
Asdecentaplanasthissounds,bigcorporationsusuallyhaveanaddedburden:costcenters.Unfortunately,BubbleCorpcan’tsimplyallowdeveloperstologintotheAWSConsole—wherewecanmanageAWSresources—andspinupenvironmentsatwill.Theyneededawayto,amongotherthings,addcostcentermetadatatotheenvironmenttohandletheirinternalbillingprocess.
Thisbringsustotheapplicationwewillbefocusingonfortheremainderofthischapter.
AWSresourcesdashboardMyteamandIweretaskedwithbuildingaweb-baseddashboardforAWS.ThisdashboardwouldallowdeveloperstologinusingtheirBubbleCorp’scredentialsand,onceauthenticated,createnewCloudFormationenvironmentsaswellasvisualizethestatusofeachindividualresourcewithinaCloudFormationstack.
Theapplicationitselfisfairlyinvolved,sowewillfocusonasubsetofit:interfacingwiththenecessaryAWSservicesinordertogatherinformationaboutthestatusofeachindividualresourceinagivenCloudFormationstack.
Oncefinished,thisiswhatoursimplifieddashboardwilllooklike:
ItwilldisplaytheID,type,andcurrentstatusofeachresource.Thismightnotseemlikemuchfornow,butgiventhatallthisinformationiscomingfromdifferent,independentwebservices,itisfartooeasytoendupwithunnecessarilycomplexcode.
WewillbeusingClojureScriptforthisandthereforetheJavaScriptversionoftheAWSSDK,whosedocumentationcanbefoundathttp://aws.amazon.com/sdk-for-node-js/.
Beforewegetstarted,let’shavealookateachoftheAWSServicesAPIswewillbeinteractingwith.
TipInreality,wewillnotbeinteractingwiththerealAWSservicesbutratherastubserverprovidedfordownloadfromthebook’sGitHubrepository.
Thereasonforthisistomakefollowingthischaptereasier,asyouwon’tneedtocreateanaccountaswellasgenerateanAPIaccesskeytointeractwithAWS.
Additionally,creatingresourcesincurscost,andthelastthingIwantisforyoutobechargedhundredsofdollarsattheendofthemonthbecausesomeoneaccidentallyleftresourcesrunningforlongerthantheyshould—trustmeithashappenedbefore.
CloudFormationThefirstservicewewilllookatisCloudFormation.ThismakessenseastheAPIsfoundinherewillgiveusastartingpointforfindinginformationabouttheresourcesinagivenstack.
ThedescribeStacksendpointThisendpointisresponsibleforlistingallstacksassociatedwithaparticularAWSaccount.Foragivenstack,itsresponselookslikethefollowing:
{"Stacks"
[{"StackId"
"arn:aws:cloudformation:ap-southeast-2:337944750480:stack/DevStack-
62031/1",
"StackStatus""CREATE_IN_PROGRESS",
"StackName""DevStack-62031",
"Parameters"[{"ParameterKey""DevDB","ParameterValue"nil}]}]}
Unfortunately,itdoesn’tsayanythingaboutwhichresourcesbelongtothisstack.Itdoes,however,giveusthestackname,whichwecanusetolookupresourcesinthenextservice.
ThedescribeStackResourcesendpointThisendpointreceivesmanyarguments,buttheonewe’reinterestedinisthestackname,which,onceprovided,returnsthefollowing:
{"StackResources"
[{"PhysicalResourceId""EC2123",
"ResourceType""AWS::EC2::Instance"},
{"PhysicalResourceId""EC2456",
"ResourceType""AWS::EC2::Instance"}
{"PhysicalResourceId""EC2789",
"ResourceType""AWS::EC2::Instance"}
{"PhysicalResourceId""RDS123",
"ResourceType""AWS::RDS::DBInstance"}
{"PhysicalResourceId""RDS456",
"ResourceType""AWS::RDS::DBInstance"}]}
Weseemtobegettingsomewherenow.Thisstackhasseveralresources:threeEC2instancesandtwoRDSinstances—nottoobadforonlytwoAPIcalls.
However,aswementionedpreviously,ourdashboardneedstoshowthestatusofeachoftheresources.WiththelistofresourceIDsathand,weneedtolooktootherservicesthatcouldgiveusdetailedinformationabouteachresource.
EC2ThenextservicewewilllookatisspecifictoEC2.Aswewillsee,theresponsesofthedifferentservicesaren’tasconsistentaswewouldlikethemtobe.
ThedescribeInstancesendpointThisendpointsoundspromising.Basedonthedocumentation,itseemswecangiveitalistofinstanceIDsanditwillgiveusbackthefollowingresponse:
{"Reservations"
[{"Instances"
[{"InstanceId""EC2123",
"Tags"
[{"Key""StackType","Value""Dev"}
{"Key""junkTag","Value""shouldnotbeincluded"}
{"Key""aws:cloudformation:logical-id","Value""theDude"}],
"State"{"Name""running"}}
{"InstanceId""EC2456",
"Tags"
[{"Key""StackType","Value""Dev"}
{"Key""junkTag","Value""shouldnotbeincluded"}
{"Key""aws:cloudformation:logical-id","Value""theDude"}],
"State"{"Name""running"}}
{"InstanceId""EC2789",
"Tags"
[{"Key""StackType","Value""Dev"}
{"Key""junkTag","Value""shouldnotbeincluded"}
{"Key""aws:cloudformation:logical-id","Value""theDude"}],
"State"{"Name""running"}}]}]}
Buriedinthisresponse,wecanseetheStatekey,whichgivesusthestatusofthatparticularEC2instance.ThisisallweneedasfarasEC2goes.ThisleavesuswithRDStohandle.
RDSOnemightbetemptedtothinkthatgettingthestatusesofRDSinstanceswouldbejustaseasyaswithEC2.Let’sseeifthatisthecase.
ThedescribeDBInstancesendpointThisendpointisequivalentinpurposetotheanalogousEC2endpointwejustlookedat.Itsinput,however,isslightlydifferent:itacceptsasingleinstanceIDasinputand,asofthetimeofthiswriting,doesn’tsupportfilters.
ThismeansthatifourstackhasmultipleRDSinstances—say,inaprimary/replicasetup—weneedtomakemultipleAPIcallstogatherinformationabouteachoneofthem.Notabigdeal,ofcourse,butalimitationtobeawareof.
OncegivenaspecificdatabaseinstanceID,thisservicerespondswiththefollowingcode:
{"DBInstances"
[{"DBInstanceIdentifier""RDS123","DBInstanceStatus""available"}]}
Thefactthatasingleinstancecomesinsideavectorhintsatthefactthatfilteringwillbesupportedinthefuture.Itjusthasn’thappenedyet.
DesigningthesolutionWenowhavealltheinformationweneedtostartdesigningourapplication.WeneedtocoordinatefourdifferentAPIcallsperCloudFormationstack:
describeStacks:TolistallavailablestacksdescribeStackResources:ToretrievedetailsofallresourcescontainedinastackdescribeInstances:ToretrievedetailsofallEC2instancesinastackdescribeDBInstances:ToretrievedetailsofallDB2instancesinastack
Next,Iwouldlikeyoutostepbackforamomentandthinkabouthowyouwoulddesigncodelikethis.Goahead,I’llwait.
Nowthatyou’reback,let’shavealookatonepossibleapproach.
Ifwerecallthescreenshotofwhatthedashboardwouldlooklike,werealizethat,forthepurposesofourapplication,thedifferencebetweenEC2andRDSresourcescanbecompletelyignoredsolongaseachonehastheattributesID,type,andstatus.
Thismeanswhateveroursolutionmaybe,ithastosomehowprovideauniformwayofabstractingthedifferentresourcetypes.
Additionally,apartfromdescribeStacksanddescribeStackResources,whichneedtobecalledsequentially,describeInstancesanddescribeDBInstancescanbeexecutedconcurrently,afterwhichwewillneedawaytomergetheresults.
Sinceanimageisworthathousandwords,thefollowingimageiswhatwewouldliketheworkflowtolooklike:
Theprecedingimagehighlightsanumberofkeyaspectsofoursolution:
WestartbyretrievingstacksbycallingdescribeStacksNext,foreachstack,wecalldescribeStackResourcestoretrievealistofresourcesforeachoneThen,wesplitthelistbytype,endingwithalistofEC2andonewithRDSresourcesWeproceedbyconcurrentlycallingdescribeInstancesanddescribeDBInstances,yieldingtwolistsofresults,oneperresourcetypeAstheresponseformatsaredifferent,wetransformeachresourceintoauniformrepresentationLastly,wemergeallresultsintoasinglelist,readyforrendering
Thisisquiteabittotakein,butasyouwillsoonrealize,oursolutionisn’ttoofaroffthishigh-leveldescription.
WecanquiteeasilythinkofthisproblemashavinginformationaboutseveraldifferenttypesofinstancesflowingthroughthisgraphofAPIcalls—beingtransformedasneededinbetween—untilwearriveattheinformationwe’reafter,intheformatwewouldliketoworkwith.
Asitturnsout,agreatwaytomodelthisproblemistouseoneoftheReactive
abstractionswelearnedaboutearlierinthisbook:Observables.
RunningtheAWSstubserverBeforewejumpintowritingourdashboard,weshouldmakesureourAWSstubserverisproperlysetup.ThestubserverisaClojurewebapplicationthatsimulateshowtherealAWSAPIbehavesandisthebackendourdashboardwilltalkto.
Let’sstartbygoingintoourterminal,cloningthebookrepositoryusingGitandthenstartingthestubserver:
$gitclonehttps://github.com/leonardoborges/ClojureReactiveProgramming
$cdClojureReactiveProgramming/code/chapter09/aws-api-stub
$leinringserver-headless3001
2014-11-2317:33:37.766:INFO:oejs.Server:jetty-7.6.8.v20121106
2014-11-2317:33:37.812:INFO:oejs.AbstractConnector:Started
[email protected]:3001
Startedserveronport3001
Thiswillhavestartedtheserveronport3001.Tovalidateitisworkingasexpected,pointyourbrowsertohttp://localhost:3001/cloudFormation/describeStacks.YoushouldseethefollowingJSONresponse:
{
"Stacks":[
{
"Parameters":[
{
"ParameterKey":"DevDB",
"ParameterValue":null
}
],
"StackStatus":"CREATE_IN_PROGRESS",
"StackId":"arn:aws:cloudformation:ap-southeast-
2:337944750480:stack/DevStack-62031/1",
"StackName":"DevStack-62031"
}
]
}
SettingupthedashboardprojectAswepreviouslymentioned,wewillbedevelopingthedashboardusingClojureScriptwiththeUIrenderedusingOm.Additionally,aswehavechosenObservablesasourmainReactiveabstraction,wewillneedRxJS,oneofthemanyimplementationsofMicrosoft’sReactiveExtensions.Wewillbepullingthesedependenciesintoourprojectshortly.
Let’screateanewClojureScriptprojectcalledaws-dashusingtheom-startleiningentemplate:
$leinnewom-startaws-dash
Thisgivesusastartingpoint,butweshouldmakesureourversionsallmatch.Openuptheproject.cljfilefoundintherootdirectoryofthenewprojectandensurethedependenciessectionlookslikethefollowing:
...
:dependencies[[org.clojure/clojure"1.6.0"]
[org.clojure/clojurescript"0.0-2371"]
[org.clojure/core.async"0.1.346.0-17112a-alpha"]
[om"0.5.0"]
[com.facebook/react"0.9.0"]
[cljs-http"0.1.20"]
[com.cognitect/transit-cljs"0.8.192"]]
:plugins[[lein-cljsbuild"1.0.3"]]
...
Thisisthefirsttimeweseethelasttwodependencies.cljs-httpisasimpleHTTPlibrarywewillusetomakeAJAXrequeststoourAWSstubserver.transit-cljsallowsusto,amongotherthings,parseJSONresponsesintoClojureScriptdatastructures.
TipTransititselfisaformatandasetoflibrariesthroughwhichapplicationsdevelopedindifferenttechnologiescanspeaktoeachother.Inthiscase,weareusingtheClojurescriptlibrarytoparseJSON,butifyou’reinterestedinlearningmore,IrecommendreadingtheofficialblogpostannouncementbyRichHickeyathttp://blog.cognitect.com/blog/2014/7/22/transit.
Next,weneedRxJS,which,beingaJavaScriptdependency,isn’tavailablevialeiningen.That’sOK.Wewillsimplydownloaditintotheapplicationoutputdirectory,aws-dash/dev-resources/public/js/:
$cdaws-dash/dev-resources/public/js/
$wgethttps://raw.githubusercontent.com/Reactive-
Extensions/RxJS/master/dist/rx.all.js
--2014-11-2318:00:21--https://raw.githubusercontent.com/Reactive-
Extensions/RxJS/master/dist/rx.all.js
Resolvingraw.githubusercontent.com…103.245.222.133
Connectingtoraw.githubusercontent.com|103.245.222.133|:443…connected.
HTTPrequestsent,awaitingresponse…200OK
Length:355622(347K)[text/plain]
Savingto:'rx.all.js'
100%[========================>]355,622966KB/sin0.4s
2014-11-2318:00:24(966KB/s)-'rx.all.js'saved[355622/355622]
Movingon,weneedtomakeourapplicationawareofournewdependencyonRxJS.Opentheaws-dash/dev-resources/public/index.htmlfileandaddascripttagtopullinRxJS:
<html>
<body>
<divid="app"></div>
<scriptsrc="http://fb.me/react-0.9.0.js"></script>
<scriptsrc="js/rx.all.js"></script>
<scriptsrc="js/aws_dash.js"></script>
</body>
</html>
Withallthedependenciesinplace,let’sstarttheauto-compilationforourClojureScriptsourcefilesasfollows:
$cdaws-dash/
$leincljsbuildauto
CompilingClojureScript.
Compiling"dev-resources/public/js/aws_dash.js"from("src/cljs""dev-
resources/tools/repl")...
Successfullycompiled"dev-resources/public/js/aws_dash.js"in0.981
seconds.
CreatingAWSObservablesWe’renowreadytostartimplementingoursolution.IfyourecallfromtheReactiveExtensionschapter,RxJava/RxJS/RxClojureshipwithseveralusefulObservables.However,whenthebuilt-inObservablesaren’tenough,itgivesusthetoolstobuildourown.
SinceitishighlyunlikelyRxJSalreadyprovidesObservablesforAmazon’sAWSAPI,wewillstartbyimplementingourownprimitiveObservables.
Tokeepthingsneat,wewilldothisinanewfile,underaws-dash/src/cljs/aws_dash/observables.cljs:
(nsaws-dash.observables
(:require-macros[cljs.core.async.macros:refer[go]])
(:require[cljs-http.client:ashttp]
[cljs.core.async:refer[<!]]
[cognitect.transit:ast]))
(defr(t/reader:json))
(defaws-endpoint"http://localhost:3001")
(defnaws-uri[path]
(straws-endpointpath))
Thenamespacedeclarationrequiresthenecessarydependencieswewillneedinthisfile.NotehowthereisnoexplicitdependencyonRxJS.SinceitisaJavaScriptdependencythatwemanuallypulledin,itisgloballyavailableforustouseviaJavaScriptinteroperability.
ThenextlinesetsupatransitreaderforJSON,whichwewillusewhenparsingthestubserverresponses.
Then,wedefinetheendpointwewillbetalkingtoaswellasahelperfunctiontobuildthecorrectURIs.Makesurethevariableaws-endpointmatchesthehostandportofthestubserverstartedintheprevioussection.
AllObservablesweareabouttocreatefollowacommonstructure:theymakearequesttothestubserver,extractsomeinformationfromtheresponse,optionallytransformingit,andthenemiteachiteminthetransformedsequenceintothenewObservablesequence.
Toavoidrepetition,thispatterniscapturedinthefollowingfunction:
(defnobservable-seq[uritransform]
(.createjs/Rx.Observable
(fn[observer]
(go(let[response(<!(http/geturi{:with-credentials?
false}))
data(t/readr(:bodyresponse))
transformed(transformdata)]
(doseq[xtransformed]
(.onNextobserverx))
(.onCompletedobserver)))
(fn[](.logjs/console"Disposed")))))
Let’sbreakthisfunctiondown:
observable-seqreceivestwoarguments:thebackendURItowhichwewillissueaGETrequest,andatransformfunctionwhichisgiventherawparsedJSONresponseandreturnsasequenceoftransformeditems.Then,itcallsthecreatefunctionoftheRxJSobjectRx.Observable.NotehowwemakeuseofJavaScriptinteroperabilityhere:weaccessthecreatefunctionbyprependingitwithadotmuchlikeinJavainteroperability.SinceRx.Observableisaglobalobject,weaccessitbyprependingtheglobalJavaScriptnamespaceClojureScriptmakesavailabletoourprogram,js/Rx.Observable.TheObservable’screatefunctionreceivestwoarguments.OneisafunctionthatgetscalledwithanObservertowhichwecanpushitemstobepublishedintheObservablesequence.ThesecondfunctionisafunctionthatiscalledwheneverthisObservableisdisposedof.Thisisthefunctionwherewecouldperformanycleanupneeded.Inourcase,thisfunctionsimplylogsthefactthatitiscalledtotheconsole.
Thefirstfunctionistheonethatinterestsusthough:
(fn[observer]
(go(let[response(<!(http/geturi
{:with-credentials?
false}))
data(t/readr(:bodyresponse))
transformed(transformdata)]
(doseq[xtransformed]
(.onNextobserverx))
(.onCompletedobserver))))
Assoonasitgetscalled,itperformsarequesttotheprovidedURIusingcljs-http’sgetfunction,whichreturnsacore.asyncchannel.That’swhythewholelogicisinsideagoblock.
Next,weusethetransitJSONreaderweconfiguredpreviouslytoparsethebodyoftheresponse,feedingtheresultintothetransformfunction.Rememberthisfunction,asperourdesign,returnsasequenceofthings.Therefore,allthatislefttodoispusheachitemintotheobserverinturn.
Oncewe’redone,weindicatethatthisObservablesequencewon’temitanynewitembyinvokingthe.onCompletedfunctionoftheobserverobject.
Now,wecanproceedcreatingourObservablesusingthishelperfunction,startingwiththeoneresponsibleforretrievingCloudFormationstacks:
(defndescribe-stacks[]
(observable-seq(aws-uri"/cloudFormation/describeStacks")
(fn[data]
(map(fn[stack]{:stack-id(stack"StackId")
:stack-name(stack"StackName")})
(data"Stacks")))))
Thiscreatesanobservablethatwillemitoneitemperstack,inthefollowingformat:
({:stack-id"arn:aws:cloudformation:ap-southeast-
2:337944750480:stack/DevStack-62031/1",:stack-name"DevStack-62031"})
Nowthatwehavestacks,weneedanObservabletodescribeitsresources:
(defndescribe-stack-resources[stack-name]
(observable-seq(aws-uri"/cloudFormation/describeStackResources")
(fn[data]
(map(fn[resource]
{:resource-id(resource"PhysicalResourceId")
:resource-type(resource"ResourceType")})
(data"StackResources")))))
Ithasasimilarpurposeandemitsresourceitemsinthefollowingformat:
({:resource-id"EC2123",:resource-type"AWS::EC2::Instance"}
{:resource-id"EC2456",:resource-type"AWS::EC2::Instance"}
{:resource-id"EC2789",:resource-type"AWS::EC2::Instance"}
{:resource-id"RDS123",:resource-type"AWS::RDS::DBInstance"}
{:resource-id"RDS456",:resource-type"AWS::RDS::DBInstance"})
Sincewe’refollowingourstrategyalmosttotheletter,weneedtwomoreobservables,oneforeachinstancetype:
(defndescribe-instances[instance-ids]
(observable-seq(aws-uri"/ec2/describeInstances")
(fn[data]
(let[instances(mapcat(fn[reservation]
(reservation"Instances"))
(data"Reservations"))]
(map(fn[instance]
{:instance-id(instance"InstanceId")
:type"EC2"
:status(get-ininstance["State"
"Name"])})
instances)))))
(defndescribe-db-instances[instance-id]
(observable-seq(aws-uri(str"/rds/describeDBInstances/"instance-id))
(fn[data]
(map(fn[instance]
{:instance-id(instance"DBInstanceIdentifier")
:type"RDS"
:status(instance"DBInstanceStatus")})
(data"DBInstances")))))
EachofwhichwillemitresourceitemsinthefollowingformatsforEC2andRDS,respectively:
({:instance-id"EC2123",:type"EC2",:status"running"}...)
({:instance-id"RDS123",:type"RDS",:status"available"}...)
CombiningtheAWSObservablesItseemswehaveallmajorpiecesinplacenow.Allthatislefttodoistocombinethemoreprimitive,basicObservableswejustcreatedintomorecomplexandusefulonesbycombiningthemtoaggregateallthedataweneedinordertorenderourdashboard.
Wewillstartbycreatingafunctionthatcombinesboththedescribe-stacksanddescribe-stack-resourcesObservables:
(defnstack-resources[]
(->(describe-stacks)
(.map#(:stack-name%))
(.flatMapdescribe-stack-resources)))
Startinginthepreviousexample,webegintoseehowdefiningourAPIcallsintermsofObservablesequencespaysoff:it’salmostsimplecombiningthesetwoObservablesinadeclarativemanner.
RemembertheroleofflatMap:asdescribe-stack-resourcesitselfreturnsanObservable,weuseflatMaptoflattenbothObservables,aswehavedonebeforeinvariousdifferentabstractions.
Thestack-resourcesObservablewillemitresourceitemsforallstacks.Accordingtoourplan,wewouldliketoforktheprocessinghereinordertoconcurrentlyretrieveEC2andRDSinstancedata.
Byfollowingthistrainofthought,wearriveattwomorefunctionsthatcombineandtransformthepreviousObservables:
(defnec2-instance-status[resources]
(->resources
(.filter#(=(:resource-type%)"AWS::EC2::Instance"))
(.map#(:resource-id%))
(.reduceconj[])
(.flatMapdescribe-instances)))
(defnrds-instance-status[resources]
(->resources
(.filter#(=(:resource-type%)"AWS::RDS::DBInstance"))
(.map#(:resource-id%))
(.flatMapdescribe-db-instances)))
Boththefunctionsreceiveanargument,resources,whichistheresultofcallingthestack-resourcesObservable.Thatway,weonlyneedtocallitonce.
Onceagain,itisfairlysimpletocombinetheObservablesinawaythatmakessense,followingourhigh-levelideadescribedpreviously.
Startingwithresources,wefilteroutthetypeswe’renotinterestedin,retrieveitsIDs,andrequestitsdetailedinformationbyflatmappingthedescribe-instancesanddescribe-db-instancesObservables.
Note,however,thatduetoalimitationintheRDSAPIdescribedearlier,wehavetocallit
multipletimestoretrieveinformationaboutallRDSinstances.
ThisseeminglyfundamentaldifferenceinhowweusetheAPIbecomesaminortransformationinourEC2observable,whichsimplyaccumulatesallIDsintoavectorsothatwecanretrievethemallatonce.
OursimpleReactiveAPItoAmazonAWSisnowcomplete,leavinguswiththeUItocreate.
PuttingitalltogetherLet’snowturntobuildingouruserinterface.It’sasimpleone,solet’sjustjumpintoit.Openupaws-dash/src/cljs/aws_dash/core.cljsandaddthefollowing:
(nsaws-dash.core
(:require[aws-dash.observables:asobs]
[om.core:asom:include-macrostrue]
[om.dom:asdom:include-macrostrue]))
(enable-console-print!)
(defapp-state(atom{:instances[]}))
(defninstance-view[{:keys[instance-idtypestatus]}owner]
(reify
om/IRender
(render[this]
(dom/trnil
(dom/tdnilinstance-id)
(dom/tdniltype)
(dom/tdnilstatus)))))
(defninstances-view[instancesowner]
(reify
om/IRender
(render[this]
(applydom/table#js{:style#js{:border"1pxsolidblack;"}}
(dom/trnil
(dom/thnil"Id")
(dom/thnil"Type")
(dom/thnil"Status"))
(om/build-allinstance-viewinstances)))))
(om/root
(fn[appowner]
(dom/divnil
(dom/h1nil"StackResourceStatuses")
(om/buildinstances-view(:instancesapp))))
app-state
{:target(.js/document(getElementById"app"))})
Ourapplicationstatecontainsasinglekey,:instances,whichstartsasanemptyvector.AswecanseefromeachOmcomponent,instanceswillberenderedasrowsinaHTMLtable.
Aftersavingthefile,makesurethewebserverisrunningbystartingitfromtheREPL:
leinrepl
CompilingClojureScript.
nREPLserverstartedonport58209onhost127.0.0.1-
nrepl://127.0.0.1:58209
REPL-y0.3.5,nREPL0.2.6
Clojure1.6.0
JavaHotSpot(TM)64-BitServerVM1.8.0_25-b17
Docs:(docfunction-name-here)
(find-doc"part-of-name-here")
Source:(sourcefunction-name-here)
Javadoc:(javadocjava-object-or-class-here)
Exit:Control+Dor(exit)or(quit)
Results:Storedinvars*1,*2,*3,anexceptionin*e
user=>(run)
2015-02-0821:02:34.503:INFO:oejs.Server:jetty-7.6.8.v20121106
2015-02-0821:02:34.545:INFO:oejs.AbstractConnector:Started
[email protected]:3000
#<Serverorg.eclipse.jetty.server.Server@35bc3669>
Youshouldnowbeablepointyourbrowsertohttp://localhost:3000/,but,asyoumighthaveguessed,youwillseenothingbutanemptytable.
Thisisbecausewehaven’tyetusedourReactiveAWSAPI.
Let’sfixitandbringitalltogetheratthebottomofcore.cljs:
(defresources(obs/stack-resources))
(.subscribe(->(.merge(obs/rds-instance-statusresources)
(obs/ec2-instance-statusresources))
(.reduceconj[]))
#(swap!app-stateassoc:instances%))
Yes,thisisallweneed!Wecreateastack-resourcesObservableandpassitasanargumenttobothrds-instance-statusandec2-instance-status,whichwillconcurrentlyretrievestatusinformationaboutallinstances.
Next,wecreateanewObservablebymergingtheprevioustwofollowedbyacallto.reduce,whichwillaccumulateallinformationintoavector,convenientforrendering.
Finally,wesimplysubscribetothisObservableand,whenitemitsitsresults,wesimplyupdateourapplicationstate,leavingOmtodoalltherenderingforus.
SavethefileandmakesureClojureScripthascompiledsuccessfully.Then,gobacktoyourbrowserathttp://localhost:3000/,andyoushouldseeallinstancestatuses,aspicturedatthebeginningofthischapter.
ExercisesWithourpreviousapproach,theonlywaytoseenewinformationabouttheAWSresourcesisbyrefreshingthewholepage.Modifyourimplementationinsuchawaythatitqueriesthestubserviceseverysooften—say,every500milliseconds.
TipTheintervalfunctionfromRxJScanbehelpfulinsolvingthisexercise.ThinkhowyoumightuseittogetherwithourexistingstreambyreviewinghowflatMapworks.
SummaryInthischapter,welookedatarealusecaseforReactiveapplications:buildingadashboardforAWSCloudFormationstacks.
Wehaveseenhowthinkingofalltheinformationneededasresources/itemsflowingthroughagraphfitsnicelywithhowonecreatesObservables.
Inaddition,bycreatingprimitiveObservablesthatdoonethingonlygivesusanicedeclarativewaytocombinethemintomorecomplexObservables,givingusadegreeofreusenotusuallyfoundwithcommontechniques.
Finally,wepackagedittogetherwithasimpleOm-basedinterfacetodemonstratehowusingdifferentabstractionsinthesameapplicationdoesnotaddtocomplexityaslongastheabstractionsarechosencarefullyfortheproblemathand.
ThisbringsustotheendofwhathopefullywasanenjoyableandinformativejourneythroughthedifferentwaysofReactiveProgramming.
Farfrombeingacompletereference,thisbookaimstoprovideyou,thereader,withenoughinformation,aswellasconcretetoolsandexamplesthatyoucanapplytoday.
Itisalsomyhopethatthereferencesandexercisesincludedinthisbookprovethemselvesuseful,shouldyouwishtoexpandyourknowledgeandseekoutmoredetails.
Lastly,IstronglyencourageyoutoturnthepageandreadtheAppendix,TheAlgebraofLibraryDesign,asItrulybelieveitwill,ifnothingelse,makeyouthinkhardabouttheimportanceofcompositioninprogramming.
Isincerelywishthisbookhasbeenasentertainingandinstructionaltoreadasitwastowrite.
Thankyouforreading.Ilookforwardtoseeingthegreatthingsyoubuild.
AppendixA.TheAlgebraofLibraryDesignYoumighthavenoticedthatallreactiveabstractionswehaveencounteredinthisbookhaveafewthingsincommon.Forone,theyworkas“container-like”abstractions:
FuturesencapsulateacomputationthateventuallyyieldsasinglevalueObservablesencapsulatecomputationsthatcanyieldmultiplevaluesovertimeintheshapeofastreamChannelsencapsulatevaluespushedtothemandcanhavethempoppedfromit,workingasaconcurrentqueuethroughwhichconcurrentprocessescommunicate
Then,oncewehavethis“container,”wecanoperateonitinanumberofways,whichareverysimilaracrossthedifferentabstractionsandframeworks:wecanfilterthevaluescontainedinthem,transformthemusingmap,combineabstractionsofthesametypeusingbind/flatMap/selectMany,executemultiplecomputationsinparallel,aggregatetheresultsusingsequence,andmuchmore.
Assuch,eventhoughtheabstractionsandtheirunderlyingworkingsarefundamentallydifferent,itstillfeelstheybelongtosometypeofhigher-levelabstractions.
Inthisappendix,wewillexplorewhatthesehigher-levelabstractionsare,therelationshipbetweenthem,andhowwecantakeadvantageoftheminourprojects.
ThesemanticsofmapWewillgetstartedbytakingalookatoneofthemostusedoperationsintheseabstractions:map.
We’vebeenusingmapforalongtimeinordertotransformsequences.Thus,insteadofcreatinganewfunctionnameforeachnewabstraction,librarydesignerssimplyabstractthemapoperationoveritsowncontainertype.
Imaginethemesswewouldendupinifwehadfunctionssuchastransform-observable,transform-channel,combine-futures,andsoon.
Thankfully,thisisnotthecase.Thesemanticsofmaparewellunderstoodtothepointthatevenifadeveloperhasn’tusedaspecificlibrarybefore,hewillalmostalwaysassumethatmapwillapplyafunctiontothevalue(s)containedwithinwhateverabstractionthelibraryprovides.
Let’slookatthreeexamplesweencounteredinthisbook.Wewillcreateanewleiningenprojectinwhichtoexperimentwiththecontentsofthisappendix:
$leinnewlibrary-design
Next,let’saddafewdependenciestoourproject.cljfile:
...
:dependencies[[org.clojure/clojure"1.6.0"]
[com.leonardoborges/imminent"0.1.0"]
[com.netflix.rxjava/rxjava-clojure"0.20.7"]
[org.clojure/core.async"0.1.346.0-17112a-alpha"]
[uncomplicate/fluokitten"0.3.0"]]
...
Don’tworryaboutthelastdependency—we’llgettoitlateron.
Now,startanREPLsessionsothatwecanfollowalong:
$leinrepl
Then,enterthefollowingintoyourREPL:
(require'[imminent.core:asi]
'[rx.lang.clojure.core:asrx]
'[clojure.core.async:asasync])
(defrepl-out*out*)
(defnprn-to-repl[&args]
(binding[*out*repl-out]
(applyprnargs)))
(->(i/const-future31)
(i/map#(*%2))
(i/on-success#(prn-to-repl(str"Value:"%))))
(as->(rx/return31)obs
(rx/map#(*%2)obs)
(rx/subscribeobs#(prn-to-repl(str"Value:"%))))
(defc(chan))
(defmapped-c(async/map<#(*%2)c))
(async/go(async/>!c31))
(async/go(prn-to-repl(str"Value:"(async/<!mapped-c))))
"Value:62"
"Value:62"
"Value:62"
Thethreeexamples—usingimminent,RxClojure,andcore.async,respectively—lookremarkablysimilar.Theyallfollowasimplerecipe:
1. Putthenumber31insidetheirrespectiveabstraction.2. Doublethatnumberbymappingafunctionovertheabstraction.3. PrintitsresulttotheREPL.
Asexpected,thisoutputsthevalue62threetimestothescreen.
Itwouldseemmapperformsthesameabstractstepsinallthreecases:itappliestheprovidedfunction,putstheresultingvalueinafreshnewcontainer,andreturnsit.Wecouldcontinuegeneralizing,butwewouldjustberediscoveringanabstractionthatalreadyexists:Functors.
FunctorsFunctorsarethefirstabstractionwewilllookatandtheyarerathersimple:theydefineasingleoperationcalledfmap.InClojure,Functorscanberepresentedusingprotocolsandareusedforcontainersthatcanbemappedover.Suchcontainersinclude,butarenotlimitedto,lists,Futures,Observables,andchannels.
TipTheAlgebrainthetitleofthisAppendixreferstoAbstractAlgebra,abranchofMathematicsthatstudiesalgebraicstructures.Analgebraicstructureis,toputitsimply,asetwithoneormoreoperationsdefinedonit.
Asanexample,considerSemigroups,whichisonesuchalgebraicstructure.Itisdefinedtobeasetofelementstogetherwithanoperationthatcombinesanytwoelementsofthisset.Therefore,thesetofpositiveintegerstogetherwiththeadditionoperationformaSemigroup.
AnothertoolusedforstudyingalgebraicstructuresiscalledCategoryTheory,ofwhichFunctorsarepartof.
Wewon’tdelvetoomuchintothetheorybehindallthis,asthereareplentyofbooks[9][10]availableonthesubject.Itwas,however,anecessarydetourtoexplainthetitleusedinthisappendix.
DoesthismeanalloftheseabstractionsimplementaFunctorprotocol?Unfortunately,thisisnotthecase.AsClojureisadynamiclanguageanditdidn’thaveprotocolsbuiltin—theywereaddedinversion1.2ofthelanguage—theseframeworkstendtoimplementtheirownversionofthemapfunction,whichdoesn’tbelongtoanyprotocolinparticular.
Theonlyexceptionisimminent,whichimplementstheprotocolsincludedinfluokitten,aClojurelibraryprovidingconceptsfromCategorytheorysuchasFunctors.
ThisisasimplifiedversionoftheFunctorprotocolfoundinfluokitten:
(defprotocolFunctor
(fmap[fvg]))
Asmentionedpreviously,Functorsdefineasingleoperation.fmapappliesthefunctiongtowhatevervalueisinsidethecontainer,Functor,fv.
However,implementingthisprotocoldoesnotguaranteethatwehaveactuallyimplementedaFunctor.Thisisbecause,inadditiontoimplementingtheprotocol,Functorsarealsorequiredtoobeyacoupleoflaws,whichwewillexaminebriefly.
Theidentitylawisasfollows:
(=(fmapa-functoridentity)
(identitya-functor))
Theprecedingcodeisallweneedtoverifythislaw.Itsimplysaysthatmappingtheidentityfunctionovera-functoristhesameassimplyapplyingtheidentityfunction
totheFunctoritself.
Thecompositionlawisasfollows:
(=(fmapa-functor(compfg))
(fmap(fmapa-functorg)f))
Thecompositionlaw,inturn,saysthatifwecomposetwoarbitraryfunctionsfandg,taketheresultingfunctionandapplythattoa-functor,thatisthesameasmappinggovertheFunctorandthenmappingfovertheresultingFunctor.
Noamountoftextwillbeabletoreplacepracticalexamples,sowewillimplementourownFunctor,whichwewillcallOption.Wewillthenrevisitthelawstoensurewehaverespectedthem.
TheOptionFunctorAsTonyHoareonceputit,nullreferencesarehisonebilliondollarmistake(http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare).Regardlessofbackground,younodoubtwillhaveencounteredversionsofthedreadfulNullPointerException.Thisusuallyhappenswhenwetrytocallamethodonanobjectreferencethatisnull.
ClojureembracesnullvaluesduetoitsinteroperabilitywithJava,itshostlanguage,butitprovidesimprovedsupportfordealingwiththem.
Thecorelibraryispackedwithfunctionsthatdotherightthingifpassedanilvalue—Clojure’sversionofJava’snull.Forinstance,howmanyelementsarethereinanilsequence?
(countnil);;0
Thankstoconsciousdesigndecisionsregardingnil,wecan,forthemostpart,affordnotworryaboutit.Forallothercases,theOptionFunctormightbeofsomehelp.
Theremainingoftheexamplesinthisappendixshouldbeinafilecalledoption.cljunderlibrary-design/src/library_design/.You’rewelcometotrythisintheREPLaswell.
Let’sstartournextexamplebyaddingthenamespacedeclarationaswellasthedatawewillbeworkingwith:
(nslibrary-design.option
(:require[uncomplicate.fluokitten.protocols:asfkp]
[uncomplicate.fluokitten.core:asfkc]
[uncomplicate.fluokitten.jvm:asfkj]
[imminent.core:asI]))
(defpirates[{:name"JackSparrow":born1700:died1740:ship"Black
Pearl"}
{:name"Blackbeard":born1680:died1750:ship"Queen
Anne'sRevenge"}
{:name"HectorBarbossa":born1680:died1740:shipnil}])
(defnpirate-by-name[name]
(->>pirates
(filter#(=name(:name%)))
first))
(defnage[{:keys[borndied]}]
(-diedborn))
AsaPiratesoftheCaribbeanfan,Ithoughtitwouldbeinterestingtoplaywithpiratesforthisexample.Let’ssaywewouldliketocalculateJackSparrow’sage.Giventhedataandfunctionswejustcovered,thisisasimpletask:
(->(pirate-by-name"JackSparrow")
age);;40
However,whatifwewouldliketoknowDavyJones’age?Wedon’tactuallyhaveanydataforthispirate,soifwerunourprogramagain,thisiswhatwe’llget:
(->(pirate-by-name"DavyJones")
age);;NullPointerExceptionclojure.lang.Numbers.ops
(Numbers.java:961)
Thereitis.ThedreadfulNullPointerException.Thishappensbecauseintheimplementationoftheagefunction,weenduptryingtosubtracttwonilvalues,whichisincorrect.Asyoumighthaveguessed,wewillattempttofixthisbyusingtheOptionFunctor.
Traditionally,Optionisimplementedasanalgebraicdatatype,morespecificallyasumtypewithtwovariants:SomeandNone.Thesevariantsareusedtoidentifywhetheravalueispresentornotwithoutusingnils.YoucanthinkofbothSomeandNoneassubtypesofOption.
InClojure,wewillrepresentthemusingrecords:
(defrecordSome[v])
(defrecordNone[])
(defnoption[v]
(ifv
(Some.v)
(None.)))
Aswecansee,SomecancontainasinglevaluewhereasNonecontainsnothing.It’ssimplyamarkerindicatingtheabsenceofcontent.Wehavealsocreatedahelperfunctioncalledoption,whichcreatestheappropriaterecorddependingonwhetheritsargumentisnilornot.
ThenextstepistoextendtheFunctorprotocoltobothrecords:
(extend-protocolfkp/Functor
Some
(fmap[fg]
(Some.(g(:vf))))
None
(fmap[__]
(None.)))
Here’swherethesemanticmeaningoftheOptionFunctorbecomesapparent:asSomecontainsavalue,itsimplementationoffmapsimplyappliesthefunctiongtothevalueinsidetheFunctorf,whichisoftypeSome.Finally,weputtheresultinsideanewSomerecord.
NowwhatdoesitmeantomapafunctionoveraNone?Youprobablyguessedthatitdoesn’treallymakesense—theNonerecordholdsnovalues.TheonlythingwecandoisreturnanotherNone.Aswewillseeshortly,thisgivestheOptionFunctorashort-circuitingsemantic.
TipInthefmapimplementationofNone,wecouldhavereturnedareferencetothisinsteadofanewrecordinstance.I’venotdonesosimplytomakeitclearthatweneedtoreturnaninstanceofNone.
Nowthatwe’veimplementedtheFunctorprotocol,wecantryitout:
(->>(option(pirate-by-name"JackSparrow"))
(fkc/fmapage));;#library_design.option.Some{:v40}
(->>(option(pirate-by-name"DavyJones"))
(fkc/fmapage));;#library_design.option.None{}
Thefirstexampleshouldn’tholdanysurprises.Weconvertthepiratemapwegetfromcallingpirate-by-nameintoanoption,andthenfmaptheagefunctionoverit.
Thesecondexampleistheinterestingone.Asstatedpreviously,wehavenodataaboutDavyJones.However,mappingageoveritdoesnotthrowanexceptionanylonger,insteadreturningNone.
Thismightseemlikeasmallbenefit,butthebottomlineisthattheOptionFunctormakesitsafetochainoperationstogether:
(->>(option(pirate-by-name"JackSparrow"))
(fkc/fmapage)
(fkc/fmapinc)
(fkc/fmap#(*2%)));;#library_design.option.Some{:v82}
(->>(option(pirate-by-name"DavyJones"))
(fkc/fmapage)
(fkc/fmapinc)
(fkc/fmap#(*2%)));;#library_design.option.None{}
Atthispoint,somereadersmightbethinkingaboutthesome->macro—introducedinClojure1.5—andhowiteffectivelyachievesthesameresultastheOptionFunctor.Thisintuitioniscorrectasdemonstratedasfollows:
(some->(pirate-by-name"DavyJones")
age
inc
(*2));;nil
Thesome->macrothreadstheresultofthefirstexpressionthroughthefirstformifitisnotnil.Then,iftheresultofthatexpressionisn’tnil,itthreadsitthroughthenextformandsoon.Assoonasanyoftheexpressionsevaluatestonil,some->short-circuitsandreturnsnilimmediately.
Thatbeingsaid,Functorisamuchmoregeneralconcept,soaslongasweareworkingwiththisconcept,ourcodedoesn’tneedtochangeasweareoperatingatahigherlevelofabstraction:
(->>(i/future(pirate-by-name"JackSparrow"))
(fkc/fmapage)
(fkc/fmapinc)
(fkc/fmap#(*2%)));;#<Future@30518bfc:#<Success@39bd662c:82>>
Intheprecedingexample,eventhoughweareworkingwithafundamentallydifferenttool—futures—thecodeusingtheresultdidnothavetochange.ThisisonlypossiblebecausebothOptionsandfuturesareFunctorsandimplementthesameprotocolprovidedbyfluokitten.WehavegainedcomposabilityandsimplicityaswecanusethesameAPItoworkwithvariousdifferentabstractions.
Speakingofcomposability,thispropertyisguaranteedbythesecondlawofFunctors.Let’sseeifourOptionFunctorrespectsthisandthefirst—theidentity—laws:
;;Identity
(=(fkc/fmapidentity(option1))
(identity(option1)));;true
;;Composition
(=(fkc/fmap(compidentityinc)(option1))
(fkc/fmapidentity(fkc/fmapinc(option1))));;true
Andwe’redone,ourOptionFunctorisalawfulcitizen.Theremainingtwoabstractionsalsocomepairedwiththeirownlaws.Wewillnotcoverthelawsinthissection,butIencouragethereadertoreadaboutthem(http://www.leonardoborges.com/writings/2012/11/30/monads-in-small-bites-part-i-functors/).
FindingtheaverageofagesInthissection,wewillexploreadifferentusecasefortheOptionFunctor.Wewouldliketo,givenanumberofpirates,calculatetheaverageoftheirages.Thisissimpleenoughtodo:
(defnavg[&xs]
(float(/(apply+xs)(countxs))))
(let[a(some->(pirate-by-name"JackSparrow")age)
b(some->(pirate-by-name"Blackbeard")age)
c(some->(pirate-by-name"HectorBarbossa")age)]
(avgabc));;56.666668
Notehowweareusingsome->heretoprotectusfromnilvalues.Now,whathappensifthereisapirateforwhichwehavenoinformation?
(let[a(some->(pirate-by-name"JackSparrow")age)
b(some->(pirate-by-name"DavyJones")age)
c(some->(pirate-by-name"HectorBarbossa")age)]
(avgabc));;NullPointerExceptionclojure.lang.Numbers.ops
(Numbers.java:961)
Itseemswe’rebackatsquareone!It’sworsenowbecauseusingsome->doesn’thelpifweneedtouseallvaluesatonce,asopposedtothreadingthemthroughachainoffunctioncalls.
Ofcourse,notallislost.Allweneedtodoischeckifallvaluesarepresentbeforecalculatingtheaverage:
(let[a(some->(pirate-by-name"JackSparrow")age)
b(some->(pirate-by-name"DavyJones")age)
c(some->(pirate-by-name"HectorBarbossa")age)]
(when(andabc)
(avgabc)));;nil
Whilethisworksperfectlyfine,ourimplementationsuddenlyhadtobecomeawarethatanyorallofthevaluesa,b,andccouldbenil.Thenextabstractionwewilllookat,ApplicativeFunctors,fixesthis.
ApplicativeFunctorsLikeFunctors,ApplicativeFunctorsareasortofcontaineranddefinestwooperations:
(defprotocolApplicative
(pure[avv])
(fapply[agav]))
ThepurefunctionisagenericwaytoputavalueinsideanApplicativeFunctor.Sofar,wehavebeenusingtheoptionhelperfunctionforthispurpose.Wewillbeusingitalittlelater.
ThefapplyfunctionwillunwrapthefunctioncontainedintheApplicativeagandapplyittothevaluecontainedintheapplicativeav.
Thepurposeofboththefunctionswillbecomeclearwithanexample,butfirst,weneedtopromoteourOptionFunctorintoanApplicativeFunctor:
(extend-protocolfkp/Applicative
Some
(pure[_v]
(Some.v))
(fapply[agav]
(if-let[v(:vav)]
(Some.((:vag)v))
(None.)))
None
(pure[_v]
(Some.v))
(fapply[agav]
(None.)))
Theimplementationofpureisthesimplest.AllitdoesiswrapthevaluevintoaninstanceofSome.EquallysimpleistheimplementationoffapplyforNone.Asthereisnovalue,wesimplyreturnNoneagain.
ThefapplyimplementationofSomeensuresbothargumentshaveavalueforthe:vkeyword—strictlyspeakingtheybothhavetobeinstancesofSome.If:visnon-nil,itappliesthefunctioncontainedinagtov,finallywrappingtheresult.Otherwise,itreturnsNone.
ThisshouldbeenoughtotryourfirstexampleusingtheApplicativeFunctorAPI:
(fkc/fapply(optioninc)(option2))
;;#library_design.option.Some{:v3}
(fkc/fapply(optionnil)(option2))
;;#library_design.option.None{}
WearenowabletoworkwithFunctorsthatcontainfunctions.Additionally,wehavealsopreservedthesemanticsofwhatshouldhappenwhenanyoftheFunctorsdon’thavea
value.
Wecannowrevisittheageaverageexamplefrombefore:
(defage-option(comp(partialfkc/fmapage)optionpirate-by-name))
(let[a(age-option"JackSparrow")
b(age-option"Blackbeard")
c(age-option"HectorBarbossa")]
(fkc/<*>(option(fkj/curryavg3))
abc))
;;#library_design.option.Some{:v56.666668}
TipThevarargfunction<*>isdefinedbyfluokittenandperformsaleft-associativefapplyonitsarguments.Essentially,itisaconveniencefunctionthatmakes(<*>fgh)equivalentto(fapply(fapplyfg)h).
Westartbydefiningahelperfunctiontoavoidrepetition.Theage-optionfunctionretrievestheageofapirateasanoptionforus.
Next,wecurrytheavgfunctionto3argumentsandputitintoanoption.Then,weusethe<*>functiontoapplyittotheoptionsa,b,andc.Wegettothesameresult,buthavetheApplicativeFunctortakecareofnilvaluesforus.
TipFunctioncurrying
Curryingisthetechniqueoftransformingafunctionofmultipleargumentsintoahigher-orderfunctionofasingleargumentthatreturnsmoresingle-argumentfunctionsuntilallargumentshavebeensupplied.
Roughlyspeaking,curryingmakesthefollowingsnippetsequivalent:
(defcurried-1(fkj/curry+2))
(defcurried-2(fn[a]
(fn[b]
(+ab))))
((curried-110)20);;30
((curried-210)20);;30
UsingApplicativeFunctorsthiswayissocommonthatthepatternhasbeencapturedasthefunctionalift,asshowninthefollowing:
(defnalift
"Liftsan-aryfunction`f`intoaapplicativecontext"
[f]
(fn[&as]
{:pre[(seqas)]}
(let[curried(fkj/curryf(countas))]
(applyfkc/<*>
(fkc/fmapcurried(firstas))
(restas)))))
ThealiftfunctionisresponsibleforliftingafunctioninsuchawaythatitcanbeusedwithApplicativeFunctorswithoutmuchceremony.BecauseoftheassumptionsweareabletomakeaboutApplicativeFunctors—forinstance,thatitisalsoaFunctor—wecanwritegenericcodethatcanbere-usedacrossanyApplicatives.
Withaliftinplace,ourageaverageexampleturnsintothefollowing:
(let[a(age-option"JackSparrow")
b(age-option"Blackbeard")
c(age-option"HectorBarbossa")]
((aliftavg)abc))
;;#library_design.option.Some{:v56.666668}
WeliftavgintoanApplicativecompatibleversion,makingthecodelookremarkablylikesimplefunctionapplication.Andsincewearenotdoinganythinginterestingwiththeletbindings,wecansimplifyitfurtherasfollows:
((aliftavg)(age-option"JackSparrow")
(age-option"Blackbeard")
(age-option"HectorBarbossa"))
;;#library_design.option.Some{:v56.666668}
((aliftavg)(age-option"JackSparrow")
(age-option"DavyJones")
(age-option"HectorBarbossa"))
;;#library_design.option.None{}
AswithFunctors,wecantakethecodeasitis,andsimplyreplacetheunderlyingabstraction,preventingrepetitiononceagain:
((aliftavg)(i/future(some->(pirate-by-name"JackSparrow")age))
(i/future(some->(pirate-by-name"Blackbeard")age))
(i/future(some->(pirate-by-name"HectorBarbossa")age)))
;;#<Future@17b1be96:#<Success@16577601:56.666668>>
GatheringstatsaboutagesNowthatwecansafelycalculatetheaverageageofanumberofpirates,itmightbeinterestingtotakethisfurtherandcalculatethemedianandstandarddeviationofthepirates’ages,inadditiontotheiraverageage.
Wealreadyhaveafunctiontocalculatetheaverage,solet’sjustcreatetheonestocalculatethemedianandthestandarddeviationofalistofnumbers:
(defnmedian[&ns]
(let[ns(sortns)
cnt(countns)
mid(bit-shift-rightcnt1)]
(if(odd?cnt)
(nthnsmid)
(/(+(nthnsmid)(nthns(decmid)))2))))
(defnstd-dev[&samples]
(let[n(countsamples)
mean(/(reduce+samples)n)
intermediate(map#(Math/pow(-%1mean)2)samples)]
(Math/sqrt
(/(reduce+intermediate)n))))
Withthesefunctionsinplace,wecanwritethecodethatwillgatherallthestatsforus:
(let[a(some->(pirate-by-name"JackSparrow")age)
b(some->(pirate-by-name"Blackbeard")age)
c(some->(pirate-by-name"HectorBarbossa")age)
avg(avgabc)
median(medianabc)
std-dev(std-devabc)]
{:avgavg
:medianmedian
:std-devstd-dev})
;;{:avg56.666668,
;;:median60,
;;:std-dev12.472191289246473}
Thisimplementationisfairlystraightforward.Wefirstretrieveallageswe’reinterestedinandbindthemtothelocalsa,b,andc.Wethenreusethevalueswhencalculatingtheremainingstats.Wefinallygatherallresultsinamapforeasyaccess.
Bynowthereaderwillprobablyknowwherewe’reheaded:whatifanyofthosevaluesisnil?
(let[a(some->(pirate-by-name"JackSparrow")age)
b(some->(pirate-by-name"DavyJones")age)
c(some->(pirate-by-name"HectorBarbossa")age)
avg(avgabc)
median(medianabc)
std-dev(std-devabc)]
{:avgavg
:medianmedian
:std-devstd-dev})
;;NullPointerExceptionclojure.lang.Numbers.ops(Numbers.java:961)
Thesecondbinding,b,returnsnil,aswedon’thaveanyinformationaboutDavyJones.Assuch,itcausesthecalculationstofail.Likebefore,wecanchangeourimplementationtoprotectusfromsuchfailures:
(let[a(some->(pirate-by-name"JackSparrow")age)
b(some->(pirate-by-name"DavyJones")age)
c(some->(pirate-by-name"HectorBarbossa")age)
avg(when(andabc)(avgabc))
median(when(andabc)(medianabc))
std-dev(when(andabc)(std-devabc))]
(when(andabc)
{:avgavg
:medianmedian
:std-devstd-dev}))
;;nil
Thistimeit’sevenworsethanwhenweonlyhadtocalculatetheaverage;thecodeischeckingfornilvaluesinfourextraspots:beforecallingthethreestatsfunctionsandjustbeforegatheringthestatsintotheresultmap.
Canwedobetter?
MonadsOurlastabstractionwillsolvetheveryproblemweraisedintheprevioussection:howtosafelyperformintermediatecalculationsbypreservingthesemanticsoftheabstractionswe’reworkingwith—inthiscase,options.
ItshouldbenosurprisenowthatfluokittenalsoprovidesaprotocolforMonads,simplifiedandshownasfollows:
(defprotocolMonad
(bind[mvg]))
Ifyouthinkintermsofaclasshierarchy,Monadswouldbeatthebottomofit,inheritingfromApplicativeFunctors,which,inturn,inheritfromFunctors.Thatis,ifyou’reworkingwithaMonad,youcanassumeitisalsoanApplicativeandaFunctor.
Thebindfunctionofmonadstakesafunctiongasitssecondargument.ThisfunctionreceivesasinputthevaluecontainedinmvandreturnsanotherMonadcontainingitsresult.Thisisacrucialpartofthecontract:ghastoreturnaMonad.
Thereasonwhywillbecomecleareraftersomeexamples.Butfirst,let’spromoteourOptionabstractiontoaMonad—atthispoint,OptionisalreadyanApplicativeFunctorandaFunctor:
(extend-protocolfkp/Monad
Some
(bind[mvg]
(g(:vmv)))
None
(bind[__]
(None.)))
Theimplementationisfairlysimple.IntheNoneversion,wecan’treallydoanything,sojustlikewehavebeendoingsofar,wereturnaninstanceofNone.
TheSomeimplementationextractsthevaluefromtheMonadmvandappliesthefunctiongtoit.Notehowthistimewedon’tneedtowraptheresultasthefunctiongalreadyreturnsaMonadinstance.
UsingtheMonadAPI,wecouldsumtheagesofourpiratesasfollows:
(defopt-ctx(None.))
(fkc/bind(age-option"JackSparrow")
(fn[a]
(fkc/bind(age-option"Blackbeard")
(fn[b]
(fkc/bind(age-option"HectorBarbossa")
(fn[c]
(fkc/pureopt-ctx
(+abc))))))))
;;#library_design.option.Some{:v170.0}
Firstly,wearemakinguseofApplicative’spurefunctionintheinner-mostfunction.RememberthatroleofpureistoprovideagenericwaytoputavalueintoanApplicativeFunctor.SinceMonadsarealsoApplicative,wemakeuseofthemhere.
However,sinceClojureisadynamicallytypedlanguage,weneedtohintpurewiththecontext—container—typewewishtouse.ThiscontextissimplyaninstanceofeitherSomeorNone.Theybothhavethesamepureimplementation.
Whilewedogettherightanswer,theprecedingexampleisfarfromwhatwewouldliketowriteduetoitsexcessivenesting.Itisalsohardtoread.
Thankfully,fluokittenprovidesamuchbetterwaytowritemonadiccode,calledthedo-notation:
(fkc/mdo[a(age-option"JackSparrow")
b(age-option"Blackbeard")
c(age-option"HectorBarbossa")]
(fkc/pureopt-ctx(+abc)))
;;#library_design.option.Some{:v170.0}
Suddenly,thesamecodebecomesalotcleanerandeasiertoread,withoutlosinganyofthesemanticsoftheOptionMonad.Thisisbecausemdoisamacrothatexpandstothecodeequivalentofthenestedversion,aswecanverifybyexpandingthemacroasfollows:
(require'[clojure.walk:asw])
(w/macroexpand-all'(fkc/mdo[a(age-option"JackSparrow")
b(age-option"Blackbeard")
c(age-option"HectorBarbossa")]
(option(+abc))))
;;(uncomplicate.fluokitten.core/bind
;;(age-option"JackSparrow")
;;(fn*
;;([a]
;;(uncomplicate.fluokitten.core/bind
;;(age-option"Blackbeard")
;;(fn*
;;([b]
;;(uncomplicate.fluokitten.core/bind
;;(age-option"HectorBarbossa")
;;(fn*([c](fkc/pureopt-ctx(+abc)))))))))))
TipItisimportanttostopforamomenthereandappreciatethepowerofClojure—andLispingeneral.
LanguagessuchasHaskellandScala,whichmakeheavyuseofabstractionssuchasFunctors,Applicative,andMonads,alsohavetheirownversionsofthedo-notation.However,thissupportisbakedintothecompileritself.
Asanexample,whenHaskelladdeddo-notationtothelanguage,anewversionofthecompilerwasreleased,anddeveloperswishingtousethenewfeaturehadtoupgrade.
InClojure,ontheotherhand,thisnewfeaturecanbeshippedasalibraryduetothepowerandflexibilityofmacros.Thisisexactlywhatfluokittenhasdone.
Now,wearereadytogobacktoouroriginalproblem,gatheringstatsaboutthepirates’ages.
First,wewilldefineacoupleofhelperfunctionsthatconverttheresultofourstatsfunctionsintotheOptionMonad:
(defavg-opt(compoptionavg))
(defmedian-opt(compoptionmedian))
(defstd-dev-opt(compoptionstd-dev))
Here,wetakeadvantageoffunctioncompositiontocreatemonadicversionsofexistingfunctions.
Next,wewillrewriteoursolutionusingthemonadicdo-notationwelearnedearlier:
(fkc/mdo[a(age-option"JackSparrow")
b(age-option"Blackbeard")
c(age-option"HectorBarbossa")
avg(avg-optabc)
median(median-optabc)
std-dev(std-dev-optabc)]
(option{:avgavg
:medianmedian
:std-devstd-dev}))
;;#library_design.option.Some{:v{:avg56.666668,
;;:median60,
;;:std-dev12.472191289246473}}
Thistimewewereabletowritethefunctionaswenormallywould,withouthavingtoworryaboutwhetheranyvaluesintheintermediatecomputationsareemptyornot.ThissemanticthatistheveryessenceoftheOptionMonadisstillpreserved,ascanbeseeninthefollowing:
(fkc/mdo[a(age-option"JackSparrow")
b(age-option"Blackbeard")
c(age-option"HectorBarbossa")
avg(avg-optabc)
median(median-optabc)
std-dev(std-dev-optabc)]
(fkc/pureopt-ctx{:avgavg
:medianmedian
:std-devstd-dev}))
;;#library_design.option.None{}
Forthesakeofcompleteness,wewillusefuturestodemonstratehowthedo-notationworksforanyMonad:
(defavg-fut(compi/future-callavg))
(defmedian-fut(compi/future-callmedian))
(defstd-dev-fut(compi/future-callstd-dev))
(fkc/mdo[a(i/future(some->(pirate-by-name"JackSparrow")age))
b(i/future(some->(pirate-by-name"Blackbeard")age))
c(i/future(some->(pirate-by-name"HectorBarbossa")
age))
avg(avg-futabc)
median(median-futabc)
std-dev(std-dev-futabc)]
(i/const-future{:avgavg
:medianmedian
:std-devstd-dev}))
;;#<Future@3fd0b0d0:#<Success@1e08486b:{:avg56.666668,
;;:median60,
;;:std-dev12.472191289246473}>>
SummaryThisappendixhastakenusonabrieftouroftheworldofcategorytheory.Welearnedthreeofitsabstractions:Functors,ApplicativeFunctors,andMonads.Theyweretheguidingprinciplebehindimminent’sAPI.
Todeepenourknowledgeandunderstanding,weimplementedourownOptionMonad,acommonabstractionusedtosafelyhandletheabsenceofvalues.
Wehavealsoseenthatusingtheseabstractionsallowustomakesomeassumptionsaboutourcode,asseeninfunctionssuchasalift.Therearemanyotherfunctionswewouldnormallyrewriteoverandoveragainfordifferentpurposes,butthatcanbereusedifwerecognizeourcodefitsintooneoftheabstractionslearned.
Finally,Ihopethisencouragesreaderstoexplorecategorytheorymore,asitwillundoubtedlychangethewayyouthink.AndifIcanbesobold,Ihopethiswillalsochangethewayyoudesignlibrariesinthefuture.
AppendixB.Bibliography[1]RenePardoandRemyLandau,TheWorld’sFirstElectronicSpreadsheet:http://www.renepardo.com/articles/spreadsheet.pdf
[2]ConalElliottandPaulHudak,FunctionalReactiveAnimation:http://conal.net/papers/icfp97/icfp97.pdf
[3]EvanCzaplicki,Elm:ConcurrentFRPforFunctionalGUIs:http://elm-lang.org/papers/concurrent-frp.pdf
[4]ErikMeijer,Subject/ObserverisDualtoIterator:http://csl.stanford.edu/~christos/pldi2010.fit/meijer.duality.pdf
[5]HenrikNilsson,AntonyCourtneyandJohnPeterson,FunctionalReactiveProgramming,Continued:http://haskell.cs.yale.edu/wp-content/uploads/2011/02/workshop-02.pdf
[6]JohnHughes,GeneralisingMonadstoArrows:http://www.cse.chalmers.se/~rjmh/Papers/arrows.pdf
[7]ZhanyongWan,WalidTahaandPaulHudak,Real-TimeFRP:http://haskell.cs.yale.edu/wp-content/uploads/2011/02/rt-frp.pdf
[8]WalidTaha,ZhanyongWan,andPaulHudak,Event-DrivenFRP:http://www.cs.yale.edu/homes/zwan/papers/mcu/efrp.pdf
[9]BenjaminC.Pierce,BasicCategoryTheoryforComputerScientists:http://www.amazon.com/Category-Computer-Scientists-Foundations-Computing-ebook/dp/B00MG7E5WE/ref=sr_1_7?ie=UTF8&qid=1423484917&sr=8-7&keywords=category+theory
[10]SteveAwodey,CategoryTheory(OxfordLogicGuides):http://www.amazon.com/Category-Theory-Oxford-Logic-Guides/dp/0199237182/ref=sr_1_2?ie=UTF8&qid=1423484917&sr=8-2&keywords=category+theory
[11]DuncanCoutts,RomanLeshchinskiy,andDonStewart,StreamFusion:http://code.haskell.org/~dons/papers/icfp088-coutts.pdf
[12]PhilipWadler,Transformingprogramstoeliminatetrees:http://homepages.inf.ed.ac.uk/wadler/papers/deforest/deforest.ps
IndexA
AbstractAlgebraabout/Functors
agileboardcreating,withOm/CreatinganagileboardwithOmstate/Theboardstatecomponents/Componentsoverviewlifecycle/Lifecycleandcomponentlocalstatecomponentlocalstate/Lifecycleandcomponentlocalstatemultiplecolumn-viewcomponents,creating/Remainingcomponentsutilityfunctions,adding/Utilityfunctions
ApplicativeFunctorsabout/ApplicativeFunctorspurefunction/ApplicativeFunctorsfapplyfunction/ApplicativeFunctorsvarargfunction/ApplicativeFunctorsstats,calculatingofages/Gatheringstatsaboutages
ArrowizedFRPabout/ArrowizedFRP
Asteroidsabout/Settinguptheproject
asynchronousdataflowabout/Asynchronousdataflow
asynchronousprogrammingabout/AsynchronousCompositionalEventSystem(CES)aboutprogrammingandconcurrency
AutomatonURL/ArrowizedFRP
AWSusing/InfrastructureautomationEC2/InfrastructureautomationRDS/InfrastructureautomationCloudFormation/Infrastructureautomation
AWSresourcesdashboardbuilding/AWSresourcesdashboardbuilding,withCloudFormation/CloudFormationbuilding,withEC2/EC2building,withRDS/RDSdesigning/Designingthesolutionstubserver,settingup/RunningtheAWSstubserversettingup/Settingupthedashboardproject
Observables,creating/CreatingAWSObservablesObservables,combining/CombiningtheAWSObservablesuserinterface,building/Puttingitalltogether
Bbackpressure
about/Backpressuresamplecombinator/Samplestrategies/Backpressurestrategiesreferencelink/Backpressurestrategies
Bacon.jsURL/FunctionalReactiveProgrammingabout/Asynchronousdataflow
blockingIOabout/FuturesandblockingIO
bufferingabout/Backpressurefixedbuffer/Fixedbufferdroppingbuffer/Droppingbufferslidingbuffer/Slidingbuffer
Ccatchcombinator
about/CatchCES
versusFRP/Asynchronousdataflowversuscore.async/CESversuscore.async
ChestnutURL/AtasteofReactiveProgramming
cljs-startURL/Arespondentapplication,Settinguptheproject
cljxURL/ClojureorClojureScript?about/ClojureorClojureScript?
ClojureURL,fordocumentation/Applicationcomponentsfutures/Clojurefutures
Clojurescriptabout/ClojureScriptandOm
ClojureScriptgameproject,settingup/Settinguptheprojectgameentities,creating/Gameentitiesimplementing/Puttingitalltogetheruserinput,modelingaseventstreams/Modelinguserinputaseventstreamsactivekeysstream,workingwith/Workingwiththeactivekeysstream
CloudFormationabout/Infrastructureautomationused,forbuildingAWSresourcesdashboard/CloudFormationdescribeStacksendpoint/ThedescribeStacksendpointdescribeStackResourcesendpoint/ThedescribeStackResourcesendpoint
combinatorsusing/Combinatorsandeventhandlers
complexWebUIsproblems/TheproblemwithcomplexwebUIsfeatures/TheproblemwithcomplexwebUIs
CompositionalEventSystem(CES)about/AsynchronousCompositionalEventSystem(CES)aboutprogrammingandconcurrency
CompositionalEventSystemsabout/Onemoreflatmapfortheroad
concurrencyabout/AsynchronousCompositionalEventSystem(CES)aboutprogrammingandconcurrency
Contactsapplication
building/BuildingasimpleContactsapplicationwithOmstate/TheContactsapplicationstateproject,settingup/SettinguptheContactsprojectcomponents/Applicationcomponentscursors,using/Omcursorscontacts-appcomponent,creating/Fillingintheblankscontacts-viewcomponent,creating/Fillingintheblankscontact-summary-viewcomponent,creating/Fillingintheblanksdetails-panel-viewcomponent,creating/Fillingintheblankscontact-details-viewcomponent,creating/Fillingintheblankscontact-details-form-viewcomponent,creating/Fillingintheblankscontactinformation,updating/Fillingintheblanks
core.asyncabout/core.asyncCSP/Communicatingsequentialprocessesstockmarketapplication,rewriting/Rewritingthestockmarketapplicationwithcore.asyncerrorhandling/Errorhandlingbackpressure/Backpressuretransducers/Transducersandcore.asyncfeatures/SummaryversusCES/CESversuscore.async
core.asyncchannelsusing/Intercomponentcommunication
CSPabout/CommunicatingsequentialprocessesURL/Communicatingsequentialprocesses
cursorsabout/Omcursors
Ddata
fetching,inparallel/Fetchingdatainparalleldataflowprogramming
about/DataflowprogrammingDataTransfer
referencelink/Utilityfunctionsdc-lib
URL/DataflowprogrammingdescribeDBInstancesendpoint
about/ThedescribeDBInstancesendpointdescribeInstancesendpoint
about/ThedescribeInstancesendpointdescribeStackResourcesendpoint
about/ThedescribeStackResourcesendpointdescribeStacksendpoint
about/ThedescribeStacksendpointdroppingbuffer
about/Droppingbuffer
EEC2
about/Infrastructureautomationused,forbuildingAWSresourcesdashboard/EC2describeInstancesendpoint/ThedescribeInstancesendpoint
Elmabout/First-orderFRPURL/First-orderFRP
errorhandlingabout/ErrorhandlingonErrorcombinator/OnErrorcatchcombinator/Catchretrycombinator/Retryreferencelink/Retryincore.async/Errorhandling
eventhandlersusing/Combinatorsandeventhandlers
eventsabout/Signalsandevents
Ffactorymethods
referencelink/CustomObservablesfirst-orderFRP
about/First-orderFRPfixedbuffer
about/FixedbufferFlapjax
about/ComplexGUIsandanimationsflatmap
about/Flatmapandfriendswithmultiplevalues/Onemoreflatmapfortheroad
ForkJoinPoolURL/Themoviesexamplerevisitedusing/FuturesandblockingIO
FRPabout/FunctionalReactiveProgrammingimplementationchallenges/ImplementationchallengesversusCES/Asynchronousdataflow
FRP,usecasesasynchronousprogramming/Asynchronousprogrammingandnetworkingnetworking/AsynchronousprogrammingandnetworkingcomplexGUIs/ComplexGUIsandanimationsanimations/ComplexGUIsandanimations
FunctionalProgrammingabout/Lessonsfromfunctionalprogramming
functioncurryingabout/ApplicativeFunctors
Functorsabout/FunctorsOptionFunctor/TheOptionFunctorApplicativeFunctors/ApplicativeFunctors
futuresabout/Clojurefuturescreating,inimminent/CreatingfuturesblockingIO/FuturesandblockingIO
GGraphicalUserInterfaces(GUIs)
about/Object-orientedReactiveProgramming
HHaskell
about/Monadshigher-orderFRP
about/Higher-orderFRPhistory,ReactiveProgramming
about/Abitofhistorydataflowprogramming/Dataflowprogrammingobject-orientedReactiveProgramming/Object-orientedReactiveProgrammingLANguageforProgrammingArraysatRandom(LANPAR)/ThemostwidelyusedreactiveprogramObserverdesignpattern/TheObserverdesignpatternFRP/FunctionalReactiveProgramminghigher-orderFRP/Higher-orderFRP
Iimminent
about/Imminent–acomposablefutureslibraryforClojureURL/Imminent–acomposablefutureslibraryforClojurefutures,creating/Creatingfuturescombinators,using/Combinatorsandeventhandlerseventhandlers,using/Combinatorsandeventhandlersexample/Themoviesexamplerevisited
implementationchallenges,FRPfirst-orderFRP/First-orderFRPasynchronousdataflow/AsynchronousdataflowArrowizedFRP/ArrowizedFRP
incidentalcomplexityabout/Identifyingproblemswithourcurrentapproachremoving,withRxClojure/RemovingincidentalcomplexitywithRxClojure
infrastructureautomationproblem/TheproblemwithAWS/Infrastructureautomation
intercomponentcommunicationabout/Intercomponentcommunicationagileboard,creating/CreatinganagileboardwithOm
Iteratorinterfaceabout/Observer–anIterator’sdual
JJavaInterop
URL/FillingintheblanksJSPerf
URL/TheproblemwithcomplexwebUIs
LLANguageforProgrammingArraysatRandom(LANPAR)
about/Themostwidelyusedreactiveprogramlein-cljsbuild
URL/ClojureorClojureScript?about/ClojureorClojureScript?
Mmacros
referencelink/GameentitiesManagedBlocker
referencelink/FuturesandblockingIOmap
about/ThesemanticsofmapFunctors/Functors
MimmoCosenzaURL/SettinguptheContactsproject
minimalCESframeworkabout/AminimalCESframeworkproject,settingup/ClojureorClojureScript?publicAPI,designing/DesigningthepublicAPItokens,implementing/Implementingtokenseventstreams,implementing/Implementingeventstreamsbehaviors,implementing/Implementingbehaviorsrespondentapplication,building/Arespondentapplication
Monadabout/Onemoreflatmapfortheroad
Monadsabout/Monadsbindfunction/Monadsusing/Monadsexample/Monads
monetURL/Settinguptheproject
NNetflix
about/Asynchronousprogrammingandnetworking
Oobject-orientedReactiveProgramming
about/Object-orientedReactiveProgrammingObservables
creating/CreatingAWSObservablescombining/CombiningtheAWSObservables
observablescreating/CreatingObservablescustomobservables,creating/CustomObservablesmanipulating/ManipulatingObservables
ObservableSequencesabout/Asynchronousdataflow
Observerdesignpatternabout/TheObserverdesignpattern,TheObserverpatternrevisitedIteratorinterface/Observer–anIterator’sdual
Omabout/ClojureScriptandOmContactsapplication,building/BuildingasimpleContactsapplicationwithOmagileboard,creating/CreatinganagileboardwithOm
om-starttemplateURL/SettinguptheContactsproject
OmProjectManagement/CreatinganagileboardwithOmonErrorcombinator
about/OnErrorOptionFunctor
about/TheOptionFunctorusecase/Findingtheaverageofages
PPurelyFunctionalDataStructures
about/LessonsfromfunctionalprogrammingURL/Lessonsfromfunctionalprogramming
RRDS
about/Infrastructureautomationused,forbuildingAWSresourcesdashboard/RDSdescribeDBInstancesendpoint/ThedescribeDBInstancesendpoint
React.jsabout/EnterReact.jsFunctionalProgramming/Lessonsfromfunctionalprogramming
ReactiveCocoaURL/FunctionalReactiveProgrammingabout/Asynchronousdataflow
ReactiveExtensions(Rx)about/Asynchronousdataflow,ReagiandotherCESframeworksdrawbacks/ReagiandotherCESframeworks
Reagicomparing,withotherCESframeworks/ReagiandotherCESframeworksabout/ReagiandotherCESframeworks
respondentapplicationbuilding/Arespondentapplication
retrycombinatorabout/Retry
Rxobservables,creating/CreatingObservablesobservables,manipulating/ManipulatingObservablesflatmap/Flatmapandfriends
RXerrorhandling/Errorhandling
RxClojureURL/CreatingObservables
RxJavaURL/FunctionalReactiveProgramming,CreatingObservables
RxJSURL/AtasteofReactiveProgramming
Ssamplecombinator
about/SampleScala
about/MonadsScheduledThreadPoolExecutorpool/BuildingastockmarketmonitoringapplicationSemigroups
about/Functorssignals
about/Signalsandeventssinewaveanimation
creating/AtasteofReactiveProgrammingtime,creating/Creatingtimecoloring/Morecolorsupdating/Makingitreactiveenhancing/Exercise1.1
slidingbufferabout/Slidingbuffer
stockmarketapplicationrewriting,withcore.async/Rewritingthestockmarketapplicationwithcore.asyncapplicationcode,implementing/Implementingtheapplicationcode
stockmarketmonitoringapplicationbuilding/Buildingastockmarketmonitoringapplicationrollingaverages,displaying/Rollingaveragesproblems,identifying/Identifyingproblemswithourcurrentapproachincidentalcomplexity,removingwithRxClojure/RemovingincidentalcomplexitywithRxClojureobservablerollingaverages/Observablerollingaverages
stubserversettingup/RunningtheAWSstubserver
TThreadPool
about/Fetchingdatainparalleltools.namespace
URL/Implementingeventstreamsusing/Implementingeventstreams
transducersabout/Transducersreferencelink/Transducerswithcore.async/Transducersandcore.async
transitabout/SettingupthedashboardprojectURL/Settingupthedashboardproject
TrelloURL/Intercomponentcommunication
VVirtualDOM
about/Lessonsfromfunctionalprogramming
Related Documents
![Java High Performance Reactive Programmingiproduct.org/.../04/IPT_Reactive_Programming_Java.pdf · Reactive Programming. Functional Programing Reactive Programming [Wikipedia]: a](https://static.cupdf.com/doc/110x72/5ec60814df097e0643499b13/java-high-performance-reactive-reactive-programming-functional-programing-reactive.jpg)
