Page 1
Gooda&ernoonandthanksforhavingmehere.InthistalkIwanttolookatthedesignchallengesofsystemsthatan9cipateusers’needsandthenactonthem.Thatmeansthatitsitsattheintersec9onoftheinternetofthings,userexperiencedesignandmachinelearning,whichtomeisnewterritoryfordesignerswhomayhavedealtwithoneofthosedisciplinesbefore,butrarelyallthreeatonce.Thetalkisdividedintoseveralparts:itstartswithanoverviewofhowIthinkInternetofThingsdevicesareprimarilycomponentsofservices,ratherthanbeingself-containedexperiences,howpredic9veanaly9csenableskeycomponentsofthoseservices,andthenIfinishbytryingtotoiden9fyuseexperienceissuesaroundpredic9vebehaviorandsugges9onsforpaDernstoamelioratethoseissues.Acoupleofcaveats:-Ifocusalmostexclusivelyontheconsumerinternetofthings.Althoughpredic9veanaly9csisanimportantpartoftheIndustrialInternetofThingsforthingslikepredic9vemaintenance,Ifeelit’sREALLYkeytotheconsumerIoTbecauseofwhatexperiencesitcreatesforpeople.-IwanttopointoutthatfewifanyoftheissuesIraisearenew.Thoughtheterm“internetofthings”ishotrightnow,theideashavebeendiscussedinresearchcirclesformorethan20years.Searchfor“ubiquitouscompu9ng,”“ambientintelligence,”and“pervasivecompu9ng”andit’llhelpyoukeepfromreinven9ngthewheel.-Finally,mostofmyslidesdon’thavewordsonthem,soI’llmakethecompletedeckwithatranscriptavailableassoonI’mdone.
0
Page 2
Let me begin by telling you a bit about my background. I�m a user experience designer. I was one of the first professional Web designers. This is the navigation for a hot sauce shopping site I designed in the spring of 1994.
1
Page 3
I’vealsoworkedontheuserexperiencedesignofalotofconsumerelectronicsproductsfromcompaniesyou’veprobablyheardof.
2
Page 4
Iwroteacoupleofbooksbasedonmyexperienceasadesigner.Oneisacookbookofuserresearchmethods,andtheseconddescribeswhatIthinkaresomeofthecoreconcernswhendesigningnetworkedcomputa9onaldevices.
3
Page 5
Ialsocofoundedacoupleofcompanies.Thefirst,Adap9vePath,you’refamiliarwith,andwiththesecondone,ThingM,Igotdeepintodevelopinghardware.
4
Page 6
TodayIworkforPARC,thefamoushardware,so&wareandAIresearchlab,asaprincipalinitsInnova9onServicesgroup,whichisPARC’sconsul9ngarm.Wehelpcompaniesreducetheriskofadop9ngnoveltechnologiesusingamixofethnographicresearch,userexperiencedesignandinnova9onstrategy.
5
Page 7
IwantstartbyfocusingonwhatIfeelisakeyaspectofconsumerIoTthat’so&enmissedwhenpeoplefocusonthehardwareoftheIoT,whichisthatconsumerIoTproductshaveaverydifferentbusinessmodelthantradi9onalconsumerelectronics.Tradi9onally,acompanymadeanelectronicproduct,sayaturntable,theyfoundpeopletosellitforthem,theyadver9seditandpeopleboughtit.Thatwastradi9onallytheendofthecompany’srela9onshipwiththeconsumerun9lthatpersonboughtanotherthing,andallofthevalueoftherela9onshipwasinthedevice.WiththeIoT,thesaleofthedeviceisjustthebeginningoftherela9onshipandholdsalmostnovalueforeitherthecustomerorthemanufacturer.Letmeexplain…
6
Page 8
As value shifts to services, the devices, software applications and websites used to access it—its avatars—become secondary. A camera becomes a really good appliance for taking photos for Flickr, while a TV becomes a nice Flickr display that you don’t have to log into every time, and a phone becomes a convenient way to take your Flickr pictures on the road.
Hardware becomes simultaneously more specialized and devalued as users see “through” each device to the service it represents. The hardware exists to get better value out of the service.
7
Page 9
Amazonreallygetsthis.Here�satellingolderadfromAmazonfortheKindle.It’ssaying�Look,usewhateverdeviceyouwant.Wedon�tcare,aslongyoustayloyaltoourservice.Youcanbuyourspecializeddevices,butyoudon�thaveto.�
8
Page 10
WhenFirewasreleased3yearsago,JeffBezosevencalleditaservice.
9
Page 11
NetFlixisanothergoodmediaexample.Itfeelsnaturaltopauseamovieononedeviceandcon9nueitonanotherbecausefromyourperspec9vethere’sonlyONENeflix.Dropboxcreatesthisforfiles,Evernotefornotes,andAngryBirdsforscoresynchroniza9on.TheserviceiswhereyouraDen9onis,thedeviceistheretogiveyouaccesstotheservice,butotherthanaconvenientformfactor,thehardwareislargelydisposable.
10
Page 12
Mostlarge-scaleIoTproductsareserviceavatars.Theyusespecializedsensorsandactuatorstosupportaservice,buthaveliDlevalue—ordon’tworkatall—withoutthesuppor9ngservice.SmartThingsclearlystateditsserviceofferingrightupfrontontheirsite.Thefirstthingtheysayabouttheirproductlineisnotwhatthefunc9onalityis,butwhateffecttheirservicewillachievefortheircustomers.Theirhardwareproducts’func9onality,howtheywilltechnicallysa9sfytheservicepromise,isalmostana&erthought.
11
Page 13
ComparethattoX10,theirspiritualpredecessorthat’sbeeninthebusinessformorethan20years.AllthatX10tellsisyouiswhatthedevicesare,notwhattheservicewillaccomplishforyou.Idon’tevenknowifthereISaservice.WhyshouldIcarethattheyhave“modules”?Ishouldn’t,andIdon’t.
12
Page 14
Sowhatdotheseservicesoffer?
13
Page 15
Simpleconnec9vityhelpswhenyou’retryingtomaximizetheefficiencyofafixedprocess,butthat’snotaproblemthatmostpeoplehave.We’vebeenabletosimplyconnectvariousdevicestoacomputersinceaTandyColorComputerscouldlightsoffandonoverX10in1983.Thatwasn’tveryusefulthen,andit’snotveryusefulnow.YoucanreplacetheTandywithaniPhoneandthelampwithawashingmachineandyougetthevalueproposi9onofmostsimpleconnecteddevices.That’snotinteres9ng.
14
Page 16
Ithinktherealconsumervalueconnectedservicesofferistheirabilitytomakesenseoftheworldonpeople’sbehalf,toreducepeople’scogni9veload,ratherthanincreasingit,byallowingthemtointeractwithdevicesatahigherlevelthansimpletelemetryandcontrol.Fundamentally,humansaregoodpaDernmatchersatcertainthings,butwe’renotbuilttocollectandmakesenseofhugeamountsofdataortoar9culateourneedsascomplexsystemsofmutuallyinterdependentcomponents.Computersaregreatatit.Theycanmakesta9s9calmodelsfrommanydatasourcesacrossspaceand9meandthentrytomaximizestheprobabilityofadesiredoutcome.Apersonprogrammingadevicecanexpresswhatthey’refamiliarwith,ortrytocreateanabstrac9onbasedontheirpastexperience,some9meswithconsiderableskill,butamodellearnedfromtheoutcomeofthousandsofsitua9onsacrossmanypeopleandlongperiodsof9mecancompensateformuchwidervarietyofsitua9onsinamorenuancedwaythananindividual’sperspec9vewilleverbeableto.
15
Page 17
Predic9onisattheheartofthevalueproposi9onmanyofthemostcompellingIoTproductsoffering,star9ngwiththeNest.TheNestsaysthatitknowsyou.Howdoesitknowyou?Itpredictswhatyou’regoingtowantbasedonyourpastbehavior.
16
Page 18
Amazon’sEchospeakersaysit’scon9nuallylearning.Howisthat?Predic9veanaly9cs,predic9vemachinelearning.
17
Page 19
TheBirdismartsmokealarmsaysitwilllearnover9me,whichisagainthesamething.
18
Page 20
Jaguarcomesrightoutwithit.Theyevenobliquelyreferencethe40yearsofar9ficialintelligenceresearchthatpowerspredic9veanaly9csbycallingtheircarnotjustlearning,butlearningANDintelligent.
19
Page 21
TheEdynplantwateringsystemadaptstoeverychange.Whatisthatadapta9on?Predic9veanaly9cs.
20
Page 22
Canary,ahomesecurityservice.
21
Page 23
Here’sfoobot,anairqualityservice.
22
Page 24
Predic9vebehaviorcreatesapreDyseduc9veworldofespressomachinesthatstartbrewingasyou’rethinkingit’sagood9meforcoffee,andreorderyourfavoriteblend;officelightsthatdimwhenit’ssunny,powerischeapandyou’renotdoinganythingthatneedsthem;andfoodtruckcaravansthatshowupjustasthecrowdintheparkisgennghungry.Theproblemisthatalthoughthevalueproposi9onisofabeDeruserexperience,it’sunspecificinthedetails.Exactlyhowwillourexperienceoftheworld,ourabilitytouseallthecollecteddata,becomemoreefficientandmorepleasurable?NowI’dliketooffersomeini9althoughtsonuserexperiencedesignforpredic9veanaly9csfortheinternetofthings.We’res9llearlyinourunderstandingofdesignforpredic9vedevices,sorightnowtheproblemsareworsethansolu9onsandIwanttostartbyar9cula9ngtheissuesI’veobservedinourwork.
23
Page 25
We’veneverhadmechanicalthingsthatmakesignificantdecisionsontheirown,thefirstmajorissueisaroundexpecta9ons.Asdevicesadapttheirbehavior,howwilltheycommunicatethatthey’redoingso?Dowetreatthemlikeanimals?Dowes9ckasignonthemthatsays“adap9ng”,likethelightonavideocamerasays“recording”?Shouldmychairvibratewhenadjus9ngtomyposture?Howwillusers,orjustpassers-by,knowwhichthingsadaptandwhichmerelybehave?Icouldendupsinnguncomfortableforalong9mebeforerealizingmychairdoesn’tadaptonitsown.Howshouldsmartdevicessettheexpecta9onthattheymaybehavedifferentlyinwhatappearstopeopleasaniden9calsetofcircumstances?ChairbyRaffaelloD'Andrea,MaDDonovanandMaxDean.
24
Page 26
Theironyinpredic9vesystemsisthatthey’repreDyunpredictable,atleastatfirst.Whenmachinelearningsystemsarenew,they’reo&eninaccurateandunpredictable,whichisnotwhatweexpectfromourdigitaldevices.60%-70%accuracyistypicalforafirstpass,buteven90%accuracyisn’tenoughforapredic9vesystemtofeelright,sinceifit’smakingdecisionsallthe9me,it’sgoingtobemakingmistakesallthe9me,too.It’sfineifyourhouseisacoupleofdegreescoolerthanyou’dlike,butwhatifyourwheelchairrefusestogotoadrinkingfountainnexttoadoorbecauseit’sbeentrainedondoorsanditcan’ttellthat’snotwhatyoumeaninthisoneinstance?Forallthe9mesasystemgetsitright,it’sonthemistakesthatwejudgeitandacouplesuchinstancescanshaDerpeople’sconfidence.AliDledoubtaboutwhetherasystemisgoingtodotherightthingisenoughtoturnaUXthat’srightmostofthe9meintoonethat’smoretroublethanit’sworth.Whenthathappens,you’vemorethanlikelylostyourcustomer.Soonerthanwethink,inaccuratepredic9vebehaviorisn’tgoingtobeanisolatedincident,it’sgoingtobethenorm.Whenthereare100connecteddevicessimultaneouslyac9ngonpredic9onsandeachis99%accurate,thenoneisalwayswrong.Sotheproblemis:Howcanyoudesignauserexperiencetomakeadevices9llfunc9onal,s9llvaluable,s9llfun,evenwhenit’sspewingjunkbehavior?Howcanyoudesignforuncertainty?
25
Page 27
Thelastissuecomesasaresultoftheprevioustwo:control.Howcanwecontrolthesedevices,whentheirbehaviorisbydefini9onsta9s9calandunpredictablebyhumans?Ontheonehandyoucanmangleyourdevice’spredic9vebehaviorbygivingittoomuchdata.WhenIvisitedNestoncetheytoldmethatnoneoftheNestsintheirofficeworkedwellbecausethey’reconstantlyfiddlingwiththem.Inmachinelearningthisiscalledovertraining.Theotherhand,ifIhavenodirectwaytocontrolitotherthanthroughmyownbehavior,howdoIadjustit?AmazonandNeflix’srecommenda9onsystems,whichisakindofpredic9veanaly9cssystem,giveyousomecontextaboutwhytheyrecommendedsomething,butwhatdoIdowhenmyonlyinterfaceisagardenhose?
26
Page 28
Hereare4paDernsI’veobservedindevelopingpredic9vesystemsthatIthinkmaptotheIoT.FormostoftheseI’mgoingtobeusingexamplesfromNestandrecommendersystemslikeAmazon’s,Google’sandNeflix’swhichhavebeenusingsimilarpredic9vetechnologiesforyearsandhaveaddressedsomeoftheseissues.
27
Page 29
MyfirstpaDernisametapaDern.Pleasehaveausermodel,auser-facingstory,foreverystageofthemachinelearningandpredic9onprocess,evenifit’sastepthatisinvisibletousersandcustomers.[Acquire]Howwillyouincen9vizepeopletoadddatatothesystematall?WhyshouldIuploadmycar’sdashcamvideotoyourtrafficpredic9onsystemEVERYDAY?[Extract]Howwillyoucommunicateyou’reextrac9ngfeatures?Googlespeechtotextshowspar9alphrasesasyou’respeakingintoit,andvisiblycorrectsitself.ThatUIthatsimultaneouslytellsusersit’spullinginforma9onoutoftheirspeechandittrainstheminhowtomeetthealgorithmhalfway.[Classify]Howdomachine-generatedclassifica9onscomparetopeople’sorganiza9onofthesamephenomena?EtceteraEtceteraBecausemachinelearningandpredic9onaresonovelandsomanyofthestepsareintertwined,youneedtocareabouttheUXineverysinglestepoftheprocess.
28
Page 30
Whendealingwithapersonorananimal,unpredictablebehaviorisexpectedandtolerated,buttodayweexpectdigitalsystemswillbehaveconsistentlyandthereasonsfortheirbehaviorwillbeclear.NeitheroftheseistruefortheUXofpredic9vesystems,whichdon’tnecessarilybehaveiden9callyinsimilarcircumstances,whichchangetheirbehaviorover9me,andinwhichthereasonsforthebehaviormaynotbeobvious.Apredic9veUXfirstneedstoexplainthenatureofthedevice,todescribeitistryingtopredict,thatit’stryingtoadapt,thatit’sgoingtosome9mesbewrong,toexplainhowit’slearning,andhowlongit’lltakebeforeitcrossesoverfromcrea9ngmoretroublethanbenefit.GoogleNowdescribeswhyacertainkindofcontentwasselected,whichsetstheexpecta9onthatthesystemwillrecommendotherthingsbasedonotherkindsofcontentyou’verequested.Nest’sFAQexplainsyoushouldn’texpectyourthermostattomakeamodelofwhenyou’rehomeornotun9lit’sbeenopera9ngforaweekorso.
29
Page 31
Predic9vebehavioraboutsequencesofac9vi9es.Manypredic9veUXissuesaroundexpecta9onsanduncertaintyhave9meastheirbasis:whatwereyouexpec9ngtohappenandwhy.Ifitdidn’thappen,why?Ifsomethingelsehappened,orithappenedatanunexpected9me,whydidthathappen?Tellingthatadevicehasactedonyourbehalf,andthatit’sgoingtoact—andHOWit’sgoingtoact—inthefuturegivespeopleamodelofhowit’sworkingandreducesuncertainty.Nest,forexample,hasacalendarofitsexpectedbehavior,anditshowsthatit’sac9ngonyourbehalftochangethetemperature,andwhenyoucanexpectthattemperaturewillbereached.
30
Page 32
Youhavetogivepeopleaclearwaytoteachthesystemandtellitwhenitsmodeliswrong.Sta9s9calsystems,bydefini9on,don’thavesimplerulesthatcanbechanged.Therearen’tobvioushandlestoturnordialstoadjust,becauseeverythingisprobabilis9c.Ifthemodelismadefromdatacollectedbyseveraldevices,whichdeviceshouldIinteractwithtogetittochangeitsbehavior?GoogleNowaskswhetherIwantmoreinforma9onfromasiteIvisited,Amazonshowsaexplana9onofwhyitgavemeasugges9on.MappingthistotheconsumerIoTmeanswaymoreexplana9onthanwe’recurrentlygenng,whichiseitherthatathinghashappened,orithasn’t.
31
Page 33
Finally,don’tautomate.Thesesystem’sshouldn’ttrytoreplacepeople,buttosupportthem,toaugmentandextenttheircapabili9es,nottoreplacethem.MeshfireisasocialmediaengagementtoolthathasamachinelearningassistantcalledEmberthatdoesn’ttrytoreplacethesocialmediamanager.Insteaditmanagesthemanager’stodolist.Itaddsthingsthatitthinksaregoingtobeinteres9ng,deletesoldthings,andrepriori9zesthemanager’slistbasedonwhatitthinksisimportant.Emberaugmentsthecapabili9esofthesocialmediamanager.Ithelpsthatpersonfocusonwhat’simportantsothattheycanbesmarterabouttheirdecisions.Itdoesn’ttrytobesmarterthantheyare.
32
Page 34
Finally,formetheIoTisnotaboutthethings,buttheexperiencecreatedbytheservicesforwhichthethingsareavatars.
33
Page 35
Machinelearningalgorithmsusedtobestrictlybehind-the-scenes,butintheIoTtheyareactorsinourlives,soasdesignersit’sourresponsibilitytounderstandthesitua9onswherethealgorithmsandthedevicestheycontrolinteractwithpeople’slives,especiallysincethere’sadeepsymbio9crela9onshipbetweenthedatathatcomprisesthemodels,thebehaviorthosemodelsinduceandthepeoplewhoaretheintendedbeneficiaries.Ul9matelyweareusingthesetoolstoextendourcapabili9es,tousethedigitalworldasanextensionofourminds.Todothatwellwehavetorespectthatasinteres9ngandpowerfulasthesetechnologiesare,theyares9llintheirinfancy,andourjobasentrepreneurs,developersanddesignerswillbetocreatesystems,services,thathelppeople,ratherthanaddingextraworkinthenameofsimplis9cautoma9on.Whatwewanttocreateisasymbio9crela9onshipwherewe,andourpredic9vesystems,worktogethertocreateaworldthatprovidesthemostvalue,fortheleastcost,forthemostpeople,forthelongest9me.
34