The d.r.e.a.m. device Alex Do Ken Langford Therese Peffer Colleen Whitney.

the d.r.e.a.m. the d.r.e.a.m. devicedevice

Alex DoAlex DoKen LangfordKen LangfordTherese PefferTherese PefferColleen WhitneyColleen Whitney

Project GoalsProject Goals

Develop a Develop a ddemand emand rresponse esponse eenabled nabled aappliance ppliance mmanager… aka the d.r.e.a.m. anager… aka the d.r.e.a.m. device!device!

The device will:The device will:– Keep track of electricity use and spendingKeep track of electricity use and spending– Provide an electricity “price forecast”Provide an electricity “price forecast”– Control heating and cooling (thermostat)Control heating and cooling (thermostat)

The d.r.e.a.m. deviceThe d.r.e.a.m. device

I.I. Heuristic EvaluationHeuristic Evaluation

II.II. Current DesignCurrent Design

III.III. Pilot StudyPilot Study

IV.IV. What We Plan To DoWhat We Plan To Do

V.V. What We LearnedWhat We Learned

I. Heuristic I. Heuristic EvaluationEvaluation

SylViA’s EvaluationSylViA’s Evaluation

First interactive prototype:First interactive prototype: Java appletJava applet

Two marks of Severity 4: Two marks of Severity 4: usability catastropheusability catastrophe imperative to fiximperative to fix

– Temperature “comfort zones”Temperature “comfort zones”– Consistency in usage graphsConsistency in usage graphs

Eight marks of Severity 3: Eight marks of Severity 3: major usability problem; important to fixmajor usability problem; important to fix

– Clutter of left-hand screenClutter of left-hand screen– Use of graphs in right-hand screenUse of graphs in right-hand screen– Secondary bottom-tabsSecondary bottom-tabs

Left screenLeft screen

Arrows do not work, Arrows do not work, too close to right sidetoo close to right side

Comfort zone colors Comfort zone colors confusing and confusing and overwhelmingoverwhelming

Usage GraphsUsage Graphs

Today vs. Last 7 or Last 30 days – graph formatsToday vs. Last 7 or Last 30 days – graph formats Hard to read information and translateHard to read information and translate

Bottom TabsBottom Tabs

GUI BlooperGUI Blooper 3 different heuristics:3 different heuristics:

– H6 Recognition rather H6 Recognition rather than recall;than recall;Tabs not obviousTabs not obvious

– H8 Aesthetics and H8 Aesthetics and minimalist designminimalist design

– H2 Match between H2 Match between system and real world;system and real world;Violate metaphor of Violate metaphor of tabbed file cardstabbed file cards

SettingsSettings

Too many radio Too many radio buttonsbuttons

Not strong Not strong connection connection between between temperature and temperature and scheduleschedule

ConfusingConfusing

II. Current DesignII. Current Design

22ndnd Interactive Prototype Interactive Prototype

Prototyping language changed to combination of Prototyping language changed to combination of HTML & JavaScriptHTML & JavaScript

Comfort Zone & Temp ControlComfort Zone & Temp Control

Comfort zone eliminated; triangle changes color to match current pricing

Arrows moved

GraphsGraphs

Grouped basedon content

New graph created

Tabs & ClutterTabs & Clutter

Created nested tabs

Date/time info to toolbar

Forecast & PricingForecast & Pricing

New pricing info

Settings - TemperatureSettings - Temperature

Settings – ScheduleSettings – Schedule

Not Implemented in Second Not Implemented in Second Interactive PrototypeInteractive Prototype

Interactive setup Interactive setup wizardswizards

HelpHelp VideosVideos

Help & Videos

Wizards

III. Pilot StudyIII. Pilot Study

Test SubjectsTest Subjects

Who: 3 people who were similar to our personasWho: 3 people who were similar to our personas– Particularly interested in finding someone over 60Particularly interested in finding someone over 60

“Mabel”

60+, marriedLives in condominiumDifficulty w/small type

“Alison”

Mother of 2Live in fairly large house*Original test subject from low-fidelity testing

“Tim”

Married w/ 3 childrenSome collegeComfortable with electronics, timers

Main TasksMain Tasks

ThermostatThermostat– Info: Find current temperatureInfo: Find current temperature– Action: Set temperature preferences (for dynamic Action: Set temperature preferences (for dynamic

prices)prices)– Action: Set schedule of temperature preferencesAction: Set schedule of temperature preferences

Dynamic PricingDynamic Pricing– Info: Find current priceInfo: Find current price– Decision: When to use appliances?Decision: When to use appliances?

Budget and UsageBudget and Usage– Info: How deep into spending target?Info: How deep into spending target?– Decision: How to conserve?Decision: How to conserve?

Test MeasuresTest Measures

How long it took to accomplish each task How long it took to accomplish each task

Which pathways were taken (how much did Which pathways were taken (how much did they struggle to find what they needed?)  they struggle to find what they needed?) 

Level of certainty and satisfaction with task Level of certainty and satisfaction with task completion (how clear was the feedback?) completion (how clear was the feedback?)

Amount of prompting neededAmount of prompting needed

Results: GeneralResults: General

Overall, results per task were qualified as significant, slight, Overall, results per task were qualified as significant, slight, or no problemor no problem– Of 12 total tasks:Of 12 total tasks:

Significant Significant ProblemsProblems

Slight Slight ProblemsProblems

““Tim”Tim” 33 22

““Alison”Alison” 00 55

““Mabel”Mabel” 11 44

Main problems resulted from using “Away” switch, determining Main problems resulted from using “Away” switch, determining current price, and setting “Away” temperaturecurrent price, and setting “Away” temperature

Post-test survey results indicated that the device generally Post-test survey results indicated that the device generally performed to the users’ satisfaction and that they were able to performed to the users’ satisfaction and that they were able to understand dynamic pricing by using the deviceunderstand dynamic pricing by using the device

Results: Hold / Normal / AwayResults: Hold / Normal / Away

Switch to away modeSwitch to away mode– 2/3 subjects didn’t know 2/3 subjects didn’t know

what they were looking what they were looking for, other had already for, other had already seen it in lo-fi testseen it in lo-fi test

– Attention focused on Attention focused on right screen rather than right screen rather than left sideleft side

Results: Price IndicatorResults: Price Indicator

Find current priceFind current price– None of subjects used price indicator on left sideNone of subjects used price indicator on left side– All used price forecast by looking at “Cost” screenAll used price forecast by looking at “Cost” screen

Results: Temp SettingResults: Temp Setting

Well understood:Well understood:– Navigation: “Temperature” Navigation: “Temperature”

screen under “Settings” was screen under “Settings” was easy to findeasy to find

– Setpoints: dragging triangles Setpoints: dragging triangles worked wellworked well

– Profiles: Task specified setting Profiles: Task specified setting for Away, and all identified for Away, and all identified correct profilecorrect profile

Problem:Problem:– Per-price behavior: All had Per-price behavior: All had

difficulty setting the temperature difficulty setting the temperature specifically for the High price specifically for the High price period (specified in task)period (specified in task)

– Could be interface problem, or Could be interface problem, or could be conceptual problemcould be conceptual problem

IV. What We Plan to IV. What We Plan to DoDo

Interface ChangesInterface Changes

Left side worked well. Left side worked well. Minor modifications Minor modifications needed:needed:– Make better visual proxy for Make better visual proxy for

“knob”“knob”– Better feedback for Better feedback for

away/hold/override modesaway/hold/override modes– Visual difference between Visual difference between

sets of radio buttonssets of radio buttons

Interface ChangesInterface Changes

Revise tabs to show Revise tabs to show highlighting more highlighting more clearlyclearly

Incorporate price Incorporate price display into “time display into “time bar”bar”

Interface ChangesInterface Changes

Price forecast, budget Price forecast, budget and past usage and past usage screens very effectivescreens very effective– Reduce visual clutterReduce visual clutter

Problems with Problems with navigation among navigation among screensscreens– Card sortingCard sorting

Interface ChangesInterface Changes

Relationship of settings Relationship of settings screens, affordances screens, affordances clear clear – Add wizardsAdd wizards– Add help videoAdd help video

Problem with Problem with temperature settings: temperature settings: poor understanding of poor understanding of demand response?demand response?– Formal testingFormal testing

Other Issues for TestingOther Issues for Testing

Alternatives for displaying usage information: Alternatives for displaying usage information: – pie chartspie charts– bar chartsbar charts– iconsicons

Differences in device use, display preferences by:Differences in device use, display preferences by:– ageage– technical abilitytechnical ability– language proficiencylanguage proficiency– education leveleducation level

Interaction between radio buttons on left-hand sideInteraction between radio buttons on left-hand side

V. What We LearnedV. What We Learned

Lessons LearnedLessons Learned

Implementation: required level of fidelityImplementation: required level of fidelity– Easier to modify HTML/Javascript prototype Easier to modify HTML/Javascript prototype

than the initial Java versionthan the initial Java version

Testing: order of questions, minor wording Testing: order of questions, minor wording choices matterchoices matter

Analysis: difficult to tease out problems with Analysis: difficult to tease out problems with comprehension of new pricing model vs. UI comprehension of new pricing model vs. UI problemsproblems

Questions?Questions?