ArchMiner An exploratory data analysis tool for the Center for the Built Environment Stephanie Hornung Leah Zagreus Masters Candidates 2003
ArchMiner
An exploratory data analysis tool for the Center for the Built Environment
Stephanie HornungLeah ZagreusMasters Candidates 2003
What is CBE?
• Center for the Built Environment (CBE)– Research group in the College of
Environmental Design – Aims to inform the building industry
of new building technologies and design strategies that improve efficiency and occupant comfort
Occupant Satisfaction Survey
• Web-based survey • Measures occupant satisfaction
with a variety of metrics (such as air quality, acoustics and lighting)
It’s cold at my desk, I think there’s a draft
There’s a terrible glare on my
screen in the afternoonEveryone can hear what I say on
the phone, and sometimes its
personalThe air would be so much better if we could open a window!
Benchmarking
• Survey data used to correlate occupant comfort with building technologies – How do the occupants in this building feel
about the temperature in their workspace?– Is the occupants’ feeling about the
temperature related to having a thermostat?
– How do the results from this building compare with other buildings in the same geographical area?
Analysis Process
• View survey results using in-house reporting tool
• Create many charts and pivot tables using Microsoft Excel to try to find interesting relationships
Process is inefficient and very time consuming!
Reporting Tool
• Provides easy viewing of survey results
• New filtering capability allows some exploration– How do occupants with
individual thermostats feel about the temperature in their workspace?
But:• Side-by-side comparison is
not easy– Is the occupants’ feeling about
the temperature related to having a thermostat?
Microsoft Excel
• Researchers make charts and pivot tables looking for interesting relationships– Can result in many
charts, with only a few yielding results
• Chart-creation process is time consuming
• Raw data difficult to parse
Researcher’s Need
A fast, simple way to explore relationships in the survey data without having to manually create each chart or table — fill gap between viewing survey results and doing statistical analysis
Solution:• A visual crosstab design that requires little set-up and yields useful information
Our Design Process
• Two rounds of low-fi prototype testing– Designs are easy to make and change, and
even easier to discard
• Heuristic Evaluation done by colleagues in is213
• Pilot usability test
Implementation Tools
• Java 2SE v1.4.1– Swing UI components– Threads– JDBC
• MS SQL Server 2000• Visual Mining NetCharts Server 4.0
Evaluation and Feedback
• CBE partner meeting demo• Pilot usability study
– 3 typical users: 1 researcher, 2 GSRs– 7 typical tasks– Observation, “think aloud” techniques– Recorded time, task completion, error
rate – Post-test questionnaire with Likert
Scale followed by interview
Quotes
“Powerful!”
“Yahoo! When can I use it?”
“Easy to use”
“Everything is where I expect it to be”
“Very useful”
Pilot Usability Study Results
• All participants completed 5 of 7 tasks in ~ 2 min or less
• Overall: Easy to use, powerful, and intuitive interaction flow
General Question Average Score [1-Disagree] to [5-Agree]
The resulting charts and data would be useful for my research.
5
The workflow was natural. 5
I was satisfied with ArchMiner as a data exploratory tool.
4.7
Pilot Usability Study Results
General Question Average Score [1-Disagree] to [5-Agree]
It was difficult for me to select the survey(s) I was interested in. 1.3
It was difficult for me to select the question(s) I was interested in.
2.3
It was difficult for me to navigate the control panel on the left. 2.7
It was difficult for me to understand the correlation between the controls on the left and the grid displayed on the right.
1.7
It was difficult for me to see the relationship between the data in the column and the row.
2.3
Functions in the top menu bar were difficult to access. 2
After having generated some charts, it was difficult to add to the results.
2.3
It was difficult to navigate through the results using the scroll bars.
2
After having generated some charts, it was difficult to clear the results I did not want to see.
1.3
Pilot Usability Study Results
Problem Description Planned Interface Change
The order of the categories Currently displayed alphabetically were not familiar with seeing/ using them in this order
List categories in the same order as survey
Show/hide responses Biggest problem. No user selected the “View Columns by Responses” or “View Rows by Responses” choices in the menu bar
Change semantics on menu choices. May need more usability testing
Improve question selection Users could not see the whole question in one view
Better question selection. Wrap text. Still researching other options
Keeping one viewable query and comparing it with later built queries
Users have to rebuild old queries or scroll a lot to see comparisons from first and last queries built
Keep viewable query and compare it with later built queries
No shortcuts Would like quicker access to functionalities, other than just in the menu bar
Add shortcuts
Acknowledgements
• Dr. Marti Hearst• TAs Maggie Law and Kaichi Sung• Research Specialist Charlie
Huizenga• Center for the Built Environment
213 teammates Myra Liu and Anita Wilhelm
More Information
• ArchMinerhttp://dream.berkeley.edu/CBEdatamining
• Center for the Built Environmenthttp://www.cbe.berkeley.edu
• Occupant Satisfaction Surveyhttp://www.cbesurvey.org