Smithsonian
Learning
Lab
learninglab.si.edu
Los Angeles County Museum of Art
April 1, 2015
Darren Milligan
Senior Digital Strategist
Smithsonian Center for Learning and Digital Access
Smithsonian Institution
@darrenmilligan / [email protected]
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
Source: Frontline Design, http://www.frontlinedesign.org/project/night-at-the-museum-2-battle-of-the-smithsonian
Image from Flickr user Mike Procario,
https://www.flickr.com/photos/procario/11060000573/, used under a CC BY-ND 2.0license.
Image adapted from the Department of Education,
http://www.flickr.com/photos/departmentofed/9602545478/, used under a CC BY 2.0 license.
Smithsonian (in 2014)
19 Museums and Galleries & National Zoo
137.7M Museums Objects & Specimens
1.9M Library Volumes
136,194 Cubic feet of archival material
Smithsonian (in 2014)
19 Museums and Galleries & National Zoo
137.7M Museums Objects & Specimens
1.9M Library Volumes
136,194 Cubic feet of archival material
6,373 Employees
721 Research Fellows
9,817 Volunteers
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
1995
23.6 million physical visits to museums
72,942 digital visits
2014
26.7 million physical visits to museums
99 million digital visits
1995
23.6 million physical visits to museums
72,942 digital visits
2014
26.7 million physical visits to museums
99 million digital visits
Physical: 26,700,000-23,600,000 /
23,600,000 X 100 = 13.16% increase
Digital: 99,000,000-72,942 / 72,942 X 100
= 135,624.27% increase
1995
23.6 million physical visits to museums
72,942 digital visits
2014
26.7 million physical visits to museums
99 million digital visits
Physical: 26,700,000-23,600,000 /
23,600,000 X 100 = 13.16% increase
Digital: 99,000,000-72,942 / 72,942 X 100
= 135,624.27% increase
Pew Research Teachers Survey Report February 2013
92%: Internet has “major impact” on their ability to
access content, resources, and materials for their
teaching
90%: use search engines to find info
84%: use Internet weekly to find content that will
engage students
80%: use Internet weekly to help them create
lessons
Image: Haxorjoe at en.wikipedia, http://commons.wikimedia.org/wiki/File:Nokia1100_new.jpg, used under a Creative Commons Attribution-Share Alike 3.0 Unported, 2.5 Generic,
2.0 Generic and 1.0 Generic license.
The Problem
Fragmented Brand
Outdated Platform
Lack of Detailed DataWho is using our resources?
Why are they using them?
How are they using them?
Where are they using them?
How would they prefer to access them?
In what format would they prefer them?
Are they using them as designed?
Are students learning from them…?
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
Remedial Evaluation of the Materials
Distributed at the Smithsonian
Institution’s Annual Teachers’ Night
(2010)
Literature review and evaluation to produce
generalizable guidelines for the design and
development of museum-based lesson plans and
investigated classroom educator methodologies for
incorporating museum-based lessons into
classrooms.
Methodology
Phase One
Extensive literature review
Phase Two
Analysis of comments on specific
Smithsonian lesson plans
Phase Three
Focus groups and in-depth interviews
with classroom educators
Results: Literature Review
Usability and navigability of websites are important.
Common usability problems for visitors (including
classroom educators) who are non-museum
professionals when using museum websites:
• Frustration with overloading of content
• Distracting graphical user interfaces
• Browsing not conducive to understanding specific
topics
• Difficulties with certain terminology
• Disconnect of museum websites to the physical
museums
Results: Literature Review
Key requirements classroom educators need for
museum material to be incorporated into their
teaching:
• Aligned to curriculum standards
• Updated
• Interdisciplinary
• Related to big concepts
• Educational
• Not dependent on museum visits
Results: Analysis and Focus Groups
Key requirements classroom educators need for
museum material to be incorporated into their
teaching:
• Enjoyable for their students
• Interdisciplinary
• Adaptability
• Alignment with curriculum standards
• Flexibility to accommodate a diversity of students
Results
Educators reported that they used
Smithsonian materials acquired at
Teachers’ Night as resource starting points
for their teaching. Museum materials do not
always fit into their teaching (museum
materials must be deconstructed and then
reconstructed).There was no definite
indication that participants from the
focus groups used the materials as
intended by the Smithsonian.
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
Capturing the Voice of Customer,
Satisfaction Insight Review of
SmithsonianEducation.org (2011)
Collected from more than seven thousand surveys
completed by visitors to the central Smithsonian
Education website, the makeup of this audience,
their motivations for site visitation, their activities
while on the website, and their sources of
dissatisfaction were explored.
Methodology
User satisfaction survey (using the methodology of
the American Customer Satisfaction Index)
conducted via SmithsonianEducation.org.
A total of 7,470 surveys were completed during the
twenty-four month span of data collection through a
popup window presented to website visitors. The
survey consisted of twenty-one numerically scored
model questions, as well as ten multiple choice and
five open-ended custom questions developed by
SCLDA.
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
Digital Learning Resources Project
(2012)
To assist the Smithsonian to better understand the
educational uses of Smithsonian digital resources
and provide a road map for future digital
development.
Research objectives focused on classroom
educators’ ability to identify, analyze, and extract
digital content, with the ultimate goal of enabling all
users to achieve their own personal learning
objectives through the Smithsonian’s resources.
Intended Outcomes
Short-term: to increase classroom educators’ skills
in identifying, analyzing, and extracting specific
Smithsonian digital learning content
Medium-term: to increase skills to make strategic
use of digital learning content
Long-term: to foster online users who are active
creators of digital resources personalized for
learning in their own classroom.
Methodology
Phases One/Two
Focus groups with twenty classroom
educators in Northern California combined
with user analytics; literature review &
environmental scan
Phase Three
Prototypes were developed and tested by
group of sixty-nine classroom educators
(grade-level and geographically diverse)
Results: Phases One/Two
Search and Visualization Tools
Museums need to make resources more findable and to
generate assets that are personalized and accessible
anytime, anywhere, and on multiple platforms. Classroom
educators also asked for:
• Search results with thumbnails, previews, tag clouds,
and rating systems that allow them to easily identify what
is useful and what is not
• Personalized search hints
• Search capabilities that can be either highly filtered or
extremely broad
Results: Phases One/Two
Engaging, Standards-aligned, Learner-centered Content
Classroom educators put student interest and engagement
at the top of their list and need content that aligns with
learning goals and standards.
• Engage students
• Allow for student interaction and adaptation
• Afford accessibility for various learning styles and levels
• Offer coherence with the lesson and multidisciplinary
opportunities
• Support problem-based learning goals
• Support standards-based teaching goals
Results: Phases One/Two
Instructional Tools
When extracting resources, classroom educators want:
• Flexible technologies for a diversity of devices and
delivery methods
• Tools to assess learning
• Tools to adjust reading level of text
• Ways for classroom educators to upload their self-
authored components into a saved file, or resources
from other sites or collections
• Specific pedagogical tools
Results: Phase Three
Search and Visualization (Identifying)
• Search by entering a general search term, then filtering
further if needed. Educators also preferred the gallery
view to review their search results. Participants want
more intelligence in their searches and results to guide
them toward the most valuable resources.
• Use a diversity of locations to find what they need and
have little loyalty to one site in particular
• Use the Facebook Share option, but the most popular
method of sharing was emailing the link to themselves or
a colleague.
Results: Phase Three
Authentication, Saving, and Storing (Analyzing)
• To save resources that they find useful.
• The flexibility to organize and annotate resources
according to their own schemas.
• Flexibility in the types of viewing methods available: one
for whole-class interaction and one for individual
interaction.
• The ability to allow students to use the site and its tools
as much as the classroom educator.
• Content that is aligned with Common Core State
Standards.
Results: Phase Three
Instructional Tools (Extracting)
• Were excited about the use of “interactives” with the
resources found in the Smithsonian collection.
• Appreciated the search functionality of the site but want
better visibility of the tools, including prompts and
explanations for their use.
• Liked being able to upload resources from other sources
to augment their collections and appreciated being given
tools that make this easier to accomplish within the site.
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
Piloting Tools to Enable Active and
Participatory Learning for Middle
School Students: Facilitating Digital
Learning with Smithsonian Digital
Resources (2014)
Classroom educators have indicated that relevance
to students’ needs are a top priority. We are now
testing directly with students to better understand
how they use digital museum assets (specifically
digitized collection objects) and to document the
types of scaffolds necessary to enable active and
participatory learning using them.
Methodology
Prototype presented to middle school students in College
Park, Maryland and in Chico, California.
Testing was conducted using a predefined set of
instructions and tasks. Each testing session included an
activity where students were first introduced to the
Smithsonian, the research project, and their role as testers.
Students were then shown the prototype and walked
through how to search, save, and edit collections.
Individually, they were then asked to conduct the same
search. Finally they were instructed to search, create, and
edit a collection based on their own interests. The group
was then interviewed for suggestions of improvements.
Results
Challenges for Student Users
• Unintelligible descriptions: some of the students found
that the descriptions were difficult to understand
• Spelling limitations: some of the students could not find
the items they were looking for because they could not
spell the search terms correctly
• Loss of authority: some of the participants thought that
the information in their collections was unreliable if they
(or anyone else) were able to change the information
from the original Smithsonian descriptions (a feature of
the prototype)
Results
Feature Requests
• Auto-correct to assist with spelling
• Predictive searches/Recommended searches
• Ability to adjust fonts
• Draw on objects
• Put external images into their collection
• Share collections through social media and email
• Export their collection
• Create unique collages from multiple images
• Child-accessible collection descriptions
Image adapted from the Department of Education,
http://www.flickr.com/photos/departmentofed/9602545478/, used under a CC BY 2.0 license.
META TIME!
Conclusions Focused on
• Educator search preferences
• Factors that contribute to difficult
analysis and resistance to use learning
resources
• Educators’ preferences for learning
resources
• Educators’ preferences for platform
content and functionality
Educator Search Preferences
• Searching assist via autocomplete
and/or spelling assist
• Search results that allow for both
browsing and filtering
• Scannable grade level and subject
information
• Resources from a wide variety
of sources
Factors that Contribute to
Difficult Analysis and Resistance to Use
Learning Resources
• Too many search results; too difficult to
browse quickly for relevant content
• Distracting user interfaces
• Unfamiliar terminology and/or lack of
contextual information
Educators’ Preferences for
Learning Resources
• Interdisciplinary and/or multidisciplinary
• Connection to students’ interests
• Alignment to teaching standards and/or
relationship to big ideas
• Highly Adaptable
• Downloadable format
Educators’ Preferences for
Platform Content and Functionality
• Content available from more than one
one producer/supplier
• Tools available within the platform for
student interaction with the resources
• Variety of sharing options
• Ability to save and structure resources
within the platform for later review/use
This Morning
1. The Smithsonian and Outreach
2. Why Digital?
3. Research
- Evaluation of Learning Materials
- Audience Survey
- Digital Learning Resources Project
- Tools for Middle Schoolers
4. Learning Lab
Smithsonian Learning Lab Process
Technical Specifications
Personas
Wireframes
Style Tiles
Style Guide
Mockups
Alpha
Beta
Beta Launch
Smithsonian Learning Lab Process
Technical Specifications
Personas
Wireframes
Style Tiles
Style Guide
Mockups
Alpha
Beta
Beta Launch
Smithsonian Learning Lab Process
Technical Specifications
Personas
Wireframes
Style Tiles
Style Guide
Mockups
Alpha
Beta
Beta Launch
Smithsonian Learning Lab Process
Technical Specifications
Personas
Wireframes
Style Tiles
Style Guide
Mockups
Alpha
Beta
Beta Launch
Smithsonian Learning Lab Process
Technical Specifications
Personas
Wireframes
Style Tiles
Style Guide
Mockups
Alpha
Beta
Beta Launch
< User Testing
< User Testing
< User Testing
< User Testing
< User Testing
< User Testing
A Minimum Viable Product
has only the core features that
enable the product to be useful,
and nothing more.
A Minimum Viable Product
has only the core features that
enable the product to be useful,
and nothing more.
A Minimum Viable Product
has only the core features that
enable the product to be useful,
and nothing more.
Smithsonian Learning Lab
Public Beta
Launch
September 2015
Follow Along
And Share Your
Thoughts
learninglab.si.edu