Building Learning Ecosystems with the Experience API Mike Hruska President/CEO Problem Solutions [email protected] @mikehruska
Jan 17, 2016
Building Learning Ecosystems with
the Experience API
Mike HruskaPresident/CEO
Problem [email protected]
@mikehruska
The LandscapeAdaptive Learning:
“Adaptive learning systems will make education much more tailored to each individual student. These systems use artificial intelligence to customize lessons that match the individual student’s progress.”1
Experiential Learning: “Experiential learning to build capabilities is one of the most important elements of a successful company transformation.” 2
70/20/10 Rule: “Redefining the blend to bring learning closer to the workplace—and provide appropriate “scaffolding” for the learner’s needs—is still a struggle for most organizations. 3
http://www.mckinsey.com/insights/mgi/in_the_news/empowering_teachers_and_trainers_through_technologyhttp://www.mckinsey.com/insights/operations/experiential_learning_whats_missing_in_most_change_programshttp://insights.ccl.org/wp-content/uploads/2015/04/BlendedLearningLeadership.pdf
Current State
• Many companies required learning is beyond capabilities of current platforms
• The LMS is not “satisfying” needs• Learning is beyond formal (experiential and
informal)• Companies face cost and time challenges to
develop human capital and manage talent• Data from systems could drive learning
experiences, but data isn’t typically connected
the BIG QUESTIONS…
How can we leverage performance data to save time and money training personnel?
How can we increase training effectiveness by using data
collected along the continuum of training?
What are learning ecosystems?
“Learning ecosystems provide a combination of technologies and
support resources to help individuals learn within an environment“
Source: David Kelly: http://twist.elearningguild.net/2013/11/what-is-a-learning-ecosystem/
What are learning ecosystems?
6
Personalized and Brokered Content•Just-in-Time•Just-for-You•RIGHT TIME EXPERIENCES
Access FromAny Device
Learn From:•Recommendations•Intelligent Tutors•Mentors and peers via social networks•Self-Discovery
Learn Using:•Simulations•LMS•Web•Games•Virtual Worlds•Intelligent content
Ecosystem Elements
Actors Resources Events
Signals Sensors
Flows Patterns
AFFORDANCES
• Evaluate incoming student competencies to inform planned instruction
• Use data analytics to discover trends over time
• Discover which methods of training have had the most impact
• Make sound recommendations for the next training events based on performance data.
• Report performance data to the larger organization.
• Discover which training interventions caused the greatest transfer to the operational environment.
• Dynamically optimize training events in real-time based on performance.
• Assess capability and proficiency levels of individuals, teams, and groups
• Reveal competency gaps and their sources:
- Trainee skills - System faults
Incoming Data Current Data Outgoing Data
Use Cases
Historical proficiency / Performance over time
Live performance view
Macro adaptation (learning path)
Micro adaptation (performance based)
Trends analysis
RECOMMENDATIONS
• Is this person capable to do X?• What do they need to be ready to do X?• Are my people capable/ready to do X? If
not, how can we prepare them?• Who are the best people at X ? And why
are they so awesome?• What does someone need to do more
of? Less of?
Key Questions
Method of uniformly defining and describing experience and context to assess learning and performance
Ability to adapt training across a variety of environments, systems, and modalities
Observe, assess, evaluate, or assert performance by systems or observers
What is Interoperable Performance Assessment (IPA)?
Our [Research] ApproachBuild lightweight and useful tools that:
– Uniformly capture performance data AND context
– Enable learning ecosystem functionality• Allow the data to become immediately visible• Enable adaptive systems to respond to the data
– Allow multiple lines of research efforts to build upon work
– Inform early adopters and lower implementation risk
Army Research Laboratory (ARL) Supported Programs
Soldier Centered Army Learning Environment• Developed by Army Research Lab (ARL) and Army Research Institute
(ARI)• Data-driven architecture and web service-based means to allow the
integration with new technologies• Supports training and education across multiple hardware platforms
Generalized Intelligent Framework for Tutoring• Developed by Learning in Intelligent Tutoring Environments (LITE)
Laboratory• Supports Army’s vision of more efficient and effective learning • Computer-based tutoring framework to evaluate adaptive tutoring
concepts, models, authoring capabilities, and instructional strategies• Provides a generic tutoring capability, including remediation strategies
based on learner performance, to integrated learning environments
Human Performance Measurement Language
(HPML) Constructs• Experience • Position• Platform• Training Environment• Training
Characteristics• Measure• Assessment
• Project/Mission• Task• Subtasks• Competency• Objective• Standard• Knowledge/Skills
Reference: Stacy, W., Ayers, J., Freeman, J., & Haimson, C. (2006). Representing Human Performance with Human Performance Measurement Language. Washington, DC. Aptima, Inc.
Elements of an xAPI Statement• Actor• Verb• Object• Context• Results• Extensions
xAPI and HPML
HPML Constructs
Data via web
services
Import Tools
Games
Mobile
A/R
Virtual Worlds
Example Ecosystem
Multiple Simulators
AdaptationPersonalization
Tailoring
AnalyticsVisualization
EfficacyEfficiency
Effectiveness
• Encoding/data collection library• Reduces complexity to support xAPI• Implements encoding best practices• Describes rich performance context• Dynamic Link Library (.dll)• Individual and group performance
encoding support • www.pipelinexapi.com
TOOLS
19
Current Research
• Efficacy research • Using Pipeline with
gunnery and crew simulators
• Encoding individual data/team
using xAPI• Macro-adaptation• Show effectiveness and
ROI basis for adaptation
Effectiveness Research
• Recently completed human subject research using xAPI to adapt individual to drive team outcomes
• In the team training, participants in the Adaptive condition took nearly 40% less time to and nearly 60% fewer scenarios to achieve excellent qualification scores.
• Results will be published in Q4 2015 in research journal – I/ITSEC
Lessons LearnedCompetencies• Profiles should be built around competencies.• Systems should use competencies as an anchor for tracking
learner performanceGranularity• A balance exists between capturing large amounts of data and
capturing meaningful data that is useful to other systems. Standards and Interoperability• Though data may be captured by a given system, it may or may
not be relevant to other systems. • Most systems that adapt to the learner do so in a black box
fashion using proprietary models of the learner, domain, and data. • Highly adaptive systems are typically complex and designed as
isolated systems that do not communicate or interoperate with other digital learning systems.
Mike Hruska President/CEO Problem Solutions
[email protected] @mikehruska
Questions or want more info?
www.problemsolutions.net
APPENDIX SLIDES
Our work..
Support technical activities in and around learning technologies
Span applied research to product development
Build startups and early stage technologies
Design and build products and solutions
Design and deploy new technologies
Things We’ve DoneBuilt the Experience API with Gov’t and Industry over last 5+ years
Built more open source tools for xAPI than any other gov’t program
Brandon Hall Gold Award Winning xAPI Product on xAPI
Thought Leader Articles in Learning Solutions Magazine (Feature in June 2015) on Learning Ecosystems, xAPI and Design
CoAuthor of “Learning on Demand – ADL and the Future of e-Learning”
CoAuthor of US Dept of Education “Ed Tech Startup Guide” (April 2015)
CoAuthor of Increasing Access through Mobile Learning (2014)
Industry ArticlesLearning Solutions MagazineLearning Ecosystems, Analytics, and xAPI-http://www.learningsolutionsmag.com/articles/1766/elearning-authoring-taking-the-next-step-with-xapi -http://www.learningsolutionsmag.com/articles/1761/amplifying-the-experience-api-xapi-camp-at-devlearn-2015 -http://www.learningsolutionsmag.com/articles/1745/are-you-an-isd-a-business-process-engineer-or-both -http://www.learningsolutionsmag.com/articles/1693/learning-ecosystems-and-the-experience-api-xapi-camp-recap
-http://www.learningsolutionsmag.com/articles/1722/xapi-and-analytics-measuring-your-way-to-success --http://www.learningsolutionsmag.com/articles/1523/ten-steps-to-plan--communicate-your-xapi-design-to-a-web-developer
Training Magazine What is the xAPI?-http://www.trainingmag.com/content/what-experience-api
Elearning News - xAPI-http://elmezine.epubxp.com/t/112009/33
ASTD T&D Magazine - SCORM Evolution-http://www.astd.org/Publications/Magazines/TD/TD-Archive/2013/04/A-SCORM-Evolution
Future Learning in the DoD-http://www.kmimediagroup.com/military-training-technology/440-articles-mtt/unlimited-access-learning
Published Research on xAPIHruska, M., Medford, A., Murphy, J. (2015). Learning Ecosystems Using the Generalized Intelligent Framework for Tutoring (GIFT) and the Experience API. 17th International Conference on Artificial Intelligence in Education (AIED 2015). Madrid, Spain. June 2015.
Amburn, C., Goodwin, G., Michael, H., Murphy, J. (2015). Developing Interoperable Data for Training Effectiveness Assessment in Army Marksmanship Training. MODSIM World 2015.
Goodwin, G., Hruska, M., Murphy, J. (2015). Developing Persistent, Interoperable Learner Models in GIFT. GIFT Sym3. Orando, FL, June 2015
Hruska, M., Long, R., Amburn, C. (2014). Human Performance Interoperability via xAPI: Current Military Outreach Efforts. Simulation. Fall Simulation Interoperability Workshop, 14F-SIW-035, Orlando, FL, September, 2014.
Hruska, M., Long, R., Amburn, C., Kilcullen, T., Poeppelman, T. (2014). Experience API and Team Evaluation: Evolving Interoperable Performance Assessment. The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC). Volume: 2014.
Poeppelman, T., Hruska, Long, R., Amburn, C. (2014). Interoperable Performance Assessment for Individuals and Teams Using Experience API. 2nd Annual Generalized Intelligent Framework for Tutoring (GIFT) Users Symposium (GIFTSym). Pittsburgh, PA, June, 2014.
Hruska, M., Poeppelman, T. R., Dewey, M., Paonessa, G., Paonessa, M., Nucci, C., Ayers, J. (2013). Interoperable Performance Tracking to Support Tailored Learning (Final Report). U.S. Army RDECOM Army Research Laboratory (ARL) – Simulation Training Technology Center (STTC).
Poeppelman, T., Ayers, J., Hruska, Long, R., Amburn, C., Bink, M. (2013). Interoperable Performance Assessment using the Experience API. The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC). Volume: 2013.
{ "id": "9a9c6f5f-5bb7-4ed1-b993-96e9cee3cd2f", "actor": { "objectType": "Agent", "name": "PVT Jacob Daniels", "mbox": "mailto:[email protected]" }, "verb": { "display": { "en-US": "experienced" }, "id": "http://adlnet.gov/verbs/experienced" }, "object": { "objectType": "Activity", "id": "activityId:uri/10th_CAB_AATX", "definition": { "name": { "en-US": "10th CAB AATX" }, "description": { "en-US": "On-Station - Area of Operations - Expert:15 degree safe firing range each way,Average:18 degree safe firing range each way,Novice:20 degree safe firing range each way - This measure assesses the way in which pilots approach the target when they get into the on-station area" } } }, "result": { "extensions": { "resultExtensions:uri/extensions/trainingobjective": "Meet all Time on Target objectives Meet all Time on Target objectives" }, "score": null, "success": false, "response": "Expert" }, "context": { "extensions": { "contextExtensions:uri/AVCATT/role": "FAC", "contextExtensions:uri/AVCATT/visual_scene_display_resolution": "20/10 visuals", "contextExtensions:uri/AVCATT/cockpit_content": "Content and layout exactly replicates aircraft", "contextExtensions:uri/AVCATT/visual_field_of_view": "180 horizontal FOV", "contextExtensions:uri/AVCATT/g_cueing": "No G cues", "contextExtensions:uri/AVCATT/axis_disturbance_motion_cues": "No disturbance motion cues" }, "contextActivities": { "parent": [], "grouping": [ { "id": "contextActivities:uri/01_2_5179_perform_aerial_deliberate_attack_operations", "definition": { "name": { "en-US": "01-2-5179 Perform Aerial Deliberate Attack Operations" }, "description": { "en-US": "Destroy enemy or force them to withdraw" } } }, { "id": "contextActivities:uri/support+troops+in+contact+(tic)", "definition": { "name": { "en-US": "Support Troops in Contact (TIC)" }, "description": { "en-US": "Friendly ground forces receiving effective fire" } } }, { "id": "contextActivities:uri/battlefield_communications", "definition": { "name": { "en-US": "Battlefield Communications" }, "description": { "en-US": "Right information, to the right person, at the right time" } } } ] }, "platform": "AH-64 - Apache" }, "timestamp": "04/17/2014 16:19:27", "authority": { "objectType": "Agent", "name": null, "mbox": null } }