Supporting Flexible Competency Frameworks

Supporting Flexible Competency Frameworks

Erica Melis, Arndt Faulhaber, Ahmad Salim Doost and Carsten Ullrich

DFKI GmbH, SJTU

Supporting Flexible Competency Frameworks - Carsten Ullrich

Environment - ActiveMath

Web-based Learning Environment for Mathematics

Domain/Content Model

User ModelTeaching/Pedagogical

Model



Competency Systems

Solving some exercises require different skills than others

Multitude of Systems• Bloom (Benjamin Bloom, 1956): Knowledge, Comprehension, ...

• PISA (OECD, 1999): think mathematically, argue mathematically, solve problems mathematically, ...

• Revision of Bloom (Anderson & Krathwohl et al., 2001)

• Skills (Pardos, Heffernan et al., e.g. 2007)

No "one fits all"• Different systems for different purposes / perspectives

• E.g. more coarse grained and simple to understand vs. more fine-grained and detailed but harder to use


Competency Systems II

Competency systems differ in various ways• Structure

• e.g. 1-dim Skill vs. 2-dim Pair of (Cognitive Process, Knowledge Element)

• Granularity

• e.g. amount of included cognitive processes

• Perspective

• e.g. pedagogical perspective vs. psychological perspective (i.e. focus on pedagogical goals vs. focus on cognitive processes)


Technology-Enhanced Learning and Competencies

Multitude of different competency systems to assess students' competences

Problems:• Reuse of learning objects

• Interoperability with respect to user modeling

=> Need to extend availability of LOs/usage-data via integration of different systems

=> System to handle/combine data with metadata using various Competency Systems


Goals of our Work

Integrate different learning objects based on various competency systems

Allow for combined evaluation of data annotated with different competency systems

Devise way to integrate new Competency Systems easily


Student Model Example: ActiveMath

Models the competencies of a student i.e. How well can the Student "solve" a "linear equation"

Competencies: Pairs of "Cognitive Process" and "Knowledge" e.g. "solve" + "linear equation"

Derive estimation of competency from performance in exercises Exercises have metadata for required/trained

competencies


Student Model Example: ActiveMath (ctd.)

Fine-grained Overlay Model Each pair (Cognitive Process, Knowledge Item) is

competency (variable) in the student model Many knowledge items, modeled in the domain

ontology, e.g. Add fractions with same denominator Add fractions with different denominators Subtract fractions with same denominator Subtract fractions with different denominators...

Over 30 different Cognitive Processes Recognize, recall, interpret, exemplify, transform...


Repository

“derivation”

Planner / Pedagogical Model

Course Generator


Course Generation: Pedagogical Knowledge


Basic Knowledge

Inserting references to • educational resources

• tools

Generating structure

Accessing information about• learning object

• the learner

About 70 rules


Example / Exercise Selection

About 60 methods

Take into account• competency level

• educational level

• fields of interest

• novelty

• motivation & anxiety


Formalized CG Knowledge: Scenarios

Moderate constructivist scenarios:• Discover

• Rehearse

• Connect

• Train Intensively

• Train Competencies

• Exam Simulation

Based on guidelines of instructional design• Guided Tour

≈ 300 methods and operators


Scenario “Discover”

Introduce

Develop

Prove

Practice

Connect

(discover deriv)

(:method (discover ?f) () ((!startSection Discover ?f) (descriptionScenarioSection ?f) (learnFundamentalsDiscover ?f) (reflect ?f) (!endSection)))

(:method (learnFundamentalDiscover ?c) () ((!startSection Title (?c)) (introduceWithPrereqSection ?c) (developFundamental ?c) (proveSection ?c) (practiceSection ?c) (showConnectionsSection ?c) (!endSection)))


(:method (introduceWithSection! ?c) () ((!startSection Introduction (?c)) (text Introduction (?c)) (motivate! ?c) (problem ?c) (insertIntroductionExample ?c) (!endSection)))

Scenario “Discover”

Introduce

Develop

Prove

Practice

Connect

Motivate

Problem

Illustrate

Prerequisites

(:method (introduceWithPrereqSection! ?c) () ((introduceWithSection! ?c) (learnPrerequisitesFundamentalsShort ?c)))

(introduceWithPrereqSection! deriv)


Scenario Discover

Introduce

Develop

Practice

Connect

Reflect

Motivate

Problem

Illustrate

Prerequisites

(:method (motivate! ?c) ((learnerProperty hasEducationalLevel ?el) (learnerProperty hasAnxiety ?c ?an) (?an <= 2) (GetElement ((class Exercise) (class Introduction) (relation isFor ?c) (property hasLearningContext ?el) (property hasDifficulty very_easy)))) ((insertAuxOnceIfReady! ?element)))

Problem: rules refer to specific competency system


Our Approach

Integration of additional competency systems• Have one internal competency system

• Map additional system to internal competencies

• Before processing input, translate competencies to internal competency system

Extending Planner / Tutorial Component• Parameterize with Competency System

• Assemble Courses for arbitrary Competency Systems


Extending the Student Model

Mapping information "Cognitive Process"-part of competency in system A to

system B Weighting of Competency mapping (for multiple

competencies)


ActiveMath supports three competency systems• Blooms

• PISA competencies

• ActiveMath competency system (revised Bloom)


Extending the Pedagogical Model

Competencies are used when1. inserting a sequence of exercises or examples that

cover all competencies in order to present new content in multiple contexts

2. selecting exercise or examples for a competency given as a parameter

3. selecting an exercise or example for a specific competency.


Generalizing Competencies: Old Rule


Generalizing Competencies: New Rule

Store competency system as a fact

(competency-system (c1,...,cn))

Generalized method:


Case 2: No Change Needed


Case 3: No Change Possible

Very specific, depends on used competency systemCannot be generalized without loss of qualityIn case a competency system can encode a semantically similar case, a pedagogical expert has to add a corresponding method


Conclusion

Approach to close the gap between different TEL systems / competency systems

Course Generator and Student Model can handle multiple competency systems

Data originating from learning objects based on different competency systems can be combined

Supporting Flexible Competency Frameworks

Education

specific competency

competency variable

c introducewithprereqsection

c introducewithsection

c provesection

c developfundamental

c showconnectionssection

c practicesection