Webinar Series – Session 2 Outcome-based Learning Opportunities Michael Derntl 1 , Jad Najjar 2 and Susanne Neumann 1 1 University of Vienna, Austria 2 WU Vienna, Austria
Webinar Series – Session 2
Outcome -based Learning Opportunities
Michael Derntl 1, Jad Najjar 2 and Susanne Neumann 1
1 University of Vienna, Austria2 WU Vienna, Austria
Webinar overview
� Background and motivation for outcome-based education
� Formulating learning outcomes� Learning outcome based teaching methods and learning
designs
� Models and standards for learning outcomes and opportunities
� Learning outcome motivated assessment
Using Adobe Connect
Using Adobe Connect
Why outcome-oriented teaching and learning?
� The Bologna Process� quality assurance in higher education� a unified educational system (mobility, transparency and mutual
recognition of qualification)
� Revise curricula � integrate student centeredness and learning outcome orientation
� Pedagogical issues became central: Alignment of teaching and assessment methods in accordance with learning outcomes
Background and motivation
What is your background?
� Please shortly state in the chat:
1. Your background. (e.g. higher education, PhD student, vocational training, consulting, instructional design, other…)
2. Your position or relation to teaching (if any).
3. Subject(s) you teach / PhD topic.
Background and motivation
Learning outcomes
A learning outcome is a statement of what a learner is expected to know, understand or be able to do as a result of a learning process ―European Qualification Framework for Lifelong Learning (EQF)
ALIGNMENTLearning outcomes
Teaching methods
Workload (ECTS)
Assessment
Formulating learning outcomes
Types of learning outcomes
Formulating learning outcomes
KnowledgeThe outcome of the assimilation of information through learning.
SkillsThe ability to apply knowledge and use know-how to complete tasks and solve problems.
Competence (personal and professional) The proven ability to use knowledge, skills and personal, social and/or methodological abilities.
proven abilityto use
abilityto apply
Personal, social and/or methodological
abilities
Learning outcomes
Intended vs. achieved learning outcomes
The teacher defines the intended learning outcomes for a learning opportunityThe teacher assesses and learners demonstrate the achievement of intended learning outcomes
Formulating learning outcomes
enumerate the most influential politicians of the 20th century
Sentence structure of intended learning outcomes
An intended learning outcome is formulated as a sentence that follows a basic structure:
Learners are able to Verb Object
Describes intended (cognitive) process
Describes what students are expected to acquire
or construct
Formulating learning outcomes
Example:
Learners are able to
What type of learning outcome is this?
Examples of learning outcomes and their type
Students are able to …
… explain the seven layers of the ISO/OSI Reference Model
… enumerate the most influential politicians of the 20th century
… design an interactive website that complies with Web Accessibility Guidelines
… filter out relevant literature for a given research problem.
… guide their team mates in improving the architectural model of a system.
… moderate a virtual task-force meeting
Formulating learning outcomes
Knowledge
Skills
Competence
Guidelines
1. Students are taught about software design flaws
2. Students are able to identify critical design flaws in a given software system
→ Focus on the learning rather than the teaching!
3. Students complete Unit 1 on basics of academic writing
4. Students are able to explain the basic principles of academic writing
→ Focus on the outcome, not the process!
5. Students are able to give a good presentation about their semester project
6. Students are able to convey the essential outcomes of their semester project in a five-minute slideshow presentation
→ Make sure that achievement can be assessed!
Formulating learning outcomes
Activity
Formulate and write in the chat:� If you’re a (PhD) student : a learning outcome of the program or
course you‘re taking.� If you’re a teacher/tutor : a learning outcome of a course you’re
teaching.� Indicate in parentheses whether the outcome is related to:
� knowledge (K)� skills (S) or � competence (C)
Formulating learning outcomes
Example:Students are able to explain the basic principles of academic writing (K)
Problem outline
Alignment of outcomes, methods, and workload� Here: focus on aligning teaching methods with learning outcomes� Goal: facilitating learners in achieving the intended outcomes
Teaching methods and learning designs
ALIGNMENTLearning outcomes
Teaching methods
Workload (ECTS)
Assessment
Learning outcomes
Teaching methods
ICOPER concept model
Key concepts
Sharable Educational Resources
Learning Outcome
Learning Content
TEACHING METHOD
intended result of
involves
Stored inthe OICS
imple-ments
Learning Opportunity
LEARNING DESIGN
INSTRUCTIONAL MODEL
is a
is a
Assessment Method
is a
imple-ments
instantiation of
uses
definesroles
Learning Facilitator
Learner
is a
Assessment Opportunity
includes
Assessment Design
is a
instantiationof
uses
Examples of Teaching Methods 1/5
Lecture : The teacher is the primary delivery channel for instructional content – often through presentation.
Teaching methods and learning designs
Examples of Teaching Methods 2/5
One-Minute Paper : Learners quickly write thoughts to make their level of understanding explicit.
Teaching methods and learning designs
Examples of Teaching Methods 3/5
Feedback : Reinforce or correct actions performed by learners.
Teaching methods and learning designs
Examples of Teaching Methods 4/5
Think-Pair-Share : To break long presentations, learners first think about a question individually, then discuss their thoughts with a partner, and lastly report back to the teacher.
Teaching methods and learning designs
Role Play : is a “dramatized case-study”. Its purpose is to have members of a group act out roles surrounding a situation, condition, or circumstance to make different perspectives on the issue visible.
Examples of Teaching Methods 5/5
Teaching methods and learning designs
Description of instructional models
ICOPER researched description elements that practitioners require for browsing, selecting and implementing instructional models.� General description: title, summary, rationale, subject, learning
outcomes, learner characteristics, group size, duration, setting, author/licensing
� Detailed description: graphical representation, sequence of activities, roles, assessment method, resources, variations, implementations, references
� Comments: teacher reflections, student feedback, peer review
This allows for a structured human-readable description
Teaching methods and learning designs
Description of instructional models
To describe instructional models in an interoperable machine-readable format, there are several existing standards and specifications, e.g.:� IMS Learning Design (LD)� SCORM� IMS Simple Sequencing (SS)
In ICOPER: Instructional models described with IMS LD (other formats allowed), with LOM metadata record(s) on topStorage in the Open ICOPER Content Space (OICS)� for OICS/LOM introduction see the replay of Webinar 1; for IMS LD the upcoming Webinar 3; for application demos Webinar 5.
Teaching methods and learning designs
Alignment of Learning Outcomes and Assessment
Teaching methods and learning designs
ALIGNMENTLearning outcomes
Teaching methods
Workload (ECTS)
Assessment
Alignment of Learning Outcomes and Assessment
Learning outcome:
Students are able to moderate a virtual task-force meeting.
Aligned assessment: A tutor observes a task-force meeting where a student moderates, and gives feedback to the student based on pre-defined performance criteria.
Misaligned assessment:Students take a multiple-choice test on how to moderate meetings.
Teaching methods and learning designs
Alignment of Learning Outcomes and Teaching Methods: General
General alignment: What do we mean by “alignment” when talking about teaching methods?
Teaching methods and learning designs
ALIGNMENTLearning outcomes
Teaching methods
Workload (ECTS)
Assessment
Alignment of Learning Outcomes and Teaching Methods: General
Learning outcome:
Students are able to moderate a virtual task-force meeting.
Teaching methods:� Lecture� One-minute paper� Feedback� Think-pair-share� Role play
Teaching methods and learning designs
Take a vote:Which teaching method(s) do you consider a good/best fit for this learning outcome?
“Students are able to moderate a virtual task-force meeting”
Teaching methods:� Lecture� One-minute paper� Feedback� Think-pair-share� Role play
Voting Activity
Teaching methods and learning designs
Alignment of Learning Outcomes and Teaching Methods: Specific
Use taxonomy for alignment, for instance, Anderson & Krathwohl taxonomy of educational objectives
Teaching methods and learning designs
Students are able to moderate a virtual task-force meeting.
Source: Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision of Bloom's Taxonomy of Educational Objec tives (Complete ed.). New York: Addison Wesley Longman.
LO: moderation
Alignment of Learning Outcomes and Teaching Methods: Specific
Teaching methods and learning designs
Source: Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision of Bloom's Taxonomy of Educational Objec tives (Complete ed.). New York: Addison Wesley Longman.
LO: Moderation
TM: Lecture
TM: One-Minute-Paper
TM: Feedback
TM: Think-Pair-Share
TM: Role Play
Assessment: Moderation
Teaching methods
What to look after besides learning outcome alignment when choosing teaching methods?� Allow learners to practice and do things themselves; don’t just “talk”
about it� Plan for social interactions between learners� Support learning process (feedback, scaffolding, coaching)
Teaching methods and learning designs
Learning Design
Building a learning design: Adding content and assessment to the teaching method according to learning outcome.
Teaching methods and learning designs
Generic teaching method
Integration of a whiteboard tool to collect reasons
Specific learning design integrating the teaching method
Specifications in Focus
� Learning Outcomes (IEEE RCD)� Describe and manage Learning Outcome Definitions� ICOPER LOD AP
� Learning Opportunities (CEN MLO)� Publish metadata of learning opportunities� ICOPER MLO APP
� Personal Achievements (new spec.)� Describe and share information on learners/teacher achieved learning
outcomes and associated evidence, level and context data� Personal Achieved Learning Outcomes (ICOPER PALO)� Backbone for a new CEN specification
Models and standards
Metadata for Learning Outcomes
� Describes the characteristics of a learning outcomes� Enable storage, referencing, sharing Learning outcomes data
Models and standards
Indexed large set of learning outcomes
http://oics.icoper.org/LOD/
Sources Learning Outcomes � OpenLearn (OUUK)
� All courses have defined learning outcomes� Stored in IEEE LOM metadata classification category
� IEEE/ACM Computer Science Curricula (Task Force)� All course descriptions have learning outcomes� PDF file� Used worldwide by many universities
� 21st Century Engineer (NSF project)� 36 generic learning outcomes� Engineering domain� .ppt file
� European Computer Science Driving License (ECDL)� Full list of learning outcomes per topic� Excel sheets
� WU Vienna – Business Education study programme� Local XML into LOD XML
Models and standards
Learning Opportunities
Example:
Course catalog
Models and standards
Metadata for Learning Opportunities � Describe an opportunity a learner can follow� Enables publishing metadata across universities/countries
Models and standards
Generate MLO Instances
http://oics.icoper.org/MLO/
� Course catalogs� Local XML format� Transformation into
ICOPER MLO XML� 1000s instances� Want to produce MLO
instances from your course catalogs?…Talk to us ;-)
Metadata for Personal Achievements� Represent information on achieved/required/desired learning outcomes of a learner. Also teachers
(taught outcomes)� Share data with LMSs, recruitment systems, social applications
Models and standards
PALO ExampleModels and standards
Najjar et al. (2010) A data model for describing and exchanging Personal Achieved Learning Outcomes (PALO). International Journal of IT Standards and Standardization Research, vol. 8, pp. 87-104.
Generated PALO Data
Models and standards
� Data collected acrosssystems
� ATOM/RSS feeds� Shared in
� iGoogle� Facebook� LMSs� etc.
Evidence of achievement
� Successful completion of learning opportunity
� Evidence that a learning outcome was achieved/attained
� Each learner achievement has an assessment record
� Assessment record is a summary result of one or more learning assessments
Learning outcome motivated assessment
Approaches
� Automatic� System generates assessment records
from assessment results in LMS� Submit data to learner profile when course is closed
� Semi-automatic� Student claimes achievement of
outcomes � submit assessment records to learner profile
� Teacher approves assessment records � data submitted to learner profile
Learning outcome motivated assessment
More Resources & Feedback
ICOPER Training Infrastructure:� Learning Outcome Course:
http://training.icoper.org/moodle/mod/resource/view.php?id=269� Instructional Modeling Course:
http://training.icoper.org/moodle/mod/resource/view.php?id=270
Your feedback:What do you take home from this webinar?Please share via the chat window or raise your hand to speak.
Feedback form:http://tinyurl.com/28gjny2
Closing
Upcoming Webinars
Webinar 3: “eContent for designing and delivering outcome-based learning opportunities”February 3, 2011 (next Thursday)16.00-17.30
Webinar 4: “Learning outcome-based assessment and applications”February 10, 201116.00-17.30
Webinar 5: “Applications for outcome-based learning”February 17, 201116.00-17.30
ICOPER Webinar Series