Grasping evidence of learning: e-portfolios and outcome assessment Welcome to session 3.14 Bruselas A, Level 1, Fibes 2 | Wednesday, 13 Sep 2017 (13:30 - 14:30)
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Grasping evidence of
learning: e-portfolios and
outcome assessment
Welcome to session 3.14
Bruselas A, Level 1, Fibes 2 | Wednesday, 13 Sep 2017 (13:30 - 14:30)
Grasping Evidence of Learning: E-portfolio and
Outcome Assessment
Darla DeardrorffYuichi Kondo
Shingo Ashizawa
Important Message from Darla
DISCUSS:
1) What does it look like to go beyond numbers in assessing international exchange?
2) What is evidence of change in student learning (knowledge, skills, attitudes) in international exchange?
FRAMEWORK: Program Logic Model• Where are you going?
• (Goals)
• How will you get there?
• (Learning Outcomes)
• How will you know when you’ve arrived?
• (Evidence)
Myths on Assessment
Myth #1: Send students abroad and they will come back interculturally competent.
Myth #2: Assessment is about numbers.
Myth #3: A pre /post measure is sufficient.
Myth #4: “One size fits all” for ICC assessment.
Myth #5: ICC assessment is the same as program/course evaluation.
Please describe how e-portfolio system can benefit each of the following categories.
1. Students’ Reflection
2. Peer-learning
3. Teaching and Instruction
4. Institutional Research
E-portfolio and Outcome Assessment
of a Study Abroad Program
Case from Ritsumeikan Asia Pacific University, Japan
Yuichi Kondo, PhD
Ritsumeikan Asia Pacific University
CASE 1
Students/Faculty→approx. 50% from overseas Multi-cultural, Multi-lingual campus Students：85 countries & regions, Faculty 25
Ritsumeikan Asia Pacific University St. Edward’s University（SEU）
Create Global Education Collaboratively
Austin, Texas
SEUAPU
Thailand &Malaysia
Dual Degree Programs
Learning GoalsIdentified over years
Formative/Summative
Assessment(Mixed method)
Use of evidence in e-portfolioInventory, presentation,
rubric, surveys, etc.
Experiential LearningLearning reflection & application
in e-portfolio
Improve the program
Program Outcomes & Rubric
Descriptions of outcomes*
Self-Rating
Provided examples as evidence
Curriculum mapIndicating classes/activities which aim toward the outcome
Outcomes* ally with Institution’s vision/goals.
✓Why did I study abroad?✓What did I learn?✓What do I want to do next?
After study abroad
Showcase
portfolio
Before
✓ Goal-achieved?✓ What am I learning?✓ What should I do next?
✓Why study abroad?✓My goals?✓Pre-departure survey
During
Baseline Data
Reflection/
Learning
portfolio
How we use e-portfolio
↑ Summative Assessment
↑ Formative Assessment
During Study Abroad
Experiential Learning (Kolb, 1983) framework
①Learning reflection, Meaning making
② Application of learning
③ Self-assessment/Goal-setting
What?
So what?
Now what?
Kolb,D.A.（1983）Experiential Learning: Experience as the Source of Learning and Development. New Jersey:Prentice Hall,Inc
Reflection/Learning portfolio
④ Feedback/Comments from peers, advisors, instructor
①Learning reflection, Meaning making
② Application of learning
③ Self-assessment/Goal-setting
✓ Deepen learning by reflection, application, self-assessment
✓ Collect evidence of learning abroad✓ Direct assessment ✓ Focus on learning process✓ Support integrative learning✓ Promote social learning
④ Feedback/Comments from peers, advisors, instructor
Diverse effects of this e-portfolio
60-70% daily submission rate each year
Students’ self-assessment while abroad(Regularly conducted in e-portfolio)
Self-Rating
Student provides examples as evidence
rubrice-portfolio
After studying abroad
Showcase portfolio
<In the past> Students presented their learning outcomes through their speeches
<From this year>
Based on the rubric, students will select evidence of learning and create a digital story on PowerPoint
Stored in portfolio as showcase
→ Rubric used for peer & summative assessment
21
Read each other’s portfolios
Peer Advisor Support (participants of previous year)
Career forume-portfolio feedback
Preparation workshops
Learning Community among peers
AfterStudy abroadPrep
Peer Support Systems in study abroad
Japan classDaily ReflectionSelf-assessment
US class 1
Critical Thinking
US class 2Multi-Cultural Society
US Class 3Service Learning
All teachers can… 1. Become aware of different classes in the same program
Coherence and effectiveness in teaching2. Understand students and learning progress better3. Grow professionally by learning from other classes
US-Japan Integration of all courses into portfolio
Our e-portfolio and next step
E-portfolio collects evidence of students’ learning
Provide data for direct assessment
Enable us to understand learning process
What’s next?
Improve our assessment system to become more efficient, effective, and attractive
・ Efficiency—Feedback from users (students & teachers)
・ EffectivenessーAnalyze data in mixed methods
・ AttractivenessーImprove showcase content
• Started as a philosophy academy in 1887
• 32000 enrollment (Located in central Tokyo)
CASE 2
Shingo Ashizawa
Toyo University
Faculty of Regional Development Studies (RDS) emphasize international training.
First Stage of e-portfolioToyo RDS folio(2013-2015)
Needs of e-Portfolio
Record of
Academic Work
Overviewing Academic Workof Each Students
Test Score of
Proficiency test
• Language test
score
• History of
Language Learning
Record of
Extracurricular
Activities
• Record of study abroad
activities
• Database of extracurricular
activities
Learning Goal at Toyo RDS (2000 students)
Specific area
study in English
Improving
Language
Proficiency
Cross-cultural
Understanding and
Identity as Japanese
Practical
Learning
Approach
Learning
Objectives
Certificate
Program (sub-
major)
Goal setting
for language
study
International
Exchange
Activities
English Medium
Instruction
Project Based Study
and Graduate Thesis
in English
Structure of ToyoRDS folio
Domestic
Activities
Overseas
Activities
Language
Learning
Global Activity
Point System
Sub-
Major
Certificate
Program
ToyoRDS Course
Study Abroad
Community Activities
& Seminars
Extracurricular
ESL & Test Prep
Academic Writing
Internship
Toyo RDS
Community
Student Circle
Seminars
Portfolio for each student
Institutional
Database
Information
SharingStudy Abroad &Scholarship Info
Event
Student ProfileIndividual Goal / Language Test Score
Record of Study Abroad
E-learning
Other
Individual
Activities
Student x Student （Community of Study Abroad）
Faculty
Comment
Student Peer
Comment
Student Peer
Comment
Student
Comment
Faculty x Student （Sub Major Program）
Student
Report
Faculty ①Comment
Faculty ②Comment
Faculty-led Program on ToyoRDS folio
Global Activity Point System
Information Sharing
Goal Setting
Creating Participants Community
Pre-Departure Program Follow up
Journal and Self Review
Final Report
Final Presentation
IDI (pre) IDI (post)
Individual Counseling / Coaching
ＩＤＩ(Intercultural Development Inventory)
Pre Test Study Abroad Post Test
Pre IDI
IDI
Model１：to study student growth in cultural sensitivity
Study Abroad
Model 2：In addition to model 1 purpose, evaluate study abroad programs
IDI for Faculty
Members
IDI
Post IDI
IDI is a cross-culturally valid assessment tool for analyzing cultural competence.
Usage of e-Portfolio (2013-2015)
0,0%
10,0%
20,0%
30,0%
40,0%
50,0%
60,0%
70,0%
80,0%
90,0%
100,0%
2012 Enrolled
2013 Enrolled
2014 Enrolled
TOEIC ScoreAnnouncement
TOEIC Score Announcement
Information Session
Demonstrationat required
class
Student Questionnaire ResultsRespondents：2nd-3rd graders in Faculty of
Regional Development Studies, Toyo UniversityPeriod: 20th May – 30th June, 2014Answered Numbers：106
convenient40%
Soso convenient43%
a little inconvenient
6%
inconvenient4%
neither 7%
Do You Think ToyoRDS folio Is Convenient?
Becoming an Asian hub university for global leaders
• Selected as one of 37 Top Global Universities
• Towards comprehensive internationalization
Evidence of SuccessSecond Stage of e-portfolioRenewal of ToyoNet Ace (2016- )
6
1. Globalization related
Foreigner teachers and teachers
who obtained a degree overseas
Possible Graduation Courses
held in a foreign language
Subjects held in English
Numbering of lectures and conversion
of syllabus into English
Status of introduction
of GPA
International availability
Flexible academic schedule
2. Governance related
3. Education reform efforts related
4. Others (Publication of education information)
5. Original university achievement index related
More than 50% (400
persons) of all teachers
17 courses or more
(5,000persons)
1,500 subjects
All 9,820 subjects
Graduation requirement
GPA 2.0 or higher
Establish a Foreign Credential
Evaluation Exclusive Department
Quarter system in a total of 14 faculties
Quick decision-makingPreparation of a scheme for president
governance
Scholarship supply Transmission at admission 500 persons
Practical learning
by the studentsAll-university survey by the IR
room
TOEFL external
test, etc.Entrance exam using an external
test 888persons
Publication of
education information
Participation in the university portrait.
Information analysis / publication by
the IR office
TGLTraining
Rubric
TGL Gold Certification 460 persons/year
TGL Silver Certification 1,000 persons/year
TGL Bronze Certification 2,500 persons/year
International availabilityStudents who enter this university
using the international transfer
system300persons/year
Sustainable education
Platform
TOYO-UCLA Continuous Education
Center-
Open courses:500courses
University Renovation to Increase
International Competitiveness
To further be an open university
Introduced
Multifaceted selection
of enrolled studentsBaccalaureat AO entrance examall
faculties
Student co-ed dormitory International hall / co-ed type500persons
More than 35 Numerical Targets
【Individual Evaluation Item3-B】 Special efforts 2) Toyo TGL program
Toyo Global Leader (TGL) Project (Certificate)
Renewal of e-portfolio (2016-)
Individual Record of Global Learning
Journal of Study Abroad (Monthly
Report)
History of Academic Work
English Proficiency
Course Work in English
Toyo Global Leader (TGL) Points
Study Abroad Experience History of English
Proficiency Test
History of Course Work in English
Distribution of Standardized Test Score
0
4
8
172
699
1308
1790
2019
1773
1220
820
511
339
195
116
77
69
31
12
6
0 1000 2000 3000
11
6
21
565
1551
2237
2679
2600
1994
1246
705
364
248
126
92
50
38
22
12
7
0100020003000
0-49
50-99
100-149
150-199
200-249
250-299
300-349
350-399
400-449
450-499
500-549
550-599
600-649
650-699
700-749
750-799
800-849
850-899
900-949
950-999
人
スコア
Male Students ：N=14,574
Femal StudentsN= 11,169 人
Based on TOEIC (March 2017)
Toyo Global Leader (TGL) Award
What should students collect?
Examples of student work• Papers / assignments• Test results• Peer assessments• Attitude / interest surveys• Statements of students’ goals• Notes from faculty, staff, supervisors• Internship supervisor evaluations• Capstone projects
Evidence of learning processes and growth• Work completed prior to the course• Drafts• Work logs
Reflection page / journal• Faculty’s evaluative summary of the portfolio• Reflections on each item in the portfolio & Faculty
comments
Future Goal of Assessment
Working Portfolio:CollectReflect
Project-based portfolio:
CollectReflect
Showcase Portfolio:SelectReflect
Assessment:StudentProgram
Institutional
Resources;ICC Frameworks…
Found at: • http://www.nafsa.org/_/file/_/theory_connections_intercultural_competence.p
df
Published in The Sage Handbook of Intercultural Competence (Sage, 2009) edited by Deardorff, in Building Cultural Competence (Stylus, 2012) edited by Berardo and Deardorff
Berardo, K. & Deardorff, D.K. (2012). Building Cultural Competence (Stylus)
Byram, M. (1998). Teaching and Assessing Intercultural Communicative Competence. Multilingual Matters.
Deardorff, DK (2009). The SAGE Handbook of Intercultural Competence, (Sage)
Deardorff, DK (2015) Demystifying Outcomes Assessment for International Educators (Stylus)
Deardorff, D & Arasaratnam-Smith, L (2017). Intercultural Competence in International Higher Education (Routledge)
For additional articles on these topics, please contact Darla K. Deardorff at [email protected]
Resources;