Improving Student Reflections in Electronic Portfolio Environments
Evidence-based Learning, Writing Instruction, Reflection
Dr. David Denton
Concept Map Overview
Study
Questions
What instructional practices improve students’ electronic portfolio entries?
Does metadata (tags and categories) predict performance?
Writing
Writing
“writing ability is a source of construct-irrelevant variance” (Mislevy et la., 2004)
bPortfolio
Evidence-Based Learning
• Artifacts
• Program Standards
Method
• Context and Participants
– Convenience Sample
– 11 Undergraduates
• Mostly females
Method
• Repeated measures
– Three portfolio entries
First Entry Second Entry Third Entry
8 months before
intervention
1 week before
intervention During
intervention
Method
• Measure: – Writing Quality Rubric
• Adapted from AACU VALUE rubrics • .82 inter-rater reliability
First Entry Second Entry Third Entry
8 months before
intervention
1 week before
intervention
During intervention
t-test
5
Method
Method
Writing Quality Score
Number of Tag Cloud Terms
Results
• Statistically significant improvement
M = 5.82 M = 2.36 M = 0.55
t(10) = 4.99, p < .001, d = 3.16, 95% CI = 1.91 to 5.00
First Entry Second Entry Third Entry
8 months before
intervention
1 week before
intervention
During intervention
t-test
5
Results
• Statistically significant correlation between
– Tag cloud terms –writing quality
– Total portfolio entries – writing quality
Discussion
• Intervention improved writing quality on third entry
• Use of metadata to predict general writing quality
1 Intervention
• Explicit directions on content and format
– Prompt
– Instructions
2 Intervention
• Communication of Assessment Criteria
– Rubric
3 Intervention
• Evaluating evidence
– Questions
4 Intervention
• Instructor and peer feedback
– Strengths and weaknesses
• Submitted to instructor
• Read aloud to peer
5 Intervention
• Revising
– Initial submission
• Instructor feedback
• Peer feedback
– Final submission
References
Yao, Y., Aldrich, J., Foster, K., & Pecina, U. (2009). Preservice teachers’ perceptions of an electronic portfolio as a tool for
reflection and teacher certification. Journal of Educational Research & Policy Studies, 9, 25-43.
Yao, Y., Thomas, M., Nickens, N., Downing, J., Burkett, R. S., & Lamson, S. (2008). Validity evidence of an electronic portfolio
for preservice teachers. Educational Measurement: Issues and Practice, 27, 10-24.
Mislevy, R. J., Almond, R. G., Lukas, & J. F. (2004). A brief introduction to evidence-centered design. CSE Report 632. U.S.
Department of Education. Los Angeles, CA: National Center for Research on Evaluation
Ayan, D., & Seferoglu, G. (2011). Using electronic portfolios to promote reflective thinking in language teacher education.
Educational Studies, 37(5), 513-521.
