Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton

Improving Student Reflections in Electronic Portfolio Environments

Evidence-based Learning, Writing Instruction, Reflection

Dr. David Denton

Concept Map Overview

Study

Questions

What instructional practices improve students’ electronic portfolio entries?

Does metadata (tags and categories) predict performance?

Writing

Writing

“writing ability is a source of construct-irrelevant variance” (Mislevy et la., 2004)

bPortfolio

Evidence-Based Learning

• Artifacts

• Program Standards

Example

Method

• Context and Participants

– Convenience Sample

– 11 Undergraduates

• Mostly females

Method

• Repeated measures

– Three portfolio entries

First Entry Second Entry Third Entry

8 months before

intervention

1 week before

intervention During

intervention

Method

• Measure: – Writing Quality Rubric

• Adapted from AACU VALUE rubrics • .82 inter-rater reliability

First Entry Second Entry Third Entry

8 months before

intervention

1 week before

intervention

During intervention

t-test

5

Method

Method

Writing Quality Score

Number of Tag Cloud Terms

Results

• Statistically significant improvement

M = 5.82 M = 2.36 M = 0.55

t(10) = 4.99, p < .001, d = 3.16, 95% CI = 1.91 to 5.00

First Entry Second Entry Third Entry

8 months before

intervention

1 week before

intervention

During intervention

t-test

5

Results

• Statistically significant correlation between

– Tag cloud terms –writing quality

– Total portfolio entries – writing quality

Discussion

• Intervention improved writing quality on third entry

• Use of metadata to predict general writing quality

1 Intervention

• Explicit directions on content and format

– Prompt

– Instructions

2 Intervention

• Communication of Assessment Criteria

– Rubric

3 Intervention

• Evaluating evidence

– Questions

4 Intervention

• Instructor and peer feedback

– Strengths and weaknesses

• Submitted to instructor

• Read aloud to peer

5 Intervention

• Revising

– Initial submission

• Instructor feedback

• Peer feedback

– Final submission

References

Yao, Y., Aldrich, J., Foster, K., & Pecina, U. (2009). Preservice teachers’ perceptions of an electronic portfolio as a tool for

reflection and teacher certification. Journal of Educational Research & Policy Studies, 9, 25-43.

Yao, Y., Thomas, M., Nickens, N., Downing, J., Burkett, R. S., & Lamson, S. (2008). Validity evidence of an electronic portfolio

for preservice teachers. Educational Measurement: Issues and Practice, 27, 10-24.

Mislevy, R. J., Almond, R. G., Lukas, & J. F. (2004). A brief introduction to evidence-centered design. CSE Report 632. U.S.

Department of Education. Los Angeles, CA: National Center for Research on Evaluation

Ayan, D., & Seferoglu, G. (2011). Using electronic portfolios to promote reflective thinking in language teacher education.

Educational Studies, 37(5), 513-521.