What do international assessments measure: PISA Raymond J. Adams Washington DC, May 30 2008 This paper is intended to promote the exchange of ideas among researchers and policy makers. The views expressed in it are part of ongoing research and analysis and do not necessarily reflect the position of the National Center for Education Statistics, the Institute of Education Sciences, or the U.S. Department of Education.
21
Embed
What do international assessments measure: PISA Raymond J. Adams Washington DC, May 30 2008 This paper is intended to promote the exchange of ideas among.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
What do international assessments measure: PISA
Raymond J. Adams
Washington DC, May 30 2008
This paper is intended to promote the exchange of ideas among researchers and policy makers. The views expressed in it are part of ongoing research and analysis and do not necessarily reflect the position of the National Center for Education Statistics, the Institute of Education Sciences, or the U.S. Department of Education.
2
What is PISA?
Programme for International Student Assessment
Funded by the Organisation for Cooperation and Development (OECD)
International data collection has been managed and led by ACER since its inception
3
Developing PISA 1995: OECD countries sought
• comparative assessment of yield of education systems
• a basis for defining educational standards cross-nationally
1997: OECD countries adopted strategy and financial framework to...• monitor a broad range of curricular and cross-
curricular outcomes...…within a comparative framework established
collaboratively by countries 1998: Contract let
4
Participation 2000
• OECD 28, non-OECD 4• PISA+, 11 non-OECD 170,000 students
2003• OECD 30, non-OECD 12 250,000 students
2006• OECD 30, non-OECD 27• 14365 schools• 398,750 students (representing a
Unit for the Analysis of Educational Systems and Practices. University of Liege, Belgium
Westat, USA
National Institute for Educational Research, Japan
The German Institute for International Educational Research
cApStAn Linguistic Quality Control
12
Separating policy and science
Core B• Netherlands National Institute for
Educational Measurement (CITO)
•University of Jyvaskyla•University of Twente•French Ministry of Education
• Sets policy objectives and priorities
• Sets budget• Monitors adherence to policy
objectives• Guides analysis and reporting of
results• Establishes quality standards• Adjudicates on breaches of
quality standards
Consortia are commissioned to run PISA
15
Innovations in PISA International, intergovernmental
ownership• primary focus on public policy issues• sustained commitment• collaborative development
Age not grade-based target population Substance
• not common denominator of national curricula
• not whether students have learned• whether they can use what they have
learned• criterion-based reporting
16
PISA’s Literacy Orientation Projective (rather than reflective)
approach to instrument development
Not specifically related to curriculum• It is about authentic application of
what is learned• In contrast to assessments that are
like instructional exercises A broadening, not narrowing notion
17
Literacy Orientation
Domain definitions include two components:• each domain defined, in terms of
knowledge and skills needed in adult life, not merely in terms of mastery of the school curriculum.
• emphasis is placed on the mastery of processes, the understanding of concepts and the ability to function in various situations related to real life.
18
Basic Methodology Target population: 15-year-olds in school
• Students are spread over classes and grades Target sample sizes of 5250
• 150 schools of 35 students• 50 schools required for national sub-entities
Rotated two-hour written tests• 13 booklets• 3 domains (two minor and one major)• About 200 items in total• About 11 responses per item in each school• Students respond to an average of 13 minor
domain items and 30 major domain items• Some students are not assessed in minor
domains
19
Basic Methodology Student questionnaires School questionnaires Parent questionnaires No teacher questionnaires
• Students from multiple classrooms
• Focus is not limited to taught subjects
20
Focus of the Results Yield for well-defined populations and
domains• Reading, Mathematics, and Science every three
years• Sub-domains every nine years
National sub-group performance Profiles encompassing broad domain