Assessment in mathematics education: The interface between the multiplicative conceptual field and the Rasch measurement model Caroline Long, CEA, University.

Assessment in mathematics education: The interface between the multiplicative conceptual field and the

Rasch measurement model

Caroline Long, CEA, University of Pretoria,

[email protected]

• Challenges for mathematics education• Theoretical tools

• Theory of conceptual fields (Vergnaud, 1983)• Threshold concepts (Meyer & Land, 2005)• Zone of proximal development (Vygotsky, 1962, 1978)

• Research study• Rasch measurement theory• Reflections on the outcomes

• Educational implications• Application of the Rasch model in mathematics education

settings

• Further developments

Presentation outline

Challenges for mathematics teaching

• The transition from working with whole number to rational numbers (and real numbers), is noted as a critical mathematical development in the transition years, Grades 6 to 10 (Usiskin, 2005), which in large part determines the mathematical future for learners.

• Fractions, ratio and related topics like percent, are taught procedurally without regard for their complex interconnections and their applications in everyday situations (Kieren, 1976; Parker & Leinhardt, 1995; Lamon, 2007).

• It is the assimilation and accommodation of the component subconstructs comprising rational number that are required for synthesis in the predicative form (Kieren, 1976; Vergnaud, 1983; Lamon, 2007).

• The danger is that “tricks” are introduced to avoid real engagement with the concept (Davis, 2010).

Theory of conceptual fields (Vergnaud, 1983 - )

• Mathematics is developed through encountering concepts embedded in problem contexts

• No one-to-one mapping from mathematical concept to problem situation. In fact all such mappings are many-to-many

• Concepts are never learned singly but in relation to other concepts

• The multiplicative conceptual field include• at base multiplication and division, but also • the problem situations which are modelled by multiplication and division

and • the various representations that may be required.

• The purpose of mathematics education is to transform extant implicit and local conceptions of individual learners encountered in single problem situations into generalised mathematics concepts and theorems that may be applied to a class of related problems – from operational to predicative form

Threshold concepts (Meyer & Land, 2005)

• An historical perspective • increasingly abstract systems, radical conceptual shifts (Dantzig, 2007) • parallel conceptual reorganisation in the classroom (Sfard, 1991) • resonates with the notion of “epistemological obstacles” (Brousseau, 1983)

• A threshold concept is a critical concept which provides the gateway (to higher mathematics), and which by corollary inhibits mathematical progress where learners have not gained mastery.

• These conceptual gateways may be • transformative – “occasioning a significant shift in perception of a subject” • “troublesome” knowledge - defines “critical moments of irreversible

conceptual transformation”• irreversible – “unlikely to be forgotten”, “unlearned only through

considerable effort”, and • integrative – “exposing the previously hidden interrelatedness of concepts”

• The danger of substitution by naïve concepts which confine progress

Zone of proximal development (Vygotsky,1962)

Theory of conceptual fields builds on the work of Piaget and Vygotsky

• “(t)he discrepancy between a child’s actual mental age and the level he reaches in solving problems with assistance indicates the zone of his proximal development” (Vygotsky,1962, p. 102).

• Current interpretation - a conceptual and cognitive space comprising the distance or extent from current proficiency to a higher level, over which learning with assistance from teachers or peers may occur .

There may be hierarchies of concepts that build on one another and that some concepts may be within the learner’s current “zone of proximal development”. More abstract concepts may require prior mastery or at least partial mastery. For example an understanding of ratio may precede an understanding of time distance and speed.

Assessment instruments

The contextual features discussed lead us to assert that educational experiences in mathematics are ordered somewhat hierarchically and aligned with learners current proficiency levels.

How do we identify for individual learners, or groups of learners a suitable current conceptual space, an educational locale (or zone of proximal development) which is neither too tedious because the associated problem solutions are obvious, nor too difficult for the learner and hence frustrating or demotivating?

Can assessment instruments be designed that make explicit a conceptual and cognitive pathway that provides insights for the teacher, in the preparation of learning sequences and subsequent engagement with individual learners? And in this process, can instruments help identify threshold concepts?

Research design

Instrument construction, 36 items (18 MC and 12 polytomous items)• TIMSS 2003 released items, adapted slightly in some cases• Matrix design (12 common items and a further 24

distributed across 4 groups)

Learners at two schools, across three grades, 7, 8 and 9, 16 classes, and 330 learners

Reasonably well-functioning schools, each covering a range of South African demographics

Follow-up interviews on 4 items of graded difficulty with 6 learner groups (21 learners in all, located at specified proficiency levels as determined from a Rasch analysis).

Person-Item Map

Persons and items on the same scale, according to probabilistic estimates.

Selected ratio, rate and proportion items are described on the right.

Levels* aligned with logits are demarcated by bands from (- ,-2)

[-2, -1), [-1,0), [0, 1), [1,2), [3, + )

PERSONS [locations=estimated proficiency] ITEMS [locations=estimated difficulty] o | 7.0 | | 34 | 6.0 | | | | 5.0 | 26 | | | o | 4.0 o | | | | 27 | 3.0 o | | o | o | oo | 35 2.0 | 31 30 ooo | o | oo | oo | 1.0 o | 28 ooooo | 7 ooooo | 32 oooooooooo | 29 10 ooooooo | 0.0 ooooo | 18 19 ooooooooooooo | 9 8 15 oooooooooooo | 33 16 oooooooooooo | 14 25 5 oooooooooo | -1.0 ooooooooooooooooooo | 3 21 ooooooooooo | 23 oooooo | 4 24 ooooooooo | 20 6 12 11 ooooooooo | -2.0 ooooooooo | 22 oooo | ooooo | 2 17 o | 13 1 ooooo | -3.0 | o | |

o = 2 Persons

20. Selects fraction representing part to whole ratio.

30. Find the average speed given distance, 160 km, and time, 2.5 hours.

10. 2.4 litres for 30 hours, how many litres for 100 hours?

1. Given a proportional problem, able to apply multiplicative reasoning, 1:3:: 9:x.

5. Identifies proportional share of amount 45 000 divided in the ratio 2:3:4.

A note on terminology

“Levels” • overused, a place holder, • familiar (Griffin, 2006; Pisa 2009; TIMSS)• denotes a defined position on a unidimensional line

“Locale”• an area, a domain, or range on the 1D line• a difficulty locale (for a subset of problem situations) • a proficiency locale (for a subset of learners currently

exhibiting a similar proficiency)

Criterion zone (Wilson, 2005)

Item analysis by level and analytic category

Description Context Mathematical

situation Notation Range Mathematical structure Response process

Lev

el 1

[-

∞, -

2)

1. Given a proportional problem, apply multiplicative reasoning,

Collecting bottles

Proportional sharing

Natural language, whole numbers

Less than 30

1: 3 :: 9: ? M1

1 9

M2

3 x

Identify multiplicative relationship

Lev

el 2

[-

2,-1

)

20. Selects fraction representing part to whole ratio.

Birthday Part-whole comparison Inclusive ratio

Fraction notation

30

16 of total (16 + 14)

30

16

1416

16

M1

(time) first half

total

M2

(children) 16 30

Selects part-whole relationship

Lev

el 3

[-1

, 0) 5. 45 000 zeds

shared in unequal proportions, of 2, 3 and 4.

Family sharing money

Proportional sharing

Natural numbers in thousands, fraction and ratio notation

45000 (1000)

2 : 3 : 4

9

4 of 45 000 M1

4 9

M2

x 45

Identifies proportional relationship

Lev

el 4

[0

,1)

10. 2.4 litres to 30 hours, how many for 100 hours?

Capacity- time

Proportion Multiplicative comparison

Decimal notation

100 f (x) = 30

4.2 M1 (capacity)

2,4 x

M2

(hours) 30 100

Identifies proportional relationship and multiplicative operator

Lev

el 6

[1

,L 2

)

30. Average speed given distance and time.

Car rally, Distance, time, speed

Rate calculation

Decimal notation

160 h5.2

km160 =

64km/h st

d

M1

(hours) 2.5 1

M2

(km) 160

x

Identifies proportional relationship and calculates unit rate

Distractor analysis

Item analysis

Selected items (Item 1, Item 20, Item 5, Item 10, Item 30) at graded levels of difficulty,

levels of proficiency: means of 4 quartile groups (LQ, MLQ, MHQ, HQ).

The key mathematical concepts for success on the item noted below.

Multiple choice distractor plots: percentage of quartile group selecting each false choice

Item 1 (n=330)

Quartile Low Q ML Q MH Q High Q

Mean % 58 84 93 98

Simple ratio

Item 20 (n=83)

Quartile Low Q ML Q MH Q High Q

Mean % 28 70 81 86

Part-whole comparison

Item 5 (n=330)

Quartile Low Q ML Q MH Q High Q

Mean % 24 35 55 85

Multiplicative relationship, unequal proportions

Item 10 (n=330)

Quartile Low Q ML Q MH Q High Q

Mean % 8 19 28 65

Finding the unit rate

Item 30 (n =330)

Quartile Low Q ML Q MH Q High Q

Mean % 0 0 5 41

Understanding the relationship of distance speed and time

Summary information

MCF concepts embedded in the problem situations at each level.

Associated errors by level

Mean percentages correct for items by level and by quartile

MCF concepts Errors LQ MLQ MHQ HQ(-1.9) (-1.1) (0) (1.1)

• Part-whole of discrete and continuous quantities

• Both fraction and ratio meaning of fraction notation

• Multiplicative relationship between sets of ratios

• Fraction equivalence

• Part-part and part-whole ratios

• Percent concept and notation

• Connect probability with fraction measure

• Covariant relationships

• Comparative relationships

• Rational number, operator subconstruct

• Identification of ratio

• Multiplicative comparison (operator construct)

• Percentage increase (operator subconstruct)

• Applying ratio operator construct to find the sample

• Multiplicative comparison

• Fraction measures, addition (subtraction), multiplication (division)

• Ratio and rate concepts

• Multiplication (division) of decimals

• Probability and statistics concepts, “sample”, “random”

• Rate and ratio

• Percent, identifying referents

• Covariant relationships

• Reasoning with unknowns

• 2 step problems

29%

Lev

el 5

to

7

( >

1)

• Confusion with percent language and referents

0% 3% 9%

78%

Lev

el 4

[0,

1)

•

••

•

Consider only the numeratorAdditive reasoningLack of fluency with multiplication and divisionConfusion with terminology

14% 22% 28% 58%

Lev

el 3

[-1

,0)

•

•

•

Confusion with operator constructConfusion of additive and multiplicative relationshipIgnoring part of the problem

16% 35% 57%

94%

Lev

el 2

[-2

-1)

••••

Natural number confusionJust add the percent sign %Language difficultyIgnoring part of the problem

37% 61% 79% 91%

Lev

el 1

[-3,

2)

• Confusion between fraction measure and ratio meaning and with fraction notation

53% 88% 93%

Interviews: Item level by learner proficiency

Interview data

Interviews with the middle high proficiency group, on Item 5.

Written responses on the left, explanations of thinking about the problem on the right.

Item 5: Three brothers Thabo, Samuel and Dan, receive a gift of 45 000 zeds from their father. The money is shared between the brothers in proportion to the number of children each one has. Thabo has two children, Samuel has three children, and Dan has 4 children. How many zeds does Dan get?

Carola, (0.18 logits),

I divided all of them

4 500 divide by 2 …

…

For Sam, for Thabo, and I did the same for Samuel (divide by 3) and Dan (divide by 4).

…

I don’t know why I did that (laughs, and group laughs with her).

Shiluba, (0.26 logits)

I didn’t really completely understand but I got a part answer. My logic behind it … I divided 3 into the R45 000 … And this gave me R15 000. … I think I then timesed by two over one which gave me 30, which I then divided by three which then gave me R10 000.

Linda, (0.12 logits)

I realise I did it wrong … really bad … I worked out all of them together. All the children. And I divided them by the zeds. Then I got 5 000. Then I think you would divide the 5 000 by 4.

…

I did that wrong, I don’t know why.

Kate,(0.12 logits)

I said 45 000 divide by 3 and I got 9 000. And then I don’t know what I did and I got 5 000

I never worked out how much Dan alone gets.

If I had divided 5 000 by 4, I would have got the answer.

You mean multiply by 4.

I have no idea (and laughs)

Interview transcript

Item 5, middle high quartile group transcript

Discussion occurred after individual explanations of reasoning about the problem.

Speaker Transcript

Interviewer OK, so has anybody got a picture of what is happening here?

Shiluba We were supposed to add up the children and divide it into 45 000. And then we would get 5 000.

And what does that 5 000 mean. 5 000 for … Dan?

No, 5 000 for each of the children

For each of the children …

(Murmurs of realisation from the group)

You get 15 000.

(The group corrects her.)

You get 20 000.

Interviewer OK. Did you get that?

(Confirmation and chuckling from the group)

So where was the problem? Was it in the reading?

Shiluba In the “How many zeds does Dan get?” Because I didn’t really, … I kind of shut off … Which is that part?

The middle part?

Ja, “The money is shared between the brothers in proportion to the number of children each one has”.

OK,

So I disregarded the children.

Composite summary

Composite information from summaries of item analyses at levels of difficulty and summaries of interview data at levels of proficiency.

On the horizontal axis the proficiency levels are arranged from high to low.

On the vertical axis, the key concepts identified are listed from lesser difficulty to greater difficulty.

Common errors are noted in brackets,

Levels High ( L 5,6,7)

Middle high (L4)Middle low

(L3)Low

(L1, 2)

Ad

d a

nd

su

btr

act Fluency with addition

and subtractionFluency with addition and subtraction

(Mistakes, use of addition where multiplication more efficient)

(Use of addition where multiplication more efficient)

Mu

ltip

ly

and

div

ide

Fluency with multiplication and division, proportional sharing (Confusing multiplication and division in proportional shares)

Some fluency with multiplication and division, sharing (Lack of fluency with multiplication and division;Abandon sense making Applying incorrect algorithm)

Some fluency with multiplication and division, sharing (Confusing multiplication and division;Incorrect conception of proportional sharing)

Some fluency with multiplication and division(Misreading of the problem)

Fra

ctio

ns,

d

ecim

als,

op

erat

ion Conversions, operations

with decimal fractionsConversions of measurement units

Conversions from fraction to percentMultiplying decimal fractions

Fluency of operations with decimal fractions

Conversions from percent to decimals

Per

cen

t,

con

cep

t,

oper

atio

ns

Percent change - percent increasePercent - fraction typeConverting ratio relationship to a percentage

Finding percent increase (finding percent decrease) Percent, fraction type (Mixing objects, adding percents and quantities)

(Confusing percent with amount, rather than ratio. Difficulty changing referent when converting ratio to percent)

Usi

ng

rati

o an

d r

ate

con

cep

ts Using ratio understanding to find proportions, finding unit rate

Reasoning with ratio and rates (Lack of fluency, multiplication and division)

Iden

tify

va

riab

les

and

re

lati

onsh

ips Identifying variables

and relationshipsIdentifying scalar and function operators

Identifying variables and relationships (Confuses relationships incorrect use of algorithms)

(Incorrect identification of relationships, working back from the answer)

(Incorrect identification of relationships; working backwards from the answer)

Fluency with algorithms(Applying algorithms in conceptually clumsy ways, using inappropriate algorithms Misuse of equals sign)

Description of concepts and skills in bold, (errors enclosed in brackets)

Alg

orit

hm

s, s

ymb

olic

n

otat

ion

(Incorrect use of algorithms, Mistakes with cancelling, when applying algorithms)

(Rote use of algorithm, sometimes correct)

(Rote use of algorithms)

Reflections: theory of conceptual fields and Rasch measurement

• The insights gleaned from such a study have application in their local context, for example for the population targeted in the study, Grades 7, 8 and 9 in South African schools.– While some learners have yet to achieve fluency with

multiplication and division, others need extension at the higher end of the spectrum of multiplicative structures, for example, rate and double proportion problems

– The selection of proximate items not yet correctly mastered by learners is easily executed given the outcomes of the Rasch analysis, in conjunction with the theoretical analysis of the multiplicative conceptual field.

– The method advanced in this study follows that advocated Wilson (2005) for the most part, though further cycles of refinement may be implemented for any context.

The interactive relationship of theory and measurement

• This development and validation may elicit joint orderings of sets of items within a conceptual field, and respondents, the distances between which have a meaningful interpretation and permit the specification of a ZPD.

• We may infer from these relationships, positing educationally pertinent interventions.

• We claim that, given theoretical analysis, we may infer a degree of hierarchical ordering that, with further cycles replicated in these grades, may prove to be a helpful instrument in the planning and support of teaching.

• Provided that the instrument has been carefully constructed to include potential threshold concepts, these thresholds may be hypothesised and perhaps identified.

Concluding remarks

The theory of conceptual fields raises some questions about current articulations of the intended curriculum

From relationships of item difficulty and learner ability, educationally pertinent interventions can be targeted at groups of similar learners, as a means of reaching all learners efficiently – notion of ZPD

The identification of threshold concepts - somewhat more complicated • From an historical perspective, the topics which caused particular

consternation, for example the critical concepts that led to the construction of new number systems, the zero, the integer, the rational number etc., may be the starting point.

• The theoretical insights (see for example, Dantzig, 2007) into the topics which appear particularly troublesome and which require attention

Critical for threshold concepts to be hypothesised and then included in assessment research

Further developments: A model of assessment (Bennett & Gitomer, 2009)

Accountability component

Formative component

Professional development component

Accessible and powerful assessment

Accountability• A process of teacher involvement at the various phases of

the item construction• Accountability becomes intrinsic to the teacher

Professional development component• Deep domain specific knowledge • Expert input on identified problem areas

Formative assessment• A classroom component where teachers develop,

experiment and reflect on the suitability of current assessment in relation to the knowledge domain

• Professional learning communities (PLCs)

Critical questions for future research

How may the theoretical notions of conceptual fields (threshold concepts and zones of proximal development) inform educational assessment and measurement?

And reciprocally, how may measurement theory highlight and extend insights into conceptual fields, threshold concepts and zones of proximal development as they manifest in the progressive development of learning?

Thank you for your attention

[email protected]