Modelling early mathematical competencies and misconceptions Erasmus Intensive Seminar Graz 2005 Gisela Dösinger.

Modelling early mathematical competencies and misconceptions

Erasmus Intensive Seminar Graz 2005

Gisela Dösinger

Objective

Development of an instrument Adaptive assessment Broad range of early mathematical knowledge Including misunderstandings Against the background of internalisation Remedial instruction

Motivation

Analysis of existing instruments Neglecting prenumerical knowledge Only limited sub-domains assessed Not varying presentation format Only coarsely defining which competencies assessed Not covering misunderstandings Redundant assessment

Young children and children with disabilities

Methodology

For modelling correct knowledge Competence - Performance Theory Korossy (1993)

For modelling misconceptions Information System Based Approach Scott (1982) Applied by Lukas (1997)

Investigation 1

Brief introduction to competence - performance theory

Performance: observable solution behaviour Competence: underlying knowledge Latent level explains manifest level Levels related to each other

Competence - Performance Theory

Investigation 1

Performance structure (Q,P) Competence structure (E,K)

Interpretation function k: Q (K)kq: { 1, 2,…,r }

Representation function p: K (Q)

Competence - Performance Theory

Investigation 1

Step 1. Define solution ways Step 2. Represent solution steps by competencies

L = { f(q)|qQ} = { {3},{1,2},{2,4},{3,4} , ... , {2,3,4,6}}

q f(q)a {3},{1,2}b {2,4},{3,4}c {2},{1,3}d {2,5},{3,5}e {1,2,4,6},{2,3,4,6}

Competence - Performance Theory

Investigation 1

Step 3. Construct competence space 3.1 Apply surmise function

3.2 Close basis under union

e ke s(e)1 {1,2},{1,3},{1,2,4,6} {1,2},{1,3}2 {1,2},{2,4},{2},{2,5},{1,2,4,6},{2,3,4,6} {2}3 {3},{3,4},{1,3},{3,5},{2,3,4,6} {3}4 {2,4},{3,4},{1,2,4,6},{2,3,4,6} {2,4},{3,4}5 {2,5},{3,5} {2,5},{3,5}6 {1,2,4,6},{2,3,4,6} {1,2,4,6},{2,3,4,6}

Competence - Performance Theory

Investigation 1

Step 4. Apply interpretation function

q kq

a {3},{3,5},{3,4},{1,2},{2,3},{1,2,3},{3,4,5},{1,2,5} …b {2,4},{3,4},{3,4,5},{1,2,4},{2,3,4},{1,3,4},{1,2,3,4} …c {2},{2,5},{2,4},{1,2},{2,3},{1,3},{1,2,3} …d {2,5},{3,5},{3,4,5},{2,4,5},{1,2,5},{2,3,5} …e {1,2,4,6},{2,3,4,6},{1,2,3,4,6},{1,2,4,5,6} …

Competence - Performance Theory

Investigation 1

Step 5. Apply representation function

k p(k)

… …{1,3,4} {a,b,c}{3,4,5} {a,b,d}{1,2,3,4,5} {a,b,c,d}... …

Competence - Performance Theory

Investigation 1

Step 6. Construct performance space 6.1 Apply surmise function

6.2 Close basis under union

q pq s(q)

a {a},{a,d},{a,b},{a,c},{a,b,d},{a,c,d},{a,b,c},{a,b,c,d},{a,b,c,e},{a,b,c,d,e} {a}b {a,b},{b,c},{a,b,d},{a,b,c},{a,b,c,d},{a,b,c,e},{a,b,c,d,e} {a,b},{b,c}c {c},{a,c},{c,d},{b,c},{a,c,d},{a,b,c},{b,c,d},{a,b,c,d},{a,b,c,e},{a,b,c,d,e} {c}d {a,d},{c,d},{a,b,d},{a,c,d},{b,c,d},{a,b,c,d},{a,b,c,d,e} {a,d},{c,d}e {a,b,c,e},{a,b,c,d,e} {a,b,c,e}

Competence - Performance Theory

Investigation 1

State of research on early mathematical knowledge Proto-quantitative schemata Enumerative processes Calculation: addition and subtraction Internalisation

Twenty-six competencies and three internalisation levels Derive dependencies among competencies

Early mathematical knowledge

Investigation 1

Problem construction: making competencies accessible Take a competency Search available instruments for problems Describe solution way Represent solution steps by competencies Supplement competencies by prerequisites Copy problem to other internalisation levels

Forty-nine problems in eighteen problem classes

Problem construction

Investigation 1

A

B

F C

G

R={(A,B),(A,C),(A,F),(A,G),(B,C),(B,F),(B,G),(F,G),(C,G)}P={{A},{A,B},{A,B,F},{A,B,C},{A,B,C,F},{A,B,C,F,G}}

Deriving problem order

Investigation 1

Hypotheses For each pair element of the relation it is expected

that more difficult problem…

…is solved less frequent than or as frequent as less difficult problem

…is not solved if the less difficult problem is not solved

The performance states are expected to fit the empirical solution patterns

Hypotheses

Investigation 1

Method Ninety-four kindergarteners Mean 62.64, standard deviation 9.89 Problems partitioned into subsets 13 problems each Overlapping substructures Subjects tested individually

Method

Investigation 1

Solution frequency Solution frequencies in accordance with hypothesis 1

A .. 74%

B .. 79%

F .. 42%

C .. 89%

G .. 37%

Results

Investigation 1

Gamma-Index Derived from -Index (Goodman & Kruskal, 1954) Measure of association indicating whether two

classifications are ordered likely or unlikely

Results

Investigation 1

Gamma-Index

Gamma-Indices varying from 0.64 to 0.96, significantly differing from 0, thus supporting hypothesis 2

dc

dc

NN

NNG

Results

qj

1 0qi 1 N c

0 N d

dc

dc

NN

NN

2

2

Investigation 1

Symmetric distance

Distance distribution, average symmetric distance, and standard deviation are calculated

Distance distributions compared to 'random' ones ‘Random‘ distributions obtained by using power set as

data set

Results

MNNMNMNMd \\min,min

Investigation 1

Symmetric distance

Symmetric distances ranging from 0.13 to 0.94, with distance distributions significantly differing from 'random‘ ones ,supporting hypothesis 3

Results

Distance Random data Empirical dataAbs. frequency Rel. Frequency Obs. frequency Exp. frequency Chi²

0 11 0,3438 54 34,38 11,201 16 0,5000 29 50,00 8,822 5 0,1562 17 15,62 0,12

32 1,0000 100 100,00 20,14

Investigation 1

Distance Agreement Coefficient Proposed by Schrepp (1993) For comparing fit of different knowledge structures

The smaller, the better the fit

pot

dat

d

dDA

Results

Investigation 1

Distance Agreement Coefficient Adapted for testing relative fit of one knowledge

structure Compare DA to its maximal possible value which is

got when ddat is set to its maximal possible value

Distance Agreement Coefficient ranging from 0.05 to 0.32, much smaller than DAmax which was about 2, thus supporting hypothesis 3

Results

Investigation 1

Reproducibility Coefficient Proposed by Guttman (1944) Another measure explaining extent of concordance

between data and hypothesised structure Proportion of cells explained by model Example: a b c d

np

dREP

1a b c d

Subject 1 1 1 1 0Subject 2 0 1 0 1Subject 3 1 0 0 0Subject 4 1 1 0 0

Results

Investigation 1

Reproducibility Coefficient Reproducibility coefficient ranging from 0.93 to 0.99,

thus supporting hypothesis 3

Results

Investigation 1

Substructures proved valid Adaptive assessment Problems on different internalisation levels and on proto-

quantitative/quantitative level not varying in difficulty Use of abstract materials: difficulty to transfer knowledge

from concrete, everyday materials Reason for large number of competencies: broad range

intended to be covered and fine grained dissolution required

Discussion

Investigation 2

Brief introduction to information system based approach Manifest level: correct solutions and bugs Latent level: competencies and misconceptions

Two main concepts on the latent level Implication Incompatibility

Information system based approach

Investigation 2

Implication and incompatibility impose an algebraic structure

Information system A is a structure

‹D,Con, › where

D is set of data objects XCon is a set of finite consistent subsets of D is a binary relation ConD

Information system based approach

Investigation 2

On manifest level there is a polytomous response format: correct responses, bugs, and slips

For every qQ there is a set of possible responses

Rq = {q0,q1,q2,…,qn}

If R = Rq \ q0 a response pattern T is a subset of R

Information system based approach

Investigation 2

Relating latent and manifest level

Gq: A Rq

G: A (R)

G(x) = Gq(x)|q0

Information system based approach

Investigation 2

Identify set of knowledge entities D and their structure

Invariance principle a Additive principle b Spatial distortion c

b a c incompatible with a and b

Information system based approach

Investigation 2

Derive information system A Build power set Cancel inconsistent subsets Cancel subsets incompatible with

Con = {{},{a},{c},{a,b}}

a b c

0 0 0

1 0 0

0 1 0

0 0 1

1 1 0

1 0 1

0 1 1

1 1 1

Information system based approach

Investigation 2

Construct problems and determine responses Problem A

Spatial, number conserving change: row-to-circle

A1…same number, A2…more, A3…less Problem B

Spatial, number conserving change: spread-row

B1…same number, B2…more Problem C

Splitting set into subsets: partition-set

C1…same number, C2…more

Information system based approach

Investigation 2

Relate responses to elements xof information system

Determine response patterns

R q \q 0 x

A1 {a}

B1 {a}

C1 {a,b}

A2 {c}

A3 {c}

B2 {c}C2 {c}

Information system based approach

x T{} {}{a} {A1,B1}

{c} {A2,B2,C2},{A3,B2,C2}

{a,b} {A1,B1,C1}

Investigation 2

Measure for validating structure: discrepancy between empirical and hypothetical response patterns Number of problems in which patterns disagree

- 2

Comparison to 'random' case: randomly generated response patterns

U-Test for testing statistical significance

21,1 ,cba

Measure for validation

112 ,, cba

Investigation 2

Method Sixty-four kindergarteners Mean 61.84, standard deviation 9.40 Problems partitioned into subsets 12 problems each Subjects tested individually

Method

Investigation 2

Results Proportions of correct and buggy solutions Discrepancy

Discouraging output Re-modelling

Results

Investigation 2

No valid model found, but… Set of misconceptions identified Problems able to provoke their application designed Empirical evidence proven

No age effect Application of misconceptions seems to depend from

kind of problem Bugs arising from perceptual distraction play important

role

Discussion

Investigation 2

Misconceptions are not stable better use a probabilistic approach

Information system based approach: implications need to be neglected, because only excluding responses can be contained in response pattern

Discussion

Thank you for your attention!