Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure

Asymmetric Triangulation Scaling:

Asymmetric MDS for Visualizing Inter-Item Dependency Structure

SHOJIMA KojiroThe National Center for University Entrance Examinations

shojima@rd.dnc.ac.jp

Purpose of Research

• Development of method for visualizing inter-item dependency structure– Especially important for analyzing math test data

• Proposal: ATRISCAL– Asymmetric Triangulation Scaling

– An asymmetric multidimensional scaling

– Data: Conditional correct response rate matrix

Joint correct response rate matrix

• n×n symmetry matrix• The j-th diagonal element P(j,j)=P(j) – Correct response rate of item j

• The ij-th off-diagonal element P(i,j) – Joint correct response rate of items i and j– symmetry P(i,j)=P(j,i)

Item 1 Item 2 ⋯ Item nItem 1 P(1,1) P(1,2) ⋯ P(1,n)Item 2 P(2,1) P(2,2) ⋯ P(2,n)⋮ ⋮ ⋮ ⋱ ⋮

Item n P(n,1) P(n,2) ⋯ P(n,n)

Conditional correct response rate matrix

• n×n asymmetry matrix• The j-th diagonal element P(j|j)=P(j)/P(j)=1.0• The ij-th off-diagonal element P(j|i)=P(i,j)/P(i)– The correct response rate of item j when item i is

answered correctly– P(i|j)≠P(j|i): Usually asymmetric

Item 1 Item 2 ⋯ Item nItem 1 P(1,1) P(1,2) ⋯ P(1,n)Item 2 P(2,1) P(2,2) ⋯ P(2,n)⋮ ⋮ ⋮ ⋱ ⋮

Item n P(n,1) P(n,2) ⋯ P(n,n)

Item 1 Item 2 ⋯ Item nItem 1 P(1,1)/P(1) P(1,2)/P(1) ⋯ P(1,n)/P(1)Item 2 P(2,1)/P(2) P(2,2)/P(2) ⋯ P(2,n)/P(2)⋮ ⋮ ⋮ ⋱ ⋮

Item n P(n,1)/P(n) P(n,2)/P(n) ⋯ P(n,n)/P(n)

Item 1 Item 2 ⋯ Item nItem 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮

Item n P(1|n) P(2|n) ⋯ 1

Multidimensional scaling (MDS)

X12

X3X9

X7

X13

X10

X8

X11

X2

X4

X6

X5

X14

X1

X15

Q1

QM Q2

O

Asymmetry conditional correct response rate matrixItem 1 Item 2 ⋯ Item n

Item 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮

Item n P(1|n) P(2|n) ⋯ 1

𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ⊥𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ

𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ= 𝑘𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ+ሺ1− 𝑘ሻ𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ (0 ≤ 𝑘 ≤ 1)

|𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ| = 𝑃(𝑖) |𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ| = 𝑃(𝑗)

|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ| = 𝑃(𝑖,𝑗)

Item i Item jItem i 1 P(j|i)Item j P(i|j) 1

X ｊ

Xi

O

Xij

𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ= −𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ∙𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ|𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ| 𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ+ 𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ

Relationship betweenitems i and j

|𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ| = 𝑃(𝑖) |𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ| = 𝑃(𝑗)

|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ| = 𝑃(𝑖,𝑗)

Relationship betweenitems i and j

Item i Item jItem i 1 P(j|i)Item j P(i|j) 1

X ｊ

Xi

O

Xij

𝑃ሺ𝑖ȁ�𝑗ሻ= 𝑃(𝑖,𝑗)𝑃(𝑗)

𝑃ሺ𝑗ȁ�𝑖ሻ= 𝑃(𝑖,𝑗)𝑃(𝑖)

|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ||𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ|

|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ||𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ|

𝑐𝑜𝑠∠𝑋𝑖𝑂𝑋𝑖𝑗

𝑐𝑜𝑠∠𝑋𝑗𝑂𝑋𝑖𝑗

Asymmetric correct response rate matrix

• The asymmetric matrix lacks information about the correct response rate of each item

• So we add the imaginary n+1-th item whose correct response rate is 1.0

– P(j|n+1)=P(j,n+1)/P(n+1)=P(j)– P(n+1|j)=P(j,n+1)/P(j)=1.0

Item 1 Item 2 ⋯ Item nItem 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮

Item n P(1|n) P(2|n) ⋯ 1

Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) P(n+1|1)Item 2 P(1|2) 1 ⋯ P(n|2) P(n+1|2)⋮ ⋮ ⋮ ⋱ ⋮ ⋮

Item n P(1|n) P(2|n) ⋯ 1 P(n+1|n)Item n+1 P(1|n+1) P(2|n+1) ⋯ P(n|n+1) 1

Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) 1Item 2 P(1|2) 1 ⋯ P(n|2) 1⋮ ⋮ ⋮ ⋱ ⋮ ⋮

Item n P(1|n) P(2|n) ⋯ 1 1Item n+1 P(1) P(2) ⋯ P(n) 1

Expanded

Stress function

)1(

)|(1

)(,

nn

jipp

n

jiji 　

||

||)|(

j

ij

OX

OXji

)()(

)()( X

X

XX U

T

SF

1

)(,

2)|()|()(n

jiji ij jijipλS　

X

1

)(,

2)|()(n

jiji ijij pjiλδT　

X

21

)(,arccosexp)|(1)|(1)( ij

n

jijiijpjipU

　X

Xij

δ (delta)Well-formed triangle

• The perpendicular foot from O falls on line segment XiXj

Not well-formed triangle

• The foot from O does NOT fall on line segment XiXj

Xi

X ｊ

O

Xij

XiX ｊ

O

δij=δji=1

δij=δji=0

λ （ lambda ）

Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) 1Item 2 P(1|2) 1 ⋯ P(n|2) 1⋮ ⋮ ⋮ ⋱ ⋮ ⋮

Item n P(1|n) P(2|n) ⋯ 1 1Item n+1 P(1) P(2) ⋯ P(n) 1

0.5

0.51

1

otherwisen

njiifij 1

,1 　

Spatial indeterminacy and fixed coordinates

• Number of dimensions=3

• Coordinates of item n+1– (xn+1=0, yn+1=0, zn+1=1)

• Coordinate of item k, which has the lowest correct response rate– (xk=0, yk>0, zk)

• Coordinate of item l, which has a moderate P( ・ |k)– (xl>0, yl, zl)

Optimization of Stress Function

• Two-stage optimization– Stage 1: Simple genetic algorithm (SGA)

– Stage 2: Steepest descent method (SDM)

The author is grateful to Dr. Akinori Okada (Tama University) for speaking about this strategy at the spring seminar of the Behaviormetric Society of Japan (Gotemba, Japan) in March 2010.

Demonstration of exametrika

14

www.rd.dnc.ac.jp/~shojima/exmk/index.htm

Result of Analysis: Radial Map

• Red dots– Estimated coordinates

• Orange dots – Points of intersections

of extensions of red line segments and the surface of the hemisphere

Relationship betweenimaginary item n+1and item j

• P(j)→1.0• P(k)→0.0

Item j Item n+1Item j 1 1

Item n+1 P(j) 1

Xj

P(j)1

P(k)

Xk

Xn+1

O

Relashinship between items i and j

• P(i)<P(j)• P(i|j)→1.0• P(i|j)→0.0

Item i Item jItem i 1 P(j|i)Item j P(i|j) 1

Xn+1

Xi

O

Xj

Topographic Map

• The coordinates of orange points are projected onto the XY plane

• Voronoi tessellation

• Lift each Voronoi region by the length of the orange line segment

– Separate height with different colors

Mastery Maps

• For each examinee

Demonstration of exametrika

20

www.rd.dnc.ac.jp/~shojima/exmk/index.htm

Thank you for listening.

SHOJIMA KojiroThe National Center for University Entrance Examinations

Tokyo Institute of Technology

shojima@rd.dnc.ac.jp