Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure SHOJIMA Kojiro The National Center for University Entrance Examinations [email protected]
Dec 30, 2015
Asymmetric Triangulation Scaling:
Asymmetric MDS for Visualizing Inter-Item Dependency Structure
SHOJIMA KojiroThe National Center for University Entrance Examinations
Purpose of Research
• Development of method for visualizing inter-item dependency structure– Especially important for analyzing math test data
• Proposal: ATRISCAL– Asymmetric Triangulation Scaling
– An asymmetric multidimensional scaling
– Data: Conditional correct response rate matrix
Joint correct response rate matrix
• n×n symmetry matrix• The j-th diagonal element P(j,j)=P(j) – Correct response rate of item j
• The ij-th off-diagonal element P(i,j) – Joint correct response rate of items i and j– symmetry P(i,j)=P(j,i)
Item 1 Item 2 ⋯ Item nItem 1 P(1,1) P(1,2) ⋯ P(1,n)Item 2 P(2,1) P(2,2) ⋯ P(2,n)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1) P(n,2) ⋯ P(n,n)
Conditional correct response rate matrix
• n×n asymmetry matrix• The j-th diagonal element P(j|j)=P(j)/P(j)=1.0• The ij-th off-diagonal element P(j|i)=P(i,j)/P(i)– The correct response rate of item j when item i is
answered correctly– P(i|j)≠P(j|i): Usually asymmetric
Item 1 Item 2 ⋯ Item nItem 1 P(1,1) P(1,2) ⋯ P(1,n)Item 2 P(2,1) P(2,2) ⋯ P(2,n)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1) P(n,2) ⋯ P(n,n)
Item 1 Item 2 ⋯ Item nItem 1 P(1,1)/P(1) P(1,2)/P(1) ⋯ P(1,n)/P(1)Item 2 P(2,1)/P(2) P(2,2)/P(2) ⋯ P(2,n)/P(2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1)/P(n) P(n,2)/P(n) ⋯ P(n,n)/P(n)
Item 1 Item 2 ⋯ Item nItem 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
Multidimensional scaling (MDS)
X12
X3X9
X7
X13
X10
X8
X11
X2
X4
X6
X5
X14
X1
X15
Q1
QM Q2
O
Asymmetry conditional correct response rate matrixItem 1 Item 2 ⋯ Item n
Item 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ⊥𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ= 𝑘𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ+ሺ1− 𝑘ሻ𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ (0 ≤ 𝑘 ≤ 1)
|𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ| = 𝑃(𝑖) |𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ| = 𝑃(𝑗)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ| = 𝑃(𝑖,𝑗)
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
X j
Xi
O
Xij
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ= −𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ∙𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ|𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ| 𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ+ 𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ
Relationship betweenitems i and j
|𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ| = 𝑃(𝑖) |𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ| = 𝑃(𝑗)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ| = 𝑃(𝑖,𝑗)
Relationship betweenitems i and j
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
X j
Xi
O
Xij
𝑃ሺ𝑖ȁ�𝑗ሻ= 𝑃(𝑖,𝑗)𝑃(𝑗)
𝑃ሺ𝑗ȁ�𝑖ሻ= 𝑃(𝑖,𝑗)𝑃(𝑖)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ||𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ|
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ||𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ|
𝑐𝑜𝑠∠𝑋𝑖𝑂𝑋𝑖𝑗
𝑐𝑜𝑠∠𝑋𝑗𝑂𝑋𝑖𝑗
Asymmetric correct response rate matrix
• The asymmetric matrix lacks information about the correct response rate of each item
• So we add the imaginary n+1-th item whose correct response rate is 1.0
– P(j|n+1)=P(j,n+1)/P(n+1)=P(j)– P(n+1|j)=P(j,n+1)/P(j)=1.0
Item 1 Item 2 ⋯ Item nItem 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) P(n+1|1)Item 2 P(1|2) 1 ⋯ P(n|2) P(n+1|2)⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 P(n+1|n)Item n+1 P(1|n+1) P(2|n+1) ⋯ P(n|n+1) 1
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) 1Item 2 P(1|2) 1 ⋯ P(n|2) 1⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 1Item n+1 P(1) P(2) ⋯ P(n) 1
Expanded
Stress function
)1(
)|(1
)(,
nn
jipp
n
jiji
||
||)|(
j
ij
OX
OXji
)()(
)()( X
X
XX U
T
SF
1
)(,
2)|()|()(n
jiji ij jijipλS
X
1
)(,
2)|()(n
jiji ijij pjiλδT
X
21
)(,arccosexp)|(1)|(1)( ij
n
jijiijpjipU
X
Xij
δ (delta)Well-formed triangle
• The perpendicular foot from O falls on line segment XiXj
Not well-formed triangle
• The foot from O does NOT fall on line segment XiXj
Xi
X j
O
Xij
XiX j
O
δij=δji=1
δij=δji=0
λ ( lambda )
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) 1Item 2 P(1|2) 1 ⋯ P(n|2) 1⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 1Item n+1 P(1) P(2) ⋯ P(n) 1
0.5
0.51
1
otherwisen
njiifij 1
,1
Spatial indeterminacy and fixed coordinates
• Number of dimensions=3
• Coordinates of item n+1– (xn+1=0, yn+1=0, zn+1=1)
• Coordinate of item k, which has the lowest correct response rate– (xk=0, yk>0, zk)
• Coordinate of item l, which has a moderate P( ・ |k)– (xl>0, yl, zl)
Optimization of Stress Function
• Two-stage optimization– Stage 1: Simple genetic algorithm (SGA)
– Stage 2: Steepest descent method (SDM)
The author is grateful to Dr. Akinori Okada (Tama University) for speaking about this strategy at the spring seminar of the Behaviormetric Society of Japan (Gotemba, Japan) in March 2010.
Demonstration of exametrika
14
www.rd.dnc.ac.jp/~shojima/exmk/index.htm
Result of Analysis: Radial Map
• Red dots– Estimated coordinates
• Orange dots – Points of intersections
of extensions of red line segments and the surface of the hemisphere
Relationship betweenimaginary item n+1and item j
• P(j)→1.0• P(k)→0.0
Item j Item n+1Item j 1 1
Item n+1 P(j) 1
Xj
P(j)1
P(k)
Xk
Xn+1
O
Relashinship between items i and j
• P(i)<P(j)• P(i|j)→1.0• P(i|j)→0.0
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
Xn+1
Xi
O
Xj
Topographic Map
• The coordinates of orange points are projected onto the XY plane
• Voronoi tessellation
• Lift each Voronoi region by the length of the orange line segment
– Separate height with different colors
Mastery Maps
• For each examinee
Demonstration of exametrika
20
www.rd.dnc.ac.jp/~shojima/exmk/index.htm
Thank you for listening.
SHOJIMA KojiroThe National Center for University Entrance Examinations
Tokyo Institute of Technology