Recruitment of Small Synergistic Movement Makes a Good Pianist Beth Jelfs 1,2 Shengli Zhou 1,2,3 Bernard K.Y. Wong 1 Chung Tin 2,4 Rosa H.M. Chan 1,2 1 Department of Electronic Engineering City University of Hong Kong 2 Centre for Biosystems, Neuroscience & Nanotechnology City University of Hong Kong 3 School of Astronautics Northwestern Polytechnical University 4 Department of Mechanical & Biomedical Engineering City University of Hong Kong Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 1 / 12
29
Embed
Recruitment of Small Synergistic Movement Makes a Good …
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Recruitment of Small Synergistic Movement Makesa Good Pianist
Beth Jelfs1,2 Shengli Zhou1,2,3 Bernard K.Y. Wong1
Chung Tin2,4 Rosa H.M. Chan1,2
1Department of Electronic EngineeringCity University of Hong Kong
2Centre for Biosystems, Neuroscience &Nanotechnology
City University of Hong Kong
3School of AstronauticsNorthwestern Polytechnical University
4Department of Mechanical & BiomedicalEngineering
City University of Hong Kong
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 1 / 12
What Can We Learn From Pianists?
Interested in coordinationHow does practice manifest inexperienced pianists?
Are There Differences in How the Fingers Move ...Individually? - jointsTogether? - simultaneous movementIndependently? - sequential movement
Aim:Can we identify fundamental differences in movement of novice &experienced pianists
Subjects:5 novices - little or no experience5 experienced - either grade 8 (1-8) or concert level
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 2 / 12
What Can We Learn From Pianists?
Are There Differences in How the Fingers Move ...Individually? - jointsTogether? - simultaneous movementIndependently? - sequential movement
Aim:Can we identify fundamental differences in movement of novice &experienced pianists
Subjects:5 novices - little or no experience5 experienced - either grade 8 (1-8) or concert level
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 2 / 12
What Can We Learn From Pianists?
Are There Differences in How the Fingers Move ...Individually? - jointsTogether? - simultaneous movementIndependently? - sequential movement
Aim:Can we identify fundamental differences in movement of novice &experienced pianists
Subjects:5 novices - little or no experience5 experienced - either grade 8 (1-8) or concert level
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 2 / 12
What Can We Learn From Pianists?
Are There Differences in How the Fingers Move ...Individually? - jointsTogether? - simultaneous movementIndependently? - sequential movement
Aim:Can we identify fundamental differences in movement of novice &experienced pianists
Subjects:5 novices - little or no experience5 experienced - either grade 8 (1-8) or concert level
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 2 / 12
Movement Synergies
Fundamental patterns of movementServe as building blocksReduce complexity (degrees offreedom) to be controlledRequires few components to explainvariance in movement
KinematicsMovement recorded usingdata glove14 sensors
2 per finger (MCP, PIP)4 abduction
Rate of flexion analyzed
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 3 / 12
Movement Synergies
Fundamental patterns of movementServe as building blocksReduce complexity (degrees offreedom) to be controlledRequires few components to explainvariance in movement
KinematicsMovement recorded usingdata glove14 sensors
2 per finger (MCP, PIP)4 abduction
Rate of flexion analyzed
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 3 / 12
Music
Practical Exercises for Beginners, Op.599Carl Czerny
Taken from Czerny Op. 599 - designed to improve fingering ofnovice pianistsStatic hand using all 5 fingers
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 4 / 12
Music
Practical Exercises for Beginners, Op.599Carl Czerny
For analysis:Select 3 consecutive notesCapture transitions before & after the key press of interest
Identify finger playing middle noteGroup all sets of 3 notes by the finger playing the middle note
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 4 / 12
Music
Practical Exercises for Beginners, Op.599Carl Czerny
For analysis:Select 3 consecutive notesCapture transitions before & after the key press of interestIdentify finger playing middle note
Group all sets of 3 notes by the finger playing the middle note
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 4 / 12
Music
Practical Exercises for Beginners, Op.599Carl Czerny
For analysis:Select 3 consecutive notesCapture transitions before & after the key press of interestIdentify finger playing middle noteGroup all sets of 3 notes by the finger playing the middle note
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 4 / 12
Synergy Extraction via Principal Component Analysis
Subject 1
Subject 10
Sequence1-3-1
Sequence2-3-1
MiddleFinger
θi(1,1)̇
θi(L,K)̇θi(L,1)̇
θi(1,K)̇
Trial i
Samples k
Joint
s l
θj(1,1)̇
θj(L,K)̇θj(L,1)̇
θj(1,K)̇
Trial j
Samples k
Joint
s l
Normalize�θ(l)
PCACovarianceCij
Extraction performed on all subjects simultaneously1 data set per fingerExtract time varying synergies with fixed weighting coefficients
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 5 / 12
Variance Accounted For
1 2 3 4 5 6 7 8 9 1030
40
50
60
70
80
90
100
Component Number
VA
F (
%)
ThumbIndexMiddleRingLittle
First 4 components account for more than 60% of the variance forall fingers
What about the differences?We can successfully identify fundamental patterns in the motion ofall subjectsBut what can we learn from the differences between experienced& novice pianists?
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 6 / 12
Variance Accounted For
1 2 3 4 5 6 7 8 9 1030
40
50
60
70
80
90
100
Component Number
VA
F (
%)
ThumbIndexMiddleRingLittle
What about the differences?We can successfully identify fundamental patterns in the motion ofall subjectsBut what can we learn from the differences between experienced& novice pianists?
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 6 / 12
Classification
EmptySubset
EMGMM
ImprovedResult?
1st UnselectedComponent
FinalSubset
SelectBest
Repeat EachUnselected Component
Add Componentto Subset
Yes
No
Classify the weightings of the PCsExpectation Maximization algorithm
Fit a 2 class Gaussian Mixture Model
Iterative forward selection of components
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 7 / 12
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 8 / 12
Reconstructed Signals for Middle Finger
0 0.5 1
−0.010
0.010.02
ThumbP
IP J
oint
0 0.5 1
−0.010
0.010.02
Index
0 0.5 1
−0.010
0.010.02
Middle
0 0.5 1
−0.010
0.010.02
Ring
0 0.5 1
−0.010
0.010.02
Little
0 0.5 1
−0.010
0.010.02
MC
P J
oint
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
Abd
uctio
n
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
NovicesExperienced
Reconstructed signals using only the selected componentsThe movement relating to the greatest separability
Comparison:Focus on four long fingers as movement of thumb is different
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 9 / 12
Reconstructed Signals for Middle Finger
0 0.5 1
−0.010
0.010.02
ThumbP
IP J
oint
0 0.5 1
−0.010
0.010.02
Index
0 0.5 1
−0.010
0.010.02
Middle
0 0.5 1
−0.010
0.010.02
Ring
0 0.5 1
−0.010
0.010.02
Little
0 0.5 1
−0.010
0.010.02
MC
P J
oint
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
Abd
uctio
n
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
0 0.5 1
−0.010
0.010.02
NovicesExperienced
Comparison:Focus on four long fingers as movement of thumb is different
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 9 / 12
Variability in Reconstructed SignalsIndex Finger
0
0.005
0.01
0.015
Var
ianc
e
Thum
b M
CP
Thum
b IP
T/I Abd
Inde
x MCP
Inde
x PIP
I/M A
bd
Midd
le M
CP
Midd
le PIP
M/R
Abd
Ring M
CP
Ring P
IP
R/L A
bd
Little
MCP
Little
PIP
NovicesExperienced
Ring Finger
0
0.01
0.02
0.03
0.04
0.05
0.06
Var
ianc
e
Thum
b M
CP
Thum
b IP
T/I Abd
Inde
x MCP
Inde
x PIP
I/M A
bd
Midd
le M
CP
Midd
le PIP
M/R
Abd
Ring M
CP
Ring P
IP
R/L A
bd
Little
MCP
Little
PIP
NovicesExperienced
Middle Finger
0
0.002
0.004
0.006
0.008
0.01
0.012
Var
ianc
e
Thum
b M
CP
Thum
b IP
T/I Abd
Inde
x MCP
Inde
x PIP
I/M A
bd
Midd
le M
CP
Midd
le PIP
M/R
Abd
Ring M
CP
Ring P
IP
R/L A
bd
Little
MCP
Little
PIP
NovicesExperienced
Little Finger
0
0.005
0.01
0.015
0.02
0.025
0.03
Var
ianc
e
Thum
b M
CP
Thum
b IP
T/I Abd
Inde
x MCP
Inde
x PIP
I/M A
bd
Midd
le M
CP
Midd
le PIP
M/R
Abd
Ring M
CP
Ring P
IP
R/L A
bd
Little
MCP
Little
PIP
NovicesExperienced
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 10 / 12
In summary...
ConclusionsPatterns of movement common to all our subjectsDifferences between novice & experienced pianists in componentsaccounting for smaller varianceDistinctions between groups seem to be in how fingers interactwith each otherIs variance accounted for a sufficient measure for coordinatedmovement?
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 11 / 12
The End
Thank you for listening
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 12 / 12
Thumb Reconstruction
0 0.5 1
−0.010
0.010.020.03
Thumb
PIP
Joi
nt
0 0.5 1
−0.010
0.010.020.03
Index
0 0.5 1
−0.010
0.010.020.03
Middle
0 0.5 1
−0.010
0.010.020.03
Ring
0 0.5 1
−0.010
0.010.020.03
Little
0 0.5 1
−0.010
0.010.020.03
MC
P J
oint
0 0.5 1
−0.010
0.010.020.03
0 0.5 1
−0.010
0.010.020.03
0 0.5 1
−0.010
0.010.020.03
0 0.5 1
−0.010
0.010.020.03
0 0.5 1
−0.010
0.010.020.03
Abd
uctio
n
0 0.5 1
−0.010
0.010.020.03
0 0.5 1
−0.010
0.010.020.03
0 0.5 1
−0.010
0.010.020.03
NovicesExperienced
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 12 / 12
Index Finger Reconstruction
0 0.5 1
−0.05
0
0.05
Thumb
PIP
Joi
nt
0 0.5 1
−0.05
0
0.05
Index
0 0.5 1
−0.05
0
0.05
Middle
0 0.5 1
−0.05
0
0.05
Ring
0 0.5 1
−0.05
0
0.05
Little
0 0.5 1
−0.05
0
0.05
MC
P J
oint
0 0.5 1
−0.05
0
0.05
0 0.5 1
−0.05
0
0.05
0 0.5 1
−0.05
0
0.05
0 0.5 1
−0.05
0
0.05
0 0.5 1
−0.05
0
0.05
Abd
uctio
n
0 0.5 1
−0.05
0
0.05
0 0.5 1
−0.05
0
0.05
0 0.5 1
−0.05
0
0.05
NovicesExperienced
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 12 / 12
Ring Finger Reconstruction
0 0.5 1
−0.1
0
0.1
Thumb
PIP
Joi
nt
0 0.5 1
−0.1
0
0.1
Index
0 0.5 1
−0.1
0
0.1
Middle
0 0.5 1
−0.1
0
0.1
Ring
0 0.5 1
−0.1
0
0.1
Little
0 0.5 1
−0.1
0
0.1
MC
P J
oint
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
Abd
uctio
n
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
NovicesExperienced
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 12 / 12
Little Finger Reconstruction
0 0.5 1
−0.1
0
0.1
Thumb
PIP
Joi
nt
0 0.5 1
−0.1
0
0.1
Index
0 0.5 1
−0.1
0
0.1
Middle
0 0.5 1
−0.1
0
0.1
Ring
0 0.5 1
−0.1
0
0.1
Little
0 0.5 1
−0.1
0
0.1
MC
P J
oint
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
Abd
uctio
n
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
0 0.5 1
−0.1
0
0.1
NovicesExperienced
Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 12 / 12