dfff

Assessment of a Three-Year Argument Skill Development Curriculum

Amanda Crowell

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy

in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY

2011

2011 Amanda Crowell

All Rights Reserved

Abstract

Assessment of a Three-Year Argument Skill Development Curriculum

Amanda Crowell

This study examines whether middle-school students dense, extended

engagement in an argumentation curriculum promoted development of argument skills,

specifically increased use of direct counterargument and improved argument evaluation

skill. A total of 56 students in two classes participated twice a week for three years

(grades 6, 7, and 8) as part of their regular school curriculum. Students attended an urban

middle school affiliated with a large university and were predominantly Hispanic and

African-American and from lower and lower-middle socioeconomic backgrounds; 20%

were from middle-class Caucasian families. In addition to its central element

electronically conducted pair dialogs on social issues - the curriculum encompassed a

range of activities including small group preparation of arguments and reflective

activities. A third class of 23 served as a comparison group; they also met twice a week

over the same time period. They addressed similar social issues in more traditional

whole-class discussion and wrote essays.

Assessments of dialogic argumentation skill and argument evaluation skill

initially and at the end of each of the three years indicated that that the curriculum

promoted the use of counterargument generally and the direct counterargument skill

specifically. Performance of the experimental group increased over time in both respects

and exceeded that of the comparison group. Students participating in the intervention also

engaged in more sustained direct counterargument sequences than did students in the

comparison group at the final assessment. Parallel improvements in argument evaluation

skill of the experimental group relative to the comparison group suggest that evaluation

skill responds to practice much the same way as does argumentation performance.

Theoretical implications for our understanding of developmental mechanisms are

considered, as well as educational implications.

!

! "!

Table of Contents

Introduction and Literature Review 1

The Present Study: Rationale 15

Method 18

Participants 18

Initial, Continuing and Final Assessments 20

Individual Argumentation Skill 20

Argument Evaluation Test 21

Intervention 22

Opinion Assessment 24

Year One Procedure 24

Team method 24

Electronic method 28

The Showdown and Debrief 30

Phase Four 32

Year Two Procedure 33

The Pre-game 34

The Game 35

The End Game 35

Year Three Procedure 36

Comparison Group Curriculum 39

Results 41

Transcript Coding 41

Counterargument Use 42

! ""!

Sustained Direct Counterargument 45

Argument Evaluation 47

Discussion 49

Research Question: How does dense and extended practice in dialogic argumentation influence counterargument skill?

49

The Ceiling Effect 50

The Qualitative Case Study 54

Research Question: Do students in the intervention group sustain direct counterargument with an opponent for longer than students in a comparison group?

61

Research Question: Do students in the intervention group become more skilled in the evaluation of arguments relative to students in a comparison group?

61

Theoretical implicationsMechanisms 62

Practical and Educational Implications 64

Limitations and next steps 66

Conclusions 68

References 71

Appendix A: Experimental Group Equivalence 77 Appendix B: Capital Punishment Opinion Poll 78 Appendix C: Adaptive Response Schematic 79 Appendix D: Complete Description of Intervention Topics 80 Appendix E: Reflection Sheets 83 Appendix F: Showdown Preparation Sheets 85 Appendix G: Example of Year One Electronic Showdown Transcript 87 Appendix H: Complete List of Questions Provided and Answered 88 Appendix I: Example of Year Two and Three Showdown Transcript 95 Appendix J: Argument Game Script 100 Appendix K: Coding Scheme 101 Appendix L: Year-to-Year Change for Counterargument and Direct Counterargument

102

Appendix M: Argument Test 9-item Results 103 Appendix N: Indirect Counterargument Use 104 Appendix O: Full dialogs for students 32 and 62 105

! """!

List of Tables

Table 1: Sample details #$!

Table 2: Individual Argumentation Assessment Timeline %#!

Table 3: Summary of Phase Topics %&!

Table 4: Year One Session Activities by Group &&!

Table 5: Example of the Argument game &'!

Table 6: Frequencies and Proportion Use for All Codes Accounting for at Least 1% of Total Codes

(%!

Table 7: Verbatim examples of counter-alternative and counter-critique! (&!

Table 8: Mean of proportion counterargument use by group and time (SD) (&!

Table 9: Mean of proportion counter-critique use by group and time (SD) ((!

Table 10: Mean Rebuttal Sequence Length (Reduced N in parentheses) ()!

Table 11: Mean of Score on 15 item test (SD) (*!

Table 12: Ceiling Effectt +#!

Table 13: Proportion Usage of Direct Counter Change Across Years and Subgroups

+&!

! ",!

List of Figures

figure 1: Year one schematic! !

figure 2: Super Card &+!

figure 3: Mean of proportion counterargument use by group and time ((!

figure 4: Mean of proportion counter-critique use by group and time (+!

figure 5: Mean of Rebuttal Sequence Length by group and time ()!

figure 6: Mean Score on 15 item argumexnt evaluation assessment ('!

Figure 7: Ceiling Effect +#!

figure 8: Case Study Student Trajectories )-!

figure 9: Split Intervention Group Trajectories )-!

! ,!

Acknowledgements

Thank you to my sponsor, Dr. Deanna Kuhn, whose expertise and guidance has shaped so much of my professional identity. Thank you to my committee members: Dr. John Black, Dr. Matthew Johnson, Dr. Jose Maldonado, Dr. Clark Chinn, and Dr. Michael Preston. Thank you to the members of the education for thinking lab (past and present!) without whom this project would have been a literal impossibility.

! ,"!

Dedicaton

For Jane who loved, Floyd who believed, David who supports and Abigail who inspires.!

!

!

!

"!!

!"#$%&'(#)%"*+"&*,)#-$+#'$-*.-/)-0*

!"#$%&'(')*$)+$,#-#.%)#$%-/$#0%(1%)#$2+1-/$%.,13#-)2$"%2$.#4#'0#/$'-4.#%2'-,$.#4+,-')'+-$

%2$%$51-/%3#-)%($%26#4)$+5$,++/$)"'-7'-,$%-/$%2$%-$#/14%)'+-%($,+%(8$$!"#$-#9$:+33+-$:+.#$

;)%-/%./2

!

!

!

#!!

)"%)$2)%-/2$4+-).%.*$)+$+-#U2$6+2')'+-SI8$!+1(3'-

!

!

!

$!!

('7#(*$+&B#4)'+-28$$!+$#-4+36%22$)"'2$4+36(#O')*

!

!

!

%!!

1-(#22$:".'2$'2$9'(('-,$)+$/.'0#$)+$%$4"'(/4%.#$4#-)#.$5%.$%9%*$5.+3$9"#.#$2"#$('0#2$%-/$9+.728$$

!"#$5'5)"$4+36+-#-)

!

!

!

&!!

4+-)%'-'-,$4+1-)#.%.,13#-)2$%.#$,'0#-$"',"#.$.%)'-,2$+5$%.,13#-)$2).#-,)"$Dc+(5#\I$6.+0'/#2$'-2',")$'-)+$"+9$%.,13#-)%)'+-$6.+3+)#2$4+,-')'0#$

/#0#(+63#-)8$$c"#-$6.#2#-)#/$9')"$-#9$'-5+.3%)'+-

!

!

!

'!!

/'2#Y1'('&.'13$6.#0%'(2

!

!

!

(!!

2166+.)$+-#U2$6.'+.$('#52I$6.+0'/#2$51.)"#.$'-2',")$'-)+$)"'2$/#5'4'#-4*8$!"#*$21,,#2)$%$

4+-)'-113$+5$.#26+-2#2$+-#$4%-$)%7#$)+9%./2$%-%3+(+12$/%)%$%-/$%$-13.$+5$'-)#.%4)'-,$

0%.'%&(#2$)"%)$9+1(/$6.#/'4)$9"#.#$+-#$5%((2$+-$)"%)$4+-)'-1138$$K2$%-$#O%36(#

!

!

!

)!!

#264'%((*$'5$+-#$"%2$%$.#(%)'0#(*$1-/#0#(+6#/$#6'2)#3+(+,'4%($1-/#.2)%-/'-,$+5$24#'-4#8$$!"%)$'2

!

!

!

*!!

1-/#.2)%-/'-,`$)"#$%&2+(1)'2)$(#0#(I

!

!

!

"+!

4+-2).14)$4"%-,#$&*$6.+0'/'-,$)"#$3+)'0%)'+-$-#4#22%.*$)+$2##7$+1)$%-/$%44+33+/%)#$%()#.-%)'0#$

6#.26#4)'0#28$W-$2"+.)

!

!

!

""!

[1)(#.

!

!

!

"#!

M#%2+-'-,

!

!

!

"$!

!"12F?$+5$1))#.%-4#2$/'.#4)(*$%//.#22'-,$%-$+66+-#-)U2$%.,13#-)8$$!%7#-$)+,#)"#.$)"'2$

#0'/#-4#$'-/'4%)#2$)"%)$9"'(#$3%)1.%)'+-$%(+-#$3%*$6.+0'/#$)"#$-#4#22%.*$5+1-/%)'+-$5+.$27'((#/$

!

!

!

"%!

%.,13#-)%)'+-$/#0#(+63#-)

!

!

!

"&!

L'0#-$)"#$6+)#-)'%((*$1-2)%&(#$-%)1.#$+5$6#+6(#U2$1-/#.2)%-/'-,$+5$#0'/#-4#

!

!

!

"'!

W-$)"#$#%.('#2)$2)1/'#2$/+-#$)+$/#0#(+6$2)1/#-)2U$4+1-)#.%.,13#-)$27'((2

!

!

!

"(!

6%'.$%-/$)"#-$.#%2+-$9')"'-$)"#$/'%(+,$')2#(5I

!

!

!

")!

1-#2%&$

!"#$'-)#.0#-)'+-$#-,%,#/$2)1/#-)2$'-$/#-2#$%.,13#-)%)'+-$6.%4)'4#$'-0+(0'-,$#(#0#-$

)+6'42$+0#.$%$)".##X*#%.$6#.'+/8$$!"#$.#2#%.4"$#36(+*#/$%$6.#X6+2)$.#6#%)#/$3#%21.#2$/#2',-$%-/$

'-4(1/#/$%$-+-X'-)#.0#-)'+-$4+36%.'2+-$,.+16$)+$4+36%.#$)"#$#55#4)2$+5$)"#$'-)#.0#-)'+-$+-$

/'%(+,'4$%.,13#-)%)'+-$27'(($%-/$'-/'0'/1%($%.,13#-)$#0%(1%)'+-$27'((8$$

%0&)14150()'*

V%.)'4'6%-)2$9#.#$2)1/#-)2$%))#-/'-,$%-$1.&%-

!

!

!

"*!

^]lG8EE

!

!

!

#+!

2)1/#-)2$/1#$)+$3'22'-,$%22#223#-)28$$!"#$5'-%($2%36(#$5+.$)"#$%.,13#-)$#0%(1%)'+-$)#2)$'2$HR$

2)1/#-)2$Dkk$#O6#.'3#-)%(I8$

6(1)1037$82()1(+1(9$0(,$:1(03$;''#''

!

!

!

#"!

%.,13#-)$9%2$4+-/14)#/$'-$9.')'-,8$!"#$)9+$3#3.2$+5$%$6%'.$6%22#/$)9##-$)"#3$%$6%/$9')"$

31()'6(#$2"##)2$+5$6%6#.>$W-')'%($%22#223#-)$5+.$-#9$2)1/#-)2$+-(**

8%",9)4

!

!

!

##!

J##$,'-2$9')"$%$4(%'3$%&+1)$9"*$2+3#$2)1/#-)2$5%'($'-$24"++(8$$@6+-$.#%/'-,$J##U2$

%.,13#-)

!

!

!

#$!

'2$/#24.'/$'-$51(($'-$>&.%4=A$4@%$(&-,%&8$>&.%4=3%&&4@%$(&-,%&$/#24.'$%//')'+-2$)+$)"#$gE$

41..'41(13$3%/#$'-$gR$%2$2)1/#-)$27'(($/#0#(+63#-)$%/0%-4#/8$i0#.$)"#$4+1.2#$+5$)".##$*#%.2$

2)1/#-)2$#-,%,#/$'-$#(#0#-$6"%2#28$W-$#%4"$6"%2#$)"#*$%//.#22#/$%$/'55#.#-)$)+6'4$D;##$!%&(#$R$

5+.$%$2133%.*$+5$)+6'42

!

!

!

#%!

?2!#!$#4855&559)4

[#5+.#$#%4"$6"%2#$,%-

!

!

!

#&!

)"#$)+6'48$$N%4"$2)1/#-)$9%2$,'0#-$%$(%.,#$'-/#O$4%./$+-$9"'4"$)+$9.')#$"'2$+.$"#.$A3+2)$

6#.21%2'0#$.#%2+-C$5+.$9"*$"'2$+.$"#.$2'/#U2$6+2')'+-$9%2$)"#$))#.$+-#8$$;)1/#-)2$9#.#$6.+0'/#/$

9')"$)"#$.%)'+-%(#$)"%)$')$'2$'36+.)%-)$)+$,#)$)"#'.$.#%2+-2$A+1)C$2+$)"%)$)"#*$4%-

!

!

!

#'!

6.+0'/#/$9')"$/16('4%)#$2#)2$+5$)"#$.#%2+-$4%./2$%-/$%27#/$)+$2+.)$)"#3$'-)+$)".##$4%)#,+.'#2$

DA)

!

!

!

#(!

:&55!$#4LC4D&%.)!#"4($,#)&%.%",9)54)$4$)3&%5K4%&.5$#58$!"#$,#-#.%)'+-$+5$

4+1-)#.%.,13#-)2$D2)%)#3#-)2$)"%)$/'.#4)(*$4.')'Y1#/$)"#$+)"#.$2'/#U2$.#%2+-2I

!

!

!

#)!

F$3'-1)#28$$$

R&S+&()!$#4453&&)5C$c"'(#$9%')'-,$5+.$)"#$+)"#.$2'/#$)+$4+36(#)#$)"#'.$)1.-

!

!

!

#*!

)"#$+)"#.$6%'.U2$)$%.,13#-)$%-/$"+9$)"#*$4+1(/$"%0#$.#26+-/#/$3+.#$#55#4)'0#(*Q$)"#*$9#.#$

%27#/$)+$A9.')#$%$))#.

!

!

!

$+!

"%/$(#5)$+55

!

!

!

$"!

$

D)A'$-*67*H-+$*%"-*;(2-9+#)($

$

nWVl$;"+9/+9-$/+-#$'-$6#.2+-6;?914!@1033!

A.37=33.9-!

B5011!C89=D95437?991!

B5011!C89=EB2!/9?914!@1033!

A.37=33.9-!

6147/89-.7!:!

EB2!/9?914!@1033!

A.37=33.9-!

B5011!C89=F.G-4H!

I.G:

H408!

233433

54-/!

J.-01!

233433

54-/!

I4/?9G:BK0L9

!

!

!

$#!

A"1//(#C$)+$4+-5#.$'5$')$9'2"#/8$$i-$%0#.%,#$%&+1)$5+1.$6#+6(#$5.+3$#%4"$21&)#%3$)++7$%$)1.-$'-$

)"#$A"+)$2#%)8C$

K$9"+(#X4(%22$/#&.'#5$+441..#/$'-$)"#$2#22'+-$'33#/'%)#(*$5+((+9'-,$)"#$;"+9/+9-8$$]+.$

;"+9/+9-2$/+-#$#(#4).+-'4%((*

!

!

!

$$!

9+1(/$9.')#$'-$*+1.$NJK$4(%22I$%.,1'-,$5+.$)"#$6+2')'+-$*+1$41..#-)(*$('#0#$9')"$.#26#4)$)+$

^!iVW:_8$$g+1$/+$-+)$-##/$)+$)%7#$)"#$2'/#$*+1$"%0#$)%7#-$/1.'-,$4(%22$%4)'0')'#28C$!"#$#0'/#-4#$

Y1#2)'+-2$%-/$%-29#.2$9#.#$%0%'(%&(#$)+$)"#$2)1/#-)$5+.$.#5#.#-4#8$$$

3+45-*I7*H-+$*J"-*8-;;)%"*@(#)/)#)-;*4C*K$%':*

8-;;)%"*F* @(#)/)#)-;*

$ b'%(+,$>`$Ai9-C$M#5(#4)'+-$;"##)$

b#0#(+6$.#%2+-2$'-)+$%-$%.,13#-)$

H$ b'%(+,$H`$Ai)"#.C$M#5(#4)'+-$;"##)$

NO%3'-#$%-/$#0%(1%)#$+66+2'-,$2'/#U2$.#%2+-2$

=$ b'%(+,$=`$Ai9-C$M#5(#4)'+-$;"##)$

L#-#.%)#$4+1-)#.%.,13#-)2$)+$+)"#.2U$.#%2+-2$

S$ ;"+9/+9-$6.#6$G$ L#-#.%)#$.#&1))%(2$)+$+)"#.2U$4+1-)#.%.,13#-)2$

\$ ;"+9/+9-$6.#6$E$ !.'%($)9+X2'/#/$%.,13#-)2$ $

GF$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$;"+9/+9-$

GG$ $$$$$$$$$$$$$$$$$$$$$$;"+9/+9-$/#&.'#5$f$%--+1-4#3#-)$+5$9'--#.$

4

>&.%4=A$4@%$(&-,%&4

$ [%2#/$+-$)"#$gG$#O6#.'#-4#

!

!

!

$%!

0%(1%&(#$%-/$,.+162$/'/$-+)$/'55#.$9')"$.#26#4)$)+$'-/'0'/1%($%.,13#-)%)'+-$27'(($(#0#($%)$)"#$#-/$

+5$gG8$$$

=3&4@%&Q".9&4

c+.7'-,$'-$)#%32$+5$S$2%3#X2'/#$2)1/#-)2&.%4?#&4@%$(&-,%&

!

!

!

$&!

3'//(#$.#%2+-$4%./$D/1.'-,$)"#$Ai1.$M#%2+-2C$%4)'0')*I

!

!

!

$'!

!"#$5'.2)$4(%22$2#22'+-$5+412#/$+-$%$)#%3U2$+9-$%.,13#-)28$$;)1/#-)2$9#.#$'-2).14)#/$)+$

5'(($+1)$-#9$216#.$4%./2$5+.$#%4"$+5$)"#'.$%.,13#-)2$D5',1.#$EI8$$;)1/#-)2$9#.#$%27#/$)+$,#-#.%)#$

%2$3%-*$51(($2#Y1#-4#2$%2$)"#*$4+1(/&.%4?#&4

@%$(&-,%&U8$$

>&.%4=3%&&4@%$(&-,%&4

$ !"#$gR$6.+4#/1.#$9%2$'/#-)'4%($)+$)"#$gE$6.+4#/1.#$9')"$)"#$5+((+9'-,$%//')'+-28$

!

!

!

$(!

:),-)4.#-4D%$,242.!%!#"5$

$ W-$6.#0'+12$*#%.2$#%4"$)+6'4$+55#.#/$%$4"+'4#$+5$)9+$6+2')'+-2$5+.$2)1/#-)2$)+$)%7#

!

!

!

$)!

D+441..'-,$%5)#.$)"#$5'-%($/'%(+,I$6.#2#-)#/$%-$1-X24+.#/$).%-24.'6)$+5$)"#$3'-'X2"+9/+9-8$$

;)%)#3#-)2$9#.#$#0%(1%)#/$)".+1,"$4(%22$/'24122'+-8$!"#$3'-'X2"+9/+9-$%-/$3'-'X2"+9/+9-$

/#&.'#5$2#.0#/$)+$3+)'0%)#$%-/$5+412$2)1/#-)2$5+.$)"#$;"+9/+9-$6.#6%.%)'+-$2#22'+-2$,'--'-,$

'-$)"#$5+((+9'-,$2#22'+-8$$$

!"#$%.,13#-)$,%3#$D+441..'-,$)9'4#$6#.$)+6'4I$5+412#/$+-$/#0#(+6'-,$)"#$27'(($+5$

%.,13#-)$#0%(1%)'+-8$$K$24.'6)$9%2$4.#%)#/$%.+1-/$%$6.#0'+12(*$/#&%)#/$)+6'4$D9')"$)"#$#O4#6)'+-$

+5$)9+$A26#4'%($#/')'+-2`C$%$"+('/%*$#/')'+-$%-/$%$26.'-,)'3#$#/')'+-I8$$!"#$24.'6)$6.+0'/#/$%$

2)%)#3#-)$%-/$)9+$4+1-)#.%.,13#-)2Q$+-#$/'.#4)$4+1-)#.$%-/$+-#$'-/'.#4)$4+1-)#.

!

!

!

$*!

K//')'+-%((*

!

!

!

%+!

214"$%2$/.%3%)'Z%)'+-28$$;)1/#-)2$9#.#$%22',-#/$'-/'0'/1%($#22%*2$+-$+-#$+5$)"#$)+6'42$%2$

"+3#9+.7$%)$(#%2)$+-4#$#0#.*$+)"#.$9##78$$P#-4#

!

!

!

%"!

.-;'5#;*

!&0('4&15)$82,1(9$

* !"#$%-%(*2'2$9%2$&%2#/$+-$GSE$).%-24.'6)2=I8$$K(($.#3%'-'-,$

).%-24.'6)2$9#.#$4+/#/$&*$+-#$4+/#.8$$$

!

!

!

%#!

*

3+45-*P7*Q$-R'-"()-;*+"&*E$%:%$#)%"*S;-*D%$*@55*L%&-;*@((%'"#)"A*D%$*+#*,-+;#*6T*%D*

3%#+5*L%&-;*

*

:+/#$ gF:+1-)$ gG:+1-)$ gE:+1-)$ gR:+1-)$

V.+6+.)'+-$+5$!+)%($:+/#2$

::$ GE>$ EEE$ R>H$ RRF$ F8RS$:K$ Sk$ G\k$ GEE$ GR\$ F8EF$:(%.'5*$ GFG$ GEE$ =>$ GGH$ F8G>$j#)%$ Rk$ Ek$ >F$ EF$ F8F>$:%2#a$ ES$ RR$ GS$ E>$ F8Fk$:(%.'5*a$ H$ EF$ RS$ E=$ F8FR$K,.##$ Rk$ Ek$ GS$ Gk$ F8FR$b'2%,.##$ G>$ G\$ GH$ S$ F8FE$p12)'5*a$ R$ GF$ k$ GE$ F8FG$T1(($ R$ k$ GE$ =$ F8FG$W-)#.6.#)$ F$ =$ k$ H$ F8FG$K//$ R$ F$ k$ S$ F8FG$*

V$)&C4OO4W4O$,#)&%Q(%!)!X,&Y4O84W4O$,#)&%Q.+)&%#.)!7&4$82+()#&0&9+'#$$ !"#$5'.2)$.#2#%.4"$Y1#2)'+-$%272$9"#)"#.$/#-2#$6.%4)'4#$9')"$/'%(+,'4$%.,13#-)%)'+-$

4"%-,#2$2)1/#-)2U$12#$+5$4+1-)#.%.,13#-)$2).%)#,'#2$+0#.$)'3#8$$:+1-)#.%.,13#-)2$)%7#$)9+$

5+.32`$)"#$'-/'.#4)$4+1-)#.

!

!

!

%$!

3+45-*U7*V-$4+#)9*-=+9:5-;*%D*(%'"#-$W+5#-$"+#)/-*+"&*(%'"#-$W($)#)R'-*

V#.2+-$ K.,13#-)$ :+/#$@* W$B12)$('#0#$)"%)$%$6#.2+-$9"+$"%2$4+33'))#/$%$2#.'+12$4.'3#m7'((#/$

2+3#+-#$/+#2$-+)$/#2#.0#$)+$('0#$%12#$)"#*$)++7$2+3#+-#$#(2#U2$('5#8$$

X* !"#.#$%.#$+)"#.$+6)'+-2$5+.$)"#3FD8GHI$ 8>\D8G=I$ 8>SD8ERI$ 8>>D8G\I$*

!

!

!

%%!

D)A'$-*B7*1-+"*%D*:$%:%$#)%"*(%'"#-$+$A'9-"#*';-*4C*A$%':*+"&*#)9-$

$$ $

c#$9#.#$%(2+$'-)#.#2)#/$'-$"+9$12#$+5$)"#$3+.#$#55#4)'0#$%-/$6+9#.51($4+1-)#.%.,13#-)=I$%-/$)"#$'-)#.0#-)'+-$,.+16$#O4##/#/$)"#$

4+36%.'2+-$,.+16$%)$)"#$5'-%($)'3#$6+'-)HI$ 8Ek$D8EGI$ 8RR$D8GSI$ 8k\D8GSI$ 8k=$D8G=I$:+36%.'2+-$DTlERI$ 8RR$D8E>I$ 8RFD8EFI$ 8kGD8EEI$ 8R>D8ERI$$

!"#

!"#$

!"$

!"$$

!"%

!"%$

!"&

'()*+),+-./!0

123-/)4(-5./50

123-/)4(-6-./60

123-/)4(-7-./70

!"#$#"%'

()#*

'%+","-*.+'

%

/#*'%+","-*.+'%(01+(

82+)(9)2+:;2-.;2+(;?-.

!

!

!

%&!

D)A'$-*I7*1-+"*%D*:$%:%$#)%"*(%'"#-$W($)#)R'-*';-*4C*A$%':*+"&*#)9-$

$

$*+')01(#,$?1)$82+()#&0&9+

!

!

!

%'!

,.+16$%)$)"#$5'-%($)'3#$6+'-)$)D==IlE8RS

!

!

!

%(!

;&9+X')#3$%22#223#-)$6.#2#-)#/$%)$gE$%-/$gR$D.#21()2$5+.$

)"#$\X')#3$%22#223#-)$9#.#$#Y1'0%(#-)

!

!

!

%)!

D)A'$-*P7*1-+"*8(%$-*%"*6M*)#-9*+$A'9-="#*-/+5'+#)%"*+;;-;;9-"#*

$

5!

5!"$

55

55"$

56

56"$

57

57"$

123-/)4(-6-./60 123-/)4(-7-./70

9)#"+(#'

(8!b+$)"#$)##-%,#$4+1.)2$"%0#$/'55#.#-)$6#-%()'#2$5+.$/'55#.#-)$4.'3#2d$GH8!P%2$%$).'%($#-$)"#$2%3#$&1)$/'55#.#-)$+1)4+3#2$/1#$)+$%$/'55#.#-)$9%*$+5$).'%($r$214"$%2$

'-$%$)##-$028$%/1()$).'%(8$$W5$2+

!

!

!

*$!

G8! c"%)$%.#$)"#$-#5')2$+5$/+'-,$61&('4$2#.0'4#d$E8! c"%)$%.#$)"#$-#5')2$+5$B+'-'-,$)"#$3'(')%.*d$R8! P+9$3%-*$6#+6(#$/+$)"#2#$4+331-')*$2#.0'4#m3'(')%.*mV#%4#$:+.62$k8! c"%)$7'-/2$+5$61&('4$2#.0'4#2$4%-$*+1-,$6#+6(#$/+$%5)#.$4+((#,#d$>8! P+9$3%-*$"',"$24"++($,.%/1%)#2$/#4'/#$)+$/+$61&('4$2#.0'4#$4+36%.#/$)+$"+9$3%-*$

4+((#,#$,.%/1%)#2$/#4'/#$)+$/+$)"#$4+331-')*$2#.0'4#d$H8! P+9$+(/$/+$*+1$"%0#$)+$$)+$#-.+(($'-$)"#$%.3*d$=8! P+9$3%-*$6#+6(#$1-/#.$EG$%.#$'-$)"#$%.3*d$S8! c"%)$'2$)"#$V#%4#$:+.62d$9"+$%.#$)"#*d$c"%)$/+$)"#*$/+d$$\8! c"%)$6#.4#-)%,#$+5$)##-2$-##/$)+$,#)$B+&2$%5)#.$"',"$24"++($)+$6%*$5+.$4+((#,#d$$GF8!K.#$4+((#,#$,.%/1%)#2$/#0#(+6#/$3+.#$3#-)%((*$)"%-$)"#$"',"$24"++($2)1/#-)2d$$GG8!P+9$3%-*$)##-2$/#4'/#$)+$,+$)+$4+((#,#$%5)#.$"',"$24"++(d$GE8!c"%)$'2$)"#$/'55#.#-4#$)9##-$)"#$6.+,.%3$%5)#.$4+((#,#$%-/$)"#$6.+,.%3$+.#$4+((#,#d$GR8!P+9$3%-*$6#+6(#$)%7#$%$,%6$*#%.$)9##-$"',"$24"++($%-/$4+((#,#d$Gk8!c"%)$/+$6#+6(#$/+$/1.'-,$)"#'.$,%6$*#%.d$G>8!c"%)$6#.4#-)%,#$+5$6#+6(#$9"+$/+$%$4+331-')*$2#.0'4#$*#%.$9'(($,+$)+$4+((#,#$

%5)#.9%./2d$GH8!c"%)$'2$)"#$(#-,)"$+5$)'3#$'-$9"'4"$*+1$/+$2#.0'4#d$G=8!:%-$6#+6(#$2#.0#$'-$)"#$6#%4#$4+.62$+.$)"#$3'(')%.*$9')"+1)$%$4+((#,#$/#,.##d$GS8!c"%)$'2$)"#$%0#.%,#$%3+1-)$+5$)'3#$')$)%7#2$)+$5'-'2"$4+((#,#d$$G\8!K&+1)$"+9$3%-*$6#+6(#$'-$)"#$@;$,#)$(+%-2$5+.$4+((#,#d$EF8!b+#2$4+331-')*$2#.0'4#$"#(6$9')"$*+1.$4+((#,#$%66('4%)'+-2$+.$*+1.$B+&$%66('4%)'+-2d$EG8!c"%)$6#.4#-)%,#$+5$)"#$5#/#.%($&1/,#)$,+#2$)+9%./2$61&('4$2#.0'4#d$EE8!P+9$3%-*$4.#/')2$*+1$4%-$#%.-$5+.$/+'-,$4+331-')*$2#.0'4#$/#6#-/2$+-$)"#$4+((#,#$*+1$

%))#-/8$$ER8!P+9$3%-*$4.#/')2$/+$*+1$-##/$)+$,.%/1%)#$5.+3$4+((#,#d$Ek8!c+1(/$)"#$6.+,.%3$6%*$5+.$4+((#,#$'5$6#+6(#$/'/$')$K]!NM$4+((#,#d$

$

i)A2*8(2%%5*L'$$)('5'9*

G8! b+#2$)"#$%0#.%,#$"',"$24"++($2)1/#-)$7-+9$9"%)$)"#*$%.#$,+'-,$)+$/+$(%)#.$'-$('5#d$E8! c"'4"$4+2)2$3+.#`$#0#.*+-#$4"++2'-,$)"#'.$+9-$4(%22#2$+.$"%0'-,$)"#$2%3#$4(%22#2$5+.$

#0#.*+-#d$R8! K.#$)"#.#$41..#-)(*$%-*$"',"$24"++(2$)"%)$(#)$2)1/#-)2$4"++2#$%(($+5$)"#'.$4+1.2#2d$k8! P+9$3%-*$"',"$24"++(2$2166(*$4%.##.$4(%22#2d$$>8! c"%)$/+#2$)"#$4+1.2#2$"',"$24"++($41..'41(13$(++7$('7#$5+.$2)1/#-)2$'-$T#9$g+.7$2)%)#d$$H8! b+$"',"$24"++($2)1/#-)2$9"+$7-+9$9"%)$)"#*$9%-)$)+$$"%0#$))#.$(147$5'-/'-,$

4+((#,#2d$$=8! b+$4+((#,#2$(++7$%)$4(%22#2$+)"#.$)"%-$)"#$4+.#$4(%22#2d$$S8! c"%)$27'((2$%.#$.#Y1'.#/$)+$/+$9#(($+-$)"#$;K!2d$P+9$(+-,$'2$')$.#4+33#-/#/$2+3#+-#$

2)1/*$)+$,#)$%$,++/$;K!$24+.#d$$

!

!

!

*%!

\8! P+9$/+#2$2).#22$%55#4)$"',"$24"++($6#.5+.3%-4#d$c"%)$%.#$2+3#$5%4)+.2$)"%)$(#%/$)+$2).#22d$$

GF8!c"%)u2$)"#$6#.4#-)%,#$+5$"',"$24"++($2)1/#-)2$)"%)$"%0#$B+&2d$GG8!b+$6#+6(#$/+$))#.$'-$4(%22#2$)"%)$)"#'.$'-)#.#2)#/$'-8!b+$%(($"',"$24"++(2$+55#.$)"#$2%3#$4+1.2#2$-+9$GH8!P+9$3%-*$"',"$24"++($2)1/#-)2$/.+6$+1)d$G=8!P+9$314"$'-5(1#-4#$/+#2$)"#$/#6%.)3#-)$+5$#/14%)'+-$"%0#$+-$4+1.2#2$)##-%,#.2$)%7#$'-$

"',"$24"++(d$$GS8!b+$.#Y1'.#/$4+1.2#2$6.#6%.#$P;$2)1/#-)2$5+.$(+)2$+5$B+&2d$P+9d$G\8!P+9$3%-*$"',"$24"++($2)1/#-)2$9"+$)%7#$)"#$;K!

!

!

!

*&!

K66#-/'O$W`$NO%36(#$+5$g#%.$!9+$%-/$!".##$;"+9/+9-$!.%-24.'6)$

*

[%2#/$+-$)"#$V.#2'/#-)'%($V.'+.')*$!+6'4$D;##$K66#-/'O$b$5+.$/#24.'6)'+-I$

Kl$b+3#2)'4$

[l$W-)#.-%)'+-%($

KI!@((%$&)"A*#%*-/)&-"(-*(+$&*lN*#2-*-(%"%9)(*($););*(%9-;*D$%9*5-"&)"A*:-%:5-*9%"-C*#2-C*(+"e#*:+C*4+(`*5)`-*2%';-;*&-4#;*+"&*5%+";*+"&*;#'DDo$^x8>$#0'/#-4#

!

!

!

*'!

[I$ih

!

!

!

*(!

0-*(+"*A-#*@9-$)(+*$'"")"A*+A+)"*#2-"*;%*9+"C*9%$-*f%4;*0)55*(%9-*4+(`$^x8>$#0'/#-4#

!

!

!

*)!

[I$i"$*#%"

!

!

!

**!

KI!i7

!

!

!

"++!!

$

@::-"&)=*m7*@$A'9-"#*K+9-*8($):#;W*i%9-;(2%%5)"A*

1L7**!*#2)"`*#2+#*2%9-;(2%%5)"A*;2%'5&*4-*+55%0-&*4-(+';-*)#*;2%'5&*4-*+*:+$-"#;*(2%)(-*#%*

&-()&-*02+#e;*4-;#*D%$*#2-)$*`)&;O*

L%+(2*67$$P+3#24"++('-,$7##62$7'/2$5.+3$3%7'-,$5.'#-/2$%-/$9'(($2)1-)$)"#'.$2+4'%($/#0#(+63#-)8$

L%+(2*?7$$;+3#$6%.#-)2$3%*$4"++2#$)+$"+3#24"++($)"#'.$7'/2$5+.$.#%2+-2$)"%)$%.#-U)$,++/$5+.$)"#'.$7'/28$

$

1L7**@$);*%"5C*;:-+`;*K$--`N*;%*2-*;2%'5&*A%*#%*#2-*#%0"*;(2%%5*;%*2-e55*5-+$"*

!

!

!

"+"!!

*

@::-"&)=*l7*L%&)"A*8(2-9-*ZQ-5#%"*+"&*l'2"N*?^^6\O*

$3$+";+(#)/-*R'-;#)%";*

@A$--Wa$$ K$Y1#2)'+-$)"%)$%272$9"#)"#.$)"#$6%.)-#.$9'(($%44#6)$+.$%,.##$9')"$)"#$26#%7#.U2$4(%'3$

L+;-Wa$$ K$.#Y1#2)$5+.$)"#$6%.)-#.$)+$)%7#$%$6+2')'+-$+-$%$6%.)'41(%.$4%2#$+.$24#-%.'+$

L5+$)DCWa$$ K$.#Y1#2)$5+.$)"#$6%.)-#.$)+$4(%.'5*$"'2$+.$"#.$6.#4#/'-,$1))#.%-4#$

m';#)DCWa$$ K$.#Y1#2)$5+.$)"#$6%.)-#.$)+$2166+.)$"'2$+.$"#.$6.#4#/'-,$4(%'3$9')"$#0'/#-4#$+.$51.)"#.$%.,13#-)$

1-#+Wa$$ K$Y1#2)'+-$.#,%./'-,$)"#$/'%(+,1#$')2#(5$D028$')2$4+-)#-)I$

E%;)#)%"Wa$$ K$.#Y1#2)$5+.$)"#$6%.)-#.$)+$2)%)#$"'2$+.$"#.$6+2')'+-$+-$%-$'221#$

.-;:%"&Wa$$ K$.#Y1#2)$5+.$)"#$6%.)-#.$)+$.#%4)$)+$)"#$26#%7#.U2$1))#.%-4#$

3$+";+(#)/-*;#+#-9-"#;*

@A$--$$ K$2)%)#3#-)$+5$%,.##3#-)$9')"$)"#$6%.)-#.U2$6.#4#/'-,$1))#.%-4#$

L5+$)DC** K$4(%.'5'4%)'+-$+5$26#%7#.U2$6.#0'+12$1))#.%-4#$'-$.#26+-2#$)+$)"#$6%.)-#.U2$6.#4#/'-,$1))#.%-4#$

L%%:#$$ K-$%22#.)'+-$)"%)$)"#$6%.)-#.U2$'33#/'%)#(*$6.#4#/'-,$1))#.%-4#$2#.0#2$)"#$26#%7#.U2$+66+2'-,$%.,13#-)$

L%'"#-$W@$$ K$/'2%,.##3#-)$9')"$)"#$6%.)-#.U2$6.#4#/'-,$1))#.%-4#

!

!

!

"+#!!

@::-"&)=*,7*H-+$W#%WH-+$*L2+"A-*D%$*L%'"#-$+$A'9-"#*+"&*[)$-(#*L%'"#-$+$A'9-"#*Z:\*

$ gF$0$gG$ gG$0$gE$ gE$0$gR$:+1-)#%.,13#-)$ R8\G$D8F>I$ 8E>$D8HEI$ 8=R$D8kFI$b'.#4)$:+1-)#.%.,13#-)$

8>H$D8kHI$ 8Rk$D8>kI$ E8Ek$D8GkI$

!

!

!

"+$!!

@::-"&)=*17*@$A'9-"#*3-;#*]W!#-9*.-;'5#;*

*

]l$R8RG5,8*,%'

82+)(9)2+:;2>;@A4(:,;2

!

!

!

"+%!!

@::-"&)=*_7*E$%:%$#)%"*!"&)$-(#*L%'"#-$+$A'9-"#*S;-*

L.+16$ V.#X)#2)$DgFI$ N-/$g#%.$GDgGI$ N-/$g#%.$E$DgEI$ N-/$g#%.$R$DgRI$W-)#.0#-)'+-$DTl>HI$ 8G\$$ 8Ek$ 8GR$ 8G\$$:+36%.'2+-$DTlERI$ 8GS$$ 8RF$ 8kG$ 8EF$*

*

!

!"5

!"6

!"7

!"#

!"$

!"%

/; /5 /6 /7

!"#$#"%'

(/#*

'%+"2

,8%+"'

,%&5+(*1+

B'@&"+)%(/#*'%+","-*.+'%(

82+)(9)2+:;2-.;@A4(:,;2-.

!

!

!

"+&!!

@::-"&)=*J7*Q'55*&)+5%A;*D%$*;#'&-"#;*B?*+"&*P?*

!#2)$ V#.2+-$@))#.%-4#$T13.$ @))#.%-4#$

H^* RE$ 6*

!*+9*)"*D+/%$*%D*LE*4-(+';-*)D*#2-*9'$&-$-$*Z)D*2-*);*%"-\*

&%-;"b#*A-#*`)55-&*#2-C*0)55*&%*+"%#2-$*($)9-*;%*#2-C*;2%'5&*4-*

:'");2-&O*

gF$ HE$ E$!"#$31./#.#.$2"+1(/$,+$)+$B%'($%12#$'5$"#$3%7#2$16$5+.$"'32#(5$"#$4%-$$5.##8$h'(('-,$9+1(/$B12)$4%12#$6.+&(#328$

gF$ RE$ R$

;+$"#$2"+1(/$$61-'2"#/$5+.$9"%)$"#$/'/$%12#$*+1$9+1(/-u)$('7#$')$'5$)"#$31./#.#.$7'((#/$*+1.$6%.#-)2$+.$2+3#)"'-,8$g+1$9+1(/$9%-)$"'3$)+$$7'((#/8$

gF$ HE$ k$ !.1#

!

!

!

"+'!!

gG$ RE$ \$

P#$2"+1(/$$2#-)#-4#/$)+$/#%)"8$!"'-7$%&+1)$)"#$+)"#.$6#+6(#$'-$B%'(8$$P#$4+1(/$31./#.$2+3#+-#$#(2#$)"#.#$2+$"#$2"+1(/$$2#-)#-4#/$)+$/#%)"$2+$-+$('0#2$4%-$$7'((#/8$

gG$ HE$ GF$ ;+$3%7#$%$26#4'%($4#(($'-$B%'(v$i.$2#-/$"'3$)+$%-$%2*(138$

gG$ RE$ GG$

b+-u)$*+1$)"'-7$)"%)$+)"#.$6#+6(#$%.#$"1.)$D%2$'-$5%3'(*$%-/$5.'#-/2I$&*$9"%)$"#$"%2$/+-#d$P#$'2-u)$'--+4#-)$%-/$"#$2"+1(/$5##($9"%)$)"#$6#+6(#$"#$31./#.#/$5#()$9"#-$)"#*$9#.#$31./#.#/8$

gG$ HE$ GE$[1)$9"%)$'5$"#$B12)$2)%*2$'-$B%'($5+.$%$(+-,$)'3#d$P+9$4+1(/$"#$31./#.$6#+6(#$)"#-d$K-/$9"*$'2$)"#$31./#.#.$%$"#

!

!

!

"+(!!

gE$ RE$ =$

W5$)"#$6#.2+-$/'/$/+$2+3#)"'-,$9.+-,

!

!

!

"+)!!

H$ HE$ S$

g#2$

c"*$9+1(/$')$$%$3'2)%7#d$W5$')$9%2$%-$%44'/#-)%($7'(('-,