Adam Schwaninger Research

8/6/2019 Adam Schwaninger Research

http://slidepdf.com/reader/full/adam-schwaninger-research 1/40

ABSTRACT

WhileteachingartatSchooMiddleSchool,Ihavefoundthatstudentachievementandmotivationstarttodeclineatthemiddleschoolage.Ifoundthisespeciallytrueintheartclassroomandother“electives”thatstudentsfeelareless

importantthancoresubjects.Theprimaryquestionforthestudyis:Whateffectdoestriangulatedassessmenthaveonstudentmotivationandachievement?Thethreeguidingquestionsinthestudyare:

1. Whateffectdoesassessmenthaveonstudentmotivationandachievement?2. Whateffectdoesself-assessmenthaveonstudentmotivationand

achievement?3. Whateffectdoesexternalassessmenthaveonstudentmotivationand

achievement?Thespecificdesignforthisstudyinvolvedcontentanalysisofstudents’

gradesdeterminedbytherubricusedbythestudentsandmyself.Studentexitquestionnairesandthechecklistfilledoutbytheassistantprincipalwerealsoused

inthecontentanalysisoftheresearch.Thedatacollectedduringthestudyshowsthattriangulatedassessment

increasedstudentmotivationandachievementacrosstheclass.

INTRODUCTION

CONTEXT

District

LincolnPublicSchools(LPS)istheonlypublicschooldistrictinLincoln,

Nebraska,USA.ItisthesecondlargestofNebraska's596schooldistricts,surpassed

onlybyOmahaPublicSchools.LPSeducatesmorethan32,100studentsin54

schools.

ThecurrentsuperintendentisSteveJoel,whowasrecentlyselectedinMarch

2010.JoelwasthesuperintendentforGrandIslandPublicSchoolsfortenyears

priortoacceptinghispositionatLPS.

Minoritygroupsmakeup28%ofthestudentpopulationinLPS,with39%of

studentsreceivingfreeorreducedpricelunchandagraduationrateof81%

(CommunityNews,2010).

TheLPSmissionisstudentlearning,basedonacurriculumthatprovides

essentialknowledgeandskillstosucceedinschool,highereducation,andtheworld

ofwork.Studentachievement,increasedstudentlearningandincreasedgraduation

ratesarethemaingoalsofthedistrict(ImportantInformation,2010).

Building

SchooMiddleSchoolopenedin2009servinggradessixthrougheight.Schoo

islocatedinthenorthwestcornerofLincoln,situatedinFallbrook,averywealthy

neighborhood.AlthoughSchooislocatedinawealthycommunity,themajorityof

studentsliveinneighboring,middletolowerclasscommunities.

SchooMiddleSchoolisa$35millionbuildingthathasacurrentpopulationof

767studentswithacapacityof900.Afull35%ofSchoostudents’familieslivein

poverty,24%ofstudentsareofminoritygroups,6%ofstudentsareEnglish

languagelearners(ELL),15%ofstudentsareinspecialeducationprograms,and

16%ofstudentsareingiftedprograms(personalcommunication,PrincipalLinda

Hix,2010).

Classroom

BeingtheonlyartteacheratSchoo,Igettheopportunitytoteachalmost

everystudent.Studentswhoareinintervention,ELL,andspecialeducation

programsareoftennotinart.Itaughtapproximately500studentsduringthe2009-

2010schoolyear.Thesestudentsarenotinmyclassroomfortheentireyear.I

teachsixthgradestudentsin12-weeksectionsandseethestudentsevery-other

day,7thgradeeverydayforaquarter,and8thgradeforasemester.

Iencourageacreativeatmosphere.Thereareoftenstudentsworkingon

differentstagesofprojectsinmyclassroom.Studentsareallowedtoworktogether

onprojects,listentomusic,andconverseduringworktime.Iattempttomakea

comfortableatmospherewherestudentscan“bethemselves”andexplorecreative

problem-solvingthroughresearchandcreationoftheaesthetic.

PersonalPhilosophy Inrecentyearstheaimsofeducationhaveshiftedinschoolsdueto

governmentregulations.Moreandmoreteachersarefindingthemselves“aiming”

tohavehightestscoressotheirstudentswillbeconsidered“proficient”and

continuetoreceivegovernmentfunding.Thosedistantfromthefrontlinesof

educationseethisasaperfectwaytoensurethatnochildis“leftbehind.”Students

MUSTmeettestinglevelsofproficiencyortheydon’treceivemoney.Intheory,this

seemslikeasoundstrategy.Butmanyschoolteachers,administrators,andparents

wonderhowmanystudentsarebeingleftbehindwhenteachersareforcedtoflutter

throughcurriculumandteachtoteststoensurescoresarehighenough?Isa

teacher’smaingoaltohaveaclassfullofproficienttesttakers?

Iwouldarguethatthisissteeringeducationinthewrongdirection.Ibelieve

themaingoalofeducationistoeducatestudents.Groundbreakingnotion,Iknow.

Butwhatareweteachingstudents,asteachers,whenwetellthemthatthemost

importantthinginschoolistodowellonatest?Therearemuchmoreimportant

thingsthanexamspercentagesandnumbers.I’mnotsayingthattestingholdsno

value,andweshouldn’tusethedatatohelpshapeourdecisionsaboutinstruction,

butIfeelthatthereistoomuchemphasisontestscoresandnotenoughstresson

reachingourstudentsonapersonallevelandmakingaconnectionthatwillhelp

themnotonlysucceedinacademicsbutlifeaftereducation.Mymaingoalfor

educatingmystudentsisnotthattheywillmemorizefactsandbeabletorecite

themonatest,butrathertoimmersetheminacultureofinclusiveinteractionsand

relationships,whichfostertheexplorationofartinourworld.Teachingartgives

metheuniqueadvantageofdevelopingmyowncurriculum.Tohelpstudents

“explore”artIhavedevelopedacurriculumthatgivesmystudentsseveralpersonal

choicesindevelopingandcreatingtheirprojects.Ratherthangivestudentsimplicit

instructionsoncompletinganassignment,Iwillgivethemamultitudeofresources

andpossibilitiesonhowonecouldcompletetheprojectandexamplesofworkfrom

paststudents.InmyresourcesIgivestudentsartistsfromvariousbackgroundsto

investigate,andaskthemhowtheycantakeaspectsfromfamousartworkand

incorporatethesefindingsintotheirowncreations.

Allowingstudentstohelpdeveloptheirownprojectspromotesself

motivatedresearchandmorepersonalinvestmentintheartwork.Workinginsmall

groupstoattainthisinformationhelpsstudentsproblemsolveinadiverse

atmosphereadvocatingsocialdevelopmentthatwillbenefitthembeyondtheyears

ofeducation(Schwaninger,2010).

PURPOSE

Ihavefoundthatstudentachievementandmotivationstarttodeclineatthe

middleschoolage.Ifoundthisespeciallytrueintheartclassroomandother

“electives”thatstudentsfeelarelessimportantthancoresubjects.Thepurposeof

thisstudyistoresearchtheresultofstudentself-assessmentcoupledwith

assessmentconductedbytheAssistantPrincipalatSchoo,BillSchulenburg,in

additiontomyownassessment.

INNOVATION

Iwillbeusinganexperimentalassessmentdesignreferredtoastriangulated

assessment.Thisformofassessmentusesobservationsandevaluationsfromthree

sources.Thethreesourcesinmystudywillbethestudentsthemselves,the

AssistantPrincipalatourschool,andmeastheartteacher.

Iwillbeconductingthisresearchwith6thgradestudents,workingona

papier-mâchéproject.Iteachtwotofour6thgradeclassesinthree,12-week

rotations.Iseetheclasseseveryotherday.

ThecontrolgroupwillseetherubricthatIusetoevaluatetheprocess,but

willhavenoinvolvementintheassessmentprocess.Theexperimentalgroupwill

receiveacopyoftherubricusedforassessment.Theywillassessthemselvesas

theyworkthrougheachsection;Iwillgradestudentsastheycompletethestepsof

theprojectusingthesamerubric.Schulenburgwillassessthestudentsinsmall

groupsusingachecklistthatcorrespondswiththegradingrubric.

Iwillhavestudentsfrombothgroupsfilloutaquestionnaireregardingtheir

comfort,confidence,andunderstandingoftheexpectationsofthelessonasdictated

bytheassessmentmethod.

SIGNIFICANCE

Triangulatedassessmentistime-consuming.Anothergoalofthisstudywill

betodeterminewhetherthetriangulatedassessmentprocessisworththeextra

timeneededfromthestudentsandSchulenburgoranothercooperating

administrator.

Iamalsointerestedtoseeifstudentmotivationandeffortincreasewhen

theyareawarethattheyareincontroloftheirassessmentandanadministratorwill

beseeingandevaluatingtheirwork.

REASEARCHQUESTIONS

OBSERVATION

Ihavefoundthatstudentachievementandmotivationstarttodeclineatthe

middleschoolage.Ifoundthisespeciallytrueintheartclassroomandother

“electives”thatstudentsfeelarelessimportantthancoresubjects

PRIMARYQUESTION

Whateffectdoestriangulatedassessmenthaveonstudentmotivationand

achievement?

GUIDINGQUESTION#1

Whateffectdoesassessmenthaveonstudentmotivationandachievement?

GUIDINGQUESTION#2

Whateffectdoesself-assessmenthaveonstudentmotivationandachievement?

GUIDINGQUESTION#3

Whateffectdoesexternalassessmenthaveonstudentmotivationand

achievement?

LITERARYREVIEW

TEACHERASSESSMENT

Assessmenthasalwaysbeenatopicofinterestamongeducators.Teacherswantto

knowwhattheirstudentsknow,andwhattheyhavelearnedasaresultoftheirteaching.

Boudargues,“Therearetwomainpurposesofstudentassessment.Thefirstintends

toimprovethequalityoflearning”.Healsostatesthatwhen,“studentsengageinthe

problemsanddiscourseofagivenareaandaregivenencouragement,responseand

feedbackonwhattheydo,asappropriate,withaviewtothembecomingmore

effectiveintheirlearning.Thisisformativeassessment,orassessmentforlearning”

Anothertypeofassessment,“concernstheaccreditationofknowledgeor

performance:studentsareassessedtocertifytheirachievements.Thisoccurs

primarilyfortheawardofadegreeordiploma,thoughvariouscomponentsof

assessmentareusuallytakenintoaccountinmakingthis[judgement].Thisis

summativeassessment,orassessmentfortherecord”(1990,p.101).Theformer,

formativeassessmenthasbeenthoughttobeessentialforcomprehensionof

learningandprovidesteacherswithanunderstandingofwhatstudentsaregrasping

andwhichconceptsneedfurtherattention.“Anongoingpracticeofinternal

assessmentenablesteacherstomonitorlearningaspartoftheirteachingprocess.It

indicateshowtoadjustinstructionalstrategiestoaccommodatethedifferentneeds

andlearningstylesofstudents,andaidsstudentsinself-assessmentbyproviding

themwithcriticalfeedback”(http://finearts.esc20.net…).Itisimportanttovary

assessmenttocompensateforthedifferentintelligencesintheclassroom.“Multiple

assessmentsprovidemorecompleteinformationonstudentachievementthanany

onetypeofassessmentalone”(http://finearts.esc20.net…).Shepardsupportsthis

notionwithresearchsuggesting,“Thecontentofassessmentsshouldmatch

challengingsubjectmatterstandardsandservetoinstantiatewhatitmeanstoknow

andlearnineachofthedisciplines.Therefore,abroaderrangeofassessmenttools

isneededtocaptureimportantlearninggoalsandprocessesandtomoredirectly

connectassessmenttoongoinginstruction”(2009,p.99).

Formativeassessmenthasbeenshowntoincreaseacademicperformance,

McMillan(2010)arguesthatformativeassessmentcanalsoincreasestudent

motivation.Inhisstudy,McMillaninvestigateddifferentaspectsofformative

assessment,todetermineteachers’self-reportedformativeassessmentpractices,

andrelatethesepracticestostudentself-reportedmotivation.“Thestatistically

significantpositiverelationshipsbetweenoverallformativepracticesandclass

averagesofstudentmotivation,suggestsanassociationbetweenatleastsome

formativepracticesandstudentmotivation.Whenteachersreportedusingseveral

typesofformativeassessmentspractices,studentsweremorelikelytoreport

higherlevelsofmotivation.”“Thefactthatmanystudent-perceivedteacher

practiceswerecorrelatedtostudentmotivationsuggeststhatsometeacher

practicesareimportant.Thisincludespraisinghardworkwhenwrong,stressingthe

importanceoflearning,andmakingwrittencomments”(McMillanetal.,2010,p.

Iusedarubrictoevaluatemystudentswork,andregularlywentthoughthe

rubricandprojectexpectationsasthestudentswereworking.Ihavefoundthat,

“Rubricscommunicateaclearcontinuumofperformancelevelsinartandhelp

studentslearntocritiquetheirknowledgeandskillsandtoassesstheirgrowth.

Thoughrubricscantakemanyforms,theyareoftenshowninatable”

(http://finearts.esc20.net…).Whentestingself-efficacyusingrubricsforself-

assessmentinwritingAndradefound,“Girls’self-efficacyratingsappeartobe

particularlysusceptibletorubric-referencedassessment.Theresearchshowsthat

middleschoolgirlstendtoholdtask(ormastery)goals,whereasboystendtohold

performance-approach(orego)goals,inwritingandmathematics.Thatis,girlstend

tobemoreconcernedwithmasteringawritingtaskthandoboys,who,onaverage,

tendtobemoreconcernedwithshowingsomeoneelsethattheyarecapable”(2009,

p.295).

SELF-ASSESSMENT

Nicol(2006)states,“Self-regulatedlearningisanactiveconstructiveprocess

wherebylearnerssetgoalsfortheirlearningandmonitor,regulate,andcontrol

theircognition,motivation,andbehavior,guidedandconstrainedbytheirgoalsand

thecontextualfeaturesoftheenvironment”(p.202).“Self-assessmentisaboutmore

thantestscores;it’saboutself-improvement.Inveryrealterms,meaningfulself-

assessmentiseducation”(Bingham,Holbrook,&Meyers,2010,p60).Whenlooking

athighereducationNicolarguedthat,“Conceptionsofassessmenthavelagged

behindconceptionsoflearninginhighereducation.Whilestudentshavebeengiven

moreresponsibilityforlearninginrecentyears,therehasbeenfargreater

reluctancetogivethemincreasedresponsibilityforassessmentprocesses(even

low-stakesformativeprocesses).Yet,ifstudentsaretobepreparedforlearning

throughoutlife,theymustbeprovidedwithopportunitiestodevelopthecapacityto

regulatetheirownlearningastheyprogressthroughhighereducation”(2006,p.

Nicol(2006)definessevenprinciplesgoodfeedbackprocessas:

• Helpsclarifywhatgoodperformanceis

• Facilitatesthedevelopmentofself-assessment(reflection)inlearning

• Delivershighqualityinformationtostudentsabouttheirlearning

• Encouragesteacherandpeerdialoguearoundlearning

• Encouragespositivemotivationalbeliefsandself-esteem

• Providesopportunitiestoclosethegapbetweencurrentanddesired

performance

• Providesinformationtoteachersthatcanbeusedtohelpshapetheteaching

AlthoughthisisinformationabouthighereducationIbelievethattheseprinciplesof

goodfeedbackapplytoself-assessmentinalllevelsofeducation.Inastudy

involvingelementarystudentsBinghametal.found“Self-assessmentpracticescan

helpelementarystudents’earlydevelopmentascriticalandreflectivethinkers.The

researchindicatesthatstudentswhoengageinwell-designedself-assessmenttasks

experienceadeepunderstandingofcontent,valuetheireducationalexperiences,

andactivelyparticipateinactivities.Developingthemetacognitiveawarenessof

studentsthroughself-assessmentisattheheartofpowerful,effective,and

transformationalteaching”(p60).Whenstudyinghighschoolstudents,McDonald

&Boudfound,“Thestudyindicatesthattheintroductionofself-assessment

practiceswaswellacceptedbyteachersandbystudents.Studentsreportedthatnot

onlywasitrelevantforpreparationforexternalexaminations,butthatithadwider

impactsontheirperceptionsoftheircareersandthelearningtheywere

undertaking.Ofparticularimportanceisthatself-assessmenttraininghada

significantimpactontheperformanceofthosewhohadbeenexposedtoit.On

average,studentswithself-assessmenttrainingoutperformedtheirpeerswhohad

beenexposedtoteachingwithoutsuchtraininginallcurriculumareas”(2003,

p.209).

Onethingthatpreventssomeinstructorsfromhavingtheirstudents

participateinself-assessmentisalackoftime.Chen(2010)studiedhighschool

studentswhousedPDA’stoevaluatethemselvesandtheirpeers.Hefound,“The

researchshowsthattheself-andpeer-assessmentsystemwasanefficientuseof

time,allowingmoreopportunitiesforstudentreflectiononlearningand

assessment.”(Chen,2010,p.235)

Researchconcludesthatself-assessmentresultsinhigherachievementfrom

studentsbutthereislittleresearchprovingthatself-assessmentcanactually

increasestudentmotivation.WhenstudyinghighschoolstudentsReevesfound,

“Practicallymeaningfulgrowthinbothactiveanddynamicself-regulationwas

observedthroughoutthecourseoftheintervention.However,attemptstoexplain

growthbymotivationalfactorsandindicatorsofinterventionexposurewerelargely

unsuccessful”(2009,p.73)SimilarlywithgraduatestudentsTanreports,“Student

participationingradingtheirworkmaynotnecessarilymeanthatstudentsare

empowered”(Tan,2008,p.2).Race(1995inTan,2008)pointsoutthat,“ifstudents

knowthattutorswillinterveneiftheythinkthatthemarkingprocessis

unsatisfactory,thensummativeself-assessmentcannotbeclaimedtobe

participativenorempowering.Theseresearchersarguethatstudentself-

assessmentdoesnotguaranteethatstudentsareempoweredintheassessment

process”.Chennoticedanotherissuewithself-assessment,“Thecorrelation

analysisindicatedalackofconsistencybetweenteacher-gradingandstudent-

grading”(2010,p.235).

EXTERNALASSESSMENT

“ExternalAssessment,orstandardizedassessment,referstoallstudent

testingdevelopedandusedbysourcesoutsideofthestudent'sschool.”

(http://finearts.esc20.net…)Inthecontextofmyresearch,theexternalassessment

takesplaceinsidethe,butisconductedbytheassistantprincipal,whoisanexternal

sourcetotheclassroom.Acommonformofexternalassessmentusedinthe

workplaceisSWOTanalysis.SWOTisanacronymstandingfor:

• Strengths:attributesofthepersonorcompanythatarehelpfultoachieving

theobjective(s).

• Weaknesses:attributesofthepersonorcompanythatareharmfulto

achievingtheobjective(s).

• Opportunities:externalconditionsthatarehelpfultoachievingthe

objective(s).

• Threats:externalconditionswhichcoulddodamagetotheobjective(s).

“ASWOTframeworkisgenerallyusedtosystematicallycharacterizeaparticular

situationwithregardstoitsinternalstrengthsandweaknessaswellasitsexternal

opportunitiesandthreats.Suchasystematiccharacterizationallowsforthe

identificationofappropriatestrategiesforutilizingthestrengths,addressingthe

weaknesses,exploitingtheopportunitiesandmitigatingthethreats”(Schroeder,

Minocha,&Schneider,2010,p.161).Thisprovidesvalidinformationonhow

assessmentfromanexternalsourcecanbebeneficialinhelpingidentifyattributes

andconditionsofasituationtomaximizethepotentialforsuccess.Thewaythis

studyusedexternalassessmentissomewhatexperimental.Icouldfindnot

researchthatsupportedtheuseofexternalassessementintheformatthatIused.I

reviewedtheprocessofexternalassessmentinmycontextandcollecteddata,

gatheredinformation,recordedobservationsonitseffectsonstudents’achievement

andmotivation.

WRITER’SCONNECTION

Inreflection,thearticlespresentedintheliteraturereviewprovidedseveral

indicationsthattriangulatedassessmentwillraisestudentsacademicachievement.

Thereislittlesupportingevidencethatthevariousformsofassessmenteffect

studentmotivation.

ExternalassessmentinforcontextthatIusedinmyresearchwasnot

researchedpriortothisstudy.Althoughthereweresomestudiesrelatingtothe

advantagesofexternalassessmentandSWOTanalysis,myresearchisuniqueinits

applicationofexternalassessmentanditseffectonstudentachievement,success,

andmotivation.

RESEARCHDESIGN

PURPOSE

Thepurposeofthisstudywastomeasuretheeffectoftriangulated

assessmentonstudents’achievementandmotivation.Ihaveaddedtothe

knowledgebaseofinformationregardingformativeassessmentandstudentself-

assessmentthroughtheuseofarubric.Ihavegainedavoicebygathering

informationontheeffectofinformalexternalassessmentthroughtheuseof

Schulenburgevaluatingstudentswithachecklistthatmirroredthemainobjectives

oftherubric.

RESEARCHTRADITION

Ihaveusedqualitativeaswellasquantitativeresultsinthisstudy.

Qualitativeresultsarerepresentedintheformofstudentgrades.Ihave

incorporatednotonlythefinalgradegiven,butalsothegradethestudentsself-

assignedandthegradeindicatedbytheassistantprincipalintheexternal

assessment.Igatheredqualitativeinformationthroughtheuseofanecdotalnotes

andawritteninterviewcompetedbythestudents.

RESEARCHDESIGN

Thespecificdesignforthisstudyinvolvedcontentanalysisofstudents’

gradesdeterminedbytherubricusedbythestudentsandmyself.Studentexit

questionnairesandthechecklistfilledoutbytheassistantprincipalwerealsoused

inthecontentanalysisoftheresearch.

SAMPLINGPROCEDURES

Thispurposivesamplewasoftwosixthgradeclasses(experimentaland

control),incompleteenumeration,taughtinthefirst12weeksofthe2010-2011

schoolyearatSchooMiddleSchoolinLincoln,Nebraska.

RELIABILITY

Thestudentsallcompletedthesameprojectusinganidenticalrubric.The

controlgroupwasshowntherubricthatIusedtogradethemon.Theexperimental

groupwasgiventherubrictoevaluatethemselvesandwasassessedbymeusing

thisrubric.Allstudentsweregiventhesameexitquestionnairetoevaluatethe

studentsunderstandingoftheprojectexpectations,theirmotivationforcompleting

theproject,andtheirthoughtsonthegradetheyreceivedfortheproject.

VALIDITY

UsingAnderson’s(etal.)criteriaforvalidityIensureddemocraticvalidityby

takinginformationprovidedintheexitquestionnairebyeverystudentinthe

sample.Ialsousedinformationfromtheassistantprincipal’sassessmentandthe

self-assessmentofthestudents.

Outcomevaliditywasgainedthoughtheexaminationofstudentsuccess

thoughtheirgradesandmotivationinrelationtotriangulatedassessment.

Processvaliditywasconfirmedthroughconsistentuseofidenticalrubricsfor

assessmentandself-assessment.Also,studentsmotivationandunderstandingof

expectationswasmeasuredthoughidenticalexitquestionnaires.

Catalyticvaliditywasachievedbystudents’increasedmotivationand

understandingofexpectationsthroughtriangulatedassessment.

DialogicValidityhappenedthoughtheconversationswiththeassistant

principalandthepresentationoftheresultswiththeSchooMiddleSchoolstaff.

RESEARCHSTEPS&TIMELINE

Date Steps

August20th

• Introducepapier-mâchéproject

• Discussassessmentprocedure

August23rd-September3rd

• Studentworktimewithformative

assessmentusingrubric

• Experimentalgroupself-assessed

usingtherubric

September6th&7th

• Studentssharedtheirartworkin

aclasscritiqueandreceivetheir

finalgrades

• Experimentalgroupparticipated

inasmallgroupsharingactivity

andwillbySchulenburgusinga

checklist

September8th

• Studentscompletedtheexit

questionnaire

ETHICALCONSIDERATIONS

Allstudyparticipantsinformationwasusedinformulatingtheresulting

statistics.StudentsintheELLprogramwereofferedalternativewaystosharetheir

work,andwerenotforcedtospeakinfrontoftheclassduringthecritiqueifthey

werenotcomfortabledoingso.

DATACOLLECTION/INFORMATIONGATHERING

INTRODUCITON

Iusedavarietyofmethodstogatherinformationandcollectdataforthe

completionofthisprojectconcerningtriangulatedassessment.Triangulationis

accomplishedusingdiversedatasourcesandtools.(Seematrixbelow):

DATACOLLECTION/INFORMATIONGATHERINGMATRIX

PrimaryQuestion:Whateffectdoestriangulatedassessmenthaveonstudent

motivationandachievement?

AnecdotalNotes&Observations

AssessmentTools2

ExitQuestionnaire

Whateffectdoesassessmenthaveonstudentmotivationandachievement?

Anecdotalnotes&observationsduringclass

Rubric

(Completedbyinstructor)

Questionnairefilledoutby

studentregardingassessment

Whateffectdoesself-assessmenthaveonstudent

motivationandachievement?

Anecdotalnotes&observations

duringclass

Rubric

(Completedby

student)

studentregardingself-assessment

Whateffectdoesexternalassessmenthaveonstudentmotivationandachievement?

Anecdotalnotes&observationsduringclass

Checklist(Completedby

AssistantPrincipal)

studentregardingexternal

assessment

TRIANGULATIONOFDATA

Eachsectionofthedatacollection/informationgatheringmatrixisdescribed

indetailaccordingtothecorrespondingletter/numbercombinationassignedtothe

sectionsofthematrix.

(A,1) Anecdotalnotesandobservationswererecordedduringclassregardingthe

effectformative/summativerubricassessmenthasonthestudents.

(A,2) Dataregardingthestudents’performancebasedontheinstructor’s

evaluationusingtherubricwerecollected.

(A,3) Thestudentscompletedanexitquestionnaireregardingrubricassessment

completedbytheinstructor.

(B,1) Anecdotalnotesandobservationswererecordedduringclassregardingthe

effectformative/summativerubricself-assessmenthasonthestudents.

(B,2) Dataregardingthestudents’performancebasedontheirevaluationusing

therubricwerecollected.

(B,3) Thestudentscompletedanexitquestionnaireregardingrubricassessment

theycompleted.

(C,1) Anecdotalnotesandobservationswererecordedduringclassregardingthe

effectsummative,external;checklistassessmenthasonthestudents.

(C,2) Dataregardingthestudents’performance,basedtheexternalassessment

performedbytheassistantprincipal,usingthechecklistwerecollected.

(C,3) Thestudentscompletedanexitquestionnaireregardingtheexternal

assessmentcompletedbytheassistantprincipal.

DATACOLLECTION/INFORMATIONGATHERINGTOOLS

Thedatacollectionandinformationgatheringtoolsusedinthisstudyare

listedbelow:

1. Rubric(seeAppendixA)2. Checklist(seeAppendixB)3. ExitQuestionnaire,LikertScale(seeAppendixC1andC2)4. Anecdotalnotesandobservationsrecordedduringandafterclass5. Discussionwithcolleaguesandadministration

DATAANALYSIS

Thisstudyexploredtheeffectoftriangulatedassessmentonstudent

motivationandachievement.Threesourcesthatwereusedtocollectdataand

gatherinformationwere:anecdotalnotesandobservations,studentrubricgrades,

andstudents’responsesonanexitquestionnaire.Themeansoftherubricgrades

werecalculatedusingthemeasuringandcomparingcentraltendencytechnique

withintervalmeasurement.Patternsinstudentquestionnaireswerecodedto

identifycommonthemesandpatternsinthestudents’responses.Finally,dataand

observationswerecomparedandcontrastedbetweenthecontrolandexperimental

groups.

FINDINGS&DISCUSSION

Thedatacollectedintheexitquestionnaire(appendixC1/C2)providedthe

majorityoftheinformationregardingthestudents’dispositionandmotivation

concerningtheproject.Thefollowingpiechartsdepictthestudent’sanswersto

eachindividualquestionintheexitquestionnaire.Theexperimentalandcontrol

groupquestionnaireswereslightlydifferentconsideringthegroupswereassessed

differently.Thecontrolgroupwasaskedquestionsabouttriangulatedassessment

inahypotheticalmanner,toseeifthestudentsthoughttheywouldhavefelt

differentlyabouttheprojectiftheyhadbeenevaluatedusingtriangulated

assessment.Thequestionsforeachgrouparedisplayedintheheadingofeachpie

graph;thecontrolandexperimentalgroupsarelabeledaccordingly.

ExperimentalGroupControlGroup Thedatashowsthatstudentsintheexperimentalfelttheyunderstoodthe

projectrequirementsslightlymorethanthecontrolgroup.62%ofstudentsinthe

experimentalgrouprecordedthattheystronglyagreedtheyunderstoodtheproject

expectationsascomparedto35%ofstudentsinthecontrolgroupwhostrongly

agreedwiththisstatement.Thestudentsintheexperimentalgrouphadacopyof

thegradingrubricandwerecontinuallycheckingtherubrictocomplywiththeself-

assessmentportionofthetriangulatedassessment.Theextraexposureand

interactionwiththegradingrubricmostlikelyresultedinahigherpercentageof

understandingacrosstheclass.

ExperimentalGroupControlGroup

Thisquestionwasanattempttocapturetheinitiallevelofinterestand

motivationofstudentsinbothgroups.Sixty-eightpercentofstudentsinthe

experimentalgroupagreedtheywereexcitedabouttheprojectwhile69%of

studentsinthecontrolgroupmarkedagreeorstronglyagree.Thesimilarityin

responsetothisquestionwasbeneficialtotrackingtheeffectivenessofthe

triangulatedassessmentmodelbetweenclasses.

Asseeninthelastquestiontherewasastrikingsimilaritybetweenthetwo

groupsintermsofmotivation.Seventy-onepercentofstudentsintheexperimental

groupagreedtheyweremotivatedtocompletetheprojectwhile70%ofthe

studentsinthecontrolgroupagreedtobeingmotivated.

Students’responsesregardinginitialmotivationcontinuedtobestrikingly

similar.Ninetypercentofthestudentsintheexperimentalgroupagreedwiththe

statementand92%ofthestudentsinthecontrolgroupwereinagreeance.Ofthe

lastthreestatementsregardinginitialmotivation,thestudentsofbothgroups

answeredwithina2percentagepointsofeachother.

Bothgroupswereassessedbyarubric,whichiswhythequestionisidentical

oneachsurvey.Theexperimentalgrouphadmoreexposureandinteractionwith

therubricbecausetheyself-assessedusingthesamerubricthatIusedinthe

assessmentofbothgroups.Thismayexplainthedisparityinresponsesbetween

groups.Allofthestudentsintheexperimentalgroupagreedthatrubricassessment

fromtheinstructorhelpedthembesuccessful,whileonly60%ofthestudentsinthe

controlgroupagreedwiththestatement.Again,theoverwhelmingagreement

depictedintheexperimentalgroupmayhavebeenadirectcorrelationoftheir

increaseduseoftheassessmenttool.

ExperimentalGroupControlGroup Thisisthefirststatementonthequestionnairethatwasdifferentforeach

group.Thecontrolgroupwaspresentedwithahypotheticalsituationtorecorded

theirthoughtsonself-assessment.Allbutonestudent(95%)agreedthatself-

assessmentwitharubricwasinstrumentalincompletingtheprojectsuccessfully.

Ofthecontrolgroup82%agreedthatiftheyhadtheopportunitytoassess

themselveswitharubricitwouldhavehelpedthembemoresuccessful.

ExperimentalGroupControlGroup Again,thegroupswerepresentedwithdifferentstatementsonthe

questionnaire.Surprisingly,only37%ofstudentsintheexperimentalgroupand

35%ofstudentsinthecontrolgroupbelievedthatexternalassessmentformthe

assistantprincipal,Mr.Schulenburg,wasinstrumentalinincreasingtheir

achievement.

Thisstatementalongwiththetwoproceeding,arethesameasthethree

previousstatementsbuttheword“want”hasbeenadded.Thegoalofthese

statementswastocalculateachievementagainstmotivation.Thethreeprevious

statementsaskedstudentsifthedifferentmodesofassessmenthelped,orwould

havehelped,themdobetter,whilethistripletasksstudentsiftheassessment

techniquesmadethemwant,orwouldhavewanted,todobetter.

Intheexperimentalgroup,allbutonestudent(95%)recordedthatrubric

assessmentfromtheirinstructormadethemwanttodobetter.Thisisadecrease

from100%ofstudentsthatindicatedassessmentfromtheirinstructorhelpedthem

dobetter.Inthecontrolgroup,77%ofstudentsagreedwiththestatementthat

rubricassessmentfromtheirinstructormadethemwanttodobetter,whichisan

increasefrom60%thatrecordedrubricassessmentfromtheirinstructorhelped

themtodobetter.Iexpectedthelevelofagreementtobelowerwiththisstatement

thanthepreviousstatementregardingachievement.

ExperimentalGroupControlGroup Intheexperimentalgroup,thenumberofstudentsthatagreedwiththe

statementthatself-assessmentwitharubrichelpedthemachieve,agreedthatthe

evaluationofselfwitharubricalsomadethemwanttodobetter.Interestingly,

therewasonestudentwhorespondedneutraltothestatementregarding

achievement,andonestudentrecordedthattheydisagreedwiththestatement

regardingmotivation.Thequestionnaireswereanonymssoitisunclearifitwasthe

samestudentthatdidnotagreewiththetwostatements.

Similarly,therewasthesamenumberofstudentsinthecontrolgroupthat

agreedwithbothstatements.Thereweretwostudentsthatrespondedneutralto

theearlierstatementregardingachievement,whileallthatdidnotagree,disagreed

inthestatementaboveregardingmotivation.Again,nameswerenotrecordedon

thequestionnaires,soitisimpossibletotellifthesamefourstudentsdidnotagree

withbothstatements.

ExperimentalGroupControlGroup Thefinalstatementonthequestionnairedepictsthemostdrasticchange

betweenthestudents’attitudesinrelationtoexternalassessmentanditseffectson

motivation.90%ofthestudentsintheexperimentalgroupexpressedthat

assessmentfromMr.Schulenburgmadethemwanttodobetter,whileonly37%of

thesamestudentsrecordedthattheexternalassessmenthelpedthembesuccessful.

Inparallel,96%ofthestudentsinthecontrolgrouprecordedtheythought

externalassessmentfromMr.Schulenburgwouldhadmadethemwanttodobetter

whileonly35%ofthesamestudentsexpressedagreementthatassessmentfrom

Mr.Schulenburgwouldhavehelpedthemdobetter.

Thestatementsrecordedfromthestudentsregardingexternalassessment

werethemostdramaticonthequestionnaire.Itwasinterestingtoseethatalthough

studentsdidnotagreethatexternalassessmentwouldhelpthemontheproject,the

overwhelmingmajorityfoundittobeverymotivating.Dothestudentsseea

correlationbetweenmotivationandachievement?AsaneducatorIbelievethat

motivationhasagreatdealtodowiththelevelofachievementofanindividual

studentoranentireclass.Althoughthestudentsdidnotseetheexternal

assessmentashelpful,Ibelievethatinclusionoftheexternalassessmenthelpednot

onlymotivatestudentsbutalsoincreasedachievementacrosstheclassinthe

experimentalgroup.Thestudentsintheexperimentalgroupwereheldaccountable

bythemselves,theinstructor,andanexternalsource.Thetriangulatedassessment

modelprovidesamuchmorecomprehensive,summativeassessmentofthe

studentswork.

Thefollowingpiechartsdepictabreakdownofthestudent’sfinalgradesfortheproject.

Intheexperimentalgroup,17outof21(81%)studentsreceiveda90%or

higherwithanastonishing48%receivingallpointspossible.Eightypercentwasthe

lowestscoreintheclass.Inthecontrolgroup11of24(46%)studentsreceiveda

90%orhigherwith13%receivingallpointspossible.Thelowestscoreinthe

controlgroupwasa70%.

Althoughtheclassaverageisnotastoundinglydifferent,thenumberof

studentsreceivingallpointspossibleissubstantial.Theincreasedexposureand

interactionwiththeassessmentrubricandprocessincreasedthestudents’

awarenessoftheprojectexpectationsresultinginhigherachievement.Theexternal

assessmentprovidedbyMr.Schulenburgincreasedthestudents’motivationwhich

alsowasinstrumentalinproducingextremelyhighratesofsuccessamongthe

experimentalgroup.

SUMMARY&CONCLUSIONS

Inreview,theresearchmayhavebeenmorecomprehensiveifalarger

samplewereused.Includingmultiplegradelevelsanddifferentprojectswould

provideabetterlookattriangulatedassessmentinthemiddleschoolsetting.

Conductingthestudyinmultiplesubjectareaswouldalsoprovidealargerscopeof

theeffectivenessoftriangulatedassessmentinthepromotionofmotivationand

achievement.

References

Andrade,H.,Wang,X.,Du,Y.,&Akawi,R.(2009).Rubric-ReferencedSelf-

AssessmentandSelf-EfficacyforWriting.JournalofEducationalResearch,

102(4),287-302.

Astin,A.(1991). AssessmentforExcellence:ThePhilosophyandPracticeof

AssessmentandEvaluationinHigherEducation.Westport:GreenwoodPublishingGroup.

Berdrow,I.,&Evers,F.(2010).Basesofcompetence:aninstrumentforselfand

institutionalassessment. Assessment&EvaluationinHigherEducation,35(4),419-434.

Bingham,G.,Holbrook,T.,&Meyers,L.(2010).UsingSelf-Assessmentsin

ElementaryClassrooms.PhiDeltaKappan,91(5),59-61.Boud,D.(1990).Assessmentandthepromotionofacademicvalues.Studiesin

HigherEducation,15(1),101.

Chen,C.(2010).Theimplementationandevaluationofamobileself-andpeer-

assessmentsystem.Computers&Education,55(1),229-236.ExternalAssessment .(2003).Retrievedfrom

http://finearts.esc20.net/art/art_assessment/art_as_external.htmlLincolnPublicSchoolsImportantInformationBooklet.(2010).Retrivedfrom

http://www.lps.orgInternalAssessment.(2003).Retrievedfrom http://finearts.esc20.net/art/art_assessment/art_as_internal.htmlMcdonald,B.,&Boud,D.(2003).TheImpactofSelf-assessmentonAchievement:the

effectsofself-assessmenttrainingonperformanceinexternalexaminations. AssessmentinEducation:Principles,Policy&Practice,10(2),209.

McMillan,J.,Cohen,J.,Abrams,L.,Cauley,K.,Pannozzo,G.,&Hearn,J.(2010)

UnderstandingSecondaryTeachers’FormativeAssessmentPracticesand

TheirRelationshiptoStudentMotivation.MetropolitianEducaitonalResearch

Consortium,(onlinesubmission),1-20Nicol,D.,&Macfarlane-Dick,D.(2006).Formativeassessmentandself-regulated

learning:amodelandsevenprinciplesofgoodfeedbackpractice.Studiesin

HigherEducation,31(2),199-218.PerformanceAssessmentinArt.(2003).Retrievedfrom

http://finearts.esc20.net/art/art_assessment/art_as_perform.htmlReeves,T.(2009)TowardaTreatmentEffectofanIntervetiontoFosterSelf-

RegulatedLearning(SRL):AnApplicationoftheRaschModel.M.A.Thesis,

UniversityatBuffalo,StateUniversityofNewYork ,(onlinesubmission),1-136.

Roth,M.(2010).LincolnPublicSchoolsCommunityNews:Connectingwithour

Commmunity.Retrievedfromhttp://www.lps.org

Schroeder,A.,Minocha,S.,&Schneider,C.(2010).Thestrengths,weaknesses,

opportunitiesandthreatsofusingsocialsoftwareinhigherandfurthereducationteachingandlearning. JournalofComputerAssistedLearning,

26(3),159-174.Shepard,L.(2008).TheRoleofAssessmentinaLearningCulture. Journalof

Education,189(1/2),95-106.Tan,K.(2008).Qualitativelydifferentwaysofexperiencingstudentself-assessment.

HigherEducationResearch&Development ,27(1),15-29.Vallerand,R.,Koestner,R.,&Pelletier,L.(2008).ReflectionsonSelf-Determination

Theory.CanadianPsychology ,49(3)

Vallerand,R.,Pelletier,L.,Blais,M.,Brière,N.,Senécal,C.,&Vallières,E.(1993).On

theassessmentofintrinsic,extrinsic,andamotivationineducation:EvidenceontheconcurrentandconstructvalidityoftheAcademicMotivationScale.SageSocialScienceCollections,53(4),159-172.

APPENDIXA:RubricName_____________________________________________________________Period________Day________

Papier-Mâché Oaxacan Animal Rubric

Thisrubricwillbeusedtogradeyourwork.Youareresponsibleforkeepingthisrubricinyourfoldertohandinwithyourcompletedproject.Youwillgrade

yourselfaftercompletingeachstepoftheprojectbycirclingtheareathatbestdescribesyouranimal.Ifyouareinbetweencircletheblankspaceinbetweenthedescriptions.OnceyourhavecompletedandgradedeachstepIwillgradethesamesteponaseparaterubric.Onceeveryoneisdonewiththeirproject,Mr.Schulenburgwillvisitourclasstohelpusgradeourprojects.Hewillbeaskingyouquestionsinsmallgroupstomakesureyouunderstandtheproject.All3gradeswillbeusedtoformyourfinalgradefortheproject.Besuretoaskquestionsandhelpyourfriends.ProjectSteps 5 4 3 2 1

AnimalConstruction

Animalisrecognizable,coveredwitha“skin”oftape,bodypartsareattachedproperly

Animalisrecognizable,Thereisnewspapershowing,jointsbetweenpartsareweak/flimsy

Theanimalisnotrecognizable,littletapeisholdingthestructuretogether,bodypartsfalloff

Papier-mâché

Application

2layersofpapier-mâché,Paperissmooth,

coverstheentireanimal.

1layerofpapier-mâché,animalismostlycovered,

paperiswrinkledinplaces

1layerofpapier-mâchéwithseveral

wrinklesandmanyareaswheretapeisexposed,

(BaseCoat)

Animaliscompletelycoveredwithabasecoatofpaint.

Thereareafewsmallareasofwhitepapershowing

Thereareseveralareasofwhitepapershowing

Pattern&Details

Thereare5ormorepatterns

repeatedontheanimal,atleast5differentcolorsareused

Thereare3or4patternsonthe

animal,with3to4colorsusedinthepatterns

Thereare1to2patterns

ideasontheanimal,withonly1to2colorsused.

TotalScore______/20

APPENDIXB:Checklist

Studentswillbeabletousetheirrubricsduringthistime.Acheckwillbegivenifthestudentsexhibitedthetaskoransweredthequestioncorrectly.

Studentsidentified/describedthe4basicstepsoftheprojectasstatedintherubricStudentscouldpointoutanddescribethedifferentpatternsandcolorstheyusedontheiranimals.Studentsanimalswereidentifiable.Q:Whatartformisthisprojectbasedon? A’s:Oaxaca,Mexico;Wood-carvedanimals;ArtistZenyFuentes

APPENDIXC1:Questionnaire

WHATAREYOURFEELINGSABOUTTHISPROJECT?Pleasecircleyourfeelingsaboutthisprojectandthewayyouweregraded.Youdonotneedtoincludeyourname.Thiswillnotbegraded,pleasebehonest,Iwanttoknowhowyoureallyfeel.

SA=StronglyAgree,A=Agree,N=Neutral,D=Disagree,SD=StronglyDisagreeIunderstoodwhatwasexpectedofmeinthisproject.Iwasexcitedaboutthisproject.Iwasmotivatedtocompletethisproject.Iwantedtogetagoodgradeonthisproject.HavingMr.SchwaningerGrademewitharubrichelpedmedobetter.

Gradingmyselfwitharubrichelpedmedobetter.HavingMr.Schulenburggrademewithachecklisthelpedmedobetter.HavingMr.SchwaningerGrademewitharubricmademewanttodo

better.Gradingmyselfwitharubricmademewanttodobetter.HavingMr.SchulenburgGrademewithachecklistmademewanttodobetter.

SAANDSD

Adam Schwaninger Research

Documents

IC - · tiro Unser Land tiris im Tiroler Forstdienst...

Adam C. Simon Ph.D., University of Maryland, 2003 Research.....

NUOVA ADAM:...

IoT Interoperability at Bosch - Energy.gov...Wynne, Adam...

Computational Astrophysics: Research to Teaching and Beyond....

Equality Information and Tenant Satisfaction Adam Payne, ARP...

Research based, people driven CDISC ADaM Datasets - from...

1 Web-Scale Data Serving with PNUTS Adam Silberstein Yahoo!....

Adam Aluisio NIH/FIC Research Scholar University of...

Columbia University Amy Hale Won Yung Choi Yaniv Eyny...

Making the CASE Adam C. Cooper, Head of Research DCMS.

Elisabetta Ada Cavalcanti Adam - Unibs.it · Elisabetta Ada...

NanoBioMedical Centre Adam Mickiewicz …NanoBioMedical...

Nonproliferation Research & Education: Bringing the Academy...

1 Adam Reimer Postdoctoral Research Associate W.K. Kellogg.....

Energy Research Division REPORT · Energy Research and...