1 A Guidebook for Peer Evaluation Valdosta State University College of the Arts Mike Savoie
2
ValdostaStateUniversityCollegeoftheArts
AGuidebookforPeerEvaluation
PreparedbyMikeSavoie,AssistantDean
July2010
3
INTRODUCTION ThepurposeofthisguidebookistobetterunderstandthefactorsforeffectivepeerobservationandtoprovideapeerevaluationmodelfortheCollegeoftheArts.Areasonthatteachingisgenerallyundervaluedinhighereducationisthelackofinformationandevidenceinteachingperformance(Centra,2000).Asacomponentinassessingfacultyperformance,thepeerevaluationisoftenoverlookedasanimportantpartoftheevaluationprocess.Thisguidebookwillprovidesuggestedpeerevaluationinstrumentsandproceduresbasedintheliteratureoneffectiveteaching,peerobservation,andreflectivepractice.PeerevaluationofteachingisconsideredanimportantpartofthetenureandpromotionprocessatValdostaStateUniversityandisoneofthecriteriaasevidenceofeffectiveteachinginsummativeevaluation.FACTORSFORPEEREVALUATION Ininvestigatingthefactorsforsuccessfulpeerevaluation,onerequirementisafacultymember’swillingnesstoparticipateintheprocessandtoaddresstherecommendationsofthefindings.Itismoreusefulasaformativetoolbutitalsoprovidesspecificinformationforimprovementgenerallybythecolleagueswhowillinevitablyserveonpersonnelcommittees.AsreferencedinCentra’sresearchontheroleofcolleagues,hestatesthatpeersplayaroleinsummativeevaluation(tenuredecisions)butshouldalsohavearoleinformativeevaluation.Well‐informedcolleaguesareinthebestpositiontoassessqualityofinstructionalpractices(1993).Thisroleshouldbeexpandedtoimproveteachingeffectiveness.FurtherjustificationforpeerobservationispresentedinBukalski’sGuidetoFacultyAdvancement(2000).Hereferencesthatpeersarebettersuitedthanstudentstoaddressteachingconcerns.Peerevaluationslikelyincludemanymaterialsincludingsyllabi,assignments,handouts,andstatementofteachingphilosophy.Itisalsousualforfacultymemberstohaveatleasttwomajorpeerreviewsbeforecomingupforsummativeevaluation.Thesetworeviewsusuallycoincidewithpre‐tenurereviewsandthesummativetenurereview(Bukalski,2000).PEEREVALUATIONASEVIDENCEOFEFFECTIVETEACHING Inapeerevaluationprocess,thepeerobservationsmustofferpracticalinformationaswellascredibilityintheevaluationofteaching.ThesevenguidelinesestablishedbytheAmericanAssociationforHigherEducation(AAHE)recommendsthefollowingsevenprinciples:
1. Avoidcasual,unannouncedobservations;2. Makeobservationspartofaconsultationprocessincludingpreandpostmeetings;3. Linkobservationsreviewsofteachingmaterials;4. Conductseveralobservations;5. Useateamapproach;6. Observersmustbeopentolearning;and7. Informstudentsoftheprocess.
4
Theguidelinesforpeerevaluationareoutlinedinabodyofliteraturepartiallypresentedinthisstudy.Inadditiontogeneralizedstepsfortheprocess,anoverviewofthedimensionsofteachingmeritsinvestigation.Hart’sworkonTeachersObservingTeachers(1987)establishessixinterrelatedcategorieswhenconductingobservationsofcolleagues’teaching.Theyinclude:
1. TheCognitiveDimension–theuseofquestionsandactivitiestostimulatedeeperanalysisofthesubjectoramorethoroughunderstandingofthebasics.
2. TheSocio‐PoliticalDimension–thebuildingandmaintenanceofrapport.3. TheClassroomStructureandProcedures–theinstructionalmethodsand
procedures.4. TheCurricularContext–therelationshipbetweenthecourse,curriculum,and
generaleducation.5. TheEffectsofTeaching–thisishowwellstudentsarelearningincluding
activities,engagement,andassessments).6. TheRhetoricalDimension–theuseoflanguage,organization,anduseoftime.7. ThePhysical‐TemporalDimension–thisincludesthetimeofday,room,and
physicalcomforts. Thesedimensionsareimportantindevelopinganinstrumentandobjectivesforevaluation.Addressingallsevendimensionsisanimportantconsiderationinaneffectiveformativeevaluation.USINGVIDEOINTHEEVALUATION Inadditionthesedimensions,anotherhelpfultoolinpeerandself‐evaluationprocessesistheuseofvideorecordingofteachingperformance.Severalstudieshavebeenconductedontheeffectiveuseofvideorecordingasatooltoimproveteachingperformance.InthestudyUsingdigitalvideotorethingteachingpractices(2007),Girod,Bell,andMishraexaminetheusedigitalvideointeachertrainingtoinvestigatetheeffectonteacherthinkinginfiveareasincluding:
1. Instructionaloutcomes;2. Instructionaldesign;3. Classroomperformance;4. Processversusproduct;and5. Consideringtheneedsofthelearners
Theresultsofusingvideorevealedfivethemesfocusedonre‐thinkingandre‐workinginstructionalpracticesanddesignincluding:
1. Attendingtothedesiredoutcomesandpriorstudentknowledge;2. Instructionaldesignasengaginglayers(mediacomponents);3. Teachingasperformance;4. Re‐thinkingtheprocess/producttensionasinproducingaproductsuchasa
paperorvideo;and5. Designasasensitizingexperiencetolearners’needsorengagement
Thesethemesareaconcerninvalidatingtheuseofvideoasaninstructionaltoolandforuseinformativedevelopmentandreflectivepractice.(Girod,Bell,&Mishra,2007)
5
Facultyandstudentconsensussupporttheuseofpeerevaluationaspartofacomprehensiveprogramoffacultyassessment.Keig’s(2000)FormativePeerReviewofTeachingpresentsfacultyattitudesaboutparticipatinginapeerevaluationprocesstoimproveteachingratherthantomakepersonneldecisions.Thefindingssupportthatfacultyarewillingtoparticipateinpeerassessmentforformativeevaluationbuttheyarereluctanttoutilizevideoasacomponentoftheprocess.Thereissubstantialsupportthatvideorecordingofclassesisinvasiveandintimidatingtomanyinstructors(Keig,2000).Indevelopingguidelinesforapeerevaluationmodel,theinclusionofvideorecordingisrecommendedifthisisacceptabletothefacultymemberbeingevaluated.Ithasbeenproventobeusefulinengagingself‐reflectiveevaluationaswellasprovidingevidenceofsuccessfulteaching.Althoughitshouldnotberequiredforallevaluations,itisrecommendedasatoolforeffectiveevaluationandimprovement.GUIDELINESFOROBSERVATIONS Developingapeerevaluationmodelrequiresanunderstandingofthefacultyevaluationprocessspecificallytheflowofinformationtoeveryevaluator.ArreoladetailsanoverviewoftheflowofinformationinanevaluationprocessinaworkshoptitledDevelopingaComprehensiveFacultyEvaluationSystem(2004).Inatraditionalhierarchicalpeermodel,theflowofinformationisprocessedateveryleveloftheevaluationandthereareopportunitiestopresentirrelevantorbiasedinformation.Inacontextualfiltermodel,dataisinterpretedwithcontext(Arreola,2004).Contextisthemissingcomponentfromthetraditionalmodel.Aformativepeerevaluationprocessoffersacontextualfilterwithathoroughassessmentofteachingpracticeaswellasprovidinganopportunityforimprovement.Arreola’ssuggestedpeermodelrecommends:Atriadcommitteestructurewith:
1. Onememberselectedbythedepartmenthead,2. Onebythefacultymember,and3. Oneappointedatlarge.
Thecommitteeprovidesevaluativeinformationon:
1. Coursestructureandorganization;2. Currencyinthefield;3. Appropriatenessoflevel;and4. Accuracyandappropriatenessofcoursematerial
Thereshouldbeaminimumoffourconsultationmeetingsforthereviewprocess.Theconsultationsshouldinclude:1. Ageneralfacultymeetingtoclarifyvalues,standards,andcriteria;2. Anindividualpre‐conferencetoprovideafoundationforobjective,toestablish
ground‐rules,toresolveconflicts,andtoclarifyanyambiguity;3. Apre‐visitconferencethatestablishestheobservationschedule;and4. Apost‐visitconferencesoonaftertheobservationforreflectivefeedback
6
UNDERSTANDINGTEACHINGANDLEARNING Indevelopingaprocedureforevaluatingteachinganunderstandingofteachingpracticeandstudentlearningareimportantconsiderations.Indesigningamodelthatiffocusedonteaching,definingtheroleofateacherisrequisite.Inreferencetotheroleofteachersinhighereducation,Boyer’sparadigmonthescholarshipofteachingreferencesthatthepracticeinvolvesthedevelopmentofknowledge,skill,mind,character,ortheabilityofothers(1990).Hestatesthat,“teachingstimulatesactive,notpassive,learningandencouragesstudentstobecritical,creativethinkers,withthecapacitytogoonlearning.”(1990,p.23‐4)Withregardtoteachingaspractice,theseprocedures,aswellaspedagogy,mustbe“carefullyplanned,continuouslyexamined,andrelatedirectlytothesubjecttaught.”(Boyer,1990,p.24) InKolb’sresearchonLearningStylesandDisciplinaryDifferences,hefocusesondimensionsoflearningandhowthesestylesdifferentiatewithregardtothespecificacademicprograms.Hisfindingsreflectafour‐stagecycleconsistingof1)concreteexperience;2)observationandreflection;3)formationofabstractconceptsandgeneralizations;and4)testingimplicationsofconceptsinnewsituations.Kolbreferstothiscycleastheexperientiallearningtheory(1981).Heextrapolatesonthiscycleoflearningbyidentifyingtwobasicdimensionsconsistingofabstract‐concretelearningprocessesandactive‐reflectivelearningprocesses.ThesignificanceofKolb’sfindingswithregardtothisguidebookisthedifferentiationinlearningprocessesintheCollegeoftheArts.Coincidentally,artsprogramsincludingArt,Music,Journalism,Speech,andDramaticArtsarepositionedclosetothecenterofactiveandreflectivelearningwithaslightlyskewedtoreflectiveobservation.OnKolb’sabstract‐concretecontinuum,artsprogramsarecloselyclusteredinconcretelearning.Thisreflectsthesimilarlearningprocessesinartsbaseddisciplinesasmoreconcreteinfocusandlessorientedtoabstractconceptualization(Kolb,1981).WithregardtodevelopingastandardprocedurefortheCollegeoftheArts,Kolb’sfindingssuggestthatthelearningprocessissimilaracrossthedisciplinesinartsbasedprograms.ACTIONRESEARCHINTHEPEEREVALUATIONPROCESS AusefultoolinshapingthereflectivepracticeofthepeerevaluationprocessistheParticipatoryActionResearchmodeldevelopedbyKemmisandMcTaggart(2005).Asanoverview,actionresearchdoesnothaveonespecificapplicationordefinition.Theintentofthisdiscussionofactionresearchfocusesontheadaptabilityandusefulnessoftheconcept.Itaddressesthenotionthatself‐reflectivepracticeisatoolforimprovementbutnotmeanttocontrolorrestrictinnovativepractice(Altrichter,Kemmis,McTaggart&Zuber‐Skerritt,2002). Kemmisprovidesaframeworkforactionresearchasatoolinimprovingpractice‐basedpractice.Itdescribesactionresearchasacriticalandself‐criticalprocessaimedfacilitatingtransformationsinourpractices.Inadditiontochangingpractice,itcompelsindividualstofocusontheunderstandingofpractice.Inaneducationalenvironment,thisisahelpfulprocessforimprovingperformanceandeffectiveness(Kemmis,2009).Ina
7
diagrammaticalmodelforactionresearch,KemmisandMcTaggartdevelopedamodelthatconsistsoffourmomentsorphasesincluding:1)planning;2)acting;3)observing;andreflecting.Thiscycleisrepeatedforfeedbackfordevelopmentandevolutionintheirpracticesincludingunderstandingandthesituationsinwhichtheypractice(2005).Thisdiagrammaticalmodeliseffectiveforengaginginthefeedbackprovidedbythepeerevaluationprocess.Participatoryactionresearchsuggeststhatrepeatingthecycleofplanning,acting,observingandreflectingnotonlyengagestheparticipanttoreflectonpracticebuttoassessimprovement.Ininvestigatingtheliteratureonpeerevaluationguidelines,thefinalstepoftheprocessreinforcestheneedforreflection.Theactionresearchmodelprovidesaframeworkforthisactivity.Inrepeatingthecycle,oneiscompelledtofocusonimprovementmakingitusefulinprovidinganassessmentoroutcome.Asthepeerreviewprocessisrecommendedforformativeevaluation(Arreola,2004),incorporatingthisassessmentofimprovementintotheprocessmodelprovidesusefuldataforsummativeevaluation.CONCLUSION ReferencingPink’s(2009)newparadigmonmotivationinhisbookDrive,hestatesthatanindividualisintrinsicallymotivatedtoperform.Incentivebasedorextrinsicmotivationhasbeenproventobeineffectiveinimprovingperformanceorcapacity.Ourmotivationisdrivenbyuniversalhumanneedsandisbasedonautonomy,mastery,andpurpose.Anyprocedureforpeerevaluationmustbebasedontheseprinciplesandmustengageonetobuy‐intotheprocess.AsreferencedbyMcGowanandGraham,“noinstructorgrowsupthinking,Ireallywanttobeabadteacher.”(2009,p.161).Weareintrinsicallydriventoperformandimproveourpractice.Althoughself‐reflectiveevaluationisuseful,Centra’sresearchreflectsthatfacultyoverestimatestheireffectivenessinself‐evaluation(1993).Aprocessengagedinformativeevaluationbasedonpeerobservationandreflectionhasthepotentialtoenhanceperformanceaswellasdevelopfacultycohesionandmorale(Millis,2006).ThisprincipleisalsoreferencedinSenge’sFifthDiscipline(2006)inhowsystemsthinkingtransformorganizationsintolearningorganizations.InSenge’sfirstprinciple,TeamLearning,itisstatedthat,“theintelligenceoftheteamexceedstheintelligenceoftheindividualsintheteam.”(2006,p.9)Basedonone’sintrinsicdrivetoimproveandthecollectiveknowledgeofateam,apeerevaluationprocessoffersgreatpotentialintheformativeevaluationprocess. DevelopmentofapeerevaluationguidebookfortheCollegeoftheArtsmustbebasedintheliteraturerelevanttounderstandingteachingandlearningprocessesinthearts;anthoroughoverviewofthepracticesanddimensionsofformativeevaluation;thefactorsforsuccessfulteachingandimprovement;andusefulpracticesinthepeerevaluationprocess.Onthebasisofthesefindings,thisguidebookservesasacomprehensiveguideforengagingineffectivepeerobservation,evaluation,andimprovementinteachingperformance.
8
REFERENCESAltrichter,H.,Kemmis,S.,McTaggart,R.,&Zuber‐Skerritt,O.(2002).Theconceptof actionresearch.TheLearningOrganization,9(3/4),125‐131.Arreola,R.A.(2004).DevelopingaComprehensiveFacultyEvaluationSystem.CEDAWorkshop.Boyer,E.(1990).ScholarshipReconsidered.CarnegieEnsowmentfotheAdvancementofTeaching.N.J.:Princeton.Bukalski,P.J.,&Lyons,T.J.(2000).GuidetoFacultyAdvancement:AnnualEvaluation,Promotion,andTenure.UniversityofFilmandVideoAssociation.Centra,J.A.EvaluatingtheTeachingPortfolio:ARoleforColleagues.(K.E.Ryan,Ed.)EvaluatingTeachinginHigherEducation:Avisionforthefuture.,83,87‐93.Centra,J.A.(1993).ReflectiveFacultyEvaluation.SanFrancisco,CA:Jossey‐Bass.Girod,M.,Bell,J.,&Mishra,P.(2007).Usingdigitalvideotore‐thinkteachingpractices.JournalofComputinginTeacherEducation,24(1),23‐29.Hart,F.R.(1987)“Teachersobservingteachers.”InJ.H.Broderick(ed.)TeachingatanUrbanUniversity.Boston:UniversityofMassachusettsatBoston.Hutchings,P.(1996).Makingteachingcommunityproperty:Amenuforpeercollaborationandpeerreview.Washington,DC:AmericanAssociationforHigherEducation.Keig,L.(2000).Formativepeerreviewofteaching:attitudesoffacultyatliberalartscollegestowardcolleagueassessment.JournalofPersonnelEvaluationinEducation,14(1),67‐87.Keig,L.,&Waggoner,M.D.(1994).CollaborativePeerReview:TheRoleofFacultyinImprovingCollegeTeaching.theGeorgeWashingtonSchool,Schoolof EducationandHumanDevelopment.WashingtonD.C.:ASHE‐ERICHigherEducationReportNo.2.Kemmis,S.(2009).Actionresearchasapractice‐basedpractice.EducationalActionResearch,17(3),463‐473.Kemmis,S.,&McTaggart,R.(2005).ParticipatoryActionResearch.InN.K.Denzin,&Y.S.Lincoln,HandbookofQualitativeResearch,ThirdEdition(pp.559‐603).BeverlyHills,CA,USA:Sage.
9
Kolb,D.A.(1981).LearningStylesandDisciplinaryDifferences.InA.W.Chickering,TheModernAmericanCollege:Respondingtothenewrealitiesofdiversestudentandachangingsociety(pp.232‐255).SanFrancisco,CA:Jossey‐Bass.Lewis,K.G.PreparingATeachingPortfolio.UniversityofTexasatAustin,TheDivisionofInstructionalInnovationandAssessment.Austin:UniversityofTexas.McGowan,W.R.,&Graham,C.R.(2009).FactorsContributingtoImprovingTeachingPerformance.InnovativeHigherEducation,34,161‐171.Merrion,M.(2009,September/October).AProphecyfortheArtsinHigherEducation.Change,pp.17‐21.Millis,B.J.(2006).PeerObservationsasaCatalystforFaculty.InP.a.Seldin,EvaluatingFacultyPerformance.Bolton,MA:AnkerPublishingPink,D.H.(2009).Drive:TheSurprisingTruthAboutWhatMotivatesUs.NewYork:RiverheadBooks.Pratt,D.D.(1997).Reconceptualizingtheevaluationofteachinginhighereducation.HigherEducation,34(1),23‐44.Senge,P.M.(2006).TheFifthDiscipline:theArtandPracticeoftheLearning Organization.Doubleday.Svinicki,M.,Lewis,K.(n.d.)Preparingforpeerobservation:Aguidebook.RetrievedJuly9,2010,fromtheUniversityofTexasatAustin,CenterforTeachingEffectivenesswebsite:http://www.utexas.edu/academic/diia/teaching/index.php
11
APPENDIXAClassroomObservationInstrument‐TraditionalModelClassroomObservationReportInstructorevaluated_______________________Course______________________Numberofstudentspresent___________Date_____________________________Evaluator(s)_____________________________________________________________Purpose:Thepurposeofthisclassroomobservationis(1)toprovideadatabaseformoreaccurateandequitabledecisionsontenure,promotion,andmeritincreaseand(2)toimprovefacultyperformance.Instructions:Pleaseconsidereachitemcarefullyandassignthehighestscoresonlyforunusuallyeffectiveperformance.Questions12and13havebeendeliberatelyleftblank.Youandtheinstructorbeingevaluatedareencouragedtoaddyourownitems.Eachinstructorshouldbeobservedontwooccasions,andtheobserver(s)shouldremainintheclassroomforthefullclassperiod.Itissuggestedthattheobserver(s)arrangeaprevisitandpostvisitmeetingwiththeinstructor.
Highest Satisfactory Lowest NotApplicable5 4 3 2 1 n/a
______1.Definesobjectivesfortheclasspresentation.______2.Effectivelyorganizeslearningsituationstomeettheobjectivesoftheclasspresentation.______3.Usesinstructionalmethodsencouragingrelevantstudentparticipationinthelearningprocess.______4.Usesclasstimeeffectively.______5.Demonstratesenthusiasmforthesubjectmatter.______6.Communicatesclearlyandeffectivelytothelevelofthestudents.______7.Explainsimportantideassimplyandclearly.______8.Demonstratescommandofsubjectmatter.______9.Respondsappropriatelytostudentquestionsandcomments.______10.Encouragescriticalthinkingandanalysis.______11.Consideringthepreviousitems,howwouldyouratethisinstructortoothersinthedepartment?______12.______13.______14.OverallratingWouldyourecommendthisinstructortostudentsyouareadvising?Whatspecificsuggestionswouldyoumakeconcerninghowthisparticularclasscouldhavebeenimproved?Didyouhaveaprevisitconference?______postvisitconference?______Source:SuccessfulFacultyEvaluationPrograms,byP.Seldin.Crugers,N.Y.:CoventryPress,1980.Allrightsreserved.
12
APPENDIXBParticipatoryModelAfacilitatorasksfacultymembersworkingindividually,tofillinthetablebelow.WhatShouldBeEvaluated
HowItCanBeEvaluated
Rapportwithstudents StudentinterviewsClassroomobservationsStudentevaluations
Subjectmatterknowledge ReviewofcoursematerialsReviewofpublicationsClassroomobservationsInformaldiscussions
“Thisactivityservesseveralpurposes.Forone,classroomobservationsareoftencitedasavalidwayofmeasuringfacultyperformance,afactthatreinforcestheneedfordepartmentaluseofthem.Bititisimportanttonotethattherightcolumncapturesthedepartmentalviewofwhatshouldbemeasured.Thislisthelpstoidentifycriteriaforgoodteaching.Infact,theseentriescouldbecategoriesonthedepartment’sobservationinstrument.”ReferenceMillis,B.J.(2006).PeerObservationsasaCatalystforFaculty.InP.a.Seldin,EvaluatingFacultyPerformance.Bolton,MA:AnkerPublishing.
13
APPENDIXCGuidelinesforFormativePeerObservationfromtheUniversityofTexasatAustin,CenterforTeachingEffectivenessPreparingforPeerObservation:AGuidebookFormativePeerObservationProcessSomebasictypesofformativepeerobservationare:amasterfacultyprogram,mentor‐menteepairs,peerdevelopmenttriads,graduatestudentfeedback,smallgroupinstructionaldiagnosis,andtheappraisalinterview.MasterFacultyProgramKatzandHenry(1988)promotedaMasterFacultyProgramthatpairedasuccessfulseniorprofessorwithajuniorfacultymembertocollaborateonteaching;toobserveeachother’sclassestolearn;tocompareandimproveteachingmethodologies;andtofosterweeklydiscussionsabouteffectiveteaching.These“buddysystem”collaborationsprovidemanyrewardsforbothfacultymembersinvolved.Withinformationgatheredintheobservations,facultypairsmeetonceaweekorsotodiscusshowstudentlearninghasbeenfosteredorhinderedinthelearningmethodologiesandtoshareinsightsaboutimprovingteaching.Mentor‐MenteeRelationshipsRolesplayedbymentorsincludefriendshipforemotionalandpersonalsupport,careerguidanceforincreasedprofessionalvisibility,informationsourcefordiscussingdepartmentalanduniversityexpectations,and/orintellectualguidancetoprovideresearchandwritingreviews.Moredepartmentsarearrangingmentor‐menteepairingtherebygivingnewfacultythegreatestopportunitytoprovetheirworthandfulfillinstitutionalexpectations.MentorsaregenerallyselectedfromthesamedisciplinebutBoicesuggeststhateffectivementoringdoesnothavetobedisciplinebound.Boice(1992)foundthat:
• Onlyahandfulofnewhiresfoundusefulmentoringontheirown.Theyalsotendedtoteachcautiouslybyemphasizingfactsandprinciplesoveractivestudentinvolvement.
• Itwasnotnecessarytopairnewfacultymembersonlywithseniormembersfromthe
samedepartment.Thepairingofjuniorfacultymembersandmentorsfromotherdepartmentswasequallyeffective.
• Usefulmentoringdidnotdependonpairspickingeachother.Assigningmentorswas
equallyeffective.Itwasoftennecessary,however,topromptpairstomeetregularlyuntilmeetingbecamehabitual.
• Althoughmentoringwasgenerallybeneficial,manymentorswerereluctanttogive
advicetonewfacultyonteaching,scholarlyproductivity,andtimemanagement.Thus,mentoringwasnotwithoutitsdeficiencies.
MasterFacultyandmentor‐menteeprogramsareverysimilarinorganization,butmentorsarenotnecessarilydesignatedas“masterteachers”andmaybechosendirectlybythementeesfor
14
reasonotherthanteachingexpertise.PeerDevelopmentTriadsPeerdevelopmenttriadsextendthe“pairconcept”andofferadditionalopportunitiestoshareandcompareteaching/learningstrategieswithtwopeers.GraduateStudentFeedbackAnexampleofagraduatestudentfeedbackmechanismcanbefoundinTheUniversityofChicago’sGraduateSchoolofBusiness.TheydesignedaonehourMBAcourseforgraduatestudentsinwhichtheyprovidefeedbacktoinstructorsbyauditingaprofessor’sclasses,videotapingselectedpresentations,andgatheringsuggestionsfromenrolledstudentsformidsemestercoursechanges.SmallGroupInstructionalDiagnosisTheSmallGroupInstructionalDiagnosis(SGID)isanothermethodusedtoimproveinstructionwiththeaidofapeerorfacultydevelopmentconsultant.Theprocess,whichcanbeeasilylearnedbypeers,isdescribedbyBennett(1987)asfollows:
• Withahalfhourorsoleftinaclassperiod,theinstructorintroducesafacilitator(peer)asafriendwhowillgatherideasaboutthestudents’learningexperiences.Thewordevaluationisnotusedbecauseofitspejorativeconnotationtostudents.Beforeleavingtheroom,theinstructorinformstheclassthatheorshehasvoluntarilyrequestedthisSGIDandhopestolearnabouthowthecourseisgoing.
• Thefacilitatorassuresstudentsthatthegroupresultsareconfidentialandwillbeshared
onlywiththeteacher.Groupsoffourorsostudentsareformedtodiscusstheirlearningexperiencesandanotetakerforeachgroupisdesignatedbythefacilitator.Thefacilitatoralsoliststhreequestionsontheboardforeachgrouptodiscuss:Whichaspectsofinstructionhelpyoulearn?Whichdonothelp?Whatdoyousuggesttoimproveyourlearning?
• Aftertenminutesofdiscussion,thefacilitatorrecordsthestudents’responsesusing
appropriatequantifiers(“mostsaid,”“afewsaid”).Thefacilitatorsummarizesthemajorideasandsharesthesummarywiththestudentsforadditionsorcorrections.
• Thefacilitatorthensharesstudentresponseswiththeteacherassoonaspossible,using
thestudents’ownwordswheneverpossible.Ifseriousproblemshaveemerged,thefacilitatorhighlightssolutionsofferedbystudents.
• Duringthenextclassperiod,ifpossible,theinstructorrepliestothestudents’analysis.
Instructorsshouldtrytoimplementatleastoneofthesuggestionsmadebystudents;suggestionsthatareinconsistentwithcoursegoalsorateacher’sstyledonotneedtobegivenseriousconsideration.
15
AppraisalInterviewTheappraisalinterviewisusedbychairswhowanttodiscussateachingproblemwithaninstructor.Firstthechairneedstocreateasupportiveenvironmentfortheinterviewandbeginswithquestionsabouthowthingsaregoingingeneral.Thechairmaysharesomeinsightsfromher/hisclassroomobservationstoofferencouragingcommentsabouttheinstructor’spractices.Thenthechairaskswhethertheinstructorishavinganydifficulties.Iftheinstructordoesnotmentiondifficulties,thechairthencanrefertoinformationtakenfromtheirownclassroomobservationsorproblemsraisedbystudents.Finally,thechairaskshowshe/heorthedepartmentcanhelptheinstructorsolvetheproblem.Theappraisalinterviewmustbehandledcarefullyandmorethanonemeetingmayberequiredtobringaboutthenecessarymodifications.Theprimaryobjectiveofanytypeofformativeobservationprocessisimprovementofteaching.REFERENCESvinicki,M.,Lewis,K.(n.d.)Preparingforpeerobservation:Aguidebook.RetrievedJuly9,2010,fromtheUniversityofTexasatAustin,CenterforTeachingEffectivenesswebsite:http://www.utexas.edu/academic/diia/teaching/index.php
16
APPENDIXDGuidelinesforSummativePeerObservationfromtheUniversityofTexasatAustin,CenterforTeachingEffectivenessPreparingforPeerObservation:AGuidebookSummativePeerObservationProcessThethreepersoncommittee,faculty,student,and/oradministrator,ortheAdHocCommitteeonTeaching,isthemostfrequentlyusedarrangementforsummativepeerobservation.Alargercommitteebecomestoocumbersomeandasmallercommitteedoesnotprovideenoughdata.TheAdHocCommitteecanbecomposedofnominationsmadebytheinstructorandthechair/dean;thisselectionprocessisparticularlyhelpfulforpromotionandtenuredecisions.Sharednominationsprovidetheinstructortheopportunitytorecommendoneormoreobserversforthecommittee.Preferably,thecommitteememberswillremainanonymoustoeachotherandthegeneraldepartmentalfaculty,inordertoavoidcontaminationofobservations.Asummaryofthethreefaculty/administrator/studentobservationsshouldbeprovidedbythecommitteechair.Summaryreportsbasedonchecklists,ratingforms,and/orwrittenanalysesshouldincludethefollowinginformation(Centra,1993,p.130): (1)Classroomperformanceobservationforms (2)Instructionalmaterialsreview (3)Advisingactivityreview (4)Participationongraduatecommitteesandgraduateteaching (5)Specialrecognitionforteaching (6)OverallrecommendationProtocolforSummativePeerObservationCommitteeMembersItissuggestedthateachAdHocCommitteememberfollowthisprotocolforsummativepeerobservations(Braskamp&Ory,1994;Millis,1987).
• Observersmustrespecttheobservedinstructororasktoberemovedfromthecommittee.Afacultymemberwithastrongdifferenceofopinionorpersonaldislikeforapeerhasdifficultybeingafairobserver.
• Eachobservermeetsprivatelywiththeinstructorbeforetheclassroomobservationstodiscusstheinstructor’sobjectivesfortheirclassesandtoreviewcoursematerials.Theobservedinstructorisallowedtoaskquestionsabouttheprocess.
• Eachcommitteemembermakesarrangementstoobservetheequivalentofthreeorfourcompleteclasssessions.Iftheobservedfacultymemberisteachingintwoormoreteachingvenues(i.e.,largelecturesection,graduatecourses,performanceclass)theobservershouldarrangetoattendclassesinmorethanonecourse.Fewerclasseswillnotproduceabalanceofexposurefortheobserverortheobservedinstructor.
17
• Thesummaryreportprovidesoverallinformationthatclearlyrepresentsalltheobservationresults.Recommendationsshouldbeaccompaniedbyspecificexamplesorobservationparticulars.
REFERENCESvinicki,M.,Lewis,K.(n.d.)Preparingforpeerobservation:Aguidebook.RetrievedJuly9,2010,fromtheUniversityofTexasatAustin,CenterforTeachingEffectivenesswebsite:http://www.utexas.edu/academic/diia/teaching/index.php
18
APPENDIXEKeyIssuesforPeerObservationfromtheUniversityofTexasatAustin,CenterforTeachingEffectivenessPreparingforPeerObservation:AGuidebookWhataresomekeyissuestorememberaboutformativeorsummativepeerobservation?Therearekeyissuestorememberasyouenteraformativeorsummativepeerobservationprocess:
• Anyobservationsystemchosenbyadepartmentmustbewellunderstoodbyobserversandthoseobserved.Actualclassroompracticewiththeobservationinstrumentismandatoryforits’effectiveuse.Theobservermustpracticewiththepeerobservationform/sbeforeclassroomvisits.Theobservationisdefinedbytheinstrumentandtheobservermustbeabletorecordbehaviorinthecategoriesontheform.Trainingforobserversisrequiredinordertohelpthemseewhatishappeningintheclassroom.Simpleorelaboratesystemsrequireextensivetrainingtoprepareobservers.Theturnoverofadministratorsandfacultyinalldepartmentsindicatethatacyclicalpeerobservation‐trainingprogramisneeded.
• Allobservationdatacollectedarerepresentativeofoverallteacherperformanceinthe
classroom.Observersmustbeawarethatsomeclassesareatypicalsothattheywilldevoteenoughtimetosecuretypicaldataaboutinstructoractivity.Observersareawareofthecontentoftheclass,timeofday,lengthofclass,andothertemporalfactorssuchasage,gender,ethnicity,generalappearanceoftheinstructor,etc.,andthepossibleeffectofthesefactorsonobservationresults.
• Observersandtheobservedinstructorareawareoftheinstitutionalanddepartmental
contextfortheimportanceofteaching.Deansandchairsneedtomakeageneralannouncementabouttheroleofpeerobservation,theobservationinstrumentstobeused,committeeassignments,andtheintrinsicvalueofformativeandsummativepeerobservationandevaluation.
• Eachindividualfacultyobservermeetsface‐to‐facewiththeindividualinstructorsbeing
observed,rememberingthatsomefacultyhavetremendousfearsaboutbeingobserved,andthattheactofobservationwilleffecttheoverallteaching/learningenvironmenttosomedegreeoranother.
• Observersandinstructorsareawarethatobservationsdonottakeplaceinisolationand
thereforeproduceevidencewithpossiblelegalimplications.AdHocCommitteemembersneedtobeawareofthesensitivityofobservationreportsandrecommendationsmadeforsummativepeerevaluations.AsCentra(1993,p.160)stated"Facultymembersandadministratorsshouldhaveageneralawarenessoftheirlegalrightsandresponsibilities,asstatedinfederalorstatelawsandinterpretedin
19
courtcases;facultymembersarebothemployees,aboutwhomdecisionsaremade,andpeerswhositinjudgment."Itisrecommendedthatpeerclassroomobservationsbeusedassimplyonepartofthelargerpicturewithregardtoevaluationofteachingeffectiveness.Donotgivepeerobservationsundueweightinsummativeevaluationsforthefollowingreasons:
• Limitedamountoftimeobserved
• Differentviewsofteachingamongcommitteemembers
• Supplementarytoothersourcesaboutteaching
• Peersdonotobservesystematically
• Peerobservationsoftentaintedbyreputationofinstructor
• Colleaguestendtobegenerousinratings
• Lowcorrelationofratingsbetweendifferentcolleagues
• Peersgenerallyhavelimitedexperienceobservingteaching
• Historicallyfacultynottrainedtoobserveteaching
• Colleaguesbetteratjudgingresearchthanserviceorteaching
REFERENCESvinicki,M.,Lewis,K.(n.d.)Preparingforpeerobservation:Aguidebook.RetrievedJuly9,2010,fromtheUniversityofTexasatAustin,CenterforTeachingEffectivenesswebsite:http://www.utexas.edu/academic/diia/teaching/index.php
20
APPENDIXFFacultyObservationFormAfromtheUniversityofMinnesota,CenterforTeachingandLearning
ClassroomObservationFormOpenEnded–(FormA)
Faculty________________________ DateofObservation__________________PeerObserver__________________Thisform,adaptedfromtheCommunityCollegeofAurora’sMentorProgramHandbookandStaffordshireUniversity’s“GuidelinesfortheObservationofTeaching,”provides10areasforobservation.Eachareaincludespromptsregardingwhatshouldbeobserved.1. Developmentoflearningobjectives:
• Areobjectivesfortheclassgivenverbally,written,ornotatall?• Arespecificinstructionaloutcomesused?• Areobjectivesdiscussedattheendofclass?
2. Selectionanduseofinstructionalmaterials:
• Dofilms,websites,andotheraudiovisualmaterialshaveaclearpurpose?• Arehandoutsappropriateinnumberandsubject?• Sincethetextmaybepre‐selected,doesinstructorgivehelpwithreadingorusingthe
text,ifnecessary?3. Educationalclimateforlearning:
• ArestudentsANDteacherinterestedandenthusiastic?• Doestheinstructorusestudentnames?• Ishumorusedappropriately?• Doesinstructornotembarrassorbelittlestudentsinanyway?• Istheatmosphereoftheclassroomparticipative?• Didtheinstructorhaveeyecontactwithstudents?
4. Varietyofinstructionalactivities:
• Doestimingofclassroomactivitiesconsiderattentionspans?• Doesinstructorinvolvestudentsindecidingwhatissuestodiscuss?
5. Preparationforclasssession:
• Provideexamplesthatshowpreparationbyinstructor.• Dostudentsknowwhatpreparation(readingorotherassignments)theyshouldhave
completedpriortoclass?
21
6. Instructionalmethods:• Listinstructoractivities.• Didtheopeninggaintheclass’sattention?Diditestablishrapport?• Didtheopeningoutlinethetopicandpurposeofthelecture?• Isthedeliverypacedtostudents’needs?• Doestheinstructorintroducetopic,stategoals,presentmaterialoractivity
effectively,summarize,andgiveassignmentorsuggestanideatoconsiderbeforethenextclass?
• Couldtheinstructorbeseenandheard?• Werekeypointsemphasized?• Wereexplanationscleartostudents?• Wereexamples,metaphors,andanalogiesappropriate?• Wasthelecturestimulatingandthoughtprovoking?
7. Opportunityforstudentparticipation:
• Liststudents’activities.• Doesinstructorencouragestudentstosummarizeandaddtoother’summaries?• Doesinstructorhelpquieterstudentsinteractwithothers?
8. Individualizationofinstruction:
• Aretheemotional,physical,andintellectualneedsofstudentsmet?• Doestheinstructorpromptawarenessofstudents’priorlearningandexperiences?• Doestheinstructoroffer“realworld”application?• Istheinstructoravailablebeforeorafterclass?• Doestheinstructorrelateclasstocoursegoals,students’personalgoals,orsocietal
concerns?9. Responsivenesstostudentfeedback:
• Istheinstructorpayingattentiontocuesofboredomandconfusion?• Doestheinstructorencourageordiscouragequestions(dissension)?• Doestheinstructorprovidestudentsopportunitytomentionproblems/concernswith
theclass,eitherverballyorinwriting?10. Learningdifficulties:
• Doesastudentneedassistanceforatemporaryorpermanentdisability?• Areoneormorestudentsnotmotivatedorunabletofollowtheclass?• Doestheinstructorshowfavoritism?• Arestudentsabletoseevisualaids?• Doesonegroupdominatediscussionandhinderothers’participation?
22
APPENDIXGFacultyObservationReportfromtheUniversityofMinnesota,CenterforTeachingandLearning
ClassroomObservationReportInstructorevaluated_______________________________________________________Observer(s)_____________________________________________________________Numberofstudentspresent__________Course________________________________Date____________Instructions.Severaldayspriortotheclassroomvisit,theinstructorshouldprovidetheobserver(s)withacopyofthecoursesyllabuscontainingcourseobjectives,content,andorganization.Procedure.Theobserver(s)shouldmeetwiththeinstructorseveraldaysinadvanceofthevisittolearntheinstructor’sclassroomobjectivesaswellastheteachingmethodstobeused.Withinseveraldaysafterthevisit,theobserver(s)shouldmeetwiththeinstructortodiscussobservationsandconclusions.
Please feel free to use the reverse side of this page to elaborate on your comments
1. Describetheinstructor’scontentmastery,breadth,anddepth.
2. Describethemethod(s)ofinstruction.
3. Howclearandwellorganizedisthepresentation?
4. Describetheformandextentofstudentparticipation.
5. Whatspecificsuggestionswouldyoumaketoimprovethisinstructor’steaching?
23
APPENDIXHFacultyObservationScaleFormCfromtheUniversityofMinnesota,CenterforTeachingandLearning
ClassroomObservationFormScale—(FormC)
Faculty_________________________ DateofObservation_________________PeerObserver___________________
NOTE:Thisform,adaptedfromtheCommunityCollegeofAurora’sMentorProgramHandbookandStaffordshireUniversity’s“GuidelinesfortheObservationofTeaching,”provides10areasforobservation.Eachareaincludespromptsregardingwhatshouldbeobserved.Developmentoflearningobjectives:Areobjectivesfortheclassgivenverbally,written,ornotatall? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AArespecificinstructionaloutcomesused? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AAreobjectivesdiscussedattheendofclass? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ASelectionanduseofinstructionalmaterials:Dofilms,websites,andotheraudiovisualmaterialshaveaclearpurpose? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AArehandoutsappropriateinnumberandsubject?
NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ASincethetextmaybepre‐selected,doesinstructorgivehelpwithreadingorusingthetextifnecessary? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/A
24
Educationalclimateforlearning:ArestudentsANDteacherinterestedandenthusiastic? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoestheinstructorusestudentnames? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AIshumorusedappropriately? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoesinstructornotembarrassorbelittlestudentsinanyway? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AIstheatmosphereoftheclassroomparticipative? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADidtheinstructorhaveeyecontactwithstudents? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AVarietyofinstructionalactivities:Doestimingofclassroomactivitiesconsiderattentionspans? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoesinstructorinvolvestudentsindecidingwhatissuestodiscuss? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/APreparationforclasssession:Provideexamplesthatshowpreparationbyinstructor:Dostudentsknowwhatpreparation(readingorotherassignmentstheyshouldhavecompletedpriortoclass? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AInstructionalmethods:Listinstructoractivities:Didtheopeninggaintheclass’attention?Diditestablishrapport? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADidtheopeningoutlinethetopicandpurposeofthelecture? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AIsthedeliverypacedtostudents’needs? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/A
25
Doestheinstructorintroducetopic,stategoals,presentmaterialoractivityeffectively,summarize,andgiveassignmentorsuggestanideatoconsiderbeforenextclass? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ACouldtheinstructorbeseenandheard? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AWerekeypointsemphasized? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AWereexplanationscleartostudents? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AWereexamples,metaphors,andanalogiesappropriate? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AWasthelecturestimulatingandthoughtprovoking? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AOpportunityforstudentparticipation:Liststudents’activities:Doesinstructorencouragestudentstosummarizeandaddtoothers’summaries? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoesinstructorhelpquieterstudentsinteractwithother? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AIndividualizationofinstruction:Aretheemotional,physical,andintellectualneedsofstudentsmet? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoestheinstructorpromptawarenessofstudents’priorlearningandexperience? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoestheinstructoroffer“realworld”applications? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AIstheinstructoravailablebeforeorafterclass? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoestheinstructorrelateclasstocoursegoals,students’personalgoals,orsocietalconcerns? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AResponsivenesstostudentfeedback:Istheinstructorpayingattentiontocuesofboredom,confusion? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoestheinstructorencourageordiscouragequestions(dissension)? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/A
26
Doestheinstructorprovidestudentsopportunitytomentionproblems/concernswiththeclass,eitherverballyorinwriting? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ALearningdifficulties:Doesastudentneedassistanceforatemporaryorpermanentdisability? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AAreoneormorestudentsnotmotivatedorunabletofollowtheclass? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoestheinstructorshowfavoritism? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/AArestudentsabletoseevisualaids? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/ADoesonegroupdominatediscussionandhinderothers’participation? NotDemonstrated NeedsImprovement Satisfactory Outstanding N/A
27
APPENDIXIClassroomTeachingObservationfromtheUniversityofMinnesota,CenterforTeachingandLearningFacultyObserved_________________________________Rank____________________DateofObservation____________________CourseObserved_____________________ClassroomTeachingObservationRatingscale(1=verypoor,2=weak,3=average,4=good,5=excellent,NA=notapplicable)CONTENT Mainideasareclearandspecific 1 2 3 4 5(Excellent)Sufficientvarietyinsupportinginformation 1 2 3 4 5Relevancyofmainideaswasclear 1 2 3 4 5Higherorderthinkingwasrequired 1 2 3 4 5Instructorrelatedideastopriorknowledge 1 2 3 4 5Definitionsweregivenforvocabulary 1 2 3 4 5ORGANIZATIONIntroductioncapturedattention 1 2 3 4 5(Excellent)Introductionstatedorganizationoflecture 1 2 3 4 5Effectivetransitions(clearw/summaries) 1 2 3 4 5Clearorganizationalplan 1 2 3 4 5Concludedbysummarizingmainideas 1 2 3 4 5Reviewedbyconnectingtopreviousclasses1 2 3 4 5Previewedbyconnectingtofutureclasses 1 2 3 4 5INTERACTIONInstructorquestionsatdifferentlevels 1 2 3 4 5 NASufficientwaittime 1 2 3 4 5 NAStudentsaskedquestions 1 2 3 4 5 NAInstructorfeedbackwasinformative 1 2 3 4 5 NAInstructorincorporatedstudentresponses 1 2 3 4 5 NAGoodrapportwithstudents 1 2 3 4 5 NAVERBAL/NON‐VERBALLanguagewasunderstandable 1 2 3 4 5(Excellent)Articulationandpronunciationclear 1 2 3 4 5Absenceofverbalizedpauses(er,ah,etc.) 1 2 3 4 5Instructorspokeextemporaneously 1 2 3 4 5Accentwasnotdistracting 1 2 3 4 5 NAEffectivevoicequality 1 2 3 4 5Volumesufficienttobeheard 1 2 3 4 5
28
Rateofdeliverywasappropriate 1 2 3 4 5Effectivebodymovementandgestures 1 2 3 4 5Eyecontactwithstudents 1 2 3 4 5Confident&enthusiastic 1 2 3 4 5USEOFMEDIAOverheads/chalkboardcontentclear &wellorganized 1 2 3 4 5 NAVisualaidscanbeeasilyread 1 2 3 4 5 NAInstructorprovidedanoutline/handouts 1 2 3 4 5 NAComputerizedinstructioneffective 1 2 3 4 5 NASPECIALCLASSIFICATIONNOTES:STRENGTHS:(e.g.metacurriculum,useofcomparisons&contrasts,positivefeedback,opportunityprovidedforstudentquestions)WEAKNESSES:(e.g.unabletoanswerstudentquestions,overalltopicknowledge,relevanceofexamples,etc.)OVERALLEFFECTIVENESSRATING 1 2 3 4 5DateofConference_______________ObserverSignature______________________