Classroom teaching observations presentation 9-20-18 · Planning classroom teaching observations Guidance for faculty and staff Office of Assessment, Trinity College September 2018

PlanningclassroomteachingobservationsGuidanceforfacultyandstaff

OfficeofAssessment,TrinityCollegeSeptember2018

Ifyou’dliketofollowalong:http://bit.ly/teaching_obs

Ifyou’dliketofollowalong:http://bit.ly/teaching_obs

ThePowerPointandaudio(ifacceptablequality)willbepostedonourwebsitewithinaweek.

https://assessment.trinity.duke.edu/assessment-roundtable

Soundevidence

Meaningfulreflection

Culture of continuous improvement

Soundevidence

Meaningfulreflection

ClassroomobservationsCourseevaluations

Evidenceofstudentlearningoutcomes

Self-evaluationsHabitsandpracticesofreflection

Culture of continuous improvement

Weunderstandstudentlearningthrough:TestsSurveys&evaluationsRubric-scoredprojectsInterviews

Weunderstandinstructionaldeliverythrough:

Surveys&evaluationsTeachingobservations★

Weunderstandstudentlearningthrough:TestsSurveys&evaluationsRubric-scoredprojectsInterviews

Outcomesassessment

UnderstandingthedegreetowhichstudentsarelearningUnderstandingwhatstudentsknowandcandofollowingalearningexperience:

•  Basiccontentknowledge•  Comprehension&application•  Analysis&evaluation

Programevaluation

UnderstandinghowaprogramoperatesUnderstandingtheinputstolearning:

•  Areourfacilitiessufficient?•  Arelecturesandreadingsaligned?

•  Areinstructorseffective?

Outcomesassessment



Programevaluation

UnderstandinghowaprogramoperatesUnderstandingtheinputstolearning:

Outcomesassessment



•  Areourfacilitiessufficient?•  Arelecturesandreadingsaligned?

•  Areinstructorseffective?

Whymightanindividualordepartmentdesireclassroomteachingobservations?

Toensurestudentshave

effectivelearning

experiences

Toimproveyourownpractice

Toseehowanewapproachto

teachingisworking

Becausestudentscannotevaluateallaspectsofteaching

Tounderstandcourseevaluationresultsbetter

Torespondtopossibleproblems

intheclassroomAsameansofmentorship

betweenexpertandnovice

Tospurdialogueamongfaculty

Asaregularpartofthepeer

reviewprocess

INDIVIDUAL GROUP

Whatareyourobjectives?

Whatbroughtyouhere?Whatdoyouwanttoaccomplish?

Summative Formative

IntrospectionLocalizeddialogue

Internalimprovement

Annual/APTreviewsReportingoutsideunit

ModelsHybridizationispossible!

Characteristics Advantages Challenges

Department-facilitated

WHO:InstructorswithinthedepartmentWHAT:Rotateandreciprocallyevaluateteachingacrosscoursesandfacultyinthedept

Observationswillbealignedwiththemission&outcomesofthedeptShouldresultinconsensusaboutteachingexcellenceInformsprogramevaluationandself-study

RequiressubstantialinvestmentacrosstheunitMustsetorderofpriorityforcoursesorinstructorsInstructorsmayfeelvulnerable;maybelessinvestedifnotinvolvedinprocess.

Externalobserver(fromanotherinstitutionordepartment)

WHO:Individualobserver(s)unaffiliatedwiththeprogramWHAT:Planwithinstructorinadvancetheobjectives,format,rubric,desiredinformation.

Objectivity:Personalrelationshipsdon’tinfluencetheevaluationWell-trained,sufficientexperiencewithrubricsResultsmaybemorereliableandvalid

Observerdoesn’tknowtheprogramorpersonnelObservermaynotknowthefieldDifficulttoimplementacrossadepartment

Peer-to-peer(reciprocal)

WHO:Individualinstructorswhowanttoreflectontheirteachingpractice,independentofdepartmentaleffortsWHAT:Joinsmallgroupoflike-mindedcolleaguestoreciprocallyobserve/discuss

ConversationalLowrisk;emphasizesformativedevelopmentofallpartiesInsightsfromotherfieldsandteachingmethods

PossiblydivergentconceptionsofteachingexcellenceObservermaynotknowthefieldInformationdoesn’tfeedbacktothedepartment



Peer-to-peer(reciprocal)

WHO:Individualinstructorswhowanttoreflectontheirteachingpractice,independentofdepartmentaleffortsWHAT:Joinsmallgroupoflike-mindedcolleaguestoreciprocallyobserve/discuss

ConversationalLowrisk;emphasizesformativedevelopmentofallpartiesInsightsfromotherfieldsandteachingmethods

PossiblydivergentconceptionsofteachingexcellenceObservermaynotknowthefieldInformationdoesn’tfeedbacktothedepartment

https://www.facultyfocus.com/articles/faculty-development/teaching-squares-cross-disciplinary-perspectives/àhttps://learninginnovation.duke.edu/faculty-opportunities/connect-with-other-faculty/

“Ateachingsquareconsistsoffourfacultyfromdifferentdisciplineswhovisiteachother’sclasseswithinatwo-to-three-weekperiod.Aftertheclassroomvisits,thefourgatheraroundcoffeeoramealtodiscusstheteachingobserved.Theintentionofthesquareisnottocriticizeeachother’steaching.Rather,it’stogatherideasondifferentteachingapproachesthatmightbeusedinone’sownclasses.It’sanopportunityforfacultytoreflectontheirownteachinginlightofcolleagues’teachingexamples.”



Department-facilitated

WHO:InstructorswithinthedepartmentWHAT:Rotateandreciprocallyevaluateteachingacrosscoursesandfacultyinthedept

Observationswillbealignedwiththemission&outcomesofthedeptShouldresultinconsensusaboutteachingexcellenceInformsprogramevaluationandself-study

RequiressubstantialinvestmentacrosstheunitMustsetorderofpriorityforcoursesorinstructorsInstructorsmayfeelvulnerable;maybelessinvestedifnotinvolvedinprocess.

Departmentprovidesleadership,structure,andresourcestoenablesystematic,intentionalteachingobservations.Createandreinforcethemessagethatintrospectionandpeerreviewarepartofthedepartment’sculture.Thisisnotapunitiveapproachtoteachingpractice.It’suptothefacultyanddepartmentalofficerstodeterminewhatformatworksbest.Butit’snotnecessarilytop-down:Requiresdialogueandconsensusaboutexcellentteaching.

SpecificthoughtquestionsfordepartmentchairsandprogramofficersWillobserversbeadministratorssuchaschairsorprogramdirectors,seniorfaculty,orteachersatalllevelswhoarepairedtoobserveoneanother'scourses?Willyouinvitefacultyfromoutsideyourprogramortheinstitution?Ifso,whom?Willtheresponsibilityofobservingbevoluntaryormandatory?Willthefacultywhodoformativeobservationsbethesameasthosewhodosummativeobservations?Betransparentabouthowwillfeedbackbeused.Forinformalfeedback?Forteachingreviews?

Adaptedfromhttps://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Pages/teaching-observations.aspx



Externalobserver(fromanotherinstitutionordepartment)

WHO:Individualobserver(s)unaffiliatedwiththeprogramWHAT:Planwithinstructorinadvancetheobjectives,format,rubric,desiredinformation.

Objectivity:Personalrelationshipsdon’tinfluencetheevaluationWell-trained,sufficientexperiencewithrubricsResultsmaybemorereliableandvalid

Observerdoesn’tknowtheprogramorpersonnelObservermaynotknowthefieldDifficulttoimplementacrossadepartment

Notwhollyseparatefromordistinctfromtheothermodels.Simplymeansbringinginobjectivityfromoutsidetheunit,usuallyintheformofanexternalcolleagueorexpertwithinthediscipline.Mayrequireadditionalcoordinationifexternalobserversareaskedtoutilizerubrics,feedbacktools,schedules,etc.thatarepartofthedepartment’sstructuredapproachtoteachingobservations.

StructuringobservationsEachdepartmentwillhaveitsownapproach!Commonpracticesinclude:

Timing:3-4timespercourse,distributedacrossthesemester,andtouchinguponauthenticrepresentationsofthecourseexperience.

Selectinstructorsand/orcourses:Cyclicalbasis,perhapsbasedonexistingreviewschedules.You’llhavetoprioritizesomeindividualsandcourses,anddeterminefairdecisionrules.

Selectionofobservers:2colleaguesfromwithintheprogram,possibly+1fromoutsidetheprogram.Determinewhichobserver(s)bringabalanceofcontentknowledgeandteachingexpertise.Isserviceasobservervoluntaryorrequired?

Usingarubric/feedbacktool:It’shelpfultohaveastandardizedfeedbacktool,butwhichcanaccommodatedifferencesincoursetypes(e.g.,labsandlectures).Develop,share,solicitfeedback,revise.Likeallrubrics,ittakestimetodevelopconsensusamongindividualswhohaveastakeineffectiveteachingevaluations

Discussionandreporting.Promptdeliveryoffeedback,includingspecificsuggestionsforteaching,isessential.Howthisoccursisuptothedepartment.Instructorsshouldconsiderincludingintheirteachingportfoliossummariesofhowtheyuseobservationalfeedbacktodeveloptheirteachingpractice.

http://bit.ly/teaching_observation_toolshttps://assessment.trinity.duke.edu/course-evaluation-codes-trinity-college

Commondimensions

ofobservational

tools

Format,sequencing,organizationoftheclass? Rapport

betweeninstructor&students?

Studentengagementwithlearning

tasks?

Relevanceoflearning

evaluations?

Accuracyofcontent?

Relevanceofsupportingmaterials&resources?

Equityintheclassroom?

Suitabilityanduseofinstructional

tech.?

Typesofrubric/feedbacktoolsExamplescollectedhere:http://bit.ly/teaching_observation_tools

Structuredcategories,withmostlyfree-response

•  https://stanford.app.box.com/s/fe73bkt0cwxd7mr4z6kme2wt3fi7aa84•  http://www.academic.umn.edu/provost/peer_review/documents/

observation_protocol.docx•  http://www.fctl.ucf.edu/FacultySuccess/FacultyDevelopment/ClassObservations/

contents/criteria_for_peer_observation.pdf•  https://wiki.ubc.ca/images/7/75/CTLT_SGIF_Form_March_2016.pdf

Structuredcategories,withdefinedrubricratings

•  https://www.augusta.edu/pamplin/documents/peerevalteaching_rubric.pdf•  http://tdop.wceruw.org/Document/TDOP-2.1-Users-Guide.pdf

Checklist •  http://www.celt.iastate.edu/wp-content/uploads/2017/07/examplechecklistclassroomobservationform-.docx

Specialty:Learner-centricrubric

•  https://cetl.kennesaw.edu/sites/default/files/resources/Learner%20Centered%20Rubric%20for%20Classroom%20Observations.pdf

Specialty:Distanceed. •  https://www.westga.edu/~distance/roblyer32.html

Specialty:STEM •  http://www.cwsei.ubc.ca/resources/COPUS.htm•  https://www.lifescied.org/doi/full/10.1187/cbe.14-02-0023

Speciality:Humanities •  https://utop.uteach.utexas.edu/sites/default/files/utop-instrument-humanities-august2014_0.doc

Mappingobservationaltoolsacrossclassroomvisits

http://www.ben.edu/faculty-staff/ctle/fac_resoåurces/forms_teaching.cfm

Reviewingotherinstructionalartifacts

http://www.academic.umn.edu/provost/peer_review/

Whatchallengesalreadycometomind?

•  Observationsmakeinstructorsfeelvulnerable.Protecttheiragency,andrecognizetheircommitmenttoself-development.

•  Well-plannedobservationsrequireaninvestmentoffacultytime.

•  Teachingwhileobservedcanfeelunnatural,inauthentic.

•  Theremightbealackofconsensusaroundwhatconstituteseffectiveteaching……whichcanfrustrateparticipantsand…impedethedevelopmentofacommonevaluativerubric.

•  Iftheprocessisnotwell-calibrated,theresultsmaybelessreliable.Plantotriangulate:useobservationalevidencewithothermeasuresofteaching

Typicalchallengestoacknowledge

Concern Advicetotheobserver Advicetotheobservedinstructor

AdvicetoChairs&otherofficers

Instructorvulnerability

Recognizetherearemultipleeffectiveapproachestoteaching,anddifferencesincomfortwithinstructionaltechnologies.Collaborateonthefocusoftheobservation,goals,areasofconcern.Usethemtoguidefeedback.Balanceconstructivecriticismwithgenuine,specificpraise.

Beopentodiscussingyourteachingchallengesaswellasstrengths.Activelyparticipateinthedevelopmentoftheobservationprocess,especiallyfeedbacktools.

Recognizethosewhovoluntarilyparticipateinteachingobservationsandwhousefeedbacktodeveloptheirprofessionalpractice.Nurtureacultureofconstructiveintrospectionanddialogueaboutteachingexcellence.

Time&labor Planfor3-4visitsacrossthesemester.Balancethoroughnessofobservationsovertimewithlaborrequirements.Recognizethatagoodrubricorchecklistmaytakemultipleiterations.

Acknowledgetheinvestmentoffacultytime.Providereleasetimetodevelopmentgroup,ifpossible.

Authenticity Beasunobtrusiveaspossiblewhenvisitingface-to-faceclassestoputboththestudentsandtheinstructoratease.

Trynottoadjustyourteachingstyleandmethodstothepresenceoftheobserver.Feedbackshouldbebasedonyourauthenticpractice.

Communicatethatteachingobservationsemphasizeformativeimprovement.Reinforceideathateveryinstructorbringsuniquestrengthstotheclassroom.

Consensus Activelyparticipateinthedevelopmentoftheobservationprocess,especiallyfeedbacktools.Beinclusiveofdifferentstyles,assumptions,experiences,andlevelsofcomfort/confidencewithteachinginnovations.

Invitediscussionofexpectations,concerns,andgoals.Clarifypurposeoftheobservationandhowresultswillbeused.Conveneworkinggrouptodevelopfeedbacktool.

Reliability&usefulness

Focusyoursummaryremarksandfollow-updiscussiononspecificareasforimprovement.Offerrelevantstrategiesthattheinstructorcantryinthenearfuture.

Savematerialstoincludeinyourteachingportfolio.Makespecific,focusedplanstoadjustelementsofthecourseoryourteachingapproach.Askforhelpinterpreting/blendingobservationalfeedbackwithcourseevaluationfindings.

Ifimplementedacrossthedept.,encouragecalibrationofobservationsandfeedback.Providetraining.Integrateprofessionaldevelopmentopportunities.Useobservationsinconjunctionwithcourseevaluations.

Adaptedfromhttps://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Pages/teaching-observations.aspx

Adaptedfromhttps://ctl.yale.edu/ReflectiveTeaching

ObservationsareonesourceofevidenceThereareotherwaysfacultycanreflectonandenhancetheirteachingpractice.•  Reflectionjournals•  Video-recordedteachingpractices•  Teachingportfolio•  Mid-andend-of-semesterevaluations

https://assessment.trinity.duke.edu/course-evaluation-codes-trinity-college

Mid-termcourseevaluations:

Anothertypeofevidence

https://assessment.trinity.duke.edu/midterm-assessment-strategies




https://assessment.trinity.duke.edu/midterm-assessment-strategies


Lastnotes:Evaluatingunionizedfacultyhttp://admin.trinity.duke.edu/finance/non-regular-rank-union-faculty/self-assessment-portal

ConsultresourcesonFacultyAffairswebsiteForexample,http://admin.trinity.duke.edu/faculty-affairs/policies-procedures?qt-faculty_affairs_policies=4#qt-faculty_affairs_policies

DepartmentAssessmentPortfolio:ProgramObjectives

Programevaluation Outcomesassessment

https://learninginnovation.duke.edu/faculty-opportunities/connect-with-other-faculty/https://learninginnovation.duke.edu/faculty-opportunities/art-and-science-of-teaching/assessing-your-teaching/https://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Pages/teaching-observations.aspxhttps://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Documents/Classroom-Observation-Checklist-AustinCC.pdfhttps://cft.vanderbilt.edu/guides-sub-pages/peer-review-of-teaching/https://www.facultyfocus.com/articles/faculty-development/teaching-squares-cross-disciplinary-perspectives/https://ctl.yale.edu/Observation-Protocols-Teaching-Inventorieshttp://www.crlt.umich.edu/resources/peer-review

Webresources

Selectbibliography★ Arreola,R.A.(2007).Developingacomprehensivefacultyevaluationsystem:Aguidetodesigning,building,and

operatinglarge-scalefacultyevaluationsystems.AnkerPublishingCompany.Bernstein,D.(1996).Adepartmentalsystemforbalancingthedevelopmentandevaluationofcollegeteaching:A

commentaryonCavanagh.InnovativeHigherEducation,20(4),241-247.Carter,V.(2008).Fivestepstobecomingabetterpeerreviewer.CollegeTeaching,56(2),85-88.Cavanagh,R.(1996).Formativeandsummativeevaluationinthefacultypeerreviewofteaching.InnovativeHigherEducation,

20(4),235-240.★ Chism,N.,&Chism,G.(2007).Peerreviewofteaching:Asourcebook(2nded).Bolton,MA:AnkerPub.Francis,D.(1995).Thereflectivejournal:Awindowtopreserviceteachers'practicalknowledge.TeachingandteacherEducation,11(3),

229-241.Mager,D.,Kazer,M.,Conelius,J.,Shea,J.,Lippman,D.,Torosyan,R.,&Nantz,K.(2014).Development,implementationand

evaluationofapeerreviewofteaching(PRoT)initiativeinnursingeducation.InternationalJournalofNursingEducationScholarship,11.

Seldin,P.,Miller,J.,&Seldin,C.(2010).Theteachingportfolio:Apracticalguidetoimprovedperformanceandpromotion/

tenuredecisions(4thed).SanFrancisco:Jossey-Bass.Stone,D.,&Heen,S.(2014).Thanksforthefeedback:Thescienceandartofreceivingfeedbackwell(evenwhenitisoffbase,

unfair,poorlydelivered,andfrankly,you'renotinthemood).NewYork:Viking.