1 Measure What Matters (And Everything Matters): Assessing Sustainability Initiatives Robert Franco, Francisco Acoba, and Krista Hiser, Kapi‘olani Community College, Honolulu HI Assessing sustainability initiatives is a matter of focus. As the above “target” graphic illustrates, zooming in on one aspect often puts others at the periphery. What’s at the center for you? Some colleges keep great data on community impact, but don’t assess student learning about sustainability. On the other hand, assessing student learning addresses different questions and issues than assessing community impact. As the adage goes: “Measure what Matters.” However, in the case of sustainability, everything matters! What resources and methodologies are available for your assessment? What matters to you, your students, your campus leadership, your community and state, and your accreditation agencies? This chapter provides ideas for designing a streamlined multi-pronged assessment model, based on Kapi‘olani Community College’s Engagement, Learning and Achievement framework (KELA). The chapter lists available tools for monitoring the multiple impacts of campus sustainability initiatives, including the Community College Survey of Student Engagement (CCSSE), participant-observer research, pre- and post-testing, Student Assessment of their Learning Gains (SALG), the Sustainability Literacy Test (Sulitest), a reflection essay evaluation protocol, and the Carnegie Foundation. STUDENT ENGAGEMENT, LEARNING AND ACHIEVEMENT Community colleges in the U.S. are intently focused on student achievement: retention and completion numbers, liberal arts transfer, career and technical education degrees, job placement and even starting salary. But, as uniquely American institutions, from their inception
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
MeasureWhatMatters(AndEverythingMatters):AssessingSustainabilityInitiativesRobertFranco,FranciscoAcoba,andKristaHiser,Kapi‘olaniCommunityCollege,HonoluluHI
Assessingsustainabilityinitiativesisamatteroffocus.Astheabove“target”graphicillustrates,zoominginononeaspectoftenputsothersattheperiphery.What’satthecenterforyou?Somecollegeskeepgreatdataoncommunityimpact,butdon’tassessstudentlearningaboutsustainability.Ontheotherhand,assessingstudentlearningaddressesdifferentquestionsandissuesthanassessingcommunityimpact.Astheadagegoes:“MeasurewhatMatters.”However,inthecaseofsustainability,everythingmatters!Whatresourcesandmethodologiesareavailableforyourassessment?Whatmatterstoyou,yourstudents,yourcampusleadership,yourcommunityandstate,andyouraccreditationagencies?Thischapterprovidesideasfordesigningastreamlinedmulti-prongedassessmentmodel,basedonKapi‘olaniCommunityCollege’sEngagement,LearningandAchievementframework(KELA).Thechapterlistsavailabletoolsformonitoringthemultipleimpactsofcampussustainabilityinitiatives,includingtheCommunityCollegeSurveyofStudentEngagement(CCSSE),participant-observerresearch,pre-andpost-testing,StudentAssessmentoftheirLearningGains(SALG),theSustainabilityLiteracyTest(Sulitest),areflectionessayevaluationprotocol,andtheCarnegieFoundation.STUDENTENGAGEMENT,LEARNINGANDACHIEVEMENTCommunitycollegesintheU.S.areintentlyfocusedonstudentachievement:retentionandcompletionnumbers,liberalartstransfer,careerandtechnicaleducationdegrees,jobplacementandevenstartingsalary.But,asuniquelyAmericaninstitutions,fromtheirinception
2
morethan110yearsago,communitycollegesarealsopreparingtheirstudentsfortheworkofdemocracyandinformedparticipationascitizensengagingindemocraticsystemslocallyandnationally,aswellascollaboratorsinarapidlydevelopingglobalsystem.Kapi’olaniCommunityCollegeusesamulti-facetedinstitutionalevaluationframeworkknownastheKELAmodel:Kapi‘olaniEngagement,Learning,andAchievement.Thisframeworkfocusesfacultyandstaffplanningandprogramimplementationonafewcentralvariables.Theachievementandengagementvariablesarequantitative:achievementvariablesaretrackedeachsemesterwhileengagementvariablesaretrackedeverytwo-yearsthroughtheadministrationoftheCommunityCollegeSurveyofStudentEngagement(CCSSE).Courselearningoutcomesareassessedbyindividualfacultywhilegeneraleducationoutcomesareassessedthroughrubric-basedanalysesofstudentartifacts(usuallywritingassignments)conductedbyfacultycommunitiesofpractice.
3
EngagementToassessstudentengagementthecollegereliesheavilyontheresultsofitsCCSSE,whichitadministersinspringsemestersineven-numberedyears.TheCCSSEorganizesitsreportingunderfivemajorbenchmarks:
1. Faculty-StudentInteraction2. Active-CollaborativeLearning3. AcademicChallenge4. StudentEffort5. StudentSupport
Onecouldhypothesizethatcurricularandpedagogicalpracticesconsideredbestpracticesinsustainabilitylearning;thatis,AppliedLearning(campusasalivinglaboratory),service-learning,place-basedlearning,learningcommunities,undergraduateresearch,andresearchandindustryinternships,shouldresultinhigherlevelsofstudentengagementacrossallfivebenchmarks.SustainabilitypedagogiestendtobeHighImpactPractices,especiallyforstudentshistoricallyunderrepresentedinhighereducation(Kuh).Kapi‘olaniusesAASHESTARScriteriatoidentifySustainabilityFocusedcourses,whichmustidentifyahigh-impactpedagogyaspartofthecoursedesignation.AssessinglearningintheseSustainabilityFocusedcoursesprovidesanopportunitytocomparethesestudentswhocompletethemandthosewhodonot.Weshouldnotpretendthatallthecontrolsrequiredofpureexperimentalresearchcanbedevelopedandappliedinthehighlydiversestudentpopulationsinthecommunitycolleges.Weshouldinsteadstriveforconsistentlyandcyclicallyappliedassessmentandevaluationmethodsthatmeetthecriteriaof“quasi-experimental”research(seeHilletal).Theresultsofthesestudiesshouldsparkandsustainadministrativeandfacultydialogueaboutimprovement.Thecommunitycollegeenvironmentitselfprovidesuniquechallengestoassessmentdesign.Forexample,identifyinggroupsof“statisticaltwins”forpureexperimentalresearchisonerous,especiallywithcommunitycollegestudentswithdiversedemographicandlinguisticbackgrounds.Addedtothisfoundationaldiversityaredifferencesinhighschoolpreparation,accesstofinancialsupport(familyandcollege-supported),full-timeandpart-timestatus,patternsofstoppingoutandreturning,clarityaboutcourse-takingchoicesandpathways,clarityaboutemploymentaspirations,andunderstandingoftherightsandresponsibilitiesofcitizenship.Allthisvariabilityhasadirectimpactonwhetherstudentsgainasenseofbelongingatthecampus,howlongittakesthemtogainthissenseofbelonging,andthenwhethertheygainasenseofbecomingthatenablesthemtocompletecertificatesanddegrees,andattaingainfulemploymentorsuccessfultransfertoabaccalaureatecampus.Thisishowwedefineengagement.
4
Facultymustdesigneffectiveinteractionswithstudentswhoareunsuretheybelongandhavenotidentifiedwhattheywillbecome.Thebestcommunitycollegefacultyseetheircourseasacentralpartofthisbelonging-becomingcontinuum,strivetounderstandthateachoftheirstudentsaresituateddifferentlyonthiscontinuum,andtrytobuildonthissituated-ness.Mostcommunitycollegefacultyareteachingfivecoursesectionspersemester,oftenwithmultiplecoursepreparations.Facultythemselvesareonacontinuumofcaringandnurturing,andtheirworkloadsandclasssizesarenotalwaysamenabletoteachingdifferentlybyusingresearch-basedhighimpactpracticeswhichenablemoreindividuallycustomizedlearningopportunitiesfordiversestudents.Toengagemorefacultyinhighimpactandempoweringsustainabilityteachingpracticesrequiresstrongstaffsupporttomaintain,assess,andimprovecommunitycollaborationsandpartnerships.Thisiswhywemeasurewhatmatters.Sustainabilityteachingandlearning,curriculumandpedagogy,donewell,providearichandrelevantcontextformanygeneraleducation,careerandcapstonecoursesonthebelonging-becomingcontinuum.ButwhatdoesdonewellmeanforLearning,AchievementandEngagementthroughsustainabilityinitiatives?Totacklethequestionofsustainabilityeducationdonewellwewouldalsowanttodevelopmethods,bothquantitativeandqualitative,thatinformusaboutstudentsustainabilitylearning.AchievementTheKELAmodelfocusesonsevenstudentachievementmeasures:
1. CourseSuccess(percentofstudentcompletingaCorbetter)2. Re-enrollmentratesfromfalltospringsemester3. Re-enrollmentratesfromfalltosubsequentfall4. Annualnumberofcertificatesofachievementanddegreescompleted5. Annualnumberoftransfertobaccalaureatecampuses6. Percentofstudentscompletingcertificatesofachievementanddegreesina
three-yearperiod7. Percentofstudentstransferringtoabaccalaureatecampusinthreeyears
Let’senvisionhowwemightdocumenttheimpactofSustainabilitycoursesontheseachievementmeasures.Measures4-7aboveareparticularlyimportantsinceachievingthesemeasuresoftenresultsinadditionalfundingtocampusesfromstatelegislaturesandsystems.InthecurrentcurricularenvironmentatKapi‘olani,mostSustainabilitycoursesareactuallycoursesectionswithspecificcourseregistrationnumbersandaSUSTidentifier.Notallsections
5
ofacourseareSustainabilitydesignated.Forexample,onlyafewEnglish100(first-yearcomposition)courseshavethesustainabilitydesignation;notallthePsychology100courseshavethedesignation.ManycoursesaretaughtbydifferentfacultyandnotallthesefacultypursuetheSustainabilitydesignationforwhattheyperceiveastheircourse.However,thisSUSTversusnon-SUSTcourseidentificationcanbeviewedasanassessmentopportunity.Wecould,forexample,comparemeasures1and2inSUSTandnon-SUSTcoursesinthefallsemesters,andmeasures1and3inspringsemesters.Wecouldcomparemeasures1,2,and3inSUSTandnon-SUSTcoursestaughtbythesamefacultyinthesamesemester.Overmultiplesemesterswecouldtrackstudentswhocomplete2ormoreSUSTcoursesandestablishtheiraveragetocertificateordegreecompletionand/ortransfer.WecoulddetermineabaselineforstudentswhointheirfirstsemestersuccessfullycompletedaSUSTcourseandthencompletedadegreeofcertificateand/ortransferredinsixsemesters.WecoulddeterminetheaveragenumberofSUSTcourses,onadegreeortransferpathway,thatyieldedthebestcompletionandtransferresults.Inalltheseinquiries,wecouldassesswhatpedagogicalapproachesmadethemostbeneficialimpact.ThefuturecurricularenvironmentatKapi‘olaniincludesanAcademicSubjectCertificateinSustainability,andwecantrackandassessthesecertificatepursuersandcompletersonmeasures4-7above.Ifwecanshowthatthesecompletershavebetteroutcomesonthesemeasures,weshouldthenbeabletorequestandreceiveadditionalfundingfromthecampus,state,system,andexternalfunders.Learning Pre-andPost-TestSurveyFrom2014-2016,Kapi‘olani,inpartnershipwiththeCommunityCollegeNationalCenterforCommunityEngagement,andtheMaricopaCommunityCollegeSystem(AZ),leada3-yearTeagleFoundation-fundedprojectentitled,“StudentLearningforCivicCapacity:StimulatingMoral,Ethical,andCivicEngagementforLearningthatLasts”(seeteachingtobigquestions.wordpress.com).Sixcommunitycollegesparticipatedintheproject:Kapi‘olani(HI),Mesa(AZ),Delgado(LA),RaritanValley(NJ),andKingsboroughandQueensborough(NY).Allparticipatingcampusesdevelopedcommunitiesofpracticeengaginganaverageoftenfacultyperyearwhotaughttoasinglebigquestion:HowdowebuildOURcommitmenttocivicandmoralresponsibilityfordiverse,equitable,healthyandsustainablecommunities?
6
Asacentralcomponentoftheproject,wedevelopedapre-andpost-testsurveywithitemsetstargetingdiversity,equity,healthandsustainability.Eachsetaddressedstudents’civicandmoralengagement,behavior,andattitudes.Thespecificsustainabilityitemsonthesurveyarelistedbelow.Theallowablestudentresponseswere:completelydisagree,disagree,undecided,agree,andcompletelyagree.
1. Iamalreadyorwillbecomeactiveinenvironmentalprograms.2. Iamwillingtoconsumelessandgowithoutsomecomfortsifithelpstoprotectthe
environment.3. WhenIbuyaproduct,Iassessthetypeofpackagingandchooseonethatisrecyclable.4. IfIhadtochoosebetweentheconstructionofahighwaythatwillalleviatetrafficorthe
protectionofaplantspecies,Iwouldchoosethehighway.5. Evenifpublictransportweremoreefficientthanitcurrentlyis,Iwouldstillchooseto
usemyowncar.6. Solvingcurrentproblemsismoreimportantthanworryingaboutthefuture.7. Iwanttofindnewwaystolivemoresustainably.8. Climatechangeisasituationthatdemandsourimmediateattention.
Thefirstsurveywasadministeredearlyinthefall2015semesterandthenmatchedwithasecondsurveyadministeredaslateaspossibleinthatsamesemester.Kapi‘olani’sresultswereanalyzedwithaT-testmeasuringgainsfrompre-topost-testsurveyandatwo-wayAnovacomparingservice-learningandnon-service-learningstudents.Inthefirstiterationofthesurvey,analysisshowedthatservice-learningstudents,manydoingenvironment-relatedprojects,scoredhigherthannon-service-learningstudentsononesurveyitem:Iwanttofindnewwaystolivemoresustainably.Analysisalsoshowedallstudentgainsonthefollowingthreeitems:
• Iamalreadyorwillbecomeactiveinenvironmentalprograms.• WhenIbuyaproduct,Iassessthetypeofpackagingandchooseonethatisrecyclable.• Iwanttofindnewwaystolivemoresustainably.
Butanalysisalsoshowedstudentlossesonthreeitems:
• IfIhadtochoosebetweentheconstructionofahighwaythatwillalleviatetrafficortheprotectionofaplantspecies,Iwouldchoosethehighway.
• Evenifpublictransportweremoreefficientthanitcurrentlyis,Iwouldstillchoosetousemyowncar.
7
• Climatechangeisasituationthatdemandsourimmediateattention.Twoitemsshowedlittlechangeeitherway:
• Iamwillingtoconsumelessandgowithoutsomecomfortsifithelpstoprotecttheenvironment.
• Solvingcurrentproblemsismoreimportantthanworryingaboutthefuture.Thereweremethodologicalchallengesintheproject-wideuseofapre-andpost-surveylearningassessmentmethodology.First,sincetheTeagleFoundationwantedbroadpublicationanddisseminationofprojectfindings,weknewwewouldneedinstitutionalresearchboard(IRB)approvalsfromtheMaricopaCommunityCollegesystemaswellasthesixindividualcommunitycolleges.TheseIRBapprovalswereeasilyattainedatsomecampuses,butsomecampuseshadmoredifficulty,especiallyiftheywereinmulti-campussystems.Eachcollegechosewhichcoursestoincludeinthesurveyassessmentanddeterminedwhentoadministerthepreandpostsurveys.Wehadinitiallyhopedtohavethecapacitytopre-andpost-matchindividualstudentresponses.Oneofthegreatestchallengeswasobtainingsuccessfulmatchesforthesurveyadministeredearlyinthesemesterwiththeoneadministeredlateinthesemester.Everyyear,duringannualmeetingsandthroughmanyelectroniccommunications,thecollegesworkedtogethertodevisestrategiestohelpstudentsrememberwhatemailaddresstheyhadusedonthefirstsurveysothatitwouldmatchwiththelatersurvey.Inthe2015assessmentadministration,allsixcampusesadministeredthepre-surveybutonlythreecampuseshadenoughpairedresponsesinthepost-surveystoidentifystatisticallysignificantstudentlearninggains.Anotherchallengewasinterpretationofthesurveyresults.Beginningin1996,Kapi‘olaniCommunityCollegehadoriginallyconductedasurveyeverytwoyearsanditwasonlyafterthreesetsofsurveyshadbeenanalyzedthattheresultsbegantoshowclearpatterns(Renner).WehadhopedthatthreeconsecutiveyearsoftheTeaglesurveywouldalsobegintoshowclearpatterns,butthelackofcoherentpatternswithinandacrosscollegesledustodoamajorrevisionofthesurveyinthethirdyear.Ourgoalsweretoimprovethematchingofearlyandlatesurveysandthusincreasesamplesizeforeachcampus,andalsotobetteralignourquestions–andhopefullyourfindings–withthethemeswithinourTeagleproject.Wealsoincreasedthedemographicsectionofthesurvey.Ultimately,webelievethatthissurveyapproachcanyieldveryusefulinformation,especiallyiflargeenoughmatchedsamplesareobtained.Kapi‘olani’sresultsthatweredescribedabove
8
ledtodiscussionsamongfacultyandchangesinclasses.Further,patternsbasedongender,ethnicity,experienceprovidingcommunityservice,andinvolvementinundergraduateresearchandotherhighimpactpracticescanbeidentifiedandusedtohelpeducatorsguidestudentstowardthepathwaysthatwillbestassisttheminmovingtowarddeeperandmoremeaningfulengagementwiththeircommunities.Thepre-andpost-testassessmentgivesasnapshotofstudentlearningonawidescale—notjustfromspecificcourses.Inaseparatefocusgroupinterview,weheardfromstudentsthattheytendtoconnectlearningaboutsustainabilitymorewithnews,socialmedia,anddocumentariesthanwithspecificcourses.Thisindicatesthatfacultyneedtobemoreexplicitwhenteachingsustainabilityissues.StudentAssessmentoftheirLearningGains(SALG)Aninterestingalternativeinstrument,whichoffersapreandposttypeofviewbutfromamorequalitativeperspective,istheSALG:StudentAssessmentofLearningGains.WhatismostinterestingabouttheSALGisthatitacceptsstudentself-reflectionasevidenceoflearning.Thisisinalignmentwithcurrentresearchonlearning,andperhapswithacommunitycollegepedagogicalperspective.TheSALGwasdevelopedthroughfundingfromtheNationalScienceFoundation(NSF)whichcontinuestorecommenditforevaluationofeducationresearch.SALGdataisacceptedasprimafacieevidenceofstudentlearningbysomeaccreditingbodies,includingACCJC.TheSALGhasbeenusedbyover17,000collegeanduniversityinstructorssince1997,andcurrentlysupportsover14,000individualusersand2000groupusers.UseoftheSALGisfree;anyonemaysignupatwww.salgsite.org.TheSALGissimilartoastudentevaluationofthecourse,exceptitfocusesonthestudent,nottheteacher.Typically,StudentEvaluationsofTeaching(SETs)focusonteacherbehaviorsandusestudentsatisfactionasaprimaryevaluativecriterion.Extensiveresearchshowsthatinnovativeteachers/pedagogiesearnlowerscoresonSETsthantraditionalcourses,evenwhenstudentslearnmoreinthoseinnovativecourses(Seymouretal).Similartothe“negativeresults”describedinthepre-andpost-testing,whenitcomestoassessingsustainabilitylearning,traditionalSETsmayyieldnegativeevaluationsofsustainabilitycurriculum—forexample,ifstudentsdidn’tlikegoingoutside,orworkinginagroup.TheSALGwasspecificallydesignedtoremedythesedeficienciesbyfocusingexclusivelyonstudents’learninggains.Itwasbuiltaroundthreeessentialprinciples:
9
• thatassessmentofacourseshouldberelativetotheexplicitgoalsofthatcourse—i.e.,whatstudentsaresupposedtolearn;
• thatstudentshavesomethingvaluabletotellusaboutwhathelpsthemlearn(andwhatdoesn’t);and
• students’learningshouldbeenhancedbyfillingoutthecourseevaluation(byincreasingtheirmetacognitionabouttheirlearning).
SALG’sdesignusestenquestionsthatcomprisethecentralidentityoftheinstrumentandencouragesinstructorstocreateacustomizedinstrumentforeachcoursetheyteach.Forexample,manyofthequestion-stemsincludesub-questionsthataskinstructorstofillindetailsofclassactivities,assignments,keyclassconcepts,andsoon.Finally,instructorsmayaddadditionalquestionstotheoriginallistoften.Whenwritingthesenewquestions,instructorsmaychoosefromavarietyofestablishedresponsescales,ortheymaycreatetheirownscales.Thisflexibility,too,iscentraltotheidentityoftheSALGbecauseitallowsinstructorstoassesstheeffectivenessoftheirpedagogicalchoicesinhelpingstudentsreachthelearninggoalsfortheircourse.HereareafewoftheitemsusedtosurveystudentsinKapi‘olani’sSustainabilityFocusedcourses.Thechoicesare:nogains,alittlegain,moderategain,goodgain,greatgain,andnotapplicable.
Asaresultofyourworkinthisclass,whatGAINSDIDYOUMAKEinyourUNDERSTANDINGofeachofthefollowing:1.Thefollowingconceptsthathavebeenexploredinthisclass• SustainableEconomics(triplebottomline,alternativetoGDP)• EcosystemServices• EcologicalFootprint/CarbonFootprint• SustainableMaterialsManagement(CradletoCradleDesign)• ClimateChangeMitigation,Adaptation,andResilience• LocalFirst• SocialJustice&Equity2.HowthesecoreconceptsofsustainabilitymightrelatetotheclassI'mtaking.3.HowideasfromthisclassrelatetootherclassesIhavetaken.4.Howstudyingthissubjectareahelpspeopleaddressrealworldissues
TheSALGwascreatedtopromoteandsupportthoughtful,effectiveteachingandsystematic,rationalimprovementofthatteachingbyprovidingmeaningfulanddetailedfeedbackabout
10
thevalueandefficacyofclassroomteaching.Becauseitprovidesvalidandmeaningfulevidenceofstudentlearning,SALGdatacanbeusedtosupportcasesforpromotionandtenurefarmoreeffectivelythanSETinstrumentsthatfocusonstudentsatisfactionandhavescantrelevancetostudentlearninggains.Service-LearningReflectionEssayMostsustainabilitydesignatedcoursesatKapi‘olanidoservice-learning.Thisisbasedontheconvictionthatlearningaboutsustainabilitymeanspracticingitaswell.Sustainabilityteachingcanbedisempoweringandpushstudentstothinkingnothingcanbedonetofixtheseglobalproblems,thattheirfateissealed.Service-Learningempowersstudentstodosomethingnowaboutsustainabilityissuesconfrontingtheircommunities.TheKapi‘olaniServiceandSustainabilityLearningprogramrequiresasemestercapstoneessayinwhichstudentsreflectontheirexperience.Herearereflectionprompts:
• Issue:Identifythesocietalorecologicalissueyouhavehelpedtoaddressthroughyourservice.Explainhowyouhavehelped.
• Learning:Discuss3concepts/theoriesfromyourcourseworkthathavehelpedyoudoyourservice.Describeexperiencesduringyourservicethathavehelpedyouunderstandthoseconcepts/theories.
• CivicContext:Asaninformedindividualandcitizen,discusstheissueyouexplainedaboveasapublicorcommunityproblem.Whatelementsofunfairnessorinjusticedoestheproblemhave?Doyoubelievemorepeopleshouldcareabouttheproblem?Whyorwhynot?Discussoneormoresolutionstotheproblem.
• Goals:Explainhowyourcourseworkandserviceactivitieshaveshapedyourpersonal,academicorprofessionalgoals.Fromthelistbelow,selectalltheonesyouareinterestedindoinganddiscussatleastoneindetail.Explainhowtheaction(s)youwilltakewillhelpreducetheimpactoftheproblem.1. Supportfamily,friends,and/orneighborswhoareaffectedbytheproblem2. Serve,orfundraisetosupport,acommunity-basedorganizationthatisworkingon
theproblem3. Patronizebusinessesthatareactivelyworkingtolessentheseverityoftheproblem
orarecommittedtonotmakingtheproblemworse4. Conveneadialogwithpolicy-makerswhoareworkingontheproblem5. Advocatewithpublicofficialsand/orlegislatorswhoareworkingontheproblem6. Start,leadorjoinacampusgroupthatisworkingontheproblem7. Takeanothercoursetogainanewperspectiveontheproblem8. Completeadegreethatwillprovidemewiththeknowledge,skills,andattitudesto
workontheprobleminmyprofession
11
9. Other(specify)Thepromptscandemonstratesustainabilitylearningintheseways:UnderIssue,thestudentcandefineasustainabilityissue,suchasinvasivespecies.UnderLearning,sustainabilitydesignatedcourseshavesustainabilitySLOs,suchas“Measureone’simpactonthetriplebottomline:People,Planet,Profit.”Studentscanspecificallyexplainhowtheylearnedandappliedtheseconceptsaspartoftheirservice.UnderCivicContext,studentsoftenindicatesocialjusticeissues,suchasenvironmentaljustice.Forexample:howdoesclimatechangeasymmetricallyaffectPacificIslanders?UnderGoals,studentsexplaintheirplansthatsupportsustainability.Forexample:alotofbotanystudentswanttomaketheirownnativeplantgardens.Writtenresponsestothesepromptsmoredirectlydemonstratewhatstudentshavelearned.However,wellwrittenpiecestaketimeandexperiencetowrite,whichmanyofthestudentsdonothave.About2/3ofourservice-learningstudentsaretakingascienceorlanguage(Chinese,JapaneseorKorean)classinwhichsubstantialwritingorwritinginEnglishisnotamajorcomponent.Theseclassesdonotrequirefirst-yearcompositionasapre-requisite.Further,aboutathirdofourstudentsareinternational,anduptohalfspeakEnglishasasecondlanguage.Itseemstoomuchtoaskthemajorityofthesestudentstoproduceahigh-quality(English-language)essaythattheircurrentclassorpriorclasses/experienceshavenotnecessarilypreparedthemtowrite.Inanycase,essaysarecollectedbytheKapi‘olaniService&SustainabilityLearningOfficeandarandomizedselectionofessaysareassessedtwiceayear(seeteachingtobigquestions.wordpress.com).Whileworkintensive,thisassessmenthelpsfacultybetterunderstandstudentlearningachievementwhileprovidingasenseofcollegialityandconstantimprovementtotheservice-learningprogram.COMMUNITYANDCIVICENGAGEMENTToassesstheimpactofsustainabilitycurriculumandpedagogyoncommunityandcivicengagement,wecanexpandthestudentcapstonereflectionmethodologydiscussedabove.Wealsoneedtodevelopasimilarqualitativeassessmentapproachwiththecommunityandcivicpartnersweengageinoursustainabilityprogramming.AftermorethanadecadeofparticipationinthedevelopmentandimplementationoftheCarnegieFoundations’CommunityEngagementClassification,andfollowingthenationalresearchinthisarea,itisusefultoenvisionhowthatclassificationmighthelpusassessandimprovecommunitysustainabilityandresilienceefforts.HereistheCarnegieFoundationdefinitionofcommunityengagement:
12
Communityengagementdescribesthecollaborationbetweeninstitutionsofhighereducationandtheirlargercommunities(local,regional/state,national,global)forthemutuallybeneficialexchangeofknowledgeandresourcesinacontextofpartnershipandreciprocity.Thepurposeofcommunityengagementisthepartnershipofcollegeanduniversityknowledgeandresourceswiththoseofthepublicandprivatesectorstoenrichscholarship,research,andcreativeactivity;enhancecurriculum,teachingandlearning;prepareeducated,engagedcitizens;strengthendemocraticvaluesandcivicresponsibility;addresscriticalsocietalissues;andcontributetothepublicgood.
Further,Carnegiestates,
[Engagement]requiresgoingbeyondtheexpertmodelthatoftengetsinthewayofconstructiveuniversity-communitycollaboration…callsonfacultytomovebeyond‘outreach,’…asksscholarstogobeyond‘service,’withitsovertonesofnoblesseoblige.Whatitemphasizesisgenuinecollaboration:thatthelearningandteachingbemultidirectionalandtheexpertiseshared.Itrepresentsabasicreconceptualizationof…community-basedwork.(O’MearaandRice)
Kapi‘olani’s2015applicationfortheCarnegieCommunityEngagementClassificationidentifieditstop15mostimportantcommunitypartnershipsand11ofthesewereworkingwithusonsustainability-relatedprojectsandprograms.Newpartnershavebeendevelopedsince2015,andweneedtodevelopourassessmentsofthesepartnerships.IntheCarnegieClassificationprocessandongoingresearch,therearefivemajorcomponentsofauthenticpartnerships:
1. Clearlinesofcommunication2. Clearlydefinedrolesandresponsibilities3. Identifyingandelevatingboththeneedsandassetsofthecampusandthepartners4. Identifyingmutuallybeneficial,reciprocal,long-termgoals.5. Activeandcollaborativelearningforall
Thesefivecomponentswouldshapefutureassessmentandevaluationsofthecommunityandcivicengagementofoursustainabilityprogramming.
13
KAPIOLANI’SFUTUREDIRECTIONSINASSESSMENTThecollegeispilotinguseoftheInternationalSustainabilityLiteracyTest,orSulitest(seesulitest.org).ThemissionoftheSulitestisto:
[I]mproveSustainabilityLiteracyworldwidebyprovidingcitizensandorganizationswithinternationallyrecognizedandlocallyrelevanttoolstoengagelearningandtocollectmeaningfulindicatorsontheawarenessofsustainabilitychallengesandsolutions.Sharingtheseindicatorsmayallowresearchers,educatorsandotherrelevantstakeholderstotailorpedagogicalapproachesandlearningexperimentstosupportEducationforSustainableDevelopment.(“VisionandMission”)
WeareusingtheSulitestaspartofasuiteofprogram-levelassessmentsofSustainabilityFocusedcourses,comparingresultsfromstudentswhotakethosecoursesversusthosewhodonot.(Thereareapproximately20S-Focusedcoursesatthecollege,whichcanbetakenaspartofanembeddedAcademicSubjectCertificateinSustainability.)TheSALGisusedasapre-andpost-testforstudentreflection;theSALGcanalsobecustomizedbyfacultytogainstudentinputonspecificassignments,texts,orclassactivitiesandhowthoseaffectedtheirlearning.Inaddition,agroupoffivefacultyworkedtogethertodesignandpilotarubricassessmenttoolforfinalprojectsandpapersacrosssustainabilitycourses.ThisrubricisinformedbytheworkofArnimWieketal.Theyestablishedspecificcriteriaforumbrellacompetenciesinsustainability:systemsthinking,anticipatorycompetence,normativecompetence,strategiccompetence,andinterpersonalcompetence.WheretheWiekrubricestablishesNovice,Intermediate,andAdvancedlevelsofdemonstration,wecreatedalevel“below”novicetoreflectthelevelofenteringcommunitycollegestudents.Whiletheanalysisisnotyetcompleted,simplytheprocessofcollaboratingacrossdisciplinesinthedevelopmentoftherubrichashelpedustomoreeffectivelyengagewithstudentsatdifferentlevelsofability,experience,andengagementwithsustainability.Thesedialoguesaboutassessmenthelpustounderstandwhatmatters.
14
WorksCitedFranco,Robert.“TheCivicRoleofCommunityColleges:PreparingStudentsfortheWorkofDemocracy.”TheJournalofPublicAffairs,vol.6,2002,pp.119-138.www.compact.org/wp-content/uploads/resources/downloads/CCivic_role-final.pdf.Hill,YaoZhangetal."Service-Learning'sRoleinAchievingInstitutionalOutcomes:Engagement,Learning,andAchievement."Service-LearningattheAmericanCommunityCollege,editedbyAmyTraverandandZivahKatz,PalgraveMacmillan,2014,pp.169-182.Kuh,George.High-ImpactEducationalPractices:WhatTheyAre,WhoHasAccessToThem,AndWhyTheyMatter.AssociationofAmericanCollegesandUniversities,2008.O’Meara,KerryAnn,andR.EugeneRice,editors.FacultyPrioritiesReconsidered:RewardingMultipleFormsofScholarship.Jossey-Bass,2005.Renner,Tanya.“Convergingdata:DemonstratingServiceLearningEffectivenessOnOurCampus.”ThirdInternationalConferenceonService-LearningResearch,2003,SaltLakeCity,UT.Paper.Seymour,Eliseetal.“CreatingaBetterMousetrap:On-lineStudentAssessmentoftheirLearningGains.”UsingReal-WorldQuestionstoPromoteActiveLearning,NationalMeetingsoftheAmericanChemicalSocietySymposium,27March2000,SanFrancisco.Paper.www.salgsite.org/docs/SALGPaperPresentationAtACS.pdf.Wiek,Arminetal.OperationalisingCompetenciesinHigherEducationforSustainableDevelopment.Routledge,2015.“VisionandMission.”Sulitest.org,2016.www.sulitest.org/en/vision-mission.html.
Related Documents
