www.aacu.org/OnSolidGroundVALUE
Beyondthe“A”WordAssessmentthatEmpowersFacultytoTakeRiskswithPedagogicalInnovation
October
TheKeyElementsforaCompellingQualityFrameworkAlreadyAreinHand
Ø ConsensusAimsandOutcomes
Ø PracticesthatFosterAchievementANDCompletion
Ø Evidenceon“WhatWorks”forUnderservedStudents
Ø AssessmentsThatRaise– andReveal– theLevelofLearning
VALUEApproachtoAssessment
VALUEInitiativetoDate:
92 institutions submitted 21,189 student work products for assessment by 288 faculty scorers using VALUE rubrics.
VALUERubricApproach- Assumptions
Ø LearningisaprocessthatoccursovertimeØ Studentworkisrepresentationofstudentmotivatedlearning
Ø Focusonwhatstudentdoesintermsofkeydimensionsoflearningoutcomes
Ø FacultyandeducatorexpertjudgmentØ Resultsareusefulandactionableforimprovementoflearning
TheVALUEInitiativeRefinementYearResults(2016-2017)
VALUEProjectmap:TheMulti-State, Minnesota,andGreatLakesCollegesAssociation Collaboratives
Multi-state Collaborative
Multi-state and Minnesota Collaboratives
VALUEInitiativeResultsfortheRefinementYear
• IncludesallInstitutions- PublicandPrivate• 75%Completion– 2-yearinstitutions=45+CreditHours– 4-yearinstitutions=90+CreditHours
VALUEInitiativeResultsfortheRefinementYear75%Completion
• CriticalThinking:5Dimensions– 2-yearinstitutions,45+credithours:1,283Piecesofstudentwork– 4-yearinstitutions,90+credithours:2,006Piecesofstudentwork
• QuantitativeLiteracy:6Dimensions– 2-yearinstitutions,45+credithours:381Piecesofstudentwork– 4-yearinstitutions,90+credithours:748Piecesofstudentwork
• WrittenCommunication:5Dimensions– 2-yearinstitutions,45+credithours:990Piecesofstudentwork– 4-yearinstitutions,90+credithours:2,123Piecesofstudentwork
VALUEInitiativeResultsfortheRefinementYear75%Completion
VALUEInitiativeResultsfortheRefinementYear75%Completion
VALUEInitiativeResultsfortheRefinementYear75%Completion- 0removed
VALUEInitiativeResultsfortheRefinementYear75%Completion
LowestScoringDimensions
• CriticalThinking:Student’sPosition• QuantitativeLiteracy:Calculation– 2-year:Assumptions;Application/Analysis– 4-year:Representation
• WrittenCommunication:Sources/Evidence
VALUEInitiativeResultsfortheRefinementYear75%Completion
• Still75%Completion• Lookedatthreedemographicvariables– PellEligibility– Sex– UnderrepresentedMinority
PellEligibility- 2-yearinstitutions
PellEligibility- 4-yearinstitutions
Gender- 2-yearinstitutions
Gender- 4-yearinstitutions
UnderrepresentedMinority- 2yearinstitutions
Note: Underrepresented includes American Indian or Alaska Native, Pacific Islander, Hispanic, Black, Non-resident alien, and 2 or more races
UnderrepresentedMinority- 4-yearinstitutions
Note: Underrepresented includes American Indian or Alaska Native, Pacific Islander, Hispanic, Black, Non-resident alien, and 2 or more races
InProgress:ValidityProject
• Usedthe25StandardsforTestDesignandDevelopment(ch.4)fromtheStandardsforEducationalandPsychologicalTesting(AERA,APA,NCME,2014)
• Evidenceshowedthedevelopmentoftherubricstobeapurposefulandrigorousprocess
• AAC&Ushouldcreatematerialsrelatedtotestadministrationandscoring(inprogress)
NextSteps…
• Scorerreliabilitystatisticsforrefinementyear• Validityrelatedtoseverallevels:– Developmentoftherubricsasavalidapproachtotestdevelopment– Aligningassignmentswiththerubricstoensurestudentsatleasthavetheopportunitytoaddressalldimensions
– Ensuringtrainingiseffectiveandrefiningcurrentpractices• RubricRevision• VALUEInstitute
Aslowlyevolvingprocess….
2012-132007-10
2012-13
2013-14
MSC!!!2014-15
2010-12
CCSU’sAssessmentofGeneralEducation
Ø GeneralEducation– WantsandNeeds• University-wideresults• Commoncriteria• Facultydriven
• Fiscally-attainableassessmentmodel• Assessmentjustpriortograduation• Effective&sustainable
Ø Multi-StateCollaborative– Campus-WideImplementationofaSimpleModel• UseofexistingCourseassignments• Alignmentwithoneof3VALUErubrics
- CriticalThinking - QuantitativeLiteracy(Reasoning)- WrittenCommunication
• Collectionofartifactsfromseniors(90+credithours)• AssessmentretreatsforCCSUfacultytoscore
CCSUGeneralEducationAssessmentNeeds
Ø Strategy1.ParticipateintheMSC
- NationalNon-CCSUfacultyassess2.Use”MSCmodel”forCampus-Wide
Assessment- LocalCCSUfacultyassess
Ø Results• CCSUscoressimilartoother4-year
institutions(MSCdata)• MSCandCCSUscoressimilar
- Sameartifactscored
InfrastructureforCCSUGeneralEducationAssessment
LearningOutcomes:Written CommunicationQuantitativeLiteracyCriticalThinking
UGStudentMajorsRepresented
FacultyContributingArtifacts
ArtifactsCollected&Scored2x
TotalArtifacts(unduplicated)
CCSUPilotYear(2014-15)andDemonstrationYear(2015-2016)Assessments
54(86%)Assessed
30(50%Dept.)Participating
809Assessed
Comparisonof2014-2015ScoreRanges:CCSUandMSCScoresforsameartifact
Comparisonof2014-2015ScoreRanges:CCSUandMSCscoresforsame artifact Equal +/- 1 +/- 2 +/- 3
Equalorwithin1
QuantitativeReasoningOverall 27% 62% 11% 1% 88%
WrittenCommunicationOverall 21% 63% 15% 1% 84%
Critical ThinkingOverall 23% 58% 17% 3% 80%
STORIESFROMTHEFIELD
AssessmentatInverHillsCommunityCollege:LeadershipandGoals
ØAssessmentSteeringCommittee- ledbyfacultywithsolidadministrativesupport.
ØTwoAssessmentSubcommittees(oneforprogramreviewandoneforcollege-widelearningoutcomes).
ØEachsubcommitteecontainsan“assessmentcoach.”ØGoal1:Collectandusevalidassessmentdatatocontinuallyimprovestudentlearning.
ØGoal2:Promoteacultureofinquiryandassessmentoncampus.ØGoal3:Meettheaccountabilityexpectationsofallstakeholders.
AssignmentDesignCharette
What?Ø Insmallgroups,eachfacultymembersharedanassignmentandprovidedandreceivedfeedbackbasedonspecificquestions.(basedontheworkofLauraGambinoofGuttman CommunityCollege,CUNY)
Why?Ø Promoteagreatercultureofinquiryoncampus.Ø Beginan“assessmentritual”andmakeassessmentfun!Ø Improveassignmentsintermsofclarity,concision,andrelevancetostatedoutcomes.
Ø ProvidefacultyadditionalexperiencewiththeVALUErubrics.ØGenerateartifactsfortheMultistateCollaborativeandtheMNVALUEprojectthatarebetteralignedwiththeVALUErubrics.
FacultyCommentsontheAssignmentDesign Charette
Veryinformative!Greattoworkwithmanyothersfromdifferentareas!Igottobeintwoverydifferentgroups.Peopledidn'twanttoleave...greatconversations.
Willimplement2newmethodsofassessmentoflearningthisspring!
Thebestthingaboutthissessionwasfacultybeingabletogiveandreceivefeedbacktooneanother!
AssessmentSalon
What?ØFourfacultymemberssharedhowtheyareusingassessmentininnovativewaysintheircourses.
Why?ØShareideasforimprovingstudentsuccess.ØPromoteacultureofinquiryandassessmentoncampus.ØBeginan“assessmentritual”andmakeassessmentfun!
FacultyCommentsontheAssessmentSalon
Pleaserepeatinthefutureandallowmoretime.
Greattoshareideasamongfaculty.
Ialwayslearnthemostfromsessionsledbyfellowcolleagues,whethertheyfocusonresearchdonebycolleaguesand/orvariouswaysthatIcanimproveteachingandlearning(likethisone).Iwouldliketoseemoreofthesetypesofsessionsduringprofessionaldays.
TheVALUEInstitute(2017– 2018)
www.aacu.org/VALUEInstitute