IUPUI Assessment 2018 VALUE [Autosaved]Final...Assignment Design Charette What? ØIn small groups, each faculty member shared an assignment and provided and received feedback based

www.aacu.org/OnSolidGroundVALUE

Beyondthe“A”WordAssessmentthatEmpowersFacultytoTakeRiskswithPedagogicalInnovation

October

TheKeyElementsforaCompellingQualityFrameworkAlreadyAreinHand

Ø ConsensusAimsandOutcomes

Ø PracticesthatFosterAchievementANDCompletion

Ø Evidenceon“WhatWorks”forUnderservedStudents

Ø AssessmentsThatRaise– andReveal– theLevelofLearning

VALUEApproachtoAssessment

VALUEInitiativetoDate:

92 institutions submitted 21,189 student work products for assessment by 288 faculty scorers using VALUE rubrics.

VALUERubricApproach- Assumptions

Ø LearningisaprocessthatoccursovertimeØ Studentworkisrepresentationofstudentmotivatedlearning

Ø Focusonwhatstudentdoesintermsofkeydimensionsoflearningoutcomes

Ø FacultyandeducatorexpertjudgmentØ Resultsareusefulandactionableforimprovementoflearning

TheVALUEInitiativeRefinementYearResults(2016-2017)

VALUEProjectmap:TheMulti-State, Minnesota,andGreatLakesCollegesAssociation Collaboratives

Multi-state Collaborative

Multi-state and Minnesota Collaboratives

VALUEInitiativeResultsfortheRefinementYear

• IncludesallInstitutions- PublicandPrivate• 75%Completion– 2-yearinstitutions=45+CreditHours– 4-yearinstitutions=90+CreditHours

VALUEInitiativeResultsfortheRefinementYear75%Completion

• CriticalThinking:5Dimensions– 2-yearinstitutions,45+credithours:1,283Piecesofstudentwork– 4-yearinstitutions,90+credithours:2,006Piecesofstudentwork

• QuantitativeLiteracy:6Dimensions– 2-yearinstitutions,45+credithours:381Piecesofstudentwork– 4-yearinstitutions,90+credithours:748Piecesofstudentwork

• WrittenCommunication:5Dimensions– 2-yearinstitutions,45+credithours:990Piecesofstudentwork– 4-yearinstitutions,90+credithours:2,123Piecesofstudentwork

VALUEInitiativeResultsfortheRefinementYear75%Completion

VALUEInitiativeResultsfortheRefinementYear75%Completion

VALUEInitiativeResultsfortheRefinementYear75%Completion- 0removed

VALUEInitiativeResultsfortheRefinementYear75%Completion

LowestScoringDimensions

• CriticalThinking:Student’sPosition• QuantitativeLiteracy:Calculation– 2-year:Assumptions;Application/Analysis– 4-year:Representation

• WrittenCommunication:Sources/Evidence

VALUEInitiativeResultsfortheRefinementYear75%Completion

• Still75%Completion• Lookedatthreedemographicvariables– PellEligibility– Sex– UnderrepresentedMinority

PellEligibility- 2-yearinstitutions

PellEligibility- 4-yearinstitutions

Gender- 2-yearinstitutions

Gender- 4-yearinstitutions

UnderrepresentedMinority- 2yearinstitutions

Note: Underrepresented includes American Indian or Alaska Native, Pacific Islander, Hispanic, Black, Non-resident alien, and 2 or more races

UnderrepresentedMinority- 4-yearinstitutions

Note: Underrepresented includes American Indian or Alaska Native, Pacific Islander, Hispanic, Black, Non-resident alien, and 2 or more races

InProgress:ValidityProject

• Usedthe25StandardsforTestDesignandDevelopment(ch.4)fromtheStandardsforEducationalandPsychologicalTesting(AERA,APA,NCME,2014)

• Evidenceshowedthedevelopmentoftherubricstobeapurposefulandrigorousprocess

• AAC&Ushouldcreatematerialsrelatedtotestadministrationandscoring(inprogress)

NextSteps…

• Scorerreliabilitystatisticsforrefinementyear• Validityrelatedtoseverallevels:– Developmentoftherubricsasavalidapproachtotestdevelopment– Aligningassignmentswiththerubricstoensurestudentsatleasthavetheopportunitytoaddressalldimensions

– Ensuringtrainingiseffectiveandrefiningcurrentpractices• RubricRevision• VALUEInstitute

Aslowlyevolvingprocess….

2012-132007-10

2012-13

2013-14

MSC!!!2014-15

2010-12

CCSU’sAssessmentofGeneralEducation

Ø GeneralEducation– WantsandNeeds• University-wideresults• Commoncriteria• Facultydriven

• Fiscally-attainableassessmentmodel• Assessmentjustpriortograduation• Effective&sustainable

Ø Multi-StateCollaborative– Campus-WideImplementationofaSimpleModel• UseofexistingCourseassignments• Alignmentwithoneof3VALUErubrics

- CriticalThinking - QuantitativeLiteracy(Reasoning)- WrittenCommunication

• Collectionofartifactsfromseniors(90+credithours)• AssessmentretreatsforCCSUfacultytoscore

CCSUGeneralEducationAssessmentNeeds

Ø Strategy1.ParticipateintheMSC

- NationalNon-CCSUfacultyassess2.Use”MSCmodel”forCampus-Wide

Assessment- LocalCCSUfacultyassess

Ø Results• CCSUscoressimilartoother4-year

institutions(MSCdata)• MSCandCCSUscoressimilar

- Sameartifactscored

InfrastructureforCCSUGeneralEducationAssessment

LearningOutcomes:Written CommunicationQuantitativeLiteracyCriticalThinking

UGStudentMajorsRepresented

FacultyContributingArtifacts

ArtifactsCollected&Scored2x

TotalArtifacts(unduplicated)

CCSUPilotYear(2014-15)andDemonstrationYear(2015-2016)Assessments

54(86%)Assessed

30(50%Dept.)Participating

809Assessed

Comparisonof2014-2015ScoreRanges:CCSUandMSCScoresforsameartifact

Comparisonof2014-2015ScoreRanges:CCSUandMSCscoresforsame artifact Equal +/- 1 +/- 2 +/- 3

Equalorwithin1

QuantitativeReasoningOverall 27% 62% 11% 1% 88%

WrittenCommunicationOverall 21% 63% 15% 1% 84%

Critical ThinkingOverall 23% 58% 17% 3% 80%

STORIESFROMTHEFIELD

AssessmentatInverHillsCommunityCollege:LeadershipandGoals

ØAssessmentSteeringCommittee- ledbyfacultywithsolidadministrativesupport.

ØTwoAssessmentSubcommittees(oneforprogramreviewandoneforcollege-widelearningoutcomes).

ØEachsubcommitteecontainsan“assessmentcoach.”ØGoal1:Collectandusevalidassessmentdatatocontinuallyimprovestudentlearning.

ØGoal2:Promoteacultureofinquiryandassessmentoncampus.ØGoal3:Meettheaccountabilityexpectationsofallstakeholders.

AssignmentDesignCharette

What?Ø Insmallgroups,eachfacultymembersharedanassignmentandprovidedandreceivedfeedbackbasedonspecificquestions.(basedontheworkofLauraGambinoofGuttman CommunityCollege,CUNY)

Why?Ø Promoteagreatercultureofinquiryoncampus.Ø Beginan“assessmentritual”andmakeassessmentfun!Ø Improveassignmentsintermsofclarity,concision,andrelevancetostatedoutcomes.

Ø ProvidefacultyadditionalexperiencewiththeVALUErubrics.ØGenerateartifactsfortheMultistateCollaborativeandtheMNVALUEprojectthatarebetteralignedwiththeVALUErubrics.

FacultyCommentsontheAssignmentDesign Charette

Veryinformative!Greattoworkwithmanyothersfromdifferentareas!Igottobeintwoverydifferentgroups.Peopledidn'twanttoleave...greatconversations.

Willimplement2newmethodsofassessmentoflearningthisspring!

Thebestthingaboutthissessionwasfacultybeingabletogiveandreceivefeedbacktooneanother!

AssessmentSalon

What?ØFourfacultymemberssharedhowtheyareusingassessmentininnovativewaysintheircourses.

Why?ØShareideasforimprovingstudentsuccess.ØPromoteacultureofinquiryandassessmentoncampus.ØBeginan“assessmentritual”andmakeassessmentfun!

FacultyCommentsontheAssessmentSalon

Pleaserepeatinthefutureandallowmoretime.

Greattoshareideasamongfaculty.

Ialwayslearnthemostfromsessionsledbyfellowcolleagues,whethertheyfocusonresearchdonebycolleaguesand/orvariouswaysthatIcanimproveteachingandlearning(likethisone).Iwouldliketoseemoreofthesetypesofsessionsduringprofessionaldays.

TheVALUEInstitute(2017– 2018)

www.aacu.org/VALUEInstitute

[email protected]