Our Quest: Looking for answers we need without a process we’ll hate Assessment Design in the Geology Department at the University of Dayton.

Our Quest: Our Quest: Looking for answers we Looking for answers we needneed without a process we’ll without a process we’ll hatehate

Assessment Design in the Geology Assessment Design in the Geology Department at the University of Department at the University of

DaytonDayton

Why Assess?Why Assess?

Self-evaluate performance in order to Improve Self-evaluate performance in order to Improve educational outcomes for our students.educational outcomes for our students.

Facilitate communication of strengths and Facilitate communication of strengths and needs to internal and external audiences needs to internal and external audiences (Upper Level administration, alumni, funding (Upper Level administration, alumni, funding agencies, community partners)agencies, community partners)

Evaluate alignment with institutional priorities.Evaluate alignment with institutional priorities. Because Because “they”“they” (Administrators, Accreditors) (Administrators, Accreditors)

tell us we have to. tell us we have to.

Balancing Assessment with Balancing Assessment with ImprovementImprovement

Does time & effort expended in assessing Does time & effort expended in assessing outweigh benefits of comparable time & outweigh benefits of comparable time & effort expended in improvement? (But how effort expended in improvement? (But how do you know unless you assess?)do you know unless you assess?)

Does Law of Diminishing Returns apply to Does Law of Diminishing Returns apply to Assessment?Assessment?

At UD: one chair + one admin At UD: one chair + one admin We have to We have to look for “bang for buck.” What strategies look for “bang for buck.” What strategies yield the most valuable information with the yield the most valuable information with the minimum expenditure of time & energy? minimum expenditure of time & energy?

Assessment is a means, not an endAssessment is a means, not an end

Character & Mission of UDCharacter & Mission of UD

Founded by Society of Mary Founded by Society of Mary (Marianists) in 1850 (Marianists) in 1850

““A leader in American Catholic A leader in American Catholic Education”Education”

Key values: Community, inclusion, service, practical education, faith formation, strategic adaptation (“reading the signs of the times”)

Comprehensive, urban Comprehensive, urban Largest private University in Largest private University in

OhioOhio

Fr. William Joseph Chaminade, founder

of the Marianist order.

Character & Mission of Character & Mission of Geology DepartmentGeology Department

11stst Majors graduated in 1949 Majors graduated in 1949 Historical focus on undergraduate education, Historical focus on undergraduate education,

graduating 6-12 a yeargraduating 6-12 a year New grad certificate program in GISNew grad certificate program in GIS Preparation for graduate school (70% of Preparation for graduate school (70% of

students), but also for employment.students), but also for employment. Strong focus on getting students into field Strong focus on getting students into field

early early Partnering students with faculty mentors Partnering students with faculty mentors

early on in early on in developmental modeldevelopmental model.. Collaborative student-faculty research.Collaborative student-faculty research. Marianist character expressed through a Marianist character expressed through a

service orientation.service orientation.

Old Assessment FrameworkOld Assessment Framework Over-reliance on vague, qualitative Over-reliance on vague, qualitative

statements from subjective exit surveystatements from subjective exit survey Never fully implementedNever fully implemented Never systematically reported back to Never systematically reported back to

faculty!faculty! Faculty-driven “Competency-based” Faculty-driven “Competency-based”

assessment never implemented at all.assessment never implemented at all. The Good news: It shouldn’t be hard to The Good news: It shouldn’t be hard to

improve!improve!

New Assessment Framework at UDNew Assessment Framework at UDHabits of Inquiry and ReflectionHabits of Inquiry and Reflection

ScholarshipScholarship Faith TraditionsFaith Traditions CommunityCommunity DiversityDiversity Practical WisdomPractical Wisdom Critical Critical

Evaluation of the Evaluation of the timestimes

VocationVocationDepartments tasked to define own outcomes, at least some of which must

map to HIR

Measures We’ve ConsideredMeasures We’ve Considered Survey-based MeasuresSurvey-based Measures

Graduating Students.Graduating Students. Alumni three years after graduation.Alumni three years after graduation. Survey of Employers and/or Graduate Advisors of Alumni (?)Survey of Employers and/or Graduate Advisors of Alumni (?)

Evaluative MeasuresEvaluative Measures Competency-based measures derived from classroom Competency-based measures derived from classroom

evaluations, possible portfolio requirement.evaluations, possible portfolio requirement. Comprehensive 2nd Year Review.Comprehensive 2nd Year Review.

CountingCounting MeasuresMeasures Proportion of graduates going onto grad study, Proportion of graduates going onto grad study, Proportion of students completing thesis, presenting at UD Proportion of students completing thesis, presenting at UD

Research Symposium and/or Professional Meetings, Research Symposium and/or Professional Meetings, participating in summer research or internships.participating in summer research or internships.

Proportion of students participating in service projects or Proportion of students participating in service projects or team-learningteam-learning

BoardBoard ofof VisitorsVisitors How frequently? (10 yrs?) How frequently? (10 yrs?) Who? Alumni, Academics, Community Partners (esp. for new Who? Alumni, Academics, Community Partners (esp. for new

GIS program)GIS program)

Competency-based AssessmentCompetency-based Assessment

Based on ~14 broad competencies expected of Based on ~14 broad competencies expected of graduating Geology majorsgraduating Geology majors

Faculty would use rubric to assess individual Faculty would use rubric to assess individual majors at time of grade submission and submit majors at time of grade submission and submit to Dept. office which would track each student.to Dept. office which would track each student.

22ndnd Year Review would be instituted to identify Year Review would be instituted to identify student progress and needs for add’l assistance student progress and needs for add’l assistance to attain competencies (Developmental to attain competencies (Developmental Approach)Approach)

Possibility of portfolio-based assessment tied to Possibility of portfolio-based assessment tied to competencies, or a 2competencies, or a 2ndnd year comprehensive year comprehensive ‘qualifying’ exam (?)‘qualifying’ exam (?)

Information LiteracyInformation Literacy Quantitative Reasoning and InquiryQuantitative Reasoning and Inquiry Communication (Reading/Writing/Oral)Communication (Reading/Writing/Oral) Living in CommunityLiving in Community Each broad category subdivided into Each broad category subdivided into

component and subcomponent skill setscomponent and subcomponent skill sets Top three Categories map mostly to Top three Categories map mostly to

Scholarship & Practical Wisdom; “Living Scholarship & Practical Wisdom; “Living in Community” accents Marianist in Community” accents Marianist identity.identity.

Competency CategoriesCompetency Categories

1. Students will understand the structure, form, 1. Students will understand the structure, form, and access methods of recorded information and and access methods of recorded information and develop effective strategies to use information develop effective strategies to use information technologies to seek knowledge. technologies to seek knowledge.

2. Students will demonstrate the ability to 2. Students will demonstrate the ability to evaluate and analyze information gathered from evaluate and analyze information gathered from a variety of sources.a variety of sources.

3. Students will use information and information 3. Students will use information and information technology responsibly and ethically.technology responsibly and ethically.

Information LiteracyInformation Literacy

Students will demonstrate an ability to understand Students will demonstrate an ability to understand geological processes in space and time.geological processes in space and time.

Students will meet the university’s quantitative Students will meet the university’s quantitative competency standards by completing the basic competency standards by completing the basic math sequence through at least MTH149: Calculus II math sequence through at least MTH149: Calculus II

Students will demonstrate the capacity to plan and Students will demonstrate the capacity to plan and design protocols, experiments or procedures to design protocols, experiments or procedures to address scientific problems or societal needs. address scientific problems or societal needs.

Students will apply technical skills and analytical Students will apply technical skills and analytical methods to collect, record and interpret Earth methods to collect, record and interpret Earth science data in the field, lab and/or office.science data in the field, lab and/or office.

Quantitative Reasoning & InquiryQuantitative Reasoning & Inquiry

Students will develop scientific arguments Students will develop scientific arguments through the diverse modes of written, graphical through the diverse modes of written, graphical and/or spoken scientific communication. and/or spoken scientific communication.

Students will work effectively in small groups, Students will work effectively in small groups, teamwork situations or general discussions, teamwork situations or general discussions, meeting basic standards of respectful discourse. meeting basic standards of respectful discourse.

Students will demonstrate basic professional Students will demonstrate basic professional skills. skills.

Communication Communication Reading, Writing, OralReading, Writing, Oral

Students will be able to articulate an understanding Students will be able to articulate an understanding of the vocation of earth science and will demonstrate of the vocation of earth science and will demonstrate a commitment to the scientific and broader a commitment to the scientific and broader community through service. community through service.

Students will understand the evolving challenges of Students will understand the evolving challenges of our times pertaining to the sustainable habitation of our times pertaining to the sustainable habitation of humans on planet Earth. humans on planet Earth.

Students will demonstrate an appreciative Students will demonstrate an appreciative understanding of the value of diversity and inclusion understanding of the value of diversity and inclusion in the scientific community. in the scientific community.

Students will be exposed to the diverse ways that Students will be exposed to the diverse ways that earth processes and human culture and human earth processes and human culture and human vulnerability interact. vulnerability interact.

Living in CommunityLiving in Community

The Quest Continues . . .The Quest Continues . . .The Quest ContinuesThe Quest Continues

Additional Discussion & pruning or elaboration Additional Discussion & pruning or elaboration of competencies. They are skills-based; should of competencies. They are skills-based; should we also include some knowledge-based we also include some knowledge-based outcomes?outcomes?

Revisit Exit Survey; Design of Alumni surveyRevisit Exit Survey; Design of Alumni survey Establish a system of collection for “counting Establish a system of collection for “counting

measures.”measures.” Establish an Assessment ScheduleEstablish an Assessment Schedule Implementation! – We are tasked to begin Implementation! – We are tasked to begin

collecting data on at least one outcome this collecting data on at least one outcome this year!!year!!

Establish System for annual engagement of Establish System for annual engagement of faculty as well as generating reports to upper faculty as well as generating reports to upper administration.administration.