-
1
Assessment and Learning Resource Repository
ACSC Crawford Hall 230 & 231
2550 Campus Rd, Honolulu, HI 96822
manoa.hawaii.edu/assessment
Conducting Assessment to Support Teaching and Scholarship
Hill, Y. (2020, September 30). Conducting assessment to support
teaching and scholarship [Online workshop]. Assessment and
Curriculum Support Center and Center for Teaching Excellence.
University of Hawai‘i at Mānoa, Honolulu, HI, United States.
https://manoa.hawaii.edu/assessment
-
Conducting Assessment to
Support Teaching and Scholarship
A workshop for all instructional faculty/staff
Yao Z. Hill, Associate Specialist
PresenterPresentation NotesCONDUCTING LEARNING ASSESSMENT TO
SUPPORT TEACHING AND SCHOLARSHIPWe invite all teaching faculty and
staff, especially recently hired faculty, lecturers, & graduate
teaching assistants to this interactive workshop. Come and explore
powerful ways for using program learning assessment tools to
enhance teaching, collaborate with colleagues on curriculum matters
that impact all students, and contribute to the program quality and
coherence.��Participants will:Reflect on ways to use student
learning outcomes, the program curriculum map, signature
assignment, and rubrics to enhance teaching.Learn to reflect on
their assessment efforts and effective ways to describe it in their
contract renewal and tenure and promotion documents.Explore
scholarship in teaching, learning, and assessment through the
national Assignment Library and Learning Improvement
projects.��This workshop is co-sponsored by the Center for Teaching
Excellence and Assessment and Curriculum Support Center.
DATEWednesday, September 30, 2020TIME2:00 PM - 3:30
PMLOCATIONONLINE (Zoom)PRESENTED BY:Yao Zhang Hill, Assessment and
Curriculum Support Center
-
Housekeeping
• Access your individualized handout here:
http://go.hawaii.edu/3c3
PresenterPresentation
Noteshttps://drive.google.com/drive/u/0/folders/1JlHQsXrppVzh1jW39ZWKY6GhOmcVH58A
-
Session outcome
Plan 1 strategy to use program learning assessment tools &
processes to support teaching and scholarship
Student learning
outcomes
Curriculum map
Signature assignment Rubric
-
Assessment is an integral part of teaching
PresenterPresentation NotesThroughout the workshop, I want to
convey that assessment is an integral part of teaching.
-
Activity 1:
• Skim the sample narrative.• Highlight program learning
assessment tools/processes• Describe your impression
4 min individual + 6 min breakoutgo.hawaii.edu/3c3
PresenterPresentation Notes10 min activity: First, read the
sample narrative designed to be included in one’s tenure promotion
dossier. Identify the assessment tools and processes that this
faculty used in his/her reflective teaching practice, without me
telling you what these tools/processes mean. You can underline
keywords or use markers to highlight.Participants contribute and
call out answers.Can you describe your holistic impression of this
colleague? How does this narrative make you feel about this
colleague?
-
Program Learning Outcomes (PLO) Assessment
An on-going process designed to monitor and improve student
learning. Faculty: a) develop explicit statements of what students
should learn (i.e., student learning outcomes); b) verify that the
program is designed to foster this learning (alignment); c) collect
data/evidence that indicate student attainment (assessment
results); d) use these data to improve student learning (close the
loop). (Allen, M., 2008)
PresenterPresentation NotesSeeing how a colleague can use
program learning assessment to support his/her reflective teaching,
let me start with defining program learning outcomes. In this
definition of program learning outcome assessment, we can see that
the ultimate purpose is to improve student learning. The action
toward the improvement is informed by making our expectations clear
through student learning outcome statements, aligning learning
opportunities with the outcomes, collecting the evaluating learning
evidence, and using the data.
-
StudentLearning
Outcomes
LearningOpportunities
Collect & Evaluate Evidence
Interpret Results
Use Results
Assessment cycle
PresenterPresentation NotesThis diagram reflects the typical
steps that a program would go through in program learning outcome
assessment. The process provides the logic and the language to talk
about how a program is accountable for its educational quality. The
cycle starts with the program faculty articulating expectations of
student learning outcomes in terms of knowledge, skills, and
dispositions that they expect of a graduating senior; examining
whether there are sufficient learning opportunities for students to
meet faculty expectations; collecting & evaluating evidence of
student learning, and interpreting and using the results.
-
StudentLearning
Outcomes
LearningOpportunities
Collect & Evaluate Evidence
Interpret Results
Use Results
Assessment cycle
PresenterPresentation NotesStudent learning outcomes is the
first powerful reflection tool that I want to introduce.
-
SLO Definition: Action-oriented statement of…
knowledge skills values
PresenterPresentation NotesAction-oriented statement of the
knowledge, skills, and/or dispositions students are expected to
know, to be able to do or to value upon successful program
completion.
-
Domains of Learning
Cognitive Affective Psychomotor
PresenterPresentation NotesThere are three major learning
domains: cognitive domain, addressing intellectual knowledge and
skills; affective domain addressing values and attitudes, about how
students feel; and psychomotor domain, about student skills using
their body and hand movement. Let’s take a look at some
examples.
-
Example SLOs
Cognitive
• Students can explain major theories in the field of study.
• Students can critically evaluate and synthesize multiple
sources of evidence.
• Students can design and conduct undergraduate
researchprojects.
PresenterPresentation NotesCognitively, students can explain
theories, evaluate and synthesize evidence, and design and conduct
research.
-
Example SLOs
• Students can skillfully explore self-initiated projects on
XXX.
• Students seek out indigenousperspectives when addressing
community challenges.
• Students be able to advocatesustainable living principles to
the community.
Affective
PresenterPresentation NotesAffectively, students can explore
self-initiated projects, seek out perspectives, and advocate
sustainable living principles.
-
Example SLOs
Students can• Embody structural dance
movements• Play music instruments• Operate equipment
Psychomotor
PresenterPresentation NotesIn terms of psychomotor skills using
their body and especially their hands, students can embody dance
movements, play instruments like a piano, and operate
equipment.
-
Using program SLOs to support teaching—An example
PresenterPresentation NotesFor many instructional faculty,
knowing their program learning outcomes is very illuminative and
allows them to rethink their own course(s). Profound changes can
happen just by aligning one’s own course with the program outcomes.
I once taught Language Testing in Second Language Studies and I can
demonstrate how the alignment process can help instructors like me
to refine courses.
-
Make the alignment clearer
Course SLO:conduct item analysis
Program SLO 1Critical thinking
Evaluate item quality & usefulness through content &
descriptive statistical analysis
PresenterPresentation NotesIn my course, I had a course SLO that
students will be able to conduct item analysis. My task for
students is to calculate item statistics such as item difficulty in
Excel and report the results in Word. It involves the skill of
using Excel but it doesn’t necessarily address critical thinking.
To make my course more aligned with critical thinking, I could
rephrase my course outcome this way: Evaluate… through…analysis.
Now I make it clear that students have to make a judgement of the
quality and usefulness of test items. To me, that is clearly
aligned with critical thinking.
-
Address program SLOs not emphasized before
Program SLO 5: Develop and apply sound framework in
assessment
New Course SLODiagnose test items & improve a test based on
item
analysis results.
PresenterPresentation NotesI also found that Program SLO 5
speaks to applying framework in assessment, and my course is the
only course that deals with assessment. This makes me feel a great
responsibility to make sure that application happens in my course.
After this consideration, I further modified my Course SLO to
“Diagnose test items & improve a test based on item analysis
results.” This means my students not only have to analyze the test
items and make judgement on their quality, but they also have to
improve the items that show poor quality.
-
Assignment redesign to align with new SLO
Item Analysis Report
• Excel calculations• Item diagnosis• Recommendation for
revision
PresenterPresentation NotesIn seeing whether students have
achieved the learning outcomes, I realized my previous assignment
in which students do calculations in Excel was not good enough. So,
I had to modify the assignment to make it better aligned with the
new Course SLO.
-
Scaffold learning experience
• Revise instructions• Tips sheet• Annotated sample
reports• Rubrics
Photo by Bernard Hermant on Unsplash
PresenterPresentation NotesThe first time I gave out the
assignment without detailed instructions, the assignments turned
back to me varied in quality. Each time that I taught the course, I
identified where students struggled and designed scaffolding
activities/materials to help students turn in high quality
assignments.
https://unsplash.com/@bernardhermant?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyTexthttps://unsplash.com/s/photos/scaffold?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText
-
Use programs SLOs to support teaching• Align course SLOs with
program expectations• address important knowledge/skill not
emphasized before• refine assignments/tasks to clearly align with
the SLOs• provide necessary scaffolding learning experience• help
students connect their learning by communicating the
program SLOs that you address.
-
Activity 21. Locate your program SLOs2. Plan a strategy to use
program SLOs to support your teaching (handout)
PresenterPresentation Notes5 min
-
Where to find program SLOs
• Departmental Website• Assessment Office Website:
https://manoa.hawaii.edu/assessment/update2/view.php
-
Learning Outcomes
LearningOpportunities
Collect & Evaluate Evidence
Interpret Results
Use Results
Assessment cycle
-
Curriculum map:learning opportunities program SLOs
Courses SLO 1Knowledge
SLO 2Critical
thinking
SLO 3Research
SLO 4Written Comm.
SLO 5 Ethical
Reasoning
Course 101 X X X X
Course 102 X X X
Course 201 X X
Course 202 X X X
Mandatory advising X X
Capstone X X X X X
PresenterPresentation NotesIn program assessment, the curriculum
map is another powerful tool that can engage thoughtful reflection
on program coherence. The map makes the learning opportunities very
clear under each program SLO.
Imagine that I am the student in this course. Which outcome
would I likely fall short of in achievement when I graduate?
-
Curriculum map:learning opportunities program SLOs
Courses SLO 1Knowledge
SLO 2Critical
thinking
SLO 3Research
SLO 4Written Comm.
SLO 5 Ethical
Reasoning
Course 101 X X X X
Course 102 X X X
Course 201 X X
Course 202 X X X
Mandatory advising X X
Capstone X X X X X
?
PresenterPresentation NotesEthical Reasoning! It only appeared
in the capstone course, and capstone is a place for students to
organize, synthesize, and reflect to achieve a higher level of
learning. It is not a good place to introduce a new SLO. Clearly
there is a gap in the curriculum. As individual instructors, we can
think about how to fill that gap through assignments and
activities. We can also think about how to collaborate with
colleagues to collaboratively design learning activities to help
fulfill the gap.
-
Curriculum map:learning opportunities program SLOs
Courses SLO 1Knowledge
SLO 2Critical
thinking
SLO 3Research
SLO 4Written Comm.
SLO 5 Ethical
Reasoning
Course 101 X X X X
Course 102 X X X
Course 201 X X
Course 202 X X X
Mandatory advising X X
Capstone X X X X X
PresenterPresentation NotesNow, let’s take a look at SLO 2:
Critical thinking. Every course claims to address this SLO. How can
we make sure that we are not overlapping with each other and we
don’t create a big jump in student learning experience from one
course to another, so that there can be an implicit gap in the
curriculum?
-
Your course
Prior course
Subsequent course
Major theoriesResearch writingCritical thinking
PresenterPresentation NotesLook at the curriculum from another
angle. We are expecting students to gain a high level of complex
skills upon program completion.
Let’s say that you are teaching a course right in the middle of
the curriculum, it can be a 200- or 300-level course. In your
course, you require students to have gained certain foundational
knowledge or skills to be successful in your course. In almost
every discipline, if we teach middle or upper level courses, we
have assumptions of students’ prior knowledge or skills in the
discipline.
Did you talk to instructors teaching the lower level courses and
ask them what you should expect students to know or be able to do
before entering your class? Have you made your assumption of prior
knowledge clear to your students? Did you investigate what prior
knowledge or skills the students brought in? Did you try to provide
support for students to make up the knowledge/skills that were
needed before entering the class? For example, asking them to
review certain topics before coming into the class or having a
review class?
-
Use curriculum map to support teaching• identify the curriculum
gap and collaborate with
colleagues to fill it• collaboratively design learning
tasks/activities to
scaffold learning throughout the curriculum• collegially
communicate one’s expectations of
students’ prior knowledge/skills• find out colleagues’
expectations of student
knowledge/skills existing in one’s course
-
Activity 3
Locate your program’s curriculum map
Select a strategy to use the curriculum map to support your
teaching
Share in small groups.
PresenterPresentation Notes3 minutes. Share for 6 minutes
-
Breakout Room Task: Think of one strategy to implement and share
with a colleague (6 min)
-
Curriculum map
PresenterPresentation NotesCurriculum map reviews the vertical
alignment of the curriculum.
-
Signature Assignment
PresenterPresentation NotesSignature assignment helps with
horizontal alignment of the curriculum
-
Signature Assignment
A generic task, problem, case, or project that can be tailored
or contextualized in different disciplines or course contexts.
(Driscoll, 2016, p. 11).
PresenterPresentation NotesDriscoll, A. (2016, February).
Aligning direct and indirect assessment strategies with outcomes.
In Assessment 201: Advanced topics in assessment Resource Binder,
p.10-28. WASC Senior College and University Commission Educational
Programs. Chaminade University, Honolulu, HI.
-
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Eng 100
Sections of the same course
PresenterPresentation NotesThis semester, we have 47 sections of
English 100. How can we ensure comparable learning experiences
across all sections?
-
Sample Signature Assignment for Eng 100• Write 3-5 pages• Argue
for a position• Include 3 sources• Specify audience• Genres to
consider:
• Academic essay for general public• A letter to GSO• A white
paper to legislature • A transcript of a talk or media product (pod
cast)
-
Quan Survey Qual
Similar types of courses
PresenterPresentation NotesSay you are teaching graduate level
courses and in your program there is only one research method
course being required. Should students have comparable opportunity
to practice using those methods? If one course is giving out a
multiple-choice exam, and another asks students to conduct research
and write up the research report, the students in the latter class
may gain more solid learning experience and retain the skill
better.
-
Sample Research Proposal signature assignment• Students write a
5-8 page research proposal that
include• Research topic• Justification of methods selection•
Description of methods• Proposed analysis procedure
-
ChineseOC
Oral Communication (OC) designated course
Political Science
OC
EngineeringOC
PresenterPresentation NotesWe have a lot of General Education
requirements. How are we making sure that our Gen Ed requirements
are helping students achieve what we want them to gain? How can we
promote cross discipline conversations about oral communication?
What are the essential skills that we all want our students to be
able to demonstrate? How do we development signature assignments to
allow students to demonstrate those skills?
-
UH Mānoa Oral Presentation Signature AssignmentTask: Students
will prepare and deliver a formal oral presentation. Length: 5-20
minutesPurpose: persuade or informAudience: Specify intended
audienceSupporting material: Utilize appropriate supporting
materials
Oral Communication Teaching and Assessment resource for
facultyhttps://manoa.hawaii.edu/assessment/reports/gened/oral-communication/oc-resources/
PresenterPresentation NotesUH Mānoa has a signature assignment
on oral presentation.
-
Signature assignment development
SLOs
Signature Assignment
EvaluationCriteria
PresenterPresentation NotesSignature assignments need to be
developed with the learning outcomes in mind. If your outcome
includes critical thinking, your signature assignment will ask
students to argue, to evaluate, to analyze, rather than to define,
explain, and describe. Once you have the signature assignment, the
next step is to develop evaluation criteria. The criteria, in turn,
need to align and address the target learning outcomes.
-
Use signature assignment to support teaching• Collaborate with
colleagues for shared learning expectations
• Refine assignments to align with the shared expectations
• Collaborate with colleagues to design a signature
assignment
• Collaboratively brainstorm ways to enhance assignment and
teaching
PresenterPresentation NotesCollaborate with colleagues teaching
the same or similar courses to form shared learning
expectationsRefine the assignment to align with the shared
expectationsCollaborate with colleagues to design a signature
assignmentDiscuss student performance with fellow colleagues and
collaboratively brainstorm ways to enhance assignment and
teaching
-
Activity 4
Select a strategy to use a signature assignment to support your
teaching.
Plan one activity to implement.
PresenterPresentation Notes3 minutes
-
Learning Outcomes
LearningOpportunities
Collect & Evaluate Evidence
Interpret Results
Use Results
Assessment cycle
Signature Assignment
PresenterPresentation NotesOnce a signature assignment is
developed, it can serve both as learning opportunities and as
learning evidence. Using signature assignments can make assessment
efficient and promote faculty conversation and collaboration.
-
Learning Outcomes
LearningOpportunities
Collect & Evaluate Evidence
Interpret Results
Use Results
Assessment cycle
PresenterPresentation NotesOnce you have collaborated with your
colleagues either in your department or faculty teaching the same
Gen Ed designated courses to design a signature assignment, you
will need to think about how to evaluate students’ work. Very
often, we ask students to do complex performance tasks that involve
students integrating knowledge and skills at multiple dimensions.
We need an evaluation tool that can capture that complexity. A
rubric is a commonly used evaluation tool.
-
Rubric
A rubric is a scoring guide that describes the criteria that
faculty use to evaluate student performance, understanding, or
behavior.
PresenterPresentation NotesFirst thing first, let’s take a look
at the definition of the rubric.
-
VALUE Oral Presentation Rubric
Organization
Language
Delivery
Supporting Material
Central message
https://drive.google.com/file/d/1MDJRFjFrPXOw8TvFxq8RHF7tMB02hOOo/view
Capstone4
Milestone 3
Milestone 2
Benchmark 1
PresenterPresentation NotesIt is often shaped like a matrix,
which describes levels of achievement in a specific area of
performance, understanding, or behavior.The words in red boxes
represent evaluation dimensions or evaluation criteria. The words
in the green boxes represent different levels of performance. We
call them performance scales.The words in each square are called
performance descriptors.
https://drive.google.com/file/d/1MDJRFjFrPXOw8TvFxq8RHF7tMB02hOOo/view
-
How can this rubric be revised?
Rubric Categories1. Grammar2. Word choice3. Typos4. Content
organization
Program Outcome:Apply critical thinking and rhetorical skills to
produce coherent written works
Need to also evaluate critical thinking
PresenterPresentation NotesWhat you see are rubric criteria for
a writing task in a foreign language course. What is missing?
-
Collaboratively decide on the criteria, performance description,
& definition of satisfactory performance
PresenterPresentation NotesThe best way to develop a rubric is
to ask your colleagues what they think is important. Ask them to
give examples and provide descriptions.
-
Adapt an existing rubric
https://www.aacu.org/value-rubrics
https://www.aacu.org/value-rubricshttps://www.aacu.org/value-rubrics
-
Adapt an existing rubric
https://www.aacu.org/value-rubrics
https://www.aacu.org/value-rubrics
-
Adapt an existing rubric
http://manoa.hawaii.edu/assessment/resources/rubricbank.htm
http://manoa.hawaii.edu/assessment/resources/rubricbank.htm
-
Data dashboard template: http://go.hawaii.edu/AZW
PresenterPresentation NotesHow to analyze rubric results?
Present the percentage of the students who met or exceeded your
expectations. In this chart, the dark green bar represents the
percentage of the students who met the standards on the final
presentation and the light green bar showed the percentage of
students who met the expectations on the presentation rehearsal. We
can observe that students made a big improvement from the rehearsal
to the final presentation, but still only 60% of the students met
our expectations on delivery.
http://go.hawaii.edu/AZW
-
Use rubrics to support teaching
• align the rubric with program SLOs• collaboratively decide
what is important to evaluate
with colleagues• collaboratively define satisfactory
performance• collaboratively evaluate and refine the rubric
-
Activity 5
Select a strategy to use a rubric to support your teaching.
Plan one activity to implement.
PresenterPresentation Notes3 minutes
-
Learning Outcomes
LearningOpportunities
Collect & Evaluate Evidence
Interpret Results
Use Results
Assessment cycle
PresenterPresentation NotesResults from student performance on a
well-designed assignment and rubric are likely to generate
meaningful discussion about program curriculum quality and promote
the use of the results.
-
Breakout Room Task:Pick one signature assignment/rubric activity
that you plan to try. Share and give feedback to a colleague.
PresenterPresentation Notes10 minutes
-
Scholarship Opportunities
PresenterPresentation NotesExample of results and samples of use
of results. We see more and more professional organizations value
scholarship of teaching and learning. They are supporting this more
and more. You can look into your professional organization
conferences to publish on teaching and learning in addition to your
subject area.
-
Publish your assignment—locally
Assessment and Learning Resource Repository
http://go.hawaii.edu/ATb
PresenterPresentation NotesIf you want to start small, share
your work with the campus first. Our office is building an
Assessment and Learning Resource Repository. We welcome assignment
examples, rubrics, and any other resources that you think can be
useful for your colleagues. Your work remains your intellectual
property. However, it will be assigned a creative commons license.
This means that when other people use your work, they have to cite
you, but they can adapt for their own use without your permission.
It is one step toward open educational resources while still
acknowledging your intellectual contribution.
https://drive.google.com/drive/folders/16f1dxiQgU7GEfDUold9lM0dnQR04rUODhttp://go.hawaii.edu/ATb
-
Publish your assignment—nationally
https://www.learningoutcomesassessment.org/ourwork/assignment-library/Email:
[email protected]
NILOA
PresenterPresentation NotesAt the next level, consider
submitting to the national Assignment Library, for the opportunity
to receive peer feedback from experts in your field beyond UH and
make your work known globally. This is a project organized by the
National Institute for Learning Outcomes Assessment, or NILOA.
National leaders of learning assessment have long recognized the
important function that excellent assignments play in quality
assessment.
There is more work involved when publishing your assignment on
this forum. You will need to clearly state your learning outcomes,
have a well-developed set of evaluation criteria, describe how
you’ve scaffolded student learning, the assessment results, and
your reflections on how the assignment went. But once you have done
that, and this workshop is designed to prepare you for doing that,
do consider contributing to this forum. Make our work known.
Elevate UH’s reputation in quality education at the national level.
I have seen many assignments from CUNY: City University of New
York. It would be nice when I offer this workshop in the future, I
can use examples from you.
NILOA logo image source:
https://www.learningoutcomesassessment.org/wp-content/uploads/2019/05/NILOA2019Logonowords-120x120.png
https://www.learningoutcomesassessment.org/ourwork/assignment-library/https://www.learningoutcomesassessment.org/ourwork/assignment-library/mailto:[email protected]
-
https://www.learning-improvement.org/
Tell your story of learning improvement
Assess Change Re-Assess
Fulcher et al. (2004)
PresenterPresentation NotesAnd if you want to even further
pursue the scholarship in program level teaching, learning, and
assessment, the Learning Improvement Community is a
faculty-friendly space for you to share how you collaborated with
colleagues to assess student learning, using your assignment, made
adjustment to pedagogy collectively with your colleagues, and
re-assessed student learning and showed improvement. Our office is
here to support you in making your story happen and getting it
published.Scale image: Photo by Theme Inn on UnsplashPig image:
Photo by Christopher Carson on Unsplash
Fulcher et. al. (2014). A simple model for learning improvement:
Weigh pig, feed pig, weigh pig:
https://www.learningoutcomesassessment.org/documents/Occasional_Paper_23.pdf
https://www.learning-improvement.org/https://www.learningoutcomesassessment.org/documents/Occasional_Paper_23.pdf
-
Disciplinary Conferences
-
Assessment & SoTL Conferences
LocalHawaii Educational Research Association Annual Conference
(HERA) (Jan/Feb) Pacific Association for Institutional Research
(PacAIR)Hawai’i-Pacific Evaluation Association (HPEA)NationalAALHE
ConferenceIUPUI Assessment InstituteAnnual Scholarship on Teacher
and Learning Conference List
http://www.hawaii.edu/hera/conferenceinfo.htmhttp://www.pacairweb.org/https://h-pea.org/https://www.aalhe.org/http://assessmentinstitute.iupui.edu/http://www.washington.edu/teaching/sotl-annual-conferences/
-
Thank you!
Yao [email protected]
Q & AEvaluate the session here:
https://forms.gle/8wCzw4SEGXj5Ssvi7
mailto:[email protected]://forms.gle/8wCzw4SEGXj5Ssvi7
-
Assessment & Curriculum Support Center University of Hawai‘i
at Mānoa Crawford Hall 230 & 231 2550 Campus Road, Honolulu, HI
96822
[email protected] (808) 956-4283 (808) 956-6669
manoa.hawaii.edu/assessment
1
Conducting Learning Assessment to Support Teaching and
Scholarship
Dr. Yao Z. Hill ([email protected])
Activity 1: Articulate Integrating Assessment in Teaching
Practice Activity Instruction: The following narrative represents
sample language that is intended to be included in the tenure and
promotion dossier.
1. Skim the narrative and highlight learning assessment
tools/processes mentioned.
2. Write down your impression of this colleague.
My approach to teaching the course Research Ethics transformed
after I learned that this course is the only course in the BS
program that substantially addresses the ethics-related program
student learning outcome (SLO), which is: “Students will function
professionally and ethically when conducting research.” Since this
is a senior level course, students’ performance in this course no
longer just represents their achievement in my course but also as a
graduating senior from this degree program. With this realization,
I asked myself several questions:
1. Are my expectations of the students the same as the program
expectations? What does “function ethically” entail to other
faculty and to the program as a whole?
2. If another faculty teaches this course, would students learn
the same or differently? What should be the essential learning
experience and what kind of evidence can show that students have
achieved the program expectations?
3. Is this one course enough? Where else can this be addressed
in our program curriculum? Who else might be interested in
addressing ethical deliberation in their courses? How can I
collaborate with them?
In the process of seeking the answers to these questions, I
joined the Curriculum Committee, collected input from faculty
colleagues, and revised my course SLOs to explicitly reflect
program expectations of “function ethically.” For example, SLO1:
“Students will be able to follow the IRB process when conducting
their research;” SLO2: “Students will be able to perform ethical
deliberation in every step of the research process that reflects
ethical standards of the field;” and so on. Working with
colleagues, I led and coordinated the development of rubrics to
assess
mailto:[email protected]
-
Assessment & Curriculum Support Center University of Hawai‘i
at Mānoa Crawford Hall 230 & 231 2550 Campus Road, Honolulu, HI
96822
[email protected] (808) 956-4283 (808) 956-6669
manoa.hawaii.edu/assessment
2
the outcomes achievement. I provided students both the SLOs and
the rubrics right at the beginning of the course so that students
clearly know the program expectations and the path to progress
toward mastery. In addition, with the committee’s feedback, I
revised my course final assignment to make it a signature
assignment, so that instructors other than me can use it as the
culminating evidence of student ability both for the course and for
program assessment purposes. To help students achieve higher levels
of mastery, I designed many scaffolding activities throughout the
semester. For example, … Furthermore, I approached two faculty
members teaching lower level courses and used my expertise in
ethics to help them develop classroom activities and assignments
related to ethical deliberation.
Working collaboratively with faculty beyond the boundary of the
classroom helped me to make transformative improvements to the
course. Students came more prepared after going through the
learning experience in the lower level classes. The activities and
assignments made more sense for them because the expectations are
clear and aligned with program expectations. As a result, 80% of
the students performed satisfactorily on the final culminating
assignment after the course transformation, compared to 60% before
the changes. More importantly, I believe that I significantly
contributed to the curriculum coherence and quality in building
students’ ethical reasoning ability and ethical conduct in
research. My teaching and assessment work positively impacted my
class, the Ethical Research course in general, and the program.
(Disclaimer: This is constructed sample language designed for
inclusion in a faculty’s dossier. Individual faculty members should
check with their own department personnel evaluation criteria to
see whether a narrative like this would be
pertinent/appropriate.)
mailto:[email protected]
-
Assessment & Curriculum Support Center University of Hawai‘i
at Mānoa Crawford Hall 230 & 231 2550 Campus Road, Honolulu, HI
96822
[email protected] (808) 956-4283 (808) 956-6669
manoa.hawaii.edu/assessment
3
Student learning outcomes (SLOs) Definition: Action-oriented
statement of the knowledge, skills, and/or dispositions students
are expected to know, to be able to do, and to value upon
successful course/project/program completion Resource: Find program
SLOs here:
https://manoa.hawaii.edu/assessment/update2/view.php?view=programs
Activity 2: Locate the program SLOs that your course aligns with
Which program learning outcomes do you emphasize in your
course?
Select a strategy to use program SLOs to support your
teaching
❏ Clearly align the course with program SLOs [Align =
substantially and explicitly address that program SLO in my
course]
❏ Address important knowledge/skill not emphasized before ❏
Refine assignments/tasks to clearly align with the SLOs ❏ Provide
students with necessary scaffolding learning experience ❏ Help
students connect their learning by communicating the program SLOs
that you address ❏ Other:
Plan one activity to implement here:
mailto:[email protected]://manoa.hawaii.edu/assessment/resources/how-to/develop-program-student-learning-outcomes-slos/https://manoa.hawaii.edu/assessment/update2/view.php?view=programs
-
Assessment & Curriculum Support Center University of Hawai‘i
at Mānoa Crawford Hall 230 & 231 2550 Campus Road, Honolulu, HI
96822
[email protected] (808) 956-4283 (808) 956-6669
manoa.hawaii.edu/assessment
4
Curriculum Map Definition: A matrix that shows the alignment
between learning opportunities and program SLOs
Activity 3: Locate your program’s curriculum map here. Select a
strategy to use the curriculum map to support your teaching
❏ Identify a curriculum gap and collaborate with colleagues to
fill it ❏ Collaboratively design learning tasks/activities to
scaffold learning throughout the curriculum ❏ Collegially
communicate my expectations of students’ prior knowledge/skills to
instructors teaching a lower
level course ❏ Find out colleagues’ expectations of student
knowledge/skills existing from my course ❏ Explain to students my
assumptions of their prior knowledge/skills ❏ Design activities to
help students connect their prior learning to my course ❏ Refine my
course so that my students are better prepared for subsequent
courses (alternatives: future
professional tasks, community/civic engagement, graduate
school)
Plan one activity to implement here:
mailto:[email protected]://manoa.hawaii.edu/assessment/resources/how-to/curriculum-mapping-curriculum-matrix/https://manoa.hawaii.edu/assessment/update2/view.php?view=programs
-
Assessment & Curriculum Support Center University of Hawai‘i
at Mānoa Crawford Hall 230 & 231 2550 Campus Road, Honolulu, HI
96822
[email protected] (808) 956-4283 (808) 956-6669
manoa.hawaii.edu/assessment
5
Signature Assignment Definition: A generic task, problem, case,
or project that can be tailored or contextualized in different
disciplines or course contexts (Driscoll, 2016, p. 11)
Sample Signature Assignments English 100
3-5 pages in length Argue for a position Include 3 sources
Specify audience Genres to consider:
● Academic essay for general public
● A letter to GSO ● A white paper to legislature ● A transcript
of a talk or media
product (podcast)
Research Methods Course -- Research Proposal 5-8 pages in length
Includes:
● Research topic ● Justification of methods
selection ● Description of methods ● Proposed analysis
procedure
UH Mānoa Oral Presentation Signature Assignment Task: Students
will prepare and deliver a formal oral presentation. Length: 5-20
minutes Purpose: persuade or inform Audience: Specify intended
audience Supporting material: Utilize appropriate supporting
materials OC teaching and assessment resources are here.
Activity 4: Select a strategy to use a signature assignment to
support your teaching
❏ Collaborate with colleagues teaching the same or similar
courses to form shared learning expectations ❏ Refine the
assignment to align with the shared expectations ❏ Collaborate with
colleagues to design a signature assignment ❏ Discuss student
performance with fellow colleagues and collaboratively brainstorm
ways to enhance
assignment and teaching
Plan one activity to implement here:
mailto:[email protected]://manoa.hawaii.edu/assessment/reports/gened/oral-communication/oc-resources/
-
Assessment & Curriculum Support Center University of Hawai‘i
at Mānoa Crawford Hall 230 & 231 2550 Campus Road, Honolulu, HI
96822
[email protected] (808) 956-4283 (808) 956-6669
manoa.hawaii.edu/assessment
6
Rubrics Definition: A scoring guide that describes the criteria
that faculty use to evaluate student performance, understanding, or
behavior. It is often shaped like a matrix, which describes levels
of achievement in a specific area of performance, understanding, or
behavior. Resources
● VALUE rubrics: https://www.aacu.org/value/rubrics ● AO Rubric
Bank: http://manoa.hawaii.edu/assessment/resources/rubricbank.htm ●
ACSC Assessment and Learning Resource Repository:
http://go.hawaii.edu/ATb
Activity 5: Select a strategy to use a rubric to support your
teaching
❏ Align the rubric with program SLOs ❏ Collaboratively decide
what is important to evaluate with colleagues ❏ Collaboratively
define satisfactory performance ❏ Collaboratively evaluate and
refine the rubric ❏ Adapt the program rubrics together with your
students ❏ Share the rubric with students ❏ Train students to
self-evaluate using the rubrics ❏ Track student progress by
analyzing rubric data ❏ Use rubric results to identify SLOs to
enhance
Plan one activity to implement here:
Upcoming Rubric Use and Development Online Workshop can be
registered for here.
Activity 6: Write an email to yourself Remind yourself next time
that you are going to describe your teaching practice and how you
will describe your efforts integrating assessment to support
teaching and scholarship. Use bullet points to note down key
ideas.
mailto:[email protected]://www.aacu.org/value/rubricshttp://manoa.hawaii.edu/assessment/resources/rubricbank.htmhttp://go.hawaii.edu/ATbhttps://hawaii.zoom.us/meeting/register/tJEsc-GoqzMvHtfcp2c-UOoH5Mtn_UJKQPjx
Conduct Assessment to Supporting Teaching and Scholarship
HandoutConduct Assessment to Support Teaching and Scholarship Cover
Page.pdfConduct Assessment to Supporting Teaching and
ScholarshipConducting Assessment to Support Teaching and
ScholarshipHousekeeping Session outcomeAssessment is an integral
part of teachingActivity 1:Program Learning Outcomes (PLO)
AssessmentAssessment cycleAssessment cycleSLO Definition:
Action-oriented statement of…Domains of LearningExample SLOsExample
SLOsExample SLOsUsing program SLOs to support teaching—An
exampleMake the alignment clearerAddress program SLOs not
emphasized beforeAssignment redesign to align with new SLOScaffold
learning experienceUse programs SLOs to support teachingActivity
2�1. Locate your program SLOs�2. Plan a strategy to use program
SLOs to support your teaching (handout)Where to find program
SLOsAssessment cycleCurriculum map:�learning opportunities program
SLOsCurriculum map:�learning opportunities program SLOsCurriculum
map:�learning opportunities program SLOsSlide Number 26Use
curriculum map to support teachingActivity 3��Locate your program’s
curriculum map��Select a strategy to use the curriculum map to
support your teaching��Share in small groups.Breakout Room Task:
Think of one strategy to implement and share with a colleague (6
min)Slide Number 30Slide Number 31Signature AssignmentSections of
the same courseSample Signature Assignment for Eng 100Similar types
of coursesSample Research Proposal signature assignmentOral
Communication (OC) designated courseUH Mānoa Oral Presentation
Signature AssignmentSignature assignment developmentUse signature
assignment to support teachingActivity 4��Select a strategy to use
a signature assignment to support your teaching.��Plan one activity
to implement.Assessment cycleAssessment cycleRubricVALUE Oral
Presentation RubricHow can this rubric be revised?Slide Number
47Adapt an existing rubricAdapt an existing rubricAdapt an existing
rubricSlide Number 51Use rubrics to support teachingActivity
5��Select a strategy to use a rubric to support your
teaching.��Plan one activity to implement.Assessment cycleBreakout
Room Task:�Pick one signature assignment/rubric activity that you
plan to try. Share and give feedback to a colleague. Scholarship
OpportunitiesPublish your assignment—locallyPublish your
assignment—nationallySlide Number 59Disciplinary
ConferencesAssessment & SoTL Conferences�Thank you!��Yao
Hill�[email protected]
_Master Handout.pdfActivity 1: Articulate Integrating Assessment
in Teaching PracticeStudent learning outcomes (SLOs)Curriculum
MapSignature AssignmentRubricsUpcoming Rubric Use and Development
Online Workshop can be registered for here.Activity 6: Write an
email to yourself