The University of SydneyPage 1 Smart Sparrow: Using learning analytics to measure student engagement in case based learning Presented by Dr Vanessa Hughes.

The University of Sydney Page 1

Smart Sparrow: Using learning analytics to measure student engagement in case based learning  

Presented byDr Vanessa Hughes Sydney Nursing School

The University of Sydney Page 2

Smart Sparrow Case Based Studies

Cohort: First Year Bachelor of Nursing Advanced Studies Students

Nursing focus: Clinical notes, family/social history and diagnosis is provided

Content: Focuses on connecting the signs/symptoms observed with the underlying pathophysiology

The University of Sydney Page 3

The Evolution of Online Case Based Learning in NURS1005

2012 2013 2014 2015

Passive PDF Online module + videos + PDF worksheet

Smart Sparrow Adaptive Learning

Smart Sparrow technology has made it possible to:

• embed videos into the lesson

• create lessons that adapt to the students responses

• create an active learning environment

• test student knowledge and understanding

The University of Sydney Page 4

Purpose of the Online Case Study Lesson

– Replaces 1 hour of face to face teaching

– Covers important content in greater detail

– Connects bioscience theory with clinical practice

– Students receive formative feedback that enables them to:a. Check their answersb. Understand the level of detail and knowledge expectedc. Practice answering similar types of questions to the ones

included in their Pathophysiology Workbook Assessment

The University of Sydney Page 5

1. Trigger Video

2. Clinical Notes

3. MCQ or Short Answer Question

Model AnswerGuidance

Correct AnswerIncorrect Answer

The University of Sydney Page 6

The University of Sydney Page 7

The University of Sydney Page 8

Using Learning Analytics to Improve Learning and Teaching

– How long did students spend watching each video? Did they watch all of it?

– Were the questions too hard/easy? What needs to be clarified?

– What are the concepts that all students struggle with? What can I do to facilitate their understanding of the content?

– How long did each exercise take? Are they disengaging at some point?

– I would never have been able to easily answer any of these questions without Smart Sparrow Analytics.

The University of Sydney Page 9

Lesson Completion Semester 2

Week 1 Week 2

96% of students attempted the lesson 85% of students attempted the lesson

The University of Sydney Page 10

Simple overview of how students engaged with the activity

The University of Sydney Page 11

Anonymous Participation Students Enrolled in Smart Sparrow

49% completed the activity

94% completed the activity

The University of Sydney Page 12

Ability to Track Learning Pathways

The University of Sydney Page 13

Short Answer Responses

The University of Sydney Page 14

Student Feedback (1= Strongly disagree, 5= Strongly Agree)

Week 1 (n=179) Week 2 (n=154)

The videos in this module helped me to better understand the content.

4.11 4.11

I feel that completing this module will help me complete my pathophysiology workbook.

4.06 4.05

I feel that the feedback provided encouraged me or helped me find the answer when I made a mistake.

4.27 4.28

The model answers provided helped me understand what is expected of me and helped me understand what I needed to know about this topic.

4.54 4.49

The University of Sydney Page 15

Using Analytics to Track Student Performance

Using the analytics students can be categorised as:– No risk– Low risk– High risk

This data can be used to:– send personalised emails to

students– identify students for follow up

using the Track & Connect program

The University of Sydney Page 16

Using analytics to provide personalised student feedbackStudents are sent a personalised email (weekly) (via SRES) based on their risk category.

No risk – congratulations on completing all activities Low risk – encouragement to improve High risk – encouragement to improve, support and advice

Students at risk are emailed with their advisors details as well as the contact details for other university support services.

The email also contains links to additional resources that address common misconceptions found in the short answer responses.

This email feedback system has lead to several student problems being identified by the end of week 3.

It is hoped that this system will lead to a reduction in absent fails and enable students who are falling behind to get the help they need earlier in the semester.

The University of Sydney Page 17

Important Points

– Numbers only tell half the story:a. Use a combination of MCQ and short answer questions.b. By quickly scanning short answer responses you can identify

common misconceptions.

– Start small and build on the lesson.

– Keep it simple, too many trap states and pathways can lead to errors.

– Guidance to correct mistakes can be simple e.g. a recommended reading, video etc.

If you have not used SRES (Student Relationship and Engagement System), you will need to contact Adam Bridgeman ([email protected]) to be registered as a user.