NERU Presentation...2013/12/09  · Lead. NERU Presentation Evaluating the Impact of a Capstone Project on Student Learning McMaster University School of Nursing December 9, 2013 Louela

Insp

ire.

Lead

. E

ngag

e.

NERU Presentation

Evaluating the Impact of a Capstone Project on Student Learning

McMaster University School of Nursing December 9, 2013

Louela Manankil-Rankin, PhD student; Ola Lunyk-Child, MScN; Lynda Bentley-Poole, MScN; Lynn Martin, EdD

Insp

ire.

Lead

. E

ngag

e.

Background and Significance

§  Nursing education today calls for creative and alternate ways to prepare our students

§  Educational trends suggest that we promote integration

of learning between classroom and clinical settings §  Need to measure the impact of such innovations on

student learning §  This project measures the impact of the new capstone

project on student learning (Benner, Sutphen, Leonard, and Day, 2010)

Insp

ire.

Lead

. E

ngag

e.

Capstone Experience §  Small groups of students identify an issue in their own

professional practice placements

§  4Q03 concepts are used to develop a quality improvement activity to implement

§  The activity is implemented and evaluated in the professional practice placement

§  Students share experiences at the capstone in poster or

presentation format

Insp

ire.

Lead

. E

ngag

e.

Purpose of the study §  Determine the impact of the capstone on the students’

abilities to demonstrate: –  course ends-in-view, –  undergraduate degree level expectations, and –  entry to practice competencies

§  Describe the students’ perceptions of the capstone project

Insp

ire.

Lead

. E

ngag

e.

Objectives §  To analyze the relationship between the aggregated

grade results on the capstone project and the following variables –  depth and breadth of knowledge; –  utilization of methodologies; –  application of knowledge; –  communication skills; and –  professional capacity

§  To determine the students’ perceptions of the capstone experience through course evaluations.     

Insp

ire.

Lead

. E

ngag

e.

Research Design §  A sequential mixed methods study design

§  The unit of analysis is the capstone experience including student grades and responses on course evaluations

§  Descriptive statistics will be used to describe the grade distribution and quantitative data from course evaluations.

§  Interpretive analysis will be employed to establish the relationship between grades and the elements of interests.

§  Thematic analysis will be conducted when looking at the qualitative data from the course evaluations.

Insp

ire.

Lead

. E

ngag

e.

Grades/Marking Rubric Content relevancy of rubric based on 4 documents:

–  Rubric for Understanding (Wiggins &Mgtighe, 2005); –  Value Learning Rubric (Association of American

Colleges and Universities); –  University Undergraduate Degree Level Expectations

(DLE) (Ontario Council of Academic Vice-Presidents, Council of Ontario Universities, 2007);

–  National competencies in the context of entry level Registered Nurse practice (College of Nurses, 2009).

Insp

ire.

Lead

. E

ngag

e.

Data Collection Grades: §  Anonymized grades and course evaluation data from Fall

2012 until Fall 2014 will be collected from the Level 4 PBL/PBL Course planner (5 capstones - approx. 970 grades)

§  Grades will be provided by the course planner to the curriculum assistant who will anonymize them and then forward them to the research team

Course Evaluations: §  Course evaluation data will be compiled by the curriculum

assistant in a word document and forwarded to the research team in an anonymized format.

Insp

ire.

Lead

. E

ngag

e.

Data Analysis Step 1: Create Frequency distribution of Grades Step 2: Grades analyzed against the elements of interests: Depth and

breadth of knowledge, Application of Knowledge, Utilization of Knowledge, Communication skills, Professional Capacity.

Step 3: Frequency distribution of quantitative portion of course evaluations

Step 4: Thematic analysis of qualitative portion of course evaluations. Step 5: Data presented using tables and histograms. Qualitative

themes will be described in a report. Data will be analyzed overall to answer the research questions.

Insp

ire.

Lead

. E

ngag

e.

REB status and Funding

§  Submitted to REB – Preliminary data currently under the hat of course development. In order to publish results beyond our School of Nursing we need REB approval

§  Study funded through the Nursing Education Research Fund

Insp

ire.

Lead

. E

ngag

e.

Preliminary Results §  Quantitative Analysis

Insp

ire.

Lead

. E

ngag

e.

Preliminary Results

Insp

ire.

Lead

. E

ngag

e.

Preliminary Results

Insp

ire.

Lead

. E

ngag

e.

Preliminary Results §  Qualitative Relevance No value: in transition to new grad; no new learning §  “Capstone is ‘make work,’ how are we actually supposed to

use this as a new grad” §  “Capstone is pretty much like research, we all did it and

passed. It’s very redundant” Requires emphasis on integration §  “encourage students to be more critical and creative with

selection of capstone activity to emphasize the importance of the activity”

Insp

ire.

Lead

. E

ngag

e.

Preliminary Results Logistics Lack of knowledge regarding the Capstone project in tutors § “need more guidelines and directions” Conference disorganization §  “spaces were extremely small and crammed, did not feel professional or academic” Project: topics of interest did not coincide with every group member; difficulty meeting with group members § “it was hard to coordinate… a topic correlating to all floors” § “difficult to complete with group members because everyone’s clinical hours varied” Cost: stress to underlying financial stress (poster cost)

Insp

ire.

Lead

. E

ngag

e.

Feeling Unsupported § Lack of information translating to lack of support from PBL tutors and clinical preceptors § “my preceptors had no knowledge of this and did not show a lot of support, could we inform them better?” § Barriers to implementation: power; time; poor application § “it’s really hard for a student to implement something that they are only there for a short amount of time… we don’t have the power to say things or implement things in real practice world” § “good idea but poor application”

Insp

ire.

Lead

. E

ngag

e.

Workload Unneeded stress: unrealistic goal and expectation; uneven participation from group members; burnout § “between PBL and professional practice, there is too much expectations placed on students” § “expectation of capstone was too much and most students like myself are completely burnt out” Lack of time and added work to professional clinical practice § “felt very rushed… there was only about 8 weeks to complete it” § “there is too much going on in PBL for it to be meaningful” § “doing the 36hours a week for clinical made it difficult to meet with group… capstone is very in-depth and takes a lot of time”

Insp

ire.

Lead

. E

ngag

e.

Preliminary Analysis Combined Analysis § There is a disconnect between the results of quantitative data and qualitative

§ Students, on average, performed at a high academic level but their perception of the Capstone assignment was largely negative. As strategic learners, students are able to complete assignments without failure but unless the value of this learning is emphasized, they see little value or relevance.

Insp

ire.

Lead

. E

ngag

e.

Strengths & Limitations Strengths and Limitations Strength • Increased inter-rater reliability: Internal assessment process was completed for to ensure consistency in level of difficulty and expectation in tutors involved in marking students’ capstone projects. Limitations • There was large variation from the Fall 2012 to Winter 2013 grades for Scholarly Thought. This can be due to the Fall cohort being the first sample to undertake the Capstone Project while the Winter cohort had previous term students as a source of guidance. • Apparent difference in sample sizes. For the Fall 2012 group, there was a small sample size (n=67) which can be attributed for the large SD value.

Insp

ire.

Lead

. E

ngag

e.

Recommendations for the Future

1) Timeline adjustment –Students may benefit from introducing the project beginning of 4th year or extending deadline to a later date. 2) Reducing the number of PBL care scenarios – This will allocate more time in-class to discuss and brainstorm capstone project. Students will gain in-depth understanding and expectations for the assignment. 3) Required reading of 2 reference articles – Providing supplementary readings will better establish the relevance of the capstone project in their future professional practice.

Insp

ire.

Lead

. E

ngag

e.

Recommendations for the Future

4) Informed PBL, clinical practice tutors and involved clinical staff –Involved clinical staffs (e.g. nurse manager) should be informed about capstone and the necessary support and resource needed for the students. 5) Clear guideline and criteria evaluation method – The inconsistent expectations of tutors caused insecurity in many students and worsened their negativity towards the project. 6) Utilize capstone as a learning activity – Capstone project has greater value of learning that cannot be achieved through conventional learning activities (e.g. essay).

Insp

ire.

Lead

. E

ngag

e.

Overall Impression

§  Quantitative data indicate that students were able to meet the undergraduate degree level expectations and CNO expectation of communication

§  Thematic analysis of the course evaluations reveals the negative perception shared by the majority of the students.

§  These results reveals the need to further emphasize the significance of the project and its implication to the transition to professional practice.

Insp

ire.

Lead

. E

ngag

e.

Next Steps in our project

§  Continue Data collection/analysis (Capstone 3, 4, 5, 6 will need to be analyzed)

§  Hire a research assistant to help with SPSS analysis §  Results to be presented at NETNEP §  Study the impact of this learning activity on transition to

practice-18 month follow up. –  Collaboration with Lakehead

Insp

ire.

Lead

. E

ngag

e.

Conclusion

§  The Capstone project offers a way to assess the extent to which students achieve UDLE and entry to practice competency of communication

§  This project allows us to discern the experience of students with the project through course evaluations

§  Data from this project could be used for program evaluation §  It is snapshot of students’ performance. §  We need a more robust test for impact on transition – This is

through an 18 month follow up of our students as nurses in practice

Insp

ire.

Lead

. E

ngag

e.

Questions