Tech-Driven Education Reform: A Model for Simultaneously Improving Student Retention and Performance in STEM Disciplines

1

An ExamSoft Client Webinar

Tech-Driven Education Reform: A Model

for STEM Disciplines

2

Tech-Driven Education Reform: A Model for Simultaneously Improving

Student Retention and Performance in STEM Disciplines

Rob Petros, Ph.D. | Assistant Professor, UNT

Department of Chemistry

3

4

5

6

Roadmap of Presentation

•  The state of STEM/recent results •  Course data and information •  NextGen concepts •  Chemistry Redesign

– Engagement – Learner Centered and Strategies

•  Questions and Feedback

7

Dr. Rob Petros: Chemistry The Problem

•  Only 40% of entering college students that declare STEM majors complete degrees in STEM disciplines

•  Economic projections indicate 1 million more STEM professionals will be needed in the US than will be produced over the next decade at current rates

Why are we losing these students? 1.  Lack of inspiration in low level science classes 2.  Frustration with the ‘weed out’ courses 3.  STEM graduate programs prepare researchers

not educators

8

Problem Solved!? •  Redesign has increased enrollment from 145 to 192

(32%), while reducing actual class size from 145 to 96 ($120K+ in annual tuition).

•  Retention rate has increased dramatically ($250K?)

* first year incorporating engaged learning activities (peer group learning)

•  Percent effort for teaching course has decreased dramatically

# of students receiving D, F, or W

# of students registered

retention rate (%)

2013/2014* 39 377 90 2012/2013 83 285 71 2011/2012 83 291 71 2010/2011 87 248 65 2009/2010 77 218 65

9

Student Performance During Redesign

0

10

20

30

40

50

60

May-‐11 May-‐12 May-‐13 May-‐14† May-‐14‡ May-‐14*

% of class

Results from ACS Standardized Final Exam

Above Natl. Average BoLom quarNle

† All students taking second semester with Petros

‡ Students that were in Petros' section for both OChem I and II * Students that were in Petros' section for only OChem II

podcasts introduced

lecture eliminated

NextGen launched

10

•  A 10 question quiz covering basic concepts from OChem I was administered on the first class day of the second semester.

•  Students enrolled in Petros’ first semester course performed significantly better.

•  Remediation activities are being developed for transfer students.

Student Preparedness for OChem II

N = 172 Former Petros students – 97 Students transferring in – 75

0 10 20 30 40 50 60 70 80

Pretest

Aver

age

Scor

e OChem I w/Petros

OChem I w/someone else

11

12

Student Learning Outcomes

•  What the student is expected to know and be able to do – Useful in the classroom

•  To students •  To Faculty

– Connects to “larger” goals •  Program •  Institution •  Accreditation

13

NextGen Course Redesign

Course ObjecNves

Assessment* InstrucNonal Strategies

*Carriveau, R. S., Connecting the dots : developing student learning outcomes & outcome based assessments. Fancy Fox Publications:

Denton, TX, 2010.

14

1. Goal: The student will understand literature (by) 1.1. General Learning Outcome (GLO): The student will demonstrate

accurate, critical, analytic reading of literature (by) specific learning outcome statement (sLO): The student will (be able to) 1.1.1 Identify important and supporting details 1.1.2 Recognize assumptions and inferences 1.1.3 Identify sequence of events 1.1.4 Determine the main idea/theme of a passage or

piece of literature

Source: Carriveau, R.S. (2011), Connecting the Dots: Developing Student Learning Outcomes and Outcome-Based Assessments

Example of Three Level SLO Structure

15

Code each item to a specific learning outcome

1 Goal: The student will understand literature (particular period or genre) 1.1 General Learning Outcome (GLO): The student will demonstrate

accurate, critical, analytic reading of literature. Specific learning outcome statements (sLO): 1.1.1 Determine the main idea/theme of a passage or piece of literature. Test Item or Rubric Dimension 1.1.1 What was this passage mostly about? (Could also be CR) A. Bias and prejudice can affect intellectual growth.* B. Economic growth is impacted by bias and prejudice. C. Current thoughts on bias and prejudice lack insight.


16

The Link Between Outcomes and Items Calculating Outcome Attainment Values

Goal 1

Avg = 83


1. 87 2. 90 3. 65 4. 58 5. 63 6. 52 7. 66 8. 77 9. 84 10 93 11. 96 12. 88 13. 82 14. 88 15. 90 16. 80 17. 92 18. 81 19. 81 20. 82

Specific Outcome 1.1.1

Avg = 81


Ave = 60


Avg = 88


Avg = 85


Avg = 85

General Outcome 1.1

Avg = 81

General Outcome 1.2

Avg = 85

17

ExamSoft “Categories”

Use the “Categories” feature to create three levels of outcome statements

18

Writing OChem Course Objectives

•  What are the essential principles every student should take away from the course?

•  Who are your students and what are you preparing them for?

•  Can your course teach important lessons beyond your own discipline-specific content?

•  How will you assess student attainment?

19

Course Objectives

•  Goal 1 – Students will understand molecular structure and its implication for basic chemical reactivity

•  Goal 2 – Students will understand the concept of isomerism in organic chemistry

•  Goal 3 – Students will understand various mechanistic pathways commonly encountered in organic chemistry

•  Goal 4 – Students will understand how spectroscopy can be used to determine molecular structure

•  Goal 5 – Students will understand the relationship of organic chemistry to their everyday lives

•  Goal 6 – Students will be aware of the impact of the globalization of scientific research on US competitiveness in science and technology and be motivated to pursue and obtain degrees in STEM majors

•  Goal 7 – Students will demonstrate elements of collaboration, leadership, innovation, problem solving, creativity, teamwork, and critical thinking

20

Objectives and Professional Standards

21

22

23

Online Content

252 podcasts totaling ~23h of recorded material produced covering all topics from both semesters posted on UNT’s iTunesU site

24

STEM Incentives Program

•  Information on graduate school in the sciences

•  Careers in chemistry •  Where my classmates are now •  Need for diversity in STEM •  Science and Engineering Indicators 2012 •  Activity demonstrating the globalization of

scientific research – List authors’ home institution for first 20

articles in the first issue of JACS in 1982, 1992, 2002, and 2012

25

STEM Incentives Program

26

Engaged Learning Activities

*Office of Science and Technology Policy – Engage to Excel (2012)

27

Class Time – Small Group Work

•  192 students divided into 32 groups of 6 •  Each group meets 2hr/wk •  Can observe skills related to course goals 5-7

28

Group Assignments

29

30

Formative Assessment On Demand

31

More Robust Analysis

Learning Objective # Assessments # Items Group Average 1. Students will understand molecular structure and its implication for basic chemical reactivity

11 66 73.36%

1.1 Students will know and apply the naming system for organic compounds

11 23 71.06%

1.1.1 Students will recognize and correctly name molecules containing functional groups

9 23 71.24%

1.1.2 Students will correctly name stereoisomers 2 1 70.56% 1.2 Students will understand the relationship between structure, hybridization, resonance, and aromaticity

2 21 81.80%

1.2.1 Students will identify the correct hybridization state for C, N, O and other relevant atoms

2 18 81.19%

1.2.2 Students will identify factors that lead to stabilization in resonance structures

2 3 84.86%

1.3 Students will understand the role of acidity⁄basicity in reactions

4 23 70.67%

1.3.1 Students will predict products of an acid⁄base reaction

2 2 73.89%

1.3.2 Students will identify acid⁄base conjugate pairs 2 6 89.73% 1.3.3 Students will use pKa values to predict relative acidity

4 15 66.86%

32

Tracking Student Learning in Real Time First Half of Semester

Second Half of Semester

Number of Items

Group Average

Number of Items

Group Average

1.1.1 Students will recognize and correctly name molecules containing functional groups 10 91% 5 97%

1.3.3 Students will identify acid⁄base trends for common functional groups 10 82% 2 69%

2.1.1 Students will recognize the mechanisms for electrophilic and nucleophilic aromatic substitution reactions

2 38% 1 84%

2.1.2 Students will predict the site of substitution based on substituents present on the aromatic ring 2 44% 1 85%

2.1.5 Students will predict products of reactions given specific starting materials 2 80% 12 87%

3.1.1 Students will predict reaction products based on starting materials 35 76% 29 83%

3.1.2 Students will differentiate between reversible and irreversible addition reactions 8 87% 4 93%

5.2.1 Students will analyze spectral data to predict molecular structure 12 84% 9 88%

33

Rob Petros – Improved from 56% to 71%

Fourth Exam Performance (split into two section)

Remediation Assignment Performance (sections combined)

34

Rob Petros – Early intervention

Student Feedback •  Student feedback

released on learning outcome performance and areas of strength/weakness

•  Displayed score

upon exam exit

35

Engagement in the Class •  The purpose of the UNT Student Engagement Inventory

(SEI) is to provide information about student engagement that instructors can use to make meaningful continuous improvements to their course.

•  Part 1, Personal Internal Motivation, is a measure of the

students’ personal internal predisposition to be engaged in learning.

•  Part 2, Classroom Interaction is a measure of the students’ perception of engagement based on interactions in the teacher-created classroom learning environment.

•  Part 3, Social Interaction, is a measure of the social interaction among classmates in interactive classroom experiences.

36

Review

•  Identify relevant student and faculty data related to STEM course redesign

•  Describe effective, learner focused instructional practices •  Describe the connection between outcome based education and

improvement of instruction •  Describe outcome based assessment strategies •  Identify basic milestones for success •  Discuss how various assessment and instructional best

practices can be applied to a variety of settings and institutions •  Develop strategies for implementing instructional and

assessment best practices at course, department and institutional level.

•  Develop a basic outline of ideas for incorporating outcome based, learner centered instruction and assessment into their own courses, programs, or institutions.

37

Contact

•  Dr. Rob Petros –  [email protected]

Tech-Driven Education Reform: A Model for Simultaneously Improving Student Retention and Performance in STEM Disciplines

Education

petros students

college students

transfer students

student retention

student performance

student preparedness

stem disciplines rob

petros section