Creating a Coherent STEM Gateway at Michigan State University “The first two years of college are the most critical to the retention and recruitment of.

Creating a Coherent STEM Gateway at Michigan State

University

“The first two years of college are the most critical to the retention and recruitment of STEM majors”

- President's Council of Advisors on Science and Technology (PCAST, 2012)

A project funded by the AAU STEM Education Initiative Project

AAU STEM Initiative

AAU has launched a five-year initiative in collaboration with our member universities to improve the quality of undergraduate teaching and learning in science, technology, engineering, and mathematics (STEM) fields. This is not another study or research project on STEM education. Instead, it is an effort based on overwhelming existing research to influence the culture of STEM departments at AAU universities so that faculty members are encouraged to use student-centered, evidence-based, active learning pedagogy in their classes, particularly at the first-year and sophomore levels.

https://stemedhub.org/groups/aau

Acknowledgements: the MSU AAU team

• Melanie Cooper• Joe Krajcik• Diane Ebert-May• Danny Caballero• Lynmarie Posey• Bob Geier• Sarah Jardeleza• J.T. Laverty

• Sonia Underwood• Becky Matz• Cori Fata Hartley• Biological Sciences

Faculty• Chemistry Faculty• Physics Faculty• CNS and Lyman-Briggs

Deans

Change model: build faculty consensus around the aims and rewards of reform

through:

1. Developing a shared vision for gateway course transformation in biology, chemistry and physics

2. Developing policies and structures to support and reward reform

Structures to reward and support reform

• STEM Gateway Fellows– For faculty who excel in STEM gateway courses

(modeled on the prestigious MSU Lilly Fellowships)• STEM Alliance – An institution-wide alliance of all entities involved in

STEM education (colleges, research centers etc.), to facilitate communication and coordinate activities

• DBER postdoctoral fellows to assist faculty in implementing reform efforts

A shared vision for curriculum reform:

Engage faculty in discussions to build consensus on key issues.

– What are the core ideas in the discipline?– What scientific practices are important? – What cross-cutting concepts make connections

among disciplines

The result – three dimensional learning.

Disciplinary Core Idea:

Disciplinary significance

Explanatory Power Generative

Biology Evolution Cell Theory of Life

Chemistry Matter is composed of

atoms Molecular structure

predicts macroscopic properties

Physics Force and momentum Waves

These are examples - not the full list!

Scientific and Engineering Practices

1. Asking questions and defining problems

2. Developing and using models3. Planning and carrying out

investigations and design solutions

4. Analyzing and interpreting data

5. Using mathematics and computational thinking

6. Constructing explanations and designing solutions

7. Engaging in argument from evidence

8. Obtaining, evaluating, and communicating information

The multiple ways of knowing and doing that scientists and engineers use to study the natural world and design world.

Scientific and Engineering Practices

1. Asking questions and defining problems

2. Developing and using models3. Planning and carrying out

investigations and design solutions

4. Analyzing and interpreting data

5. Using mathematics and computational thinking

6. Constructing explanations and designing solutions

7. Engaging in argument from evidence

8. Obtaining, evaluating, and communicating information

The multiple ways of knowing and doing that scientists and engineers use to study the natural world and design world.

Crosscutting Concepts

Ideas that cut across and are important to all science disciplines1. Patterns2. Cause and effect3. Scale, proportion and quantity4. Systems and system models5. Energy and matter6. Structure and function7. Stability and change

NRC Framework for Science Education 2012

Crosscutting Concepts

Ideas that cut across and are important to all science disciplines1. Patterns2. Cause and effect3. Scale, proportion and quantity4. Systems and system models5. Energy and matter6. Structure and function7. Stability and change

NRC Framework for Science Education 2012

National Research Council. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Washington, DC: The National Academies Press, 2012.

Measuring change

• Data on persistence, grades, affective domain (motivation, attitudes, expectations) etc.– MSU will participate in the CIC STEM Learning

Analytics Initiative.• BUT… – we know that grades do not necessarily equate

with learning

Our premise:

Engaging faculty to determine the core ideas,

science practices and cross cutting concepts

promote change

leads to changes in classroom practice

and changes in assessment

practices

We will measure change by describing:

• Classroom practice – Using the Three Dimensional Learning Observation

Protocol (3D-LOP)• Course assessments.– Using the Three Dimensional Learning Assessment

Protocol (3D-LAP)

Assessing the Assessments The 3D-LAP

National Research Council. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Washington, DC: The National Academies Press, 2012.

There is "a damaging collusion between students on the one hand and faculty on the other –– a collusion in which students agreed to accept bad teaching provided that they were given bad examinations."

Quote from Peter Kennedy in “They are not dumb, they are different” Tobias, 1990.

Clearly assessments may have different purposes…

• To help students learn• To measure a students understanding• To assess a large scale program

But regardless…

“If you don’t assess what’s important, what’s assessed becomes important”

Assessment should be to “to educate and improve student performance, not merely to audit it”

Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco, CA: Jossey-Bass

We have designed and operationalized the 3 Dimensional Learning Assessment Protocol (3D-LAP) to help us to:

Identify change in assessments over time

Help faculty design 3D assessments that provide better evidence of what students

know and can do

Scientific Practices P1 (Does the item contain a practice - yes/no)(SP) P2 (If there is a practice, which practice is presented)

P3 (If there is a practice, is the practice explicit/implicit)

Crosscutting Concepts CC1 (Is there a crosscutting concept (CCC) - yes/no)

CC2 (If there is a CCC, which CCC is present)CC3 (If there is a CCC, is the CCC explicit/implicit)

Disciplinary Core Ideas DCI1 (Is there a disciplinary core idea (DCI) - yes/no)

DCI2 (If there is a DCI, which DCI is present)DCI3 (If there is a DCI, is the DCI explicit/implicti)

Part 1: Assessing 3D Learning

Scientific Practices P1 (Does the item contain a practice - yes/no)(SP) P2 (If there is a practice, which practice is presented)

P3 (If there is a practice, is the practice explicit/implicit)

Crosscutting Concepts CC1 (Is there a crosscutting concept (CC) - yes/no)(CC) CC2 (If there is a CC, which CC is present)

CC3 (If there is a CC, is the CC explicit/implicit)

Disciplinary Core Ideas DCI1 (Is there a disciplinary core idea (DCI) - yes/no)

DCI2 (If there is a DCI, which DCI is present)DCI3 (If there is a DCI, is the DCI explicit/implicti)

Part 1: Assessing 3D Learning

Scientific Practices P1 (Does the item contain a practice - yes/no)(SP) P2 (If there is a practice, which practice is presented)

P3 (If there is a practice, is the practice explicit/implicit)

Crosscutting Concepts CC1 (Is there a crosscutting concept (CC) - yes/no)(CC) CC2 (If there is a CC, which CC is present)

CC3 (If there is a CC, is the CC explicit/implicit)

Disciplinary Core Ideas DCI1 (Is there a disciplinary core idea (DCI) - yes/no)(DCI) DCI2 (If there is a DCI, which DCI is present)

DCI3 (If there is a DCI, is the DCI explicit/implicit)

Part 1: Assessing 3D Learning

Comparison of exams from a traditional and a transformed course

Implicit SP

Implicit CCC

Implicit DCI

Explicit SP

Explicit CCC

Explicit DCI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24SP CC DCI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20SPCC DCI

Part 2: Question Quality

Phenomena Does the question address a phenomenon (i.e., an observable event)?

Intent What is the intent or goal of the question?

ExplicitDoes the item elicit explicit evidence of student learning that is aligned with the intent of the question?

Learning goal Does the question address an explicit learning goal?

Question constructionDoes the question meet acceptable practices for valid item construction (e.g., appropriate level of math and reading literacy, reasonable number of choices of similar length)?

Example Chemistry Question

Dimension Present? Type

Practice No N/A

CCC No N/A

DCI No N/A

Example Physics Question

Dimension Present? Type

Practice No N/A

CCC No N/A

DCI No N/A

Sample Biology Question

Dimension Present? Type

Practice No N/A

CCC No N/A

DCI No N/A

How will we know when students understand a phenomenon?

• “To understand a phenomenon is to understand how it is caused”– Strevens. M. (2013), No Understanding without

Explanation, Studies in History and Philosophy of Science 44, 510–515

• Explanation “is the phenomenological mark of an evolutionarily determined drive”– Gopnik, A. (1998). Explanation as Orgasm*. Minds

and machines, 8(1), 101-118.

3DLAP Operating Definition of Explanation

Question asks student to explain a phenomenon, event, or observation. • Question gives or requires student to provide the target of the

explanation

• Question requires student to reference scientific principles and/or data

• Question requires student to provide reasoning linking scientific principles and/or data to phenomenon, event, or observation

Chemistry Example

Dimension Present? Type

Practice Yes Explanation

CCC Yes Cause and Effect

DCI Yes Chemical Reactions

When you mix acetic acid and methyl amine, this reaction occurs.i) What type of reaction is it?ii) Indicate what is happening at the molecular level by drawing mechanistic arrowsiii) Justify, using your knowledge of molecular structure and interactions, why this

reaction occurs

Which is a stronger base? CH3OH or CH3NH2? Why?

CH3NH2 Claim

Is it possible to design MC questions with a practice such as explanation?

Which is a stronger base? CH3OH or CH3NH2? Why?

CH3NH2 because N is less electronegative than O

Claim Scientific Principle

Which is a stronger base? CH3OH or CH3NH2? Why?

CH3NH2 because N is less electronegative than O and therefore is better able to donate a lone pair into a bond with an acid.

Claim Scientific Principle Reasoning

Mulitple Choice Exam Question

Dimension Present? TypePractice Yes (explicit) ExplanationCCC Yes (explicit) Cause and Effect

DCI Yes (explicit) Molecular Structure and properties

We have designed and operationalized the 3 Dimensional Learning Assessment Protocol (3D-LAP) to help us to:

Identify change in assessments over time

Help faculty design 3D assessments that provide better evidence of what students

know and can do

Your tasks:• Form groups of 3-4 people. • Construct an open response exam question for a gateway course

using 3DLAP as a guide. • Use this question to construct one or more multiple choice item(s)• Assess two dimensions of learning:

– Crosscutting concept: Energy and matter– Practice: Explanation

• Prepare to share your questions/describe the process with your group.

• Choose an exemplar (or two) to share with the whole STEM Alliance when we report out

• You have one hour!

Room assignments

• Physics and Astronomy,1400 – Danny Caballero

• Chemistry and Molecular Biology, 1415– Melanie Cooper

• Biology, stay here– Diane Ebert-May

• Implications for Math: 1420– JT Laverty

Anticipated Outcomes

• Transformed STEM gateway courses – that address core ideas and practices of the discipline.

• Transformed teaching practices (emerge as a consequence of changes in expectations)

• Improved learning outcomes (no inert knowledge!)

• Cultural change emerges from shared vision