Campus Wide Assessment Project

Campus Wide Assessment Campus Wide Assessment ProjectProject

Quantitative and Symbolic Quantitative and Symbolic ReasoningReasoning

Assessment Team:

David Nelson – Faculty Lead, Math Division

Janet Ash, Technology Division

Brenda Bindschatel, Business Division

Keith Clay, Science Division

Sandy Johanson, Humanities Division

Goals of the ProjectGoals of the Project

1.1. Peer review of the Learning Outcome Peer review of the Learning Outcome Tracking System (LOTS) databaseTracking System (LOTS) database

2.2. Campus-wide assessment of the QSR Campus-wide assessment of the QSR outcomeoutcome

Quantitative and Quantitative and Symbolic ReasoningSymbolic Reasoning

1.1. Evaluate and interpret quantitative and symbolic reasoning Evaluate and interpret quantitative and symbolic reasoning information/datainformation/data

2.2. Recognize which quantitative or symbolic reasoning methods Recognize which quantitative or symbolic reasoning methods are appropriate for solving a given problem, and correctly are appropriate for solving a given problem, and correctly implement those methodsimplement those methods

3.3. Demonstrate the ability to estimate a solution to a presented Demonstrate the ability to estimate a solution to a presented problemproblem

4.4. Translate data into various formats such as symbolic language, Translate data into various formats such as symbolic language, equations, graphs and formulasequations, graphs and formulas

5.5. Implement calculator/computer technology to solve problemsImplement calculator/computer technology to solve problems6.6. Demonstrate logical reasoning skills through formal and Demonstrate logical reasoning skills through formal and

informal proofs.informal proofs.

LOTS DatabaseLOTS Database

Courses are rated by instructors or Courses are rated by instructors or departmentsdepartments

Level 0: Not taught, practiced or Level 0: Not taught, practiced or assessedassessed

Level 1: taught or practiced, but not Level 1: taught or practiced, but not assessedassessed

Level 2: assessed by not taughtLevel 2: assessed by not taught Level 3: taught and assessedLevel 3: taught and assessed

Peer Review of LOTSPeer Review of LOTS

Review CARS of all courses claiming Review CARS of all courses claiming Level 3 for any of the QSR competenciesLevel 3 for any of the QSR competencies

Review of LOTS DatabaseReview of LOTS Database

The student will demonstrate proficiency in the following areas:The student will demonstrate proficiency in the following areas: Metric conversions, using dimensional analysisMetric conversions, using dimensional analysis Naming chemical elements, and identifying atomic numbers and massNaming chemical elements, and identifying atomic numbers and mass Writing chemical formulas and naming compoundsWriting chemical formulas and naming compounds Calculating molar massesCalculating molar masses Quantitative composition of compoundsQuantitative composition of compounds Writing and balancing chemical equationsWriting and balancing chemical equations Periodic propertiesPeriodic propertiesIndividual instructors will chose two (three?) of the following proficiencies as the Individual instructors will chose two (three?) of the following proficiencies as the

last unit of the class :last unit of the class : Mass relationships in a balanced equationMass relationships in a balanced equation Atomic structure of the first eighteen elementsAtomic structure of the first eighteen elements Gas lawsGas laws Chemical bondsChemical bonds Properties of liquidsProperties of liquids

CHEM 105: Marked QSR Competency 2 and Competency 5 at Level 3

Review of LOTS DatabaseReview of LOTS Database

Students will learn:Students will learn: The use of taxonomic keys to identify trees, and shrubs.The use of taxonomic keys to identify trees, and shrubs. The use and comprehension of dendrology terminology.The use and comprehension of dendrology terminology. Plant morphologyPlant morphology Identification of all required plantsIdentification of all required plants How to establish grid plotsHow to establish grid plots

NATRS 183 claims QSR 1, 2, and 4

Quantitative Reasoning: Students will measure physical, biological, and environmental parameters. Additional parameters and results will be obtained by calculations and graphing.

Review of LOTS DatabaseReview of LOTS Database

158 CARs claiming Level 3 were reviewed158 CARs claiming Level 3 were reviewed QSR documentation in 14 CARs were deemed QSR documentation in 14 CARs were deemed

inadequateinadequate LOC Chair and division rep met with lead LOC Chair and division rep met with lead

instructors and either changed LOTS rating or instructors and either changed LOTS rating or changed course syllabuschanged course syllabus

13 of the 14 have been adjusted13 of the 14 have been adjusted Issue of authority and control needs to be Issue of authority and control needs to be

decideddecided

Campus Wide Campus Wide AssessmentAssessment

Determine an appropriate assessmentDetermine an appropriate assessment Implement assessment with the aid of Implement assessment with the aid of

instructorsinstructors Analyze the dataAnalyze the data Report to the communityReport to the community

Campus Wide QSR Campus Wide QSR AssessmentAssessment

Competencies 1, 2, 4 and 5Competencies 1, 2, 4 and 5 8 classes chosen randomly for each 8 classes chosen randomly for each

competencycompetency Appropriate mix of day, night and distance Appropriate mix of day, night and distance

coursescourses Appropriate mix full-time and adjunct Appropriate mix full-time and adjunct

instructorsinstructors Embedded assessment tool – created by the Embedded assessment tool – created by the

facultyfaculty Scored according to the Community Rubric Scored according to the Community Rubric

Campus Wide QSR Campus Wide QSR AssessmentAssessmentChallenges with the DataChallenges with the Data

Several instructors did not participate or did not Several instructors did not participate or did not provide data in time for the assessment – 11 of provide data in time for the assessment – 11 of the 35 selected classesthe 35 selected classes

Minimal data collected for QSR 5 - 2 of 10 Minimal data collected for QSR 5 - 2 of 10 classes provided pre and post assessment classes provided pre and post assessment data, one provided post assessment data onlydata, one provided post assessment data only

One day class was substituted for the initial One day class was substituted for the initial selection of a night class (QSR 1)selection of a night class (QSR 1)

Difficult to distinguish between Competent & Difficult to distinguish between Competent & Mastering when there was a single problemMastering when there was a single problem

Positive shift towards masteringPositive shift towards mastering Approximately 67.6% achieve competent or Approximately 67.6% achieve competent or

mastering on the post assessmentmastering on the post assessment

Overall Total

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

Pre Assessment

Post Assessment

QSR 1

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%

Pre Assessment

Post Assessment

Evaluate and interpret quantitative and symbolic reasoning information

• 51.8% reach competent or mastering

Recognize which quantitative or symbolic reasoning methods are appropriate for solving a given problem and correctly implement those methods.

• 70.2% of students reached competent or mastering

QSR 2

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

Pre Assessment

Post Assessment

Translate data into various formats such as symbolic language, equations, graphs and formulas

• 69.4% of students achieve competent or mastering

QSR 4

0.0%10.0%20.0%30.0%40.0%50.0%60.0%

none

emer

ging

com

pete

nt

mas

tery

Pre Assessment

Post Assessment

Implement calculator/computer technology to solve problems.

• 92.2% of students were able to reach competent or mastering level

QSR 5

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

none

emer

ging

com

pete

nt

mas

tery

Pre Assessment

Post Assessment

Unusual ClassesUnusual ClassesMath 97

02468

101214

Pre Assessment

PostAssessment

B A 220

02468

101214

none

emer

ging

com

pete

nt

mas

tery

Pre Assessment

PostAssessment

Biol 100

02468

101214

none

emer

ging

com

pete

nt

mas

tery

Pre Assessment

PostAssessment

Math 97, B A 220 and BIO 100 show improvement, but not nearly as much as other courses.

MATH 97, B A 220 - QSR Competency 1

BIO 100 - QSR Competency 4

Unusual ClassesUnusual ClassesB A 110

02468

101214

none

emer

ging

com

pete

nt

mas

tery

Pre Assessment

PostAssessment

Chem 150

02468

101214

none

emer

ging

com

pete

nt

mas

tery

Pre Assessment

PostAssessment

BA 110 and Chem 150 had a large number of competent/mastering students on the pre assessment. (Competency 4)

Full-Time and Adjunct Full-Time and Adjunct FacultyFaculty

Full-Time Faculty

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Pre Assessment

PostAssessment

Part-Time Faculty

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Pre Assessment

PostAssessment

Pre Assessment

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Full-Time Faculty

Part-TimeFaculty

Post Assessment

0.0%

10.0%

20.0%

30.0%40.0%

50.0%

60.0%

Full-Time Faculty

Part-TimeFaculty

Day, Evening and Online Day, Evening and Online ClassesClasses

Day Classes

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%

none

emer

ging

com

pete

nt

mas

tery

Pre Assessment

PostAssessment

Other Classes (Distance)

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%

none

emer

ging

com

pete

nt

mas

tery

Pre Assessment

PostAssessment

Evening Classes

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%

Pre Assessment

PostAssessment

Pre Assessment

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%

Day Classes

Evening Classes

Other Classes

Post Assessment

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%

none

emer

ging

com

pete

nt

mas

tery

Day Classes

Evening Classes

Other Classes

RecommendationsRecommendations

Relatively few courses claim Competency 3 and Relatively few courses claim Competency 3 and Competency 6. Should we offer more? Do we expect Competency 6. Should we offer more? Do we expect mastering of all competencies?mastering of all competencies?

Revise Competency 2, or use better assessment Revise Competency 2, or use better assessment methods. Two different skills are listed.methods. Two different skills are listed.

Improve success rate of Competency 1Improve success rate of Competency 1 Increase communication between full-time, adjunct, Increase communication between full-time, adjunct,

day, evening and online instructorsday, evening and online instructors Look at math prerequisites for courses with a minority Look at math prerequisites for courses with a minority

of students reaching the competent or mastering level.of students reaching the competent or mastering level.

Recommendations for Recommendations for follow-up studiesfollow-up studies

Check degree requirements with CWOs. Check degree requirements with CWOs. Can a student graduate without taking Can a student graduate without taking QSR classes?QSR classes?

Use appropriate assessments. – Comp. 2Use appropriate assessments. – Comp. 2 Start earlier.Start earlier. Contact instructors of classes selected Contact instructors of classes selected

for sample earlier/more frequently to get for sample earlier/more frequently to get a better response rate.a better response rate.