Standards and Rubrics for Assessing Learning Outcomes in Mathematics GEAR Conference April 27 th – 28 th.

Standards and Rubrics for Assessing Learning Outcomes in

Mathematics

GEAR Conference

April 27th – 28th

Presenters:

Maryann Faller – Adirondack CCRalph Bertelle – Columbia-

Greene CCJack Narayan – SUNY Oswego

History

• The SUNY Board of Trustees passed a resolution creating three levels of assessment: general education, assessment of the major and system-wide assessment.

• PACGE (Provost’s Advisory Council on General Education) was formed to provide some guidance to the campuses as they submitted the courses they wanted to use for general education in mathematics.

• PACGE developed the Guidelines for the Approval of State University General Education Requirement Courses which listed the following learning outcomes for mathematics.

Students will show competence in the following quantitative reasoning skills:

• Arithmetic;• Algebra;• Geometry;• Data analysis; and• Quantitative reasoning

• GEAR (General Education Assessment Review) was formed to assist campuses in assessing the learning outcomes in general education.

• ACGE (Advisory Council on General Education) was formed to serve as the judicator for general education courses and to review/revise the learning outcomes.

• SUNY BoT passes a resolution requiring strengthened campus-based assessment in mathematics, basic communication and critical thinking.

At the request of the mathematics faculty from our campuses and the Provost, ACGE revises the learning outcomes in mathematics.

New Learning Outcomes in Mathematics

Students will demonstrate the ability to:

• interpret and draw inferences from mathematical models such as formulas, graphs, tables and schematics;

• represent mathematical information symbolically, visually, numerically and verbally;

• employ quantitative methods such as, arithmetic, algebra, geometry, or statistics to solve problems;  

• estimate and check mathematical results for reasonableness; and  

• recognize the limits of mathematical and statistical methods.

There are three options for assessing the learning outcomes in mathematics.

1. Nationally-normed standardized tests2. SUNY-normed standardized tests3. Using rubrics developed by discipline

specific panels.

The discipline panels first met at System Administration on February, 2005 to discuss the charge and other aspects of writing those rubrics.

Members of the Mathematics Discipline Panel

• Maryann Faller – Chair, Adirondack CC• Mel Bienenfeld - Westchester CC• Ralph Bertelle – Columbia Greene CC• Jack Narayan – SUNY Oswego• Michael Oppedism – Onondaga CC• Robert Rogers – SUNY Fredonia• Malcomb Sherman– SUNY Albany• William Thistleton – SUNY IT

Procedures Used In Creating a Rubric

• Determine the standard to be assessed.• Write learning objectives for that standard.• Determine the style and scale that will be

used.• Describe criteria for the highest and the

lowest levels• Describe the criteria for the levels in

between the highest and lowest.

Our rubric

After much discussion, the panel decided that we will have a matrix with 2 columns and 4 rows. The rows will represent the levels of assessment. They are:– 3: Exemplary– 2: Generally Correct– 1: Partially Correct– 0: Incorrect

The panel decided to rate the student’s response with respect to the following criteria:

– Does the student understand the problem?– Does the student use a clearly developed logical plan to solve the problem and is that plan evident in the solution?

– Is the solution totally correct?

Learning Outcome #1• Standard: Students will demonstrate the

ability to interpret and draw inferences from mathematical models such as formulas, graphs, tables and schematics .

• Learning Objectives:Given a mathematical model, the student will

be able to:• Interpret the information • Draw inferences from that model

Level

3Exemplary

•The student is able to interpret the significance of the values and/or variables given in the model.The student has an understanding of how to use the model to answer the question.The question is answered accurately and completely.

2Generally Correct

•The student is able to interpret the significance of the values and/or variables given in the model.The student has an understanding of how to use the model to answer the question.The question is not answered accurately or completely.

Level

1Partially Correct

•The student has some misunderstanding of how the model relates to the situation.The student attempts to use the model to answer the question but lacks a clear understanding of how to carry that out.The question is not answered accurately or completely.

0Incorrect Solution

•The student does not understand how the model relates to the situation.The student does not understand how to use the model to answer the question.The question is not answered accurately or completely.

Learning Outcome #2• Standard: Students will demonstrate the ability to

represent mathematical information symbolically, visually, numerically and verbally .

• Learning Objectives:

Given mathematical information, the student will be able to:

• Represent that information symbolically• Represent that information visually• Represent that information numerically• Represent that information verbally

Level

3Exemplary

•The student fully understands the mathematical information and the mode of representation. •The student understands all required aspects of the representation and clearly demonstrates the knowledge of how to develop it.•The representation of the given information is correct and accurate. It is displayed using the correct format, mathematical terminology, and/or language. Variables are clearly defined, graphs are correctly labeled and scaled, and the representation is otherwise complete as required.

Level

2Generally Correct

•The student understands the essentials of the mathematical information and the required representation(s). •The student understands most of the specific aspects of the representation and demonstrates the knowledge of how to develop it from the given information. This understanding or demonstration is lacking in a minor way.A misrepresentation of the mathematical information was given due to a minor computational/ copying error or the representation was not labeled or labeled incorrectly. The representation is incomplete in some minor way. .

Level

1Partially Correct

•The student does not fully understand the mathematical information or the mode of representation, but some understanding is shown.The student shows some knowledge of how to develop the appropriate representation, but this knowledge is incomplete in a major way.The representation(s) show some reasonable relation to the information, but they contain major flaws, use incorrect format, mathematical terminology or language. The representation is incomplete in a major way.

Level

0Incorrect Solution

•The student does not understand the mathematical information or the required representation(s). •Complete misinterpretation of the problem. The student could not represent the information in any format other than the format in which the information was given.•The representation(s) are incomprehensible or unrelated to the given information. The process of developing the representation is entirely incorrect.The student’s response does not address the question in any meaningful way or there is no response at all.

Learning Outcome #3• Standard: Students will demonstrate the ability to

employ quantitative methods such as, arithmetic, algebra, geometry, or statistics to solve problems .

• Learning Objectives:

Given a problem, the student will be able to• Identify the appropriate quantitative

method(s) necessary to solve that problem.• Use those methods to correctly solve that

problem.

Level

3Exemplary

•The student correctly understands the specific numeric, algebraic, geometric, or statistical method or equation that is needed to solve the problem.•The student completes the process or solves the equation and arrives at an accurate and complete solution of the problem.

2Generally Correct

•The student shows a general understanding of the numeric, algebraic, geometric, or statistical method or equation needed to solve the problem.•The student completes the process or solves the equation in a generally correct way, but with a minor flaw.

Level

1Partially Correct

•The student shows only a slight understanding of the numeric, algebraic, geometric, or statistical method or equation needed to solve the problem.•The student makes an attempt at a process or equation that will solve the problem, but in a way that has very little correlation with the correct solutionSome minor portion of the student’s overall solution is completed correctly.

0Incorrect Solution

•The student shows absolutely no understanding of the numeric, algebraic, geometric, or statistical method or equation needed to solve the problem.•Little or no work is shown that in any way relates to the correct solution of the problem

Learning Outcome #4• Standard: Students will demonstrate the ability to

estimate and check mathematical results for reasonableness .

• Learning Objectives:

Given a mathematical problem, the student will be able to:

• Estimate the result of that problem• Determine and justify the reasonableness of

that result given the constraints of the problem.

Level

3Exemplary

•The student can estimate and justify a mathematical result to a problem. •The student’s justification is developed and the estimate has been found using a clearly defined, logical plan•The student’s response is complete and accurate.

2Generally Correct

•The student can estimate and justify a mathematical result to a problem but the estimate or justification contains a minor flaw•The student’s justification is developed and the estimate has been found was lacking in some minor way•The student’s response addresses all aspects of the question but is lacking in some minor way.

Level

1Partially Correct

•The student can estimate and justify a mathematical result to a problem but the estimate or justification contains a major flaw. •The student’s justification is not developed and the estimate has been found was lacking in some major way•The student’s response addresses some aspect of the question correctly but is lacking in a significant way.

0Incorrect Solution

•The student cannot estimate and/or justify a mathematical result to a problem. •The student’s justification is not supported by any logic plan.•The student’s response does not address the question in any meaningful way or there is no response at all.

Learning Outcome #5

• Standard: Students will demonstrate the ability to recognize the limits of mathematical and statistical methods .

• Learning Objectives:

Given mathematical method, the student will be able to identify and articulate the limits of that mathematical method.

Level

3Exemplary

  Student indicates the assumptions/simplifications made in developing a mathematical/statistical model Student provides an accurate description how the results from the model might diverge from the real life situation it modelsStudent indicates alternative assumptions/models which might be reasonable replacements for those that were used

2Generally Correct

  Student indicates the assumptions/simplifications made in developing a mathematical/statistical model Student provides an accurate description of how the results from the model might diverge from the real life situation it modelsStudent does not indicate alternative assumptions/models which might be reasonable replacements for those that were used

Level

1Partially Correct

  Student indicates only some of the assumptions/simplifications made in developing a mathematical/statistical model  Student realizes that some of the results of the model diverge from real life but is unable to articulate these differences.Student does not indicate alternative assumptions/models which might be reasonable replacements for those that were used

0Incorrect Solution

 Student does not indicate any assumptions/simplifications made in developing a mathematical/statistical model Student fails to realize that the results of the model may diverge from real lifeStudent does not indicate alternative assumptions/models which might be reasonable replacements for those that were used he student does not understand how the model relates to the situation.

An example…  Suppose that you invest $500.00 in an account and that interest is compounded continuously according to the formula    1. If your annual rate of return is 4%,

a.       How much money will you have at the end of 10 years?b.       How long will it take your money to double?

2. What rate of return do you need in order for your money to double every 5 years?

A P ert

Level 3:

The student writes: .Substituting correctly for t demonstrates that the student understands how to use the model to answer the question.

The student writes: .Substituting correctly for P and r demonstrates that the student is able to interpret the significance of those variables given in the model .  The student writes: The balance in the account at the end of 10 years is $745.91.This is a complete and accurate answer.

A 500e0.0410

A 500e0.04 t

A P ert

Level 2:

The student writes: .Substituting correctly for P and r demonstrates that the student is able to interpret the significance of those variables given in the model . The student writes: .Substituting correctly for t demonstrates that the student understands how to use the model to answer the question. The student writes: The balance in the account at the end of 10 years is $5204.05.This is a computational error involving order of operations. It is not unusual for a student not to question a result like this.

A 500e0.04 t

A 500e0.0410

A P ert

Level 1:

The student writes: .Substituting incorrectly for P or r demonstrates that the student has some misunderstanding of how the model relates to the situation. The student writes: .The student attempts to use the model to answer the question.  The student writes: The balance in the account at the end of 10 years is 1.1769E20.This is the calculator display, which is meaningless in this situation.

A 500e4 t

A 500e410

Level 0:

The question is left blank or whatever is written is meaningless.

Questions and Answers

• Do campuses have to assess all of their courses?

• What do they do in cases where some of the learning outcomes are not covered in the courses?

• Can I write my own assessment?

• Can the same rubric be used for all courses?

Questions and Answers

• Is there money for folks at campuses to construct the rubrics?

• What are the learning outcomes?• How were the learning outcomes created? • What is a rubric? • What is a standard?• Are mathematicians using rubrics?• Is there a rubric for each learning outcome?• Does a campus have to use the same exam for all

courses?

Questions and Answers

• Can a campus use pre- and post-tests?

• What happens to the data when the system gets it?

• Can reporters access the data?

• What is the process for a campus to get its assessment plan approved by GEAR? What is the time line?

• How are testing and assessment related?