Common Definitions

Boston University Center for Excellence and Innovation in Teaching


Activities undertaken by teachers –and by their students in assessing themselves—

that provide information to be used as feedback to modify the teaching and learning activities in which

they are engaged.

Black, P.J. & Wiliam, D. (1998) Inside the Black Box: Raising standards through classroom assessment. King’s College, London.

(see http://en.wikipedia.org/wiki/Assessment_for_Learning)

Common Definitions

http://en.wikipedia.org/wiki/Assessment_for_Learning

Teaching

versus

Learning

Finding the right measures.


What’s the Goal?

Start with the end goal and work backwards!


• Assessment for learning• Assessment as learning• Assessment of learning• Evaluation• Pre-assessment• Formative assessment• Summative assessment

Types of Assessment


• Comprises two phases.• Pre-assessment. • Formative assessment.

• Is based on a variety of information sources.• Portfolios.• Works in progress.• Teacher observation.• Conversation.

• Provides feedback (verbal or written) to the student that is descriptive and emphasizes strengths, identifies challenges, and points to next steps.

• Involves instructor adjustment of instruction to keep students on track. • Does not provide grades or scores; record-keeping is anecdotal and descriptive. • Occurs throughout the learning process, from the outset of the course of study

to the time of summative assessment.

Assessment for Learning


• Requires student awareness of goals of instruction and criteria for performance.

• Involves goal-setting, monitoring progress, and reflecting on results.

• Implies student ownership and responsibility for moving his or her thinking forward (metacognition).

• Occurs throughout the learning process.

Assessment as Learning


• Is accompanied by a grade (summative)

• Compares one student’s achievement with standards

• Produces results that can be communicated to the student

• Occurs at the end of the learning unit

Assessment of Learning


A judgment made on the basis of a student’s performance.

Evaluation


• Determines what a student does and does not know about a topic

• Determine a student's learning style or preferences used to determine how well a student can perform a certain set of skills related to a particular subject or group of subjects

• Occurs at the beginning of a unit of study

• Is used to inform instruction

• Makes up the initial phase of assessment for learning

Pre-Assessment


Formative assessment• Determines a student’s knowledge and skills, including learning gaps as

they progress through a unit of study. • Is used to inform instruction and guide learning. • Occurs during the course of a unit of study. • Makes up the subsequent phase of assessment for learning.

Summative assessment• Is done at the end of a unit of study to determine the level of

understanding the student has achieved. • Grades against an expected standard.

Formative and Summative Assessment


Assessment can be conducted at various levels:

• Student• Course• Major• Degree• Program

Levels of Analysis


Teaching-as-Research

• Deliberate, systematic, reflective use of research methods to develop and implement teaching practices that advance the learning experiences and outcomes of students and teachers.

• Application of a research approach to teaching practice.

Assessing Effectiveness of Pedagogy

http://www.cirtl.net/CoreIdeas/teaching_as_research


• Learning foundational knowledge.

• Creating objectives for student learning.

• Developing an hypothesis for practices to achieve the learning objectives.

• Defining measures of success.

• Developing and implementing teaching practices within an experimental design.

• Collecting and analyzing data.

• Reflecting, evaluating, and iterating.




• What is known about the teaching practice?

• What do we want students to learn?

• How can we help students succeed with the learning objectives?

• What evidence will we need to determine whether students have achieved learning objectives?

• What will we do in and out of the classroom to enable students to achieve learning objectives?

• How will we collect and analyze information to determine what students have learned?

• How will we use what we have learned to improve our teaching?




VALUE (Valid Assessment of Learning in Undergraduate Education) assumes that:

•To achieve a high-quality education for all students, valid assessment data are needed to guide planning, teaching, and improvement;

•Colleges and universities seek to foster and assess numerous essential learning outcomes beyond those addressed by currently available standardized tests;

•Learning develops over time and should become more complex and sophisticated as students move through their curricular and cocurricular educational pathways toward a degree;

•Good practice in assessment requires multiple assessments, over time;

•Well-planned electronic portfolios provide opportunities to collect data from multiple assessments across a broad range of learning outcomes while guiding student learning and building self-assessment capabilities;

•e-portfolios and assessment of work in them can inform programs and institutions on progress in achieving expected goals.

http://www.aacu.org/value/index.cfm

Association of American Colleges and Universities: AAC&U

Example: AACSB Assurances of Learning


• Introduced outcomes assessment in 1991 standards• Changed in 2003 to reflect need for improved

accountability measures. • Emphasize direct assessments of student

learning. • Requires formulation of specific learning goals.

• Support two principles: accountability and continuous improvement.

Fundamentals


The systematic collection, review, and use of information about educational programs undertaken

for the purpose of improving student learning and development.

Assessment essentials: planning, implementing, and improving assessment in higher education, Palomba and Banta, 1999.

Outcome Assessment


1. Define student learning goals and objectives.2. Align curricula with the adopted goals.3. Identify instruments and measures to assess

learning.4. Collect, analyze, and disseminate assessment

information.5. Use assessment information for continuous

improvement.

Process


1. What will our students learn in our program? What are our expectations?

2. How will they learn it?

3. How will we know they have learned it or not?

4. What will we do if they have not learned it?

Process: Another Form


The Focus is the Program

• Course-level assessments are not the focus

• Learning goals should reflect:

• Broad educational expectations for each degree program, regardless of major.

• Major intellectual and behavioral competencies a program intends to instill in its students due to the total educational experience across a given program.

Program Focus


“Learning goals should express expectations that reflect the expected depth and breadth of student

knowledge and skills that are the sustainable foundations for life-long learning in support of their

professional and personal development.”

Program Focus


Our graduates will understand the importance of behaving ethically in their professional lives (i.e., have an ethical perspective).

Learning Goal Examples


Corresponding Objectives:

•Our students will know the professional code of conduct within their discipline.

•Our students will identify the activities/issues in their chosen profession that may present ethical challenges, and will articulate the consequences associated with unethical behavior.

•Our students will identify an ethical dilemma in a scenario case and apply an ethics model or framework to propose and defend a resolution.



Our graduates will demonstrate problem solving skills, supported by appropriate analytical and quantitative techniques.




•In a case setting, students will use appropriate analytical techniques to identify a business problem, generate and compare alternatives, and develop a solution.

•In a case setting, students will recognize and analyze ethical problems, choose, and defend a solution.



Our graduates will be effective communicators.


•Our students will produce professional quality business documents.

•Our students will deliver a professional quality presentation accompanied by appropriate technology.

•Our students will demonstrate effective interpersonal communication skills in a team setting.



Learning Objective: Students will demonstrate competency in a second language to qualify for admission.

Assessment Measure: Acceptable score on standardized examination of a second language is required for admission to a business program.

Assessment Measures Examples


Learning Objective: Students will demonstrate competency in statistical analysis and reasoning skills.

Assessment measure: A placement exam is given to measure competency as part of the admissions process.

Assessment Measures Examples (selection)


Learning Objective: Our students will produce professional quality business documents.

Assessment measure: Students’ work from a case analysis assigned in a senior capstone course is assessed for writing effectiveness.

Assessment Measures Examples (course embedded)


Learning Objective: Our students will demonstrate appropriate abilities to integrate knowledge across business disciplines in a case solution.

Assessment measure: A case analysis presentation in a business strategy class is assessed for developing a strategic recommendation that spans functional areas.

Assessment Measures Examples (course embedded)


Examples: alumni surveys, graduating student surveys, and/or employer surveys.

Information from indirect measures can complement the direct assessment processes, but they are not acceptable substitutes for direct assessment of learning.

Indirect Measures


• Course grades are aggregate measures on examinations, research papers, other projects, homework, presentations, class participation, etc. and, therefore, cannot measure students’ performance or achievement of a specific skill or knowledge set.

• Professors set the criteria and standards for their courses and these may not be completely aligned with program-level learning goals because of the focus on learning related to the specific business discipline for which the faculty member is responsible.

Course Grades Don’t Cut It


• Course level: instructors want to know whether their students are getting the “key points” of the course.

• Major

• Some majors have “disciplinary” expectations.

• Department chairs/deans want to know whether their students are achieving the learning goals for the major.

• Program

• Most programs have accreditation requirements.

• Provosts want to know whether students are achieving the learning goals for the program

Generalizability?


1. The results of assessment will be used to evaluate faculty performance. No. Faculty awareness, participation, and ownership are essential, but assessment should not be used to evaluate or judge individual faculty performance. Program assessment findings are used to improve programs.

2. Our program is working well; we don’t need to bother with assessment.Program assessment findings are used to improve programs – and there’s always room for improvement.

3. One person can do this; too many faculty will only complicate things.Having all faculty involved means bringing in different perspectives and ideas for improving the academic program – and helps with buy-in.

From University of Central Florida: http://oeas.ucf.edu/doc/acad_assess_handbook.pdf

Misconceptions About Program Assessment


4. Program assessment might be used to eliminate some programs. Program assessment is generally intended to be formative evaluation.

5. Assessment is a waste of time and does not benefit the students.Assessment “forces” articulation of objectives and learning outcomes in order to improve student learning. Is this a waste of time?

6. Not so bad; we’ll come up with a good plan and keep using it. Program assessment must be an ongoing and continuous process.

7. Okay – but it’s time-consuming and complex. Effective program assessment will take some of your time and effort, but there are ways to make it manageable.

Misconceptions About Program Assessment


Must all students be assessed?

Sampling is okay as long as there is a high degree of confidence that the data are representative, valid, and reliable.

AACSB FAQs


What is the minimum performance standard?

The goal for overall student performance on any given learning goal should be determined by each school consistent with its mission, degree programs, and student profile.

AACSB FAQs


Must all graduates meet the expected standard on all learning goals?

No, but if students are not achieving the learning goals at acceptable levels, action must be taken to strengthen the curriculum for future students.

AACSB FAQs


Can a faculty member be responsible for the assessment of a learning goal within his/her class?

Yes, but the learning goal, operational objective, and supporting rubrics for evaluating performance must be used consistently and follow the collective decisions of the faculty as a whole regarding what constitutes acceptable student performance.

AACSB FAQs


Must learning goals be measured separately?

Each learning goal should have its own performance standard, but a common method or activity can be used to gather data on more than one learning goal. For example, a case analysis may be useful for assessing analytical thinking as well as writing skills or a presentation may be used for evaluating oral communication skills and business disciplinary competence.

AACSB FAQs


Is benchmarking against other institutions required?

No.

Should student learning be assessed at multiple points in the curriculum?

This is not required – but it’s a good idea!

AACSB FAQs


If specific students do not perform well on assessments, is remediation required?

No, but again, if many students have problems, the curriculum should be addressed and evidence of that should be available.

AACSB FAQs


Can the collective work of student teams be used for assessment?

Not for individual performance, but the collective work of the team may provide a basis for assessing performance as a team member.

How many learning goals are needed?

Four to 10 learning goals for each degree program. There is no limit, but this is the guidance in order to keep the assessment program manageable.

AACSB FAQs


Must all learning goals be assessed each year?

No, but a systematic process is needed to insure all goals are assessed to support meaningful curricula change and development. Normally, each goal should be evaluated at least twice over a five-year AACSB review cycle.

AACSB FAQs


What documentation must be maintained?

Schools should maintain copies of instruments, course-embedded assignments, scoring grids or rubrics, summary of data and analyses, samples of student products used, documentation that the data was used, and documentation of the curricula actions that were taken based on assessment results.

AACSB FAQs


What are the most popular AACSB learning goals?

• Effective communication.

• Skills.

• Ethics.

• Knowledge of all business disciplines

• Critical thinking.

• Effective decision making.

AACSB FAQs


What are the most popular AACSB learning goals (continued)?

• Problem solving ability.

• Ability to integrate across business disciplines.

• Global perspective.

• Team skills.

• Competency in the major.

AACSB FAQs


Questions/Discussion?

Common Definitions

Documents

formboston university

lifelong learning

use assessment information

degree program

given program

adopted goals

focuslearning goals

teachinglearning goals