Using a standards alignment model as a framework for doctoral candidate assessment

Using a Standards Alignment Model as a Framework for Doctoral Candidate

AssessmentSketching a Road Map

CPED October 2013 Convening

Hosted by Rutgers University

Dr. Valerie A. Storey

Fall Convening, 2007Hosted by Peabody College, Vanderbilt

University

The question is…“How do we create a framework for

assessment and accountability that takes advantage of our diversity and yet helps us account for our efforts to reclaim education’s doctorates within and across programs, strands, and institutions?”

CPED Meeting at American Association of Colleges for Teacher Education (AACTE), New

Orleans, Imig & Perry, 2008

Program Design Concepts:

Capstones•Backward mapping•Use of state standards•Program design & leadership analysis

Spencer Foundation Grant, 2008Preparation of Professional Practitioners

$75,000 funding to identify outcomes of phase 1 institution programs.

Consortium identified six characteristics of graduates that should result from preparation in a CPED-influenced EdD program:

1. Equity 4. Inquiry stance

2. Commitment to continuous change 5. Community engagement/social responsiveness

3. Leadership capabilities 6. Harnessing human capital

Ref: Imig, Perry, Syed, 2009

Palo Alto, CA Convening, June 2009

Goldring & Yinger challenged consortium members to Reflect on the results of the outcome data derived from the

Spencer funded research; Consider how outcomes would be tested both by program and

candidate.

Members responded by developing principles of best practice for CPED EdD programs.

Ref: Perry & Imig, 2010

Working Principles for the Professional Practice Doctorate in Education

1.Is framed around questions of equity, ethics, and social justice to bring about solutions to complex problems of practice.

2.Prepares leaders who can construct and apply knowledge to make a positive difference in the lives of individuals, families, organizations, and communities.

3.Provides opportunities for candidates to develop and demonstrate collaboration and communication skills to work with diverse communities and to build partnerships.

4.Provides field-based opportunities to analyze problems of practice and use multiple frames to develop meaningful solutions.

5.Is grounded in and develops a professional knowledge base that integrates both practical and research knowledge, that links theory with systemic and systematic inquiry.

6.Emphasizes the generation, transformation, and use of professional knowledge and practice.

CPED Meeting at American Association of Colleges for Teacher Education (AACTE)

Atlanta Hilton - February 19, 2010

Program Design Concepts:CPED members:

Need to operationalize conceptual ideas; Need to stay faithful to CPED definitions as we move from the theoretical to the operational; Need to develop an assessment model grounded on CPED principles, and test over time.

Identify what we have and what is the gap?

Start a student-generated data base that is evidence of how we work with CPED principlesSignature assessments: Define 3 that all institutions use and study their efficacyStrong formative aspect and summative; focus on capacities of students as they develop and then leave the program; student-tracking database; Content analysis of capstone process; guiding question of what impacts our grads are making in the field

This presentation-3 broad topics

How we decided as an institution to move forward

Process of program design Development of “gatekeepers.” Design of assessment activities to provide

ongoing data for continuous improvement

We relied on two inter-related theories of action

1. Understanding institutional change as individual and collective learning growth, and capacity building grounded by CPED critical friends ( Tharp, Estrada, Dalton, Yamauchi, 2000);

2. Utilizing curriculum design as the framework for planning for institutional renewal.

Starting with the End in Mind: What Are Our Intended

Results?

Outcomes: What changes in student attitudes, behaviors, skills, status, and/or level of functioning do we intend?

Outputs: What changes in our program design do we hypothesize will lead to these outcomes?

Impact: How will our outcomes enhance educational practice,

research, and policy in the metropolitan area that the university serves?

BUT FIRST-who are our candidates? Conduct needs assessment

Soul searching… Putting the Needs Assessment data to work

Vision statement-What is the purpose of the program? What is the rationale and educational purpose of each

element of the professional practice doctoral program?-address themes, elements, and learning outcomes.

What evidence aids the answering of the above questions?

Golde, Jones, Bueschel, Walker, 2006

Cognizant of … More value in a department assessing the quality of its

program regularly and continuously when done in reference to clearly agreed upon vision for the program and its students;

Golde, Jones, Bueschel, Walker, 2006

Need to measure the growth and development of students during the doctoral program.

Golde, Jones, Bueschel, Walker, 2006

Coincidentally… University actively develops its outcomes assessment program, requires faculty in all

programs to develop statements of intended learning outcomes and gather assessment data to evaluate whether or not the desired outcomes are being achieved;

Program Advisory Committee-Educational Leadership & Interdisciplinary faculty begin a two-year process of identifying the knowledge, skills, competencies, and perspectives that they wished all students to possess upon graduation;

Create a set of standards (i.e., rubrics) that describe the characteristics of excellent work in each of the learning outcome areas.

Develop student and faculty guidelines for using LiveText to craft a Scholarly Portfolio incorporating assignment rubrics, enabling students to monitor and shape their own learning and, in the process, develop reflective and self-awareness skills.

1. Program theory of action underpins program design and identified assumptions on

candidate outcomes

What content is essential and why? Which pedagogical practices are essential? What learning experiences are created, and how are these developmental? How are Laboratories of Practice designed and supported ? What program supports (such as cohort structure, advisement, and alternative

delivery formats) facilitate program delivery? What is the role of assessment in fostering candidate development? What are the knowledge, skills, and program commitments of core faculty How does the program recruit faculty, and what values guide faculty recruitment and

selection, as a means of pursuing the program’s theory of action? What assumptions link these components (i.e., how do these components, taken

together, effectively prepare candidates who reflect the program’s core values)?

2. Program Design & Program Standards

What are the key understandings (e.g., big ideas)? What are the essential questions of the field that this course addresses? What are students expected to know and be able to do as a result of this

course? What performance tasks will students do to demonstrate their knowledge

and understanding? Core readings:

To what extent do readings engage candidates in connecting core research with the skills necessary to support recommended practices?

How are candidates asked to measurably demonstrate competency? How do these relate to the program’s theory of action?

file://localhost/Volumes/KINGSTON/Academic Assessment Planning Matrix.docx

Setting Quality Program Standards

Mapping to Program Standards

  Exemplary (3 pts) Proficient (2 pts) Unacceptable (1 pt)  

Leadership

FL-LYNN-EDD.3.1

Analyzes an authentic problem of practice in which the elements and interactions of leadership, vision, mission and goals can be maximized and lead to an inclusive action plan or agenda which is clear, easily translated into work tasks, and evaluated when completed.

Understands and, with minor difficulty, analyzes an authentic problem of practice exhibiting the inter-relatedness among leadership, vision, mission, and goals and plan development.

Is not able to analyze an authentic problem of practice or exhibit an understanding of the interactive nature between leadership, vision, mission, and goals or translate them into a coherent action plan or agenda.

Leadership

FL-LYNN-EDD.3.3

Is able to identify the situational, contextual, and cultural aspects of an organization which are necessary to attain balance and that will lead to enhanced educational practice and outcomes.

Has some understanding of the situational, contextual and cultural aspects of an organization relative to balance and enhanced educational practice and outcomes.

Is not able to identify or relate any specific organizational context, culture, or situationally unique aspects of organizational balance in any setting to enhance educational practice and outcomes.

Standard 3-Leadership for Learning,EDU 701- Leadership, Policy & Context , Critical Assignment Performance Rubric

Authentic MeasureScholarly Practitioner Portfolio (SPP)

Electronic Web-based format i.e. LiveText.

Organization of the SPP framed by the program’s standards & learning outcomes.

Incorporates Summary data, curriculum vita etc Individual reflection papers and artifacts matched against the dimensions of each

learning outcomes. Emphasis on meta cognitive development. Demonstrate via portfolio evidence the ability to address practitioner problem with

scholarly knowledge.

Admission to candidacy is predicated on successful presentation of SPP to SPP committee.

Scholarly Practitioner Electronic Portfolio

Scholarly Practitioner Portfolio presentation:

Authentic assessment evaluating the desired learning outcomes of the program, abilities and dispositions that characterize a professional practice doctorate;

(Wiggins, 1998).

Authentic tests require the performance of exemplary tasks, individual judgments, reflection, and dialogue.

(Wiggins, 1989).

Scholarly Practitioner Portfolio (SPP) Presentation

Student: Presents selected assignment that addresses a program

standard; Reflects and demonstrates via artifacts that program knowledge

applied to practitioner competencies;

SPP Faculty Committee: Individual rubrics combined for overall assessment. Verbal and later written feedback to student. Identification of program standards met –strengths and

weakness in meeting learning outcomes; Program review to address identified program weakness.

Dissertation in Practice Designed to engage students in a consultative

relationship with an educational agency e.g. school district, college, university, or other educational organization;

Demonstrate the knowledge & competencies of a

scholarly practioner;

Both the SPP & DiP designed to provide faculty with data they could use to make judgments not only about individual students’ learning, but also about students’ achievements collectively through program assessment activities i.e. a strategy that has the potential to establish a reciprocal learning and assessment relationship for both candidate and faculty.

References Imig, D., Perry, J.A., & Syed, S. (2009). Creating rubrics for the assessment of the

EdD: Narrative report to the Spencer Foundation. College Park, MD: Carnegie Project on the Education Doctorate.

Jacobs, H. H. (1997). Mapping the big picture: Integrating curriculum and assessment K-12. Alexandria, VA: Association for Supervision and Curriculum Development.

Maki, P. L., & Borkowski, N.A. (2006) (Eds). The assessment of doctoral education: Emerging criteria and new models for improving outcomes. Sterling, Virginia: Stylus Publishing.

Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco: Jossey-Bass.

Perry, J.A., & Imig, D. (2010). Final Report: The Carnegie Project on the Education Doctorate 2007-2010.

Wiggins, G., & McTighe, J. (1998). Understanding by Design. Alexandria, VA: Association for Supervisions and Curriculum Development.