WHAT DO WE KNOW ABOUT THE QUALITY OF HIGHER LEARNING IN THE UNITED STATES? Marilee J. Bresciani, Ph.D. Professor of Postsecondary Education Leadership.

WHAT DO WE KNOW ABOUT THE QUALITY OF HIGHER LEARNING IN THE

UNITED STATES?

Marilee J. Bresciani, Ph.D.Professor of Postsecondary Education Leadership at San

Diego State University

October 4, 2010

WEBINAR OVERVIEWBRESCIANI, M.J., 2010

Context for Accountability of the Quality of General Learning within the United States

Exploring Comparability of Quality of General Learning

Positing Questions for Consideration

FRAMING QUESTIONSBRESCIANI, M.J., 2010

What do you know about the quality of general learning at your institution?

How does the quality of general learning at your institution compare to the quality of learning at other institutions?

Bresciani, M.J.

CONTEXT FOR ACCOUNTABILITY(BRESCIANI, ET AL, 2009)

To improve the underperforming student

Competency Movement in Business and Industry International Trade Agreements

Bologna Declaration of 1999

Bresciani, M.J.

CONTEXT FOR ACCOUNTABILITY, CONT.(BRESCIANI, ET AL, 2009)

Government Conversation, 1985 The Higher Education Re-authorization Act Testimonies in USA, 2002 and 2006 Response to NCLB Legislation Regional Accreditation – flexibility

CRAC – 2003, 2004 Both documents focus on student learning

Bresciani, M.J.

CONTEXT FOR ACCOUNTABILITY, CONT.(BRESCIANI, ET AL, 2009)

National Commission on the Future of Higher Education

•Demand for Public Information about Performance•Transparency of outcomes and results•Comparable measures of quality•Demonstration of value-added of the entire educational experience

Bresciani, M.J.

CONTEXT FOR ACCOUNTABILITY, CONT.(BRESCIANI, ET AL, 2009)

Accountability requirements handed back to statesPerformance indicators Increased desire for transparency, comparability, and value-added

Discipline Standards still expected to be addressed by disciplines

IN OTHER WORDS…(BRESCIANI, ET AL, 2009)

We are being held accountable for general learning expectations by the public.

The disciplines are “taking care” of accountability via professional accreditation/certification.

Bresciani, M.J.

SO, THE QUESTIONS REMAIN…(BRESCIANI, ET AL, 2009)

So, how effective is our design and delivery of general learning?

How well equipped are we to evaluate that learning?

How well can we compare our quality of general learning with another institution?

Bresciani, M.J.

REPORT OUT

Bresciani, M.J.

Bresciani, M.J.

TIPS FROM GOOD PRACTICE INSTITUTIONS(BRESCIANI, 2007)

They clearly communicate the purpose of the programs that deliver their general learning

There is cross institutional buy-in to the purpose

There is cross institutional buy-in to the outcomes and in some cases assessment methods, if appropriate

Bresciani, M.J.

TIPS FROM GOOD PRACTICE INSTITUTIONS(BRESCIANI, 2007)

They clearly define their expected learning through learning outcomes

They align the process for creating the expected learning with the manner in which they evaluate the learning

They “advertise” the variances in their processes

They advertise the “success” of their general learning with evidence; they improve where they are not successful

Bresciani, M.J.

TIPS, CONT.(BRESCIANI, 2007)

They invite peer critiques, internal to their organization and external to their organization

They use testimonials to demonstrate their effectiveness

Some articulate expectations of and investments in to students and faculty

Some are mindful of their ”inputs”

SECOND SET OF QUESTIONS(BRESCIANI, ET AL, 2009)

Who claims ownership for the design, deliver, and evaluation of your general learning?

How well does the assessment data inform improvements?

How well coordinated is the evaluation and decision-making process?

Bresciani, M.J.

REPORT OUT

Bresciani, M.J.

SUMMARY OF PROS AND CONS OF COMPARABLE DATABRESCIANI, 2010

Standardized Tests Commercially designed Ability to update is limited May be able to align with

learning outcomes May be able to align with

delivery of learning Generates validity and

reliability statistics Less subjective

Rubrics Faculty designed Ability to update is dynamic May be able to align with

learning outcomes May be able to align with

delivery of learning Generates inter-rater

reliability statistics Subjective

CONSIDERATIONS FOR USING RUBRICS FOR COMPARABILITYBRESCIANI, 2010

Can you answer the previously posited questions?

Can your faculty collaborate on the design of rubrics or can your faculty adapt those developed by the AAC&U VALUES project?

Can you collaborate with other institutions to share rubric results and run inter-rater reliability data?

SESSION CITATION

Bresciani, M.J. (October 4, 2010). What do We know About the Quality of Higher Learning in the United States? EDUCAUSE Learning Initiative Webinar

REFERENCES

Bresciani, M. J. (Ed.). (2007). Good practice case studies for assessing student learning in general education. San Francisco, CA: Jossey Bass.

Bresciani, M. J., Gardner, M. M., & Hickmott, J. (2009). Demonstrating student success in student affairs. Sterling, VA: Stylus Publishing.

RESOURCES

AAC&U Leap Report - http://www.aacu.org/leap/documents/GlobalCentury_ExecSum_3.pdf

Raising the Bar: Employers' Views on College Learning in the Wake of the Economic Downturn" (January, 2010) - http://www.aacu.org/leap/documents/2009_EmployerSurvey.pdf

"How Should Colleges Assess And Improve Student Learning? Employers' Views on the Accountability Challenge" (January 2008): http://www.aacu.org/leap/documents/2008_Business_Leader_Poll.pdf

RESOURCES, CONT.

AAC&U VALUE Rubrics - http://www.aacu.org/value/rubrics/index_p.cfm?CFID=25068677&CFTOKEN=29196266

New Leadership Alliance for Student Learning and Accountability - http://www.newleadershipalliance.org/

Bresciani, M. J., Oakleaf, M., Kolkhorst, F., Nebeker, C., Duncan, K., Barlow, J., & Hickmott, J. (2009). Examining Inter-Rater Reliability for a Research Methodology Rubric. [Electronic version] Practical Assessment, Research, & Evaluation, 14(13A),1-7.

QUESTIONS?

[email protected]