Copyright © 2006 by ABET, Inc. 1 Quality and Innovation in Technical Education An ABET Perspective Bill Clark.

1

Copyright © 2006 by ABET, Inc.

Quality and Innovation Quality and Innovation in Technical Educationin Technical Education

An ABET Perspective

Bill Clark

2


OverviewOverview

ABET’s structure and accreditation process

Addressing quality and innovation through accreditation criteria

Quality improvement for ABET Q&A

3


ABET Vision:ABET Vision:

ABET will provide world leadership in assuring quality and in

stimulating innovation in applied science, computing, engineering,

and technology education.

http://www.abet.org/mission

4


ABET Mission:ABET Mission:

Accredit educational programs. Promote quality and innovation in education. Consult and assist in the development and advancement of

education worldwide in a financially self-sustaining manner. Communicate with our constituencies and the public

regarding activities and accomplishments. Anticipate and prepare for the changing environment and

the future needs of constituencies. Manage the operations and resources to be effective and

fiscally responsible.

ABET serves the public through the promotion and advancement of applied science, computing, engineering, and technology education. ABET will:

5


ABET BackgroundABET Background

Accredits over 2800 applied science, computing, engineering & technology programs at over 550 colleges & universities

Established 1932 Engineers Council for Professional

Development (ECPD) Accreditation Board for Engineering and

Technology (ABET)

6


How is ABET Structured?How is ABET Structured?

ABET is a federation of 28 professional engineering & technical societies.

Neither institutions nor individuals are Members of ABET

ABET relies on the services of approx. 1500 volunteers and 30 staff

7


ABET Board

Applied Science

Accreditation Commission

72 accredited programs at 54

institutions(+9%)

Computing


261 accredited programs at 222

institutions

(+9%)

Engineering


1797 accredited programs 364

institutions

(+2%)

Technology

AccreditationCommission

685 accreditedprograms at

230 institutions

(+3%)

Sept 2006Sept 2006

8


Accredited Engineering Technology Programs

0

50

100

150

200

250

300

350

400

450

500

Assoc

Bach

9


Team Chair

Editor 1

Editor 2

Commission Meeting

Team

Member Societies

(28)

Program Evaluation Process*Program Evaluation Process*

* For a visit* For a visit Institution Self-Study

Team Chair

Team Chair

Team Team Team Team

10


Continuous Quality Continuous Quality Improvement and Technology Improvement and Technology

AccreditationAccreditation

The Technology Accreditation Commission (TAC) has long history of continuous improvement emphasis

Before EC2000 (outcomes-based accreditation criteria for engineering programs), technology accreditation criteria included specific language on outcomes assessment

11


ABET Criteria for ABET Criteria for Engineering TechnologyEngineering Technology

circa 1990circa 1990

Highly structured – 11 pages of General Criteria – about 10K words

Quantitative and qualitative elements Considerable specificity and guidance Path to compliance reasonably clear Evaluation relatively straightforward

12


ABET Criteria for ETABET Criteria for ETcirca 1990circa 1990

Criteria virtually defined an Engineering Technology program

Contrasted ET with engineering including typical job opportunities

Quantified minimum standards for curriculum and faculty

Encouraged experimental or innovative programs

13


ABET Criteria for ETABET Criteria for ETcirca 1990circa 1990

Program goals shaped by institution goals, student body served; Industry Advisory Committees

Programs goals with measurable objectives Demonstrate success through student

outcome assessment Surveys of students, graduates, employers;

standardized test results

14


Why Change??Why Change??

1992: ABET Board established Accreditation Process Review Committee (APRC) to consider engineering accreditation (EAC processes)

Response to strong message that customer focus and continuous quality improvement essential for American industry to compete globally and that engineering education must also change

ABET’s rigid application of criteria a barrier to needed innovation

15


CQI and AccreditationCQI and Accreditation 1994: Conclusions of the APRC:

Accreditation criteria must be more flexible and include outcomes assessment

Accreditation process must be constructive for the institution and educational for all involved

Volunteer participation processes must ensure that the right people are doing the right jobs at ABET

16


CQI and AccreditationCQI and Accreditation Criteria Reform Workshop findings:

Criteria too long and very nature encourages a rigid, bean counting approach that stifles innovation

Revised criteria should focus on quality and professional preparation, offer curricular flexibility, and be applicable to diverse spectrum of institutional missions and goals

Outcome was EC2000

17


TAC Addresses CQITAC Addresses CQI

1995: Mid-cycle commission meeting to improve processes in six areas, including the basis for accreditation (criteria)

1996: TAC Executive Committee appoints special Criteria Task Group

18


TAC Moves ForwardTAC Moves Forward 1997: Task Group presented initial version

of new criteria to TAC Features compared to traditional criteria:

New structure similar to EC2000 including graduate capabilities (Section 1, a.- k.)

Shorter, about 2K words No quantitative elements for curriculum Faculty quantities and qualifications retained Goals, assessment, student outcomes, and

industry involvement emphasized

19


TC2K Focus, TC2K Focus, RequirementsRequirements

Outcomes, not inputs Students, not faculty What is learned, not what is taught Demonstrate that constituents are directly

involved in developing, assessing, and improving the program.

Demonstrate that there is a formal assessment process in place and that it’s working to improve the program.

20


Quality and Continuous Quality and Continuous ImprovementImprovement

We know what we do.

We do it well.

We can prove it!

Glass Manufacturing PlantLibbey-Owens-Ford (~1990)

20

21


ABET Initiates Own ABET Initiates Own CQI ProcessCQI Process

2002: Engineering Change: A Longitudinal Study on the Impact of EC2000 What impact, if any, has EC2000 had on

student learning outcomes in ABET-accredited engineering programs.

What impact, if any, has EC2000 had on organizational and educational policies and practices that may have improved student learning outcomes?

Next cycle to include impact of TC2K.

22


Study ConclusionStudy Conclusion

The weight of the accumulated evidence … indicates clearly that the implementation of the EC2000 accreditation criteria has had a positive, and sometimes substantial, impact on engineering programs, student experiences, and student learning.

Executive summary available at www.abet.org

23


Customer PerspectiveCustomer Perspective

Statistical results show positive change based on faculty and administrator responses, but changes have caused dissatisfaction in some areas

Increased work content Uneven application of criteria among

accreditation teams and members

24


ABET Responds with ABET Responds with ChangesChanges

Assessment expertise Added Gloria Rogers to staff Offer workshops, symposia for institutional

representatives Provide assessment perspective to ABET

commissions TEI Conferences for Engr Tech faculty Participation Project: Volunteer selection,

training, evaluation, recognition

25


The Participation ProjectThe Participation Project

Design “a comprehensive and effective program that optimizes the use of the expertise and experience of the volunteer professionals that participate in ABET’s outcomes-based accreditation process.”

Participation Project Concept Paper

26


The Program Evaluator The Program Evaluator (PEV) is the Face (PEV) is the Face

of ABETof ABET

27


The Participation ProjectThe Participation Project Enhance the value of accreditation Embrace publicly ABET’s

commitment to continuous improvement

Align PEV process with vision of outcomes-based accreditation

Complete the Reform Initiative begun in 1994

28


Design Team ProductsDesign Team Products Competency Model – sets expectations

Recruitment & Selection: Proactive recruiting processes Online application & references Selection process using new tools

Training: Adult learning based training (pre-work/F2F) Opportunities to demonstrate competencies

Performance Evaluation: Competency-based evaluation form Peer ratings

29


What are Competencies?What are Competencies?

Competencies are behaviors (which include knowledge, skills, and abilities) that define a successful PEV

Set expectations Align with vision, values, and

strategy Drive continuous improvement

30


Competency ModelCompetency ModelCompetency Effective Communicator Interpersonally skilled Professional Organized Team Oriented Technically current

Approved by ABET BoD Oct 2005

31


Recruitment & Selection PilotRecruitment & Selection Pilot

Piloted with live applicants - ASME Recruiting e-mails (targeted candidates) New competency-based application form New competency-based reference form Guide for Selection process

Piloted on-line technology with mock applicants - IEEE

32


Pro-active recruiting E-mail yielded more applicants and more

diversity

Application Applicants put in the time to complete the

lengthier application Applicants agreed to complete additional forms Open-ended questions yielded more valuable

information References validated or changed selection

team decisions


Key Findings…Key Findings…

33


Recruitment and SelectionRecruitment and SelectionWhat is the same? Societies still make

decisions re: WHO is recruited & selected & assigned.

What is different? New tools:

On-line competency based application

On-line competency based references

ABET will assist in Diversity recruiting & provide promo materials

34


Selection Selection Committee meeting is

important to the process Evaluating candidates against selection

criteria (i.e. the competencies) improved process

Some applicants that would have been selected in the past were not chosen


Key Findings…Key Findings…

35


Evaluation PilotEvaluation Pilot

Performance Appraisal forms: Describe how competencies are demonstrated

pre-visit and during visit Provide performance metrics Require comments for below “met

expectations” 360-degree feedback

Decision framework Closed loop process

36


EvaluationEvaluation

What is the same? On-line form Executive

Committees’ decision

Member society input

ABET manages logistics

What is different? Competency-based 360΄evaluation Definable rubrics Closed loop

process for recognition, remediation, removal

37


Institution’s/Program’s Institution’s/Program’s RoleRole

You are ABET’s constituents Without your feedback ABET cannot

improve Performance evaluations DO NOT

affect the accreditation decision

38


TrainingTraining

“Need standardized training where PEVs participate in a mock visit and are familiar with forms.”

“How do we structure training so the necessary attributes and skills are made obvious during training (as a result of the training itself)?”

39


Current TrainingCurrent TrainingSocietySociety Trainer Perspective…Trainer Perspective…

6-8 hours of “death by PowerPoint” Limited interaction Little opportunity to ensure consistent

knowledge of criteria and assessment process among participants

No opportunity to demonstrate and assess PEV competencies

40


Member Society selects PEV candidate

Society assigns mentor

Candidate works

preliminary modules on-line

Candidate attends

visit simulation seminar

Candidate successfully completes

visit simulation seminar

Program Evaluator

Support Facilitator

Core Facilitator

Becoming an ABET Program Evaluator

Candidate attends program specific training

(Society)

PHASE I

PHASE I I

PHASE I I I

Observer visit (optional)

Candidate successfully completes modules on-line

Society approvesPEV for

assignment

41


Training PilotTraining Pilot

Pre-Work CD with Checks for Understanding Mentor Assigned Self-Study Complete pre-visit forms

1.5 days simulating campus visit Sunday team meeting Display materials and lab interview Draft statement homework Monday night meeting

42


TrainingTraining

What is the same? ABET Commissions oversee curriculum

design and training process Member society recruitment for trainers Member Society trains on Program Criteria

43


TrainingTraining

What is different? Simulated visit based on adult learning methods Regional trainings commission specific Train-the-trainer Lead, Support Facilitators

selected by Board-appointed Volunteer Training Committee

Expanded Mentor Role Observers at Training PEV candidates only

44


Where Are We Now?Where Are We Now?

Competency Model approved for inclusion in ABET’s Rules of Procedure

Design work and pilots completed

Performance Appraisal Process approved by Accreditation Council for use 2006-2007 cycle

ABET BoD approved Steering Group to oversee implementation

45


SummarySummary ABET mission is quality assurance and

innovation in technical education Completely revamped criteria to

encourage this perspective Initiated its own quality assurance

review Committed to continuous

improvement through its actions

Copyright © 2006 by ABET, Inc. 1 Quality and Innovation in Technical Education An ABET Perspective Bill Clark.

Documents

innovation slide

engineering programs

ec2000 slide

qa slide

straightforward slide

engineering education

outcomes assessment

technology education