Academic Program Evaluation Using BCSP Criteria A Case Study at the University of Minnesota Duluth Robert Feyen, Ph.D., CPE Asst. Professor/MEHS Program Director, University of Minnesota Duluth T.W. Loushine, Ph.D., P.E., CSP, CIH Asst. Professor, University of Wisconsin - Whitewater
20
Embed
Robert Feyen, Ph.D., CPE Asst. Professor/MEHS Program Director, University of Minnesota Duluth
Academic Program Evaluation Using BCSP Criteria A Case Study at the University of Minnesota Duluth. Robert Feyen, Ph.D., CPE Asst. Professor/MEHS Program Director, University of Minnesota Duluth T.W. Loushine, Ph.D., P.E., CSP, CIH Asst. Professor, University of Wisconsin - Whitewater. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Academic Program Evaluation Using BCSP Criteria
A Case Study at the University of Minnesota Duluth
Robert Feyen, Ph.D., CPEAsst. Professor/MEHS Program Director, University of
Minnesota Duluth
T.W. Loushine, Ph.D., P.E., CSP, CIHAsst. Professor, University of Wisconsin - Whitewater
UMD MEHS programBackground
• Started in 1976 with roughly 750 graduates to date• Practitioner-based program
– Designed primarily to prepare students as generalists in EHS, but allow room for some specialization
Mission statementThe Master in Environmental Health and Safety program at the University of Minnesota Duluth will produce highly-regarded and sought-after graduates who have the requisite skills and knowledge to practice environmental health and safety effectively in a diverse range of occupations and will do so in a competent, professional and ethical manner.
Curriculum: Content (2008)Core (32 credits)• SAFE 6002—Regulatory Standards and Hazard Control (3 cr)• SAFE 6011—System Safety and Loss Control Techniques (3 cr)• SAFE 6012—Risk Management and Workers’ Comp (2 cr)• SAFE 6051—Construction Safety Management (3 cr)• SAFE 6101—Principles of Industrial Hygiene (3 cr)• SAFE 6111—Industrial Noise and Ventilation Control (3 cr) • SAFE 6301—Occupational Biomechanics (2 cr)• SAFE 6302—Occupational Ergonomics (3 cr)• SAFE 6401—Environmental Safety and Legal Implications (2 cr)• SAFE 6802—Leadership, Teamwork, Behavior in EHS (3 cr) • SAFE 6821—Organization and Administration of Safety Programs (2 cr)• SAFE 6997—Cooperative Internship Electives (7 credits)• SAFE 6102—Advanced Industrial Hygiene (2 cr)• SAFE 6121—Epidemiology and Industrial Toxicology (2 cr)• SAFE 6201—Fire Prevention and Emergency Preparedness (2 cr)• SAFE 6211—Transportation Safety (2 cr)
Curriculum: Subjective Feedback
• Internal (University-level) administration– Economic constraints (e.g., coordination of resources)– Academic “consistency” with other graduate level programs
• EHS professionals (formal and informal)– Alumni– Employers– Practitioners
• Peer graduate programs in safety and health• Funding agencies (e.g., NIOSH)• External reviewers
Curriculum: Objective Assessment
• Literature– Graduate Research Council reports (e.g., student demographics)– ASSE guidelines for ABET accreditation– Academic journals (e.g., Journal of Safety Science)– ASEE and ASSE proceedings papers on curriculum assessment– BCSP outcome assessment program
• Developed an assessment approach based on Brauer (2006)– Utilized the BCSP subject matter tables (2008)– However, no other literature could be found regarding this approach
or its use
• The (the devil) details– Took a few tries, several modifications and a GRA to obtain data on
each course– Took even more tries to figure out what to do with the data…
Method• Instructors identify knowledge items from the BCSP subject
matter tables to be used in evaluating each course taught • Develop assessment matrix of courses and relevant
knowledge items• Conduct interviews with students and instructors to rank
emphasis and rate coverage of each item• Enter item ratings and rankings for each course into
spreadsheet• Analyze and interpret data
– Aggregated over all knowledge items (n=249)– For each subject area, use ratings to determine percent of items
covered within overall program and individual courses– Within subject area, identify knowledge items not covered
thoroughly or requiring additional evaluation
Method: Subject Matter (example)
Measurement and Monitoring• Knowledge (Foundation Level)
– Methods and techniques for measurement, sampling, and analysis (D2- T1)
– Uses and limitations of monitoring equipment (D2-T1) • Knowledge (Advanced Level)
– Electronic data logging and monitoring equipment (D1-T2)– Electronic data transfer methods and data storage options (D1-
T2) – Methods and techniques for measurement, sampling, and
analysis (D1-T1, D2-T1)
Methods: Measures• Objectives:
– Identify knowledge items for evaluation within a course– Rate evidence of coverage/assessment for knowledge items
Results: Initial Assessment
• 3 knowledge items not identified for rating
Results: Master Dataset
• 14 courses (horizontal)– 4 columns each
• “rated”, “primary”, “secondary”, and “tertiary”
• 249 items (vertical)– Group by subject area
• Color-coded– Green, yellow, red– Blue – no rating
Results: Aggregate by Rank & Rating
Ranking & Rating Item Count Percent of total (N=249)
• BCSP foundation provides objectivity and flexibility– Externally validated criteria– Provides objective assessment of curriculum– Easily adapted to any EHS program*– Other degree programs with a certification or
licensing body could apply the approach*
Summary
• The UMD MEHS program needed an objective means to assess curriculum design– Modified Brauer’s proposed approach for using
the BCSP blueprints to evaluate curriculum• Using a combined ranking and rating system
for each course in the program provided an effective multi-purpose benchmarking tool