Biomedical Engineering Key Content Survey - Results from Round One of a Delphi Study

1

Biomedical Engineering Key Content Survey - Results from Round One of a Delphi Study

David W. Gatchell and Robert A. LinsenmeierVaNTH ERC for Bioengineering Educational Technologies and Northwestern University

Whitaker Foundation Biomedical Engineering Educational SummitMarch, 2005

Supported by NSF EEC 9876363

2

Why conduct a BME key content survey? Motivation and potential benefits Motivation

Establish an identity for undergraduate Biomedical Engineers

Improve communication between academic BME programs and industry Academia – Inform industry of the knowledge, skills and

training of BMEs Industry – Inform academia of the knowledge, skills and

training expected Benefits

More industrial positions for BMEs Each graduate does not have to explain curriculum Recognition that BME degree is ideal preparation for at least

some industrial positions.

3

In General: An iterative process for collecting knowledge from, and disseminating

results to, a group of experts Four steps (repeat steps #2 and #3 to attempt to reach consensus)

1. Develop a set of questions on a topic. 2. Experts give opinions on topics; suggest new ideas that were missed3. Explore and evaluate inconsistencies uncovered in step 24. Disseminate findings, or revise questions and go back to 2

Key point is that experts remain anonymous

Our Study: A set of three surveys Round 0: Select concepts from VaNTH taxonomies; reviewed by domain

experts Round 1: Survey BME industrial representatives and faculty. Asked

participants to rate relevance of concepts important for ALL undergrads in BME, and make suggestions of concepts missed

Round 2: Refine and update list of concepts and resubmit to the above groups for further evaluation

Round 3: Question proficiencies expected (e.g., using Bloom’s Taxonomy)

Delphi study - Overview

4

Utilized an online survey tool to query ~274 concepts: Eleven bioengineering domains (including design) Physiology, cellular biology, molecular biology and genetics,

biochemistry Mathematical modeling, statistics, general engineering skills (e.g.,

computer programming) Survey divided in two parts, each with half the domains:

Total number of participants, n = 136 Part one: Academia – 42, Industry – 25 Part two: Academia – 35, Industry – 23

Participants were asked to: Provide demographic information

Employer, Job Title, Responsibilities, Years of Experience Self-assess level of expertise in each domain (e.g.,

Biomechanics) Rate the importance/relevance of each concept to a BME core

curriculum Suggest concepts that had not been included

Overview of the key content survey, round one

5

Concepts rated on 5 point Likert Scale 1- very unimportant for all BMEs 5 – very important for all BMEs

Mean ratings across concepts similar for industry and academia Academia (n=77) mean and SD rating: 3.71 ± 0.52 Industry (n=48) mean and SD rating: 3.75 ± 0.41

Domains Investigated: Bioinformatics, bioinstrumentation, biomaterials,

biomechanics, biooptics, biosignals and systems, medical imaging, thermodynamics, transport (fluid, heat, mass)

Cell biology, biochemistry, molecular biology and genetics, physiology

Statistics, general engineering

Overview of the key content survey

6

Some concepts included as “Ringers” -Expected to have low rating

Concept Rating (Academia)

Rating (Industry)

Statistical Physics (e.g., Bose-Einstein statistics; Fermi-Dirac statistics)

2.32 2.58

Statistical Physics (e.g., Partition function; statistical representation of entropy;

population of states)

2.82 2.58

Comparative Genomics (e.g., ortholog and paralog genes; gene fusion events)

2.50 2.94

Dynamical Instability and Chaos 2.59 3.11

Unsteady state mass diffusion equation (e.g. Fick’s second law; production and

consumption; boundary conditions; different geometries; multiple layers)

3.42 3.29

All except unsteady state mass diffusion equation met expectations

7

Some concepts included in more than one domain to check consistency of response Two values shown are ratings when concepts are included in

different domains Generally good agreement, but rating sometimes depended on

contextConcept Ratings

(Academia)Ratings

(Industry)

Databases - Interfaces and Structures (e.g., MySQL, relational tables, simple queries, PERL,

CGI, DBI)

2.29/2.66 3.22/3.68

Signal Processing to Reduce Noise (e.g., signal-to-noise ratio; signal averaging)

4.24/3.83 4.17/4.14

Properties of Systems (e.g., boundary, surroundings, universe)

3.88/3.97 3.63/3.88

Electrochemical Potential, Nernst Potential, Fick's Law

4.09/4.23 3.54/4.00

8

Concept RatingHypothesis Testing (e.g., paired and un-paired t-tests; chi-square test) 4.69

Principles of Statics (e.g., forces; moments; couples; torques; free-body diagrams)

4.68

Descriptive Statistics (e.g., mean, median, variance, std deviation) 4.63Circuit Elements (e.g., resistors, capacitors, sources, diodes,

transistors, integrated circuits)4.56

DC and AC circuit analyses (e.g., Ohm's and Kirchoff's laws) 4.56Mathematical Descriptions of Physical Systems (e.g., functional relationships, logarithmic, exponential, power-law; ODEs; PDEs)

4.54

Strength of Materials (e.g., stress, strain; models of material behavior) 4.53

Pressure-Flow Relations in Tubes and Networks (e.g., flow rate = [change in pressure]/resistance; Poiseiulle relation; Starling resistor) 4.51

Measurement concepts (e.g. accuracy, precision, … 4.50Regression analysis 4.49

Forces and pressures in fluids (e.g. shear, normal, surface tension… 4.49

Results: Highest rated eng’g concepts – AcademiaOrange concepts are from statistics and general engineering

9

Results: Highest rated eng’g concepts – IndustryOrange concepts are from statistics and general engineering

Concept RatingDescriptive Statistics (e.g., mean, median, variance, standard

deviation) 4.76

Measurement Concepts (e.g., accuracy, precision, sensitivity; error analysis - sources, propagation of error) 4.71

Hypothesis Testing (e.g., paired and un-paired t-tests; chi-squared) 4.65Probability Distributions (e.g., normal, Poisson, binomial) 4.62

Strength of Materials (e.g., stress, strain; models of material behavior) 4.57

Fundamental Properties of Polymers, Metals and Ceramics 4.50Product Specification (e.g., requirements, design, reliability,

evolution/tracking of the product) 4.45

Principles of Statics (e.g., forces; moments; couples; torques; free-body diagrams) 4.43

Mechanical Properties of Biological Tissues (e.g., elastic; viscoelastic, hysteresis, creep, stress relaxation) 4.43

Data Acquisition (e.g., sampling rates and analog-digital conversion; Nyquist criterion; aliasing) 4.39

10

Results: lowest rated concepts some from “Ringers”

AcademiaDatabases - Interfaces and Structures (e.g., MySQL, relational tables, simple

queries, PERL, CGI, DBI)2.29

Statistical Physics (e.g., Bose-Einstein and Fermi-Dirac statistics) 2.32Artificial Intelligence (e.g., artificial neural networks, fuzzy logic) 2.33

Analysis of Phylogenetic Trees, Molecular Evolution 2.47

Comparative Genomics (e.g., ortholog and paralog genes; gene fusion events)

2.50

Structural Prediction and Molecular Design 2.53

Industry

Statistical Physics (e.g., Partition function; statistical representation of entropy; population of states)

2.58

Statistical Physics (e.g., Bose-Einstein; Fermi-Dirac statistics) 2.58Artificial Intelligence (e.g., artificial neural networks, fuzzy logic) 2.78

Storage Instruments and their properties (e.g., tape, disk, memory) 2.89Comparative Genomics 2.94

Root Locus Plots (e.g., definition, properties, sketching) 2.95

11

Results: Industry - Academia agreementDistribution of mean ratings of all concepts

2.00

3.00

4.00

5.00

2.00 2.50 3.00 3.50 4.00 4.50 5.00Academia

Indu

stry

Concept Ratings

Most concepts rated highly. Few ringers in survey. All traditional domains had some highly rated concepts. Cutoff level for inclusion in recommended undergrad curriculum

still to be determined on basis of further analysis and round two.

12

Results: Industry – Academia AgreementDifferences in means (A-I)

-1.25

-1.00

-0.75

-0.50

-0.25

0.00

0.25

0.50

0.75

0 50 100 150 200 250Concept #

Diff

eren

ces

in M

ean

Res

pons

es

Design

13

Results: Discrepancies in design concepts

Rankings - BME Design Concepts (A comparison of opinions from Academia and Industry)

1.00 2.00 3.00 4.00 5.00

Human Factors Issues/FDA

Computer-Aided Design Considerations

Risk Analysis/Hazard Analysis

Software and Process Design Considerations

Software for Design and Project Management (e.g.,flowcharting; Gannt and PERT charts)

Design for Manufacturing and Assembly

Decision Matrix Approaches to Initial Design

Mean Ranking (all participants)

IndustryAcademia

14

Results: A comparison of general engineering concepts

1.00 2.00 3.00 4.00 5.00

Measurement Concepts (e .g., accuracy, precision, sensitivity;error analysis - sources, propagation of error)

Estimation and Order of Magnitude Calculations

Competency with (at least) One Programming Environment(e.g., Matlab, Mathematica, C, C++, FORTRAN)

Generalized Ohm's Law (i.e., driving force-flow-resistanceconcept)

Numerical Differentiation and Integration

Scaling and Dimensional Analysis

Familiarity with Multiple Computing Platforms (e.g., Windows,Macintosh, LINUX, UNIX)

Artificial Intelligence (e.g., artificial neural networks, fuzzylogic, etc.)

Databases - Interfaces and Structures (e.g., MySQL, relationaltables, simple queries, PERL, CGI, DBI)

Mean Rating (all participants)

IndustryAcademia

Ringer

Significant Deltas

15

2.5

3.0

3.5

4.0

4.5

5.0

2.5 3.0 3.5 4.0 4.5 5.0Rating of Concept - Academia

Rat

ing

of C

once

pt -

Indu

stry

BiochemistryCell BiologyMolecular Biol.BioinformaticsUnity slope

Results: Biology Domains

Good agreement on the whole All biology areas important, but industry sees molecular biology

as being more important than academia Bioinformatics generally scored low, but industry feels that it is

more important than academia does

16

Concepts Academia Industry Academia - Industry

Flow of Genetic Information (i.e., DNA to RNA to Protein)

4.5 4.1 0.44

Methods for Determining Macromolecular Structure (e.g., NMR...)

3.5 4.2 -0.70

DNA Microarrays 3.4 3.8 -0.42Biological Networks (e.g., genetic

networks...)3.2 3.7 -0.46

Structural Prediction and Molecular Design (e.g., homology modeling and

prediction of macromolecular structures and interactions)

2.5 3.3 -0.72

Results: Largest biology discrepancies

17

2.5

3.0

3.5

4.0

4.5

5.0

2.5 3.0 3.5 4.0 4.5 5.0

Rating of Concept - Academia

Ratin

g of

Con

cept

- In

dust

ryPhysiology

Unity slope

Results: Physiology (82 concepts)

Very large span within domain

Generally good agreement

Cardiovascular, neural, cellular physiology concepts rated highly

Digestive, renal, parts of endocrine rated low

18

Results: Largest physiology discrepancies between academia and

industryConcepts Academia Industry Academia

- IndustryCellular Anatomy (e.g...) 4.56 4.14 0.41

Membrane Dynamics (e.g....) 4.44 3.95 0.49

Processes of the Kidney (e.g....) 4.26 3.85 0.41

Renal Filtration (e.g...) 4.03 3.60 0.43

Homeostasis of Volume and Osmolarity

4.03 3.50 0.53

Water Balance and Urine Concentration 3.73 3.25 0.48

Platelets and Coagulation (e.g....) 3.21 3.75 -0.54

Sodium Balance and the Regulation of ECF Volume (e.g....)

3.76 3.15 0.61

19

Results: Should the following foundational courses be required?

Comparison of Responses - Industry and Academia

Calculus - Differential, Integral and Multivariate

Vector Calculus

Linear Algebra

Ordinary Differential Equations

Chemistry - General

Chemistry - Organic (Semester One)

Chemistry - Organic (Semester Two)

Physics - Mechanics

Physics - Electricity and Magnetism

Physics - Waves and Optics

IndustryAcademia

"NO" "YES""UNSURE"

Agreement that second semester organic chemistry is not universally required; some uncertainty about one semester

20

Universities represented in round one of the survey

1. Arizona State University*2. Binghampton University3. Boston University*4. Columbia University5. Devry Institute of Tech6. Duke University*7. Florida International University8. IIT9. Johns Hopkins University*10. Marquette University*11. Milwaukee SOE*12. MIT13. NJIT14. NC State University*15. Northwestern University*16. RPI*17. RHIT18. Stanford University19. Syracuse University*

20. SUNY – Stony Brook21. Tulane University*22. University of Akron*23. University of Cincinnati24. University of Illinois – UC*25. University of Iowa*26. University of Memphis27. University of Michigan28. University of Minnesota*29. University of Pittsburgh*30. University of Rochester*31. University of Texas – Austin*32. University of Toledo*33. Vanderbilt University*34. VCU*

*ABET Accredited – 21 of 37 Accredited Programs Participated

21

Companies and industrial expertise represented in round one of the survey

Companies Represented Abbott Laboratories AstraZeneca Baxter Healthcare Boston Scientific Cardiodynamics Cleveland Medical Devices Datasciences, International Dentigenix, Inc. Depuy, a Johnson and

Johnson Co. ESTECH Least Invasive

Cardiac Surgery GE Healthcare Intel, Corp. Materialise, Inc. Medtronic, Inc. Tyco Healthcare Underwriter Laboratories

Areas of Expertise Biomaterials Biomechanics Bioinformatics Bioinstrumentation BioMEMS Biotransport Cellular Biomechanics Computational Modeling Control Systems

Engineering Fluid Mechanics Medical Devices Medical Imaging Medical Optics Signal Processing

22

Conclusions More analysis is required to:

Investigate variation of opinions for individual topics Correlate ratings with expertise levels Eliminate contextual bias Incorporate concepts omitted from first round

BUT, preliminary results have shown that: “Consistency checks” validate data Generally good agreement between industry and academia Industry and academia disagree on a significant number of

Design concepts Industry highly values knowledge of statistics and

probability Core biology should include all domains assessed

23

Conclusions Remaining issues

Determine level of significance for deciding what concepts can be dropped from core curriculum

Determine significance of differences between industry and academia

Launch second round – by summer

Full matrix of results by concept will be posted on www.vanth.org/curriculum