Can We Teach Design of Experiments

Can We Teach Can We Teach Design of Experiments Design of Experiments

On-Line?On-Line?

Dr. Dan Rand – Winona State University

46th Annual Fall Technical Conference - ASQ & ASA

E-mail: [email protected]

Can We Teach Design of Can We Teach Design of Experiments On-Line: AgendaExperiments On-Line: Agenda

DoE as a prime tool of Six Sigma Instructional design impact on E-learning Voice of customer – requirements for on-line

education Requirements focus on a DoE course Web Search for on-line DoE courses Assessing on-line courses Examples/ assessments from my course An Experiment Design – Our Field of Dreams Links to resources

DoE as a prime tool of Six SigmaDoE as a prime tool of Six Sigma

Design of Experiments is the most effective and most complex quantitative tool of Six Sigma Black Belts

Most Six Sigma training is light on DoEBlack Belts form the latest group that

needs supplemental DoE trainingThey add to the audience that can’t sit

for a traditional classroom course

Instructional Design to Meet Learning Needs - Instructional Design to Meet Learning Needs - Bloom’s TaxonomyBloom’s Taxonomy11

Cognitive skills:

Instructional DesignInstructional Designin Distance Educationin Distance Education22

Contextualize instruction Present lesson content

Present vocabulary Provide examples Use cueing programs Advise learner

Activate learner processing (more)

More Instructional DesignMore Instructional DesignAspects from IDDEAspects from IDDE

Assess learning Provide feedback Provide practice Test learning

Sequence instruction Logical order Prerequisite order By content organization

General Instructional Design General Instructional Design Concepts for E-LearningConcepts for E-Learning

Modular structure Learning objectives for each module Active learning Assessment / feedback

Instructional Design Instructional Design to Meet Organizational ROIto Meet Organizational ROI

4 desired outcomes / assessments of skills development training

Learn it Test at end of course

Retain it Apply it

Apply it report on application

Cause benefits ROI $

What customers want from on-What customers want from on-line educationline education

LINK – Learning Network Initiative:Creating Effective Collaboration Between Industry and University Center Rochester (MN)

An End-to-End Framework for “High Tech” Workforce Training

From “Technology Industry” Working Group:

– Drew Flaada, IBM– Warren Kemplin, Rochester Community Technical College– Brian Nass, XIOtech (Seagate)– Dr. Daniel Rand, Winona State University

1 2 3 4 5

Phases:

1 – Navigating to relevant offerings (not pertinent today)2 - Skill / knowledge gap assessment & selection of optimal offering3 - Connecting to the offering (registration, payment, getting course materials)4 - Absorbing the offering (take the class)5 - Completing the offering and post-offering assessment of remaining gaps6 - Application - Effectively using the newly acquired skill/knowledge on the job

7

6

End-to-End Learning Management System

A A A A A A A

A = assessment of skill / knowledge gap

From Phase 2 of End-to-End LMS: From Phase 2 of End-to-End LMS: Pre-assessment & SelectionPre-assessment & Selection

Problem Statement (Voice of Customer): “I don’t know which offering is the best fit for me.” Requirements (Voice of Customer):

– I need a pre-assessment in order to further filter out offerings that aren’t at the appropriate level

– I need to be able to filter the possible offerings by: Terms of payment, match from skills assessment, cost Reputation of provider (quality), certification level I require Availability, language, delivery method

– Any tools associated with this phase must be accessible anytime, from anywhere

– My personal information needs to be protected (ongoing on-line education concern)

– Pre-assessment, pre-test for pre-requisites– Identify goals from employee and employer point of view, with

“live” or e-counselor

Phase 3: “Phase 3: “Connecting to the offeringConnecting to the offering ” ”

Problem Statement:– “I don’t know how to get started and it’s difficult to find

out. The whole process is a hassle.” Requirements:

– Automate the provisioning of all facilities I need (textbooks, databases, labs, server access, web site access) – Bundle as a student kit

– Test and ensure that my workstation or browser will work with the delivery methods used before I proceed (example: what plug ins are required?)

Phase 4: Absorbing the offering Phase 4: Absorbing the offering (taking the class)(taking the class)

Problem Statements:– “I may not complete the course unless I have specific

interim goals to hit and/or peers to keep me going.”– “My day is interrupt-driven.”– “If I miss a traditional class, I can’t get caught up. I need an

alternative.”– “While in the midst of an offering, I don’t know if I’m

on target or not.”

Phase 4: Absorbing the offering Phase 4: Absorbing the offering (taking the class)(taking the class)

Requirements:– I can communicate with the instructor and peer

students either real time (chat), via email, or via a forum– I have access to supplemental resources online– I can take the course from my desk or anywhere– There is an online assessment tool available to help me

gauge my progress– The materials become customized to my specific

needs, based upon my online assessments (which point out remaining gaps)

– I can pause, resume, and replay any of the offering – The offering includes real hands-on projects that

support my learning– Estimate total effort needed for a course– I need (small) modular content

Phase 5: Final assessment & closurePhase 5: Final assessment & closure

Problem Statements:– “I have no idea how I did vs. the expected outcome

or standard.”– “I can’t drive to a classroom to take the final.”– “I’m out of town the day of the final.”

Phase 5: Final assessment & closurePhase 5: Final assessment & closure Requirements:

– Results of the final assessment need to be fed to me, my employer, the instructor

– The final assessment vehicle needs to be available online and on my schedule

– I need to be able to pause and resume the assessment vehicle

– Assuming successful completion of the assessment, any completion credentials (grade, certificate, et al) need to be automatically issued

– I need frequent checkpoint assessments given to employer (payer), like a % completed bar.

– I need a means to re-take course – free, or even remedial material.

Phase 6: ApplicationPhase 6: Application Problem Statements:

– “I don’t know where to go for ongoing help.”– “I’m unsure how this topic/knowledge should be

applied in the real world.”– “I forgot a great deal in the X months/years since

the class ended.”– “I don’t know how to extend this knowledge to

more advanced levels.”

Phase 6: ApplicationPhase 6: ApplicationRequirements:

– Give me access to all related materials (online)– Give me access to peer learners and the instructor

(online)– Point out options for follow-on instruction – Provide me with application examples– Provide me assessment vehicles and

recommendations for remediation– Keep material available to student after completion

of course. – Form graduate communities / networks to gain

knowledge of content improvements, growth.

Additional General ConcernsAdditional General Concerns

Motivating employee to completion Synthesis of training into job

performanceHow to direct students to where their

success rate is the highestNeed a disconnected mode – carry

course content away on CD-ROM

Summarized needs, Summarized needs, applied to an on-line DoE Courseapplied to an on-line DoE Course

Content and procedure– Motivation to finish – Don’t be totally dependent on high speed

Internet– Animation, bells, and whistles may de-motivate

students if they slow down processing– Interact with others to discuss and assimilate

methods– Feedback from an expert – Link to textbook

Summarized needs, Summarized needs, applied to an on-line DoE Courseapplied to an on-line DoE Course

Application– Practice actual experiments – apply it– Try software to design and analyze– Work with others in team experiment (bring

together for catapults or simulation)– Apply it to student’s problem- as project, w/

expert consultant – Provide a support group of graduates

Pragmatic (Market) LimitationsPragmatic (Market) Limitations

Even most motivated students need checkpoints

Market is relatively small for advanced content like DoE – animation and customization are expensive

On-line Design of Experiments courses – On-line Design of Experiments courses – Web SearchWeb Search

http://www.margaret.net/doe/ Online Course in Design of Experiments (DOE)

interactive tutorials, case studies and email support. access for 4 weeks, extend 2 weeks, expect 30 hours.

http://www.thequalitygroup.net/ The Quality Group has a DOE overview module in their

on-line Six Sigma Green Belt training Features– animated with Quick Time, pre-tests, interactive

quizzes, choice of video, captions, text, or still pictures

More Web SourcesMore Web Sources Eastern Michigan U sponsors ASQ course in

DOE ($1165) Stanford - open source learning management

system (LMS) – no DOE MIT – (Stellar) Open Courseware - no DOE University of Phoenix – 3 doctoral programs

added to 1 existing. 125,364 students 5/31/02 – average age=38 – no DOE ?- MBA in Technology Management

Assessing On-line CoursesAssessing On-line Courses

http://taste.merlot.org/ Multimedia Educational Resource for

Learning and Online Teaching (MERLOT) Non-profit on-line community Peer review, user comments on 3

evaluation standards– Quality of Content – Effective Teaching-Learning Tool – Ease of Use

Industrial Design of Experiments course - Industrial Design of Experiments course - online at Winona State - assessmentsonline at Winona State - assessments

Pre-test on course material. After student enrollment and course introduction,

do survey about the enrollment and start-up process.

After each homework problem with supplied answer, ask if the solution is understood, if they could do the problem independently, if they want to ask for help.

1st learning assessment- after review of basic statistics and hypothesis testing.

In conjunction with 1st learning assessment, do survey about “how its going”

DoE at Winona State –DoE at Winona State –Further assessmentsFurther assessments

Further (5) learning assessments, after each logical section of course material, including assessment of projects (catapult-by-team or individual and self-defined)

Another “how its going” survey after two-thirds of course content

Final, comprehensive learning assessment Final “how it went” survey, including

questions of what the student wants for follow-up

A pre-test question on BlackBoardA pre-test question on BlackBoard

Surface Mount Technology (SMT) Surface Mount Technology (SMT) experiment - problem solving in a experiment - problem solving in a

manufacturing environmentmanufacturing environment

2 types of defects, probably related- Solder balls- Solder-on-goldStatistician invited in for a “quick fix”

experimentHigh volume memory card product

Homework assignments and answers on Homework assignments and answers on BlackboardBlackboard