Insight research, design research and impact on practice Engineering research in education Hugh Burkhardt Shell Centre School of Education, University.

Insight research,design research and

impact on practice

Engineering research in education

Hugh BurkhardtShell Centre

School of Education, University of Nottingham

Outline

Paradigms for research and development

Delivering effective tools and processes

Mathematical literacy – a case study

Cost effectiveness

Building a design community

The Shell Centre Team Malcolm Swan, Daniel Pead, Rita Crust, Alan

Bell, HB, with many associates Does ‘engineering research’ – design and

development of: teaching materials and processes assessment tasks professional development materials and processes Tools and strategies for system change

with some associated ‘insight research’ Based in the University of Nottingham School of Education Work with many others, notably Michigan State, Berkeley

and school systems in UK and US Contact: [email protected]

www.mathshell.com

Research in Education

Three research traditions, and products:

Humanities: insight focus > papers

Science: insight focus > papers

Engineering: impact focus >new tools, processes + papers

R<–>P Theme: for research-based practice we need a better balance of research effort –

more engineering (cf also medicine,…)

Influence on policy and practice

Humanities dominates policy making (all can play)

plausibility ~ ‘common sense’ – often fails

Science dominates research, “informs” policy, but

no body of agreed results > little influence

Engineering still marginal, except for tests*

some systematic work in some places eg here

What is design research?Design Research has emerged as an accepted part

of educational research, with a strong input from Cognitive Science. Key features include:

insight focus > products and papers realistic classroom situations exploring teaching and learning theory buildingbut with atypical teachers exceptional support no claim to wider usability >no direct impactEngineering research sees the products as first

drafts

What is design research?

For more, see e.g.

Educational Design Researcheds Jan van den Akker, Koeno Gravemeier, Susan

McKenney, Nienke NieveenRoutledge 2006

“pragmatic, grounded, interactive, iterative and flexible, integrative, and contextual”

R<–>P process – key ingredients

body of reliable research

exemplars of tools and processes

funded development programs

stable design teams

systematic iterative development

clear range of effectiveness comparative evaluation-in-depth

funded by clients who understand the process see HB+AHS Educ Researcher 32 (2003)

…but the academic value system

Favours new ideas over reliable research new results over replication and extension trustworthiness over generalizability small studies over major programs personal research over team research first author over team member disputation over consensus building

papers over products and processes

What is good design?

Good design: makes users more powerful

via tools and processes that fit their purposes

gives users, and others, pleasure

does these in a cost-effective way

Types of contribution

Good design can: make something more widely available

printing press, Model-T Ford, www,..slates, paper, ‘lesson study’, …..

lead to continuous improvementcars, TV, medical treatments,…teaching materials,…..

enlarge possibilities for practicetelephones, search engines, arthroscopescomputers, geometric software,

microworlds provide existence proofs

What do good designers do?

They know how to use research results and design

skills to improve ‘best practice’ tackle new challenges effectively

pass on their knowledge to other practitioners novices

through their materials.

Approaches to Innovation

Craft-based innovation Builds on authors’ successful practice Simple and inexpensive process

Research-based innovation Builds on ‘best practice’, research, design

skills Slow, complex and expensive R&D process

that uses trialling to build robust innovations ie ambitious innovations that work well

Craft-based development model

The normal process of authorship Write draft materials Circulate to an expert group Discuss and revise Publish

Craft-based development model

Questions: How reliable is the extrapolation of

author's professional experience to others’ needs in the new situation?

How effective is the communication of the author's intention to the target community of users?

OK for small changes

Extrapolation and communication unreliable for real innovation

Research-based development model Input from:

earlier research and development worldwide craft skills of best practice

Design skill and creativity led by designers of exceptional achievement well-defined locus of ‘design control’

Systematic developmentiterative rounds of developmental trials with:

rich, detailed feedback guiding revision of the materials in each round

so that intentions and outcomes converge Continuing customer feedback

Educational design principlesHeuristic, phenomenological theory: Some based on ‘insight research’, eg

active learning constructive build multiple connections

Others design-based, eg role shifting cognitive conflict student ‘ownership’

Design theory is not often discussed in enough detail to be useful

Design beyond just principlesDesign brilliance is more than these: ‘Surprises’ that are clearly ‘right’

Handling complexity simply

Controlled innovation

Balance in all aspectsWe know it when we see it – iPod,…

The design process

Seek excellence in design. What is it? a coherent plan, influenced by research lessons that work well with target S materials that communicate to target T robustness across circumstances of use enjoyment in use by S and T

Develop in realistic circumstances Cover all the main goals Sell to many users

Mathematical Literacy exemplar development need

ML enables non-specialist adults, if they are taught how,

to benefit from using mathematics in their everyday lives to better understand the world they live in, and to make better decisions.

“The sophisticated use of, often elementary, mathematics” also called

functional mathematics, quantitative literacy, numeracy …

Post-11 mathematics is non-functional for most people

“Mathematics is useless”

Who needs it?

Plan Potential secretaries

asked to critique and complete the spreadsheet for planning a conference budget

A B C D ECollege charges Delegates@ £ each £

Monday Buffet Supper 30 17.00 0Single En-suite Accommodation 30 40.00 0

Tuesday Breakfast 30 8.00 0Morning Coffee 30 1.90 0Luncheon 30 15.00 0Afternoon tea 30 1.90 0Dinner served 30 50.00 0Single En-suite Accommodation 30 40.00 0Plenary Room 30 15.77 0Breakout rooms 2 85.10 0

WednesdayBreakfast 30 0Morning Coffee 30 0Luncheon 30 0Afternoon tea 30 0No Dinner 30 0Single En-suite Accommodation 30 0Plenary Room 30 0Breakout rooms 2 0

Thursday Breakfast 30 8.00 0Morning Coffee 30 1.90 0Luncheon 30 15.00 0Afternoon tea 30 1.90 0Dinner 30 17.00 0Single En-suite Accommodation 30 40.00 0Plenary Room 30 15.77 0Breakout rooms 2 85.10 0

Friday Breakfast 30 8.00 0Total

charges 0VAT 0Total 0

Sudden Infant Deaths = Murder?

In the population as a whole, about 1 baby in 8,000 dies in an unexplained "cot death". The cause or causes are at present unknown.

Three successive babies in one family have died. The mother is on trial. An expert witness says:

"One cot death is a family tragedy; two is suspicious; three is murder. The odds on just two deaths in one family are 64 million to 1"

Discuss the reasoning behind the expert witness' statement, noting any errors, and write an improved version to present to the jury.

ValidateValidate

SolveSolve

FormulateFormulate

InterpretInterpret

ProblemProblemReportReport

The modelling process

ValidateValidate

FormulateFormulate

InterpretInterpret

ProblemProblem

ReportReport

SolveSolve

Dysfunctional math curriculum

Oft-neglected Design Aspectsa view

Is this outward-looking mathematics? few students will become mathematicians math can give them power in their lives does this curriculum do that? for all? (cf ELA)

or is it “just math” (RPF)symptoms: all topic focus, no modelling, tasks

What ‘dimensions of engagement’? many students lack interest in math itself is “make the math interesting” all this does?symptoms: variety of activities, of tasks (cf ELA)

Oft-neglected Design Aspects a few more

Does this develop student autonomy? reliable imitation is not enough to do math what ‘transfer distances’ do the tasks cover? how long are the chains of reasoning? …involving, which problem solving phases?

symptoms: no linked phases, similar tasks together

Does this give teachers enough support? student centered teaching is difficult it is easy to overload the teacher what design tactics are used to avoid this?

symptoms: teacher in hot seat, centre-stage; no support tactics; too much innovation at once; ……

Modelling

Joe buys a six-pack of coke for $3 to share among his friends. How much should he charge for each bottle?

If it takes 40 minutes to bake 5 potatoes in the oven, how long will it take to bake one potato?

If King Henry 8th had 6 wives, how many wives had King Henry 4th?

Plan a trip: fault finding and fixing

Alison and two friends has planned a cycling trip around Derbyshire on Saturday.

Here is their plan for the day.

Read through the plan and the information sheets (next page).

If you find a mistake, or realise something has been forgotten, write it down and say how they should change the plan.

Meet at Loughborough station at 7.23 am. Buy tickets and then catch the train to Derby. This arrives at 7.51 am.

At Derby, catch the 8.20 am train to Cromford. This arrives at 8.41 am.

Here are the instructions for getting to the Cycle Hire centre:

“Turn left as you come out of Cromford station, walk along by the river and down Mill road. Cross over the A6. Walk up Cromford hill for about 1/2 mile and you will see..

Authentic information sheets

Design a tent - open task

Design a tent - scaffolded task

It must be big enough for two adults to sleep in. The sloping sides and the two ends will be made from a single, large sheet of material.

1. Write down the height and breadth (shoulder to shoulder) of a typical adult.2. Write down suitable measurements for CD, DE and AP. Remember to leave

space for baggage. 3. Draw a sketch to show how you will cut the material from a single piece. Label

your drawing with letters A to G.4. Use Pythagoras theorem to calculate the length of AD.5. Calculate angles <ADP and <ACD using trigonometry.

Design a tent - final version

1. Estimate the relevant dimensions of a typical adult.2. Estimate the dimensions of the base of your tent.

Estimate the length of the vertical tent poles you will need. 3. Show how you can make the top and sides of the tent from a single

piece of material. Show all the measurements clearly.Calculate any lengths or angles you don't know. Explain how you figured these out.

Your task is to design a tent like the one in the picture. It must be big enough for two adults to sleep in (with baggage). The tent should be big enough so that someone can move around while kneeling down. Two vertical tent poles will hold the whole tent up.

“Bowland Maths”

~20 “case studies, including: Reducing road accidents How risky is life? “You reckon?” Alien invaders

Professional development 5 module package, activity based

Delivered on-screenon web June 17 www.bowlandmaths.org.uk

Body of reliable research needed

RF: universities, journals, funders encourage:

team research with long timescales on available treatments as well as new coherent in-depth work to give diverse data

Mathematical Literacy Extend current research, which shows students need:

knowledge of concepts and skills strategies for problem solving and modelling metacognitive control of their reasoning, and the disposition to view situations mathematically

What development skills are needed?

The team needs: Systematic methods of observation Interview skills Protocols related to the design goals Methods for analysing observation reports,

student work, interviews Design skill in using this rich feedback

systematically to improve the materials.

ie as products are developed in other fields

Stable design teams need

long term coherent programs in each field

developing a community that works towards higher standards of design and development

training and career progression for designers

Wider recognition of its importance

recognising the intellectual value of designas in medicine, engineering, fine arts

Mathematical Literacy needs all of these.

Clear range of effectiveness

Trials will give an indication of conditions for successful use, including:

Prior learning of students

Teacher skills and support

Time and support for absorbing the change

Feedback from widespread use will refine this

Most research fails to explore boundaries of validity– essential for design

Comparative evaluation-in-depth

Guides choice of materials and approaches Informs improvement and future development

To do this well, evaluation needs to: look at widely available treatments across variables: users, use, outcomes compare alternatives – many dimensions

Still rare in all countries

Mathematical Literacy will need all these in due time – typical time from agreeing goals to stable curriculum implementation ~ 10-15 years

Math Literacy ~ proof of concept

For Mathematical Literacy units so far, it seems: All students succeed and enjoy the work FM narrows the range of performance Many, but not all, teachers can handle this

work with just the materials – more with live PD training

1 or 2 new three-week units per year is digestible

More research needed, across more exemplar units to warrant such general statements

Long term growth of design skill

Freudenthal Institute 40 years, 40 fte people, coherent support

US: EDC, TERC, LHS,.….. 40 years, ~ 100 people, one-off projects

no coherent support Shell Centre

40 years, ~ 5 fte people, strands of work, no coherent support

Such groups are rare worldwide currently, you need luck and cunning to survive

How much does engineering cost?

Engineering research is more expensive. NSF curriculum projects

~$1,000,000 for ~160 hours >> $6,000/ hour

Shell Centre curriculum units 10-15 hours ~ $30,000 per classroom hour.

Is this good value?

How much does engineering cost?

Analysis (US context) whole school curriculum ~15,000 hours

14 years*200 days*5 hours 3 parallel developments ~ $1.5 billion

15,000 hours*3*$30,000/hr Over 10 years ~0.05% of running cost

$1.5 Bn/($300Bn/yr*10 years)

Other changing fields? R&D typically 5-15%80% development, 20% basic research

Why is design flair ignored? Tradition that, for most, math and science

are necessary evils – and boring Focus on design principles but

details matter (as in all art) No comparative evaluation in depth of

learning or attitudes (beliefs and motivation) The ‘classics’ of math and science are not

digestible K-12 (cf music, literature), so Major burden on educational design, but No profession of educational designer

some systematic work in some places eg here

Building a Design Community

Improve standards of design by: sharing methods and experience developing theory of design providing training

Increase recognition in: government via payoff >> funding academia via argument, funding..

Better career paths – will follow ISDDE, a growing group, is making a start

International Society for Design and Development in Educationwww.ISDDE.org

ISDDE Conference 2008

FI @ Egmond aan See, NL June 30 – July 2 “Linking design research to

professional educational design” Working groups:

curriculum design designing professional development assessment design educational software

Contact Peter Boon, Conference Chair [email protected]

http://www.fi.uu.nl/isdde/welcome.html

Summary

Design and development – society needs to develop the engineering research approach

‘Functional Mathematics’ – we need to make good our claims of power for all

Build a design community to get higher standards, and improved recognition for designers