In theory, there is no difference between theory and practice, in practice there is — Jan L.A. Van de Snepscheut Design based research, understand the.

In theory, there is no difference between theory and practice, in practice there is — Jan L.A. Van de Snepscheut

Design based research, understand the world by trying to change it — Chris Hoadly





Design Based Research

November 2007

Defining research

• “. . . the systematic study of materials and sources in order to establish facts and reach new conclusions . . .”

• adjectives describing research as• disciplined • transparent • public • scientific• diligent• and accessible

• organized• problem orientated• creative• systematic• laborious

Research Paradigms

• Quantitative - discovery of the laws that govern behavior

• Qualitative - understandings from the inside

• Critical - Investigate and expose the power relationships

• Design-based - interventions, interactions and their effect

Design-Based Research

• Related to engineering and architectural research

• Focuses on the design, construction, implementation, and adoption of a learning initiative in a real context

• Related to ‘Development Research’• Closest educators have to a “homegrown”

research methodology

Design based research

• “Engineering” particular forms or learning and systematically studying those forms of learning within the context defined by the means of supporting them. • Subject to test and revision• Successive iterations play a role similar to

systematic variation in experiment

• To develop theories not just to determine “what works”• To develop a greater understanding of

“learning ecology”

Design Studies are

• Iterative • Process focused• Interventionist• Collaborative • Multi-leveled • Utility oriented• Theory driven and generative

Design based research• Theory vs. Practice

Principles of Design Based Research• Designing learning environments intertwined with

developing theories (proto-theories)• Cycles of design, enactment, analysis & redesign.

Respond to emergent features• Must lead to sharable theories (plausible causal

accounts)• Must account for how designs function in authentic

settings (document success, failure and interactions)• Relies on methods that can document and connect

processes of enactments to outcomes of interest

DBR…

• Interventionist - in that a purposeful change is made in a functioning educational context.

• Participatory - in that researchers work with practitioners.

• Theoretical - in that theory is used to design the intervention.

• Generative - in that it generates theory through studied application in the original and subsequent educational contexts in which the innovation is used.

DBR

• The 5 fold way• Develop theories - about process and means• Interventionist: test beds for innovation• Place theories in harm’s way: Prospective & reflective• Iterative - to develop explanatory framework that

specifies expectations, become focus of next round• Theories are humble: concerned with domain-specific

learning & accountable to the activity of design

• Design experiments are extended (iterative), interventionist (innovative and design-based), and theory-oriented enterprises whose “theories” do real work in practical educational contexts.

DBR: A CASE STUDY

Koehler & Mishra (or should I say Mishra & Koehler)

TPCK

• Over the years, we have developed a framework for thinking about what teachers need to know about technology

• Over this period, we have articulated a theoretical framework called technological pedagogical content knowledge (TPCK).

• Before going into what that framework is, let talk about how it began

Ancient History (1999-2000)

• Traditionally, if you want to teachers to use technology …• Teach them technology skills

(assuming this is the barrier to their use of technology)

• Hold a few workshops or training seminars, and teach them to make web-pages, use microsoft word, maybe iMovie, etc.

• Assume that armed w/ this knowledge, they will put it to good educational use in the classroom.


• In 1999 or 2000 Matt met Punya (and vice versa), and had a few conversations about this idea• We agreed that this approach

doesn’t really work• And roughly, had an idea why (ie.,

learning skills out of context)• And we both favored a more hands

on approach / learning by doing approach / learning in context approach


• So we began a few “design experiments” (Brown, 1992; Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Design Based Research Collective, 2003).

• Our teaching of ed tech masters students (and college faculty) became oriented around an approach that we called learning technology by design• People would learn about educational technology

by designing educational technology

• We have been developing theory; developing teachers; and developing a curricular approach (learning technology by design).

Learning Technology by


• Learning Technololgy design projects included:• Designing online courses that would be taught in

the college of education (ed tech students and college faculty working in groups)

• Designing a website or powerpoint to teach ______

• We thought the advantages were:• Collaboration allowed for sharing expertise• Learning about technology was always

contextualized• Rich opportunities to learn about multiple

technologies in one single design project• Engaging learning environment

History (2002)

• We collected data to see what people were learning:• Surveys

• The artifacts that teams were designing

• Transcripts of what people talked about

• Copies of writings and emails

• Interviews

History (2002)

• We were sure that it was working

• But… why? It surely had something to do about the learning technology in context, but what does that mean? What is the educational technology context anyway?

• We were struck by an very good description of teacher knowledge by Shulman (1986).

History (2002)• Shulman argued that to date, there are

roughly two things that scholars had identified that teachers need to know:• Content knowledge : Expertise about the topic they

were teaching• Pedagogical knowledge: Knowledge about teaching

techniques, assessment, etc.• But what was missing was:

• Pedagogical Content Knowledge: Knowledge about how to teach a particular subject matter, knowing about common student trajectories (and misconceptions), and ways to assess that knowledge for a particular subject matter

• This idea by Shulman is widely accepted (and cited) now by everyone

History (2002)• We thought that when you threw technology

into the mix, that traditional workshop approaches that teach technology in isolation were doing:

History (2002)

• Instead, we thought learning technology in context meant learning this:

•C : Content

•P: Pedagogy

•T: Technology

•R: Representations

Why?

Diagrammatic representations

• The tetrahedral model…

Diagrammatic representations

• The tetrahedral model…

History (2002)

• Ultimately, R (Representations) went away from our way of thinking:• Representations are usually one of the three categories• New representations afforded by technology• Representations of content or pedagogy (e.g., a formula

to represent math or physics ideas)• Shulman often wrote about representations for a subject

matter to be part of pedagogical content knowledge• It was really hard to code for• It wasn’t doing much for it

• Which leads us to 2003 and the present

Theoretical Framework

• The current framework has three, interconnected components• CONTENT• PEDAGOGY• TECHNOLOGY

An intermediate step

Still with lines… but with no way to represent TPCK


• These three components, each represent knowledge bases that ideally overlap.


• In our framework, there are multiple things that teachers need to know• Content (C) – The subject matter of what

they are teaching

• Pedagogy (P) – The methods of teaching

• Technology (T) – Tools, software, and hardware (commonplace technologies like blackboards, as well as digital computers)


• But not just the three in isolation, but their intersection as well• Pedagogical Content Knowledge (PC) – How to teach

a particular subject matter, common student understandings and misconceptions, etc. (Shulman, 1986, 1987)

• Technological Content Knowledge (TC) – how a subject matter is transformed by the application of technology

• Technological Pedagogical Knowledge (TP) – how technology can support pedagogical goals (e.g., fostering collaboration).


• But not just the three in isolation, but their intersection as well• Technological Pedagogical Content Knowledge (TPC)

– understanding and negotiating the relationships between these three components of knowledge (Bruce & Levin, 1997; Dewey & Bentley, 1949; Rosenblatt, 1978)

• Similar to ideas presented by Hughes, in press; Keating & Evans, 2001; Lundeberg, Bergland, Klyczek, & Hoffman, 2003; Margerum-Leys, & Marx, 2002

What is such a framework good for?

• Building theory• Describing phenomena• Telling you where to look for

evidence• Informing practice• Others?

2003 - Present

• Recall that our framework evolved over a period of 4 years,

• The result of design experiments where we constantly designed and redesigned a learning environment

• Closely connected to our philosophy of how to teach this stuff: Learning technology by design

Funded by

• College funds to conduct research on Learning Technology by Design seminars

• Preparing Tomorrow’s Teachers to use Technology (PT3 grant 1 and 2)• 1999 - 2007

Learning Technology by Design

• Our design seminars have focused on teams working together to build online or hybrid courses as a means of developing their knowledge of educational technology.

• Design teams typically consist of one tenure stream faculty, and several teachers seeking their masters degree in educational technology.

• The teams work over the course of a semester to develop one online course.


• The “official” site for learning is a course that we teach.

• We do very little formal lecturing instead we do more technology “demos”

• There is some reading, dealing with current issues in design or issues of online pedagogy.

• Groups tend to go in different directions, choosing different technologies and pedagogies.

• Accordingly, our role as instructors is to guide and facilitate a potentially vast field of possible scenarios (technical and pedagogical).


• Designing an online course requires inquiry research to design tangible, meaningful products (e.g., syllabus, websites, online communities, lessons, etc.).

• Teachers use technology to build something that is sensitive to the subject matter, and specific instructional goals.

• As such, every act is a weaving together of content, pedagogy, and technology


• So in short, our claim has been that unlike some approaches to teacher training, that develop technology skills in isolation. Learning technology by design develops skills in an integrated fashion.

Testing our Theory• In Koehler et al, 2004

†

we presented a case study of a college faculty member (Dr. Shaker) as she worked with her design team to create an online course.

• Our analysis revealed important changes in Dr. Shaker’s technological literacy and her thinking about her personal relationship with technology.

• In accounting for these changes, we found that the learning by design approach afforded rich opportunities for Dr. Shaker (and her other team members) to deeply consider the relationships between content, pedagogy, and technology.

† Koehler, M.J., Mishra, P., Hershey, K., & Peruski, L. (2004). With a little help from your students: A new

model for faculty development and online course design. Journal of Technology and Teacher Education, 12(1), 25-55.

An early case studyKoehler, Mishra, Hershey & Peruski (2004)

Testing our Theory II

• In Koehler et al, 2004†

we followed a different design group in detail, tracking their conversations over a course of a semester.

• A mixture of quantitative and qualitative analyses showed that:• Initially participants talked about content,

pedagogy, and technology as independent constructs

• Over time, their conversations revealed more and more connections between these bodies of knowledge

• This suggests that the learning by design approach develops knowledge consistent with the integrated TPCK theory. † Koehler, M.J., Mishra, P., Yahya, K., & Yadav, A. (2004). Successful teaching with technology: The complex interplay of

content, pedagogy, and technology. Proceedings from the Annual Meeting of the Society for Information Technology & Teacher Education, Atlanta, GA. Charlottesville, VA: Association for the Advancement of Computing in Education

Studying Design TalkKoehler, Mishra, & Yahya (2007)

The survey study

• Research on course taught by Koehler during 2003

• Design teams: 1 college faculty + teachers (ed-tech masters students) worked to design online courses

• 4 college of education faculty, 14 students• Had to design everything: syllabus,

readings, student assignment, assessment rubrics, the nature of student interactions, etc.

• Also had to figure out how technology would be used to accomplish all of the course goals

Measuring Outcomes by a Survey

• We designed a survey that measured participants attitudes and opinions about:• The learning environment (e.g., the

learning technology by design approach)

• How they thought about designing for a face to face course vs. designing for an online audience

• How participants were developing knowledge as suggested by the TPCK framework (Technology, Pedagogy, Content).

A Survey of TPCK Thinking Koehler & Mishra (2005)

“Designing an online course requires changes in what we teach and how we teach”

“Our group has chosen technologies that fit the course content and the instructors teaching philosophy.”

Conclusions

• Both at the individual, and group level participants reported significant changes in how much they thought, talked about, and learned about Content, Pedagogy, and Technology.

• Not completely unexpected given the attention to each of these topics

• However, it does agree with our earlier, more in-depth studies of single design groups

Conclusions

• Our theoretical framework, however, stresses the importance of developing knowledge at the intersection of the three components.

• It has been our hypothesis that the learning technology by design approach was ideally suited to developing this type of knowledge.

• The results of this study confirm previous results that suggest the approach does develop these integrated forms of knowledge as well.

Book based on faculty dev courses

• Faculty Development by Design: Integrating Technology in Higher Education edited by Punya Mishra, Matthew, J Koehler, Yong Zhao

Update…

• AACTE Book, goal to “situate Mishra & Koehler’s concept of TPCK in the realm of teacher education”

Back to DBR

• Long term project• Can be built around smaller

more manageable studies• Don’t do this for your

dissertation• Or better still think of your

dissertation in these terms (just don’t tell your committee right away)

In theory, there is no difference between theory and practice, in practice there is — Jan L.A. Van de Snepscheut Design based research, understand the.

Documents

research theory

designbased research

snepscheut design

architectural research

generative slide

design studies

practice principles

effect slide