And Then the Internet Happened Prospective Thoughts about Concept Mapping in the New Millennium

And Then the Internet Happened Prospective Thoughts about Concept Mapping in the New Millennium

Panel Session 599 Concept Mapping at 25:

Development, Application, and Future Directions Columbia Section 2

Friday, Oct 18, 2:40 PM to 4:10 PM

Presentation at the annual conference of the American Evaluation Association Washington, DC

William Trochim, PhD Professor, Policy Analysis & Management Cornell University

Daniel McLinden, EdD Assistant Professor, General & Community Pediatrics Cincinnati Children’s Hospital Medical Center

Special thanks Melida D. Busch, MLIS, Manager, Edward L. Pratt Medical Research Library Cincinnati Children’s Hospital Medical Center

Predicting the future is easy - Getting it right is difficult

“The future is now, and we are not impressed… Now is the time to stop wishing, to stand up, and to shout, "Where the hell is my jetpack?!”1

1 Wilson, DH. (2007). Dude, Where's My Jetpack? A look at the future that refuses to arrive. Discover, February, accessed online at http://discovermagazine.com/2007/feb/jetpack-future-technologies

2 Steve Jobs as quoted in BusinessWeek (12 May 1998) accessed online at http://www.businessweek.com/bwdaily/dnflash/may1998/nf80512d.htm.

3 xkcd accessed online at : http://xkcd.com/864/

“…people don't know what they want until you show it to them.” 2

The internet happened and it got big! • Back in the day

– In 1989, while working at CERN, Tim Berners-Lee invented a network-based implementation of the hypertext concept.4

– The first commercial dialup ISP in the United States …opened in 1989.4

– Websites in 1990 – one.5

– Concept mapping illustrates emergent internet properties of the future in 1989.

• More recently – As of today, we see a total of 147,262,819 domains across the .COM, .NET,

.ORG, .INFO, .BIZ, and .US top level domains (TLDs).1

– The Indexed Web contains at least 4.32 billion pages.2

– One trillion unique URLs3

– Internet gives rise to new models of collaboration and methods of studying collaboration.

1 "Domain Counts & Internet Statistics". Accessed online at: http://www.whois.sc/internet-statistics/. 2 "The size of the World Wide Web". Accessed online at: Worldwidewebsize.com. 3Alpert, Jesse; Hajaj, Nissan (25 July 2008). Accessed online at: "We knew the web was big...". The Official Google Blog. 4 History of the internet. Accessed online at: http://en.wikipedia.org/wiki/History_of_the_Internet#cite_note-89. 5 How we got from 1 to 162 million websites on the internet. Accessed online at: http://royal.pingdom.com/2008/04/04/how-we-got-

from-1-to-162-million-websites-on-the-internet/.

In order to increase the probability of being correct, let’s stand here and predict the future

"Who Let the Dogs Out," Baha Men

2000

The New York Yankees defeat the New York Mets in Game 5 of the 2000 World Series

Gisbourne, New Zealand population 32,754 is first city in the world to welcome in the new millennium

Looking Back

Agreement and consensus is the road that leads to the nowhere

Looking forward, concept mapping enables diverse thinking

1 Wikipedia at http://en.wikipedia.org/wiki/Crowdsourcing

2 Howe, J. (2006) ‘The rise of crowdsourcing’, Wired, vol. 14, no. 6. Accessed online at http://www.wired.com/wired/archive/14.06/crowds.html

3 Howe, J. (2008) Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business, Crown: New York, NY.

4 Benkler, Y. (2011). The Penguin and the Leviathan: The triumph of cooperation over self-interest. New York:

Crown Business.

“Crowdsourcing is the practice of obtaining needed services, ideas, or content by soliciting contributions from a large group of people.”1

“…the more power you give a single individual

in the face of complexity and uncertainty, the

more likely it is that bad decisions will get

made... (p. 220)”

Surowiecki, J. (2004). The Wisdom of Crowds. New York: Doubleday

1. diversity of points of view,

2. independence from each other’s opinions,

3. some kind of mechanism for aggregating

private judgments into a collective decision.

Tapping into the wisdom of crowds requires

1. diversity of points of view,

2. independence from each other’s opinions,

3. some kind of mechanism for aggregating private judgments into a collective decision. Surowiecki, J. (2004). The Wisdom of Crowds. New York: Doubleday

1. “…conceptualization is best when it includes a wide variety of relevant people (p.2).”

2(a). “… there should be no criticism or discussion regarding the legitimacy of statements

which are generated…also allow each participant to submit several statements

anonymously on paper so that confidentiality will be preserved (p.4-5).”

2(b). sort the cards into piles “in a way that makes sense to you (p.5).”

3 (a). “…conduct a two-dimensional nonmetric multidimensional scaling… (p.7).”

3 (b). “…hierarchical cluster analysis … to group individual statements on the map into

clusters of statements which presumably reflect similar concepts (p.8).”

Trochim, W. M. K. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and Program

Planning, 12(1), 1–16.

* Emphasis added

Similarities between now and then

Simple problems & simplifying problems.

Over Under

• 72% – hang toilet paper with the first

sheet going over the roll.

• 28%- prefer to hang the first sheet

under the roll.

“This is not rocket science.” Which is correct?

Source: Many internet sites that are

devoted to reporting this important and

fascinating result.

“It is much more difficult

than Rocket Science.”

Source: Me in response to

stakeholders wanting to

oversimplify a complex issue.

Looking forward, I predict an appreciation of

wicked problems & simple but profound insights

1 Rittel, HWJ & Webber, MM. (1973). Dilemmas in a general theory of planning. Policy Sciences,

4, 155-169.

2 Attributed to Thomas Schelling, date unknown.

“…wicked in a meaning akin to … vicious…or tricky…it

becomes morally objectionable for the planner to

treat a wicked problem as though it were a tame one,

or to tame a wicked problem prematurely, or to refuse to

recognize the inherent wickedness of social problems

(160-161).” 1

“One thing a person cannot do, no matter how rigorous

his analysis or heroic his imagination, is to draw up a list

of things that would never occur to him.” 2

Emphasis added.

1Burke, JG, O’Campo, P, Peak, GL, Gielen, AC, McDonell, KA, & Trochim, WMK. (2005). An introduction to concept mapping as a participatory public health research method. Qualitative health research, 15(10), 1392-410.

2 Rosas, SR. (2012). The utility of concept mapping for actualizing participatory research. Cuadernos Hispanoamericanos De Psycologia, 12(2), 7-24.

Concept mapping as a participatory method1,2 addressing wicked problems

Thinking together with diverse perspectives

• characteristics of your school environment that could relate in any way, good or bad, to a student’s experience of violence.1

• characteristics of neighborhoods that could relate in any way, good or bad, to women’s experience of intimate partner violence. 2

• role of drugs and alcohol in Newark’s low-income and predominantly African American neighborhoods 3

• ways to deal with the problem of bullying in children4

• ways to deal with the problem of overweight/obesity in children 4

1 Johnson, SL, Burke, JA, & Gielen, AC. (2011). Prioritizing the school environment in school violence prevention efforts. Journal of School Health. 81, 331-340.

2 O’Campo, P., Burke, J., Peak, GL, McDonnell, KA, & Gielen, AC. (2004). Uncovering neighbourhood influences on intimate partner violence using concept mapping. Journal of Epidemiology and Community Health. 59(7), 603-608.

3 Windsor L.C. (2013). Using Concept Mapping in Community-Based Participatory Research: A Mixed Methods Approach. Journal of Mixed Methods Research, 7(3), 274-293.

4 Vaughn, L.M.; Jacquez, F. & McLinden, D. (2013). The use of concept mapping to identify school-driven intervention strategies for physical and mental health. Health Promotion Practice. 14(5), 675 - 685.

For example,

Looking forward I predict that participatory will expand to include youth

1 Borden, L., Perkins, D., Villarruel, F., Carleton-Hug, A., Stone, M., & Keith, J. (2006). Challenges and opportunities to Latino youth development: Increasing meaningful participation in youth development programs. Hispanic Journal of Behavioral Sciences, 28(2), 187–208.

2 Chun, J. & Springer, D.W. (2005). Stress and coping strategies in runaway youths: An application of Concept Mapping. Brief Treatment and Crisis Intervention, 5(1), 57-74. 3 Davis, T. S., Saltzburg, S., & Locke, C.R. (2010). Assessing Community Needs of Sexual Minority Youths: Modeling Concept Mapping for Service Planning. Journal of Gay & Lesbian Social Services, 22(3), 226-249. 4 Johnson, S.L., Burke, J.G., Gielen, A.C., (2011). Prioritizing the School Environment in School Violence Prevention Efforts. Journal of School Health, 81(6), 331-340. 5 Ridings, J.W., Powel, D.M., Johnson, J.E., Pullie, C.J., Jones, C.M., Jones, R.L., et al (2008). Using concept mapping to promote community building: The African American initiative at Roseland. Journal of Community Practice, 16(1),

39-63. 6 Ries, A.V., Voorhees, C.C., Gittelsonhn, J., Roche, K.M., & Astone, N.M. (2008). Adolescents' Perceptions of Environmental Influences on Physical Activity. American Journal of Health Behavior, 32(1), 26-39. 7 Vaughn, L.M.; Jacquez, F. & McLinden, D. (2013). The use of concept mapping to identify school-driven intervention strategies for physical and mental health. Health Promotion Practice. 14(5), 675 - 685.

“(a) One of the reasons young people take part in youth

programs is ___” and (b) “One of the reasons other

young people are NOT involved in youth programs is ___

(191).”1

“…young people between the ages of 9 and 19 who

participated in some type of organized youth program and

identified their ethnicity as Latino (191).” 1

“Something GLBT youths need from their communities to

feel supported is ... (231).” 3

“…included youths ages 14–23 … identified their sexual

orientation as 55% lesbian, 35% gay, and 10% bisexual

(230).” 3

“…generate a list of items that describe characteristics

of your school environment that could relate in any way,

good or bad, to a student’s experience of violence (333).’’ 4

“…10th and 11th grade…” and “..9th and 10th

grade…(332)”4

“The youth were recruited from both high school and

elementary levels…(44).” 5

“Please tell us about a problem or issue facing young

African American males between the ages of 10 to 16 in

the Greater Roseland Community…(47)” 5

“…9th through 12th graders from 2 magnet high schools

located in Baltimore…(26).” 6 "What things in the environment, both good and bad,

might influence physical activity among adolescents

(27)?“6

“…runaway youths …ages of 13 and 18 years, had left their

homes without permission for more than 24 hours, and

currently lived in a runaway shelter (61)” 2

“… Things that stress me out are ___ …” and “…The

way I deal with stress is to ____ (60).” 2

“…college students and school stakeholders (students,

parents, teachers, administrators) (677).” 7

“Thinking of your school, list the ways to deal with the

problem of bullying in children… ((677).” 7

In the future, I speculate*…

• We can sort more than

text?

• Visualizing & analyzing will

evolve.

* I am less sure and more worried I may be predicting jet-packs

and flying cars.

Picture this… Can we do more of this? Sort other items to make meaning?

“Seventy-seven photographs with 1-

to 2-sentence captions providing a

context for the photograph were

used as input and served as the

foundation for the sorting and rating

procedures (196)”

Haque, N & Rosas, S. (2010). Concept mapping of photovoices: Sequencing and integrating

methods to understand immigrants’ perceptions of neighborhood influences on health. Family

& Community Health, 33(3), 193-206.

Accessed online at: http://www.wellesleyinstitute.com/wp-content/uploads/2012/07/Report-

20090901-Integration-of-Concept-Mapping-and-Photovoice.pdf

Visualizing hard art & maybe not so soft science

But perhaps most of all, you may

find that some of the maps are

simply “pretty” and interesting

to look at (p 88).” 1

1 Trochim, WMK. 1989). Concept Mapping: Soft science or hard art? Evaluation and Program Planning. 12, 87-110.

2 Trochim, WMK. (1989). Outcome pattern matching and program theory. Evaluation and Program Planning. 12, 355-366.

“pattern matching approach implies a

different view of data … it treats relevant

data about programs, measures,

participants, or outcomes as patterns or

as a whole rather than just as a collection

of individual measures or observations (p

358).”2

Emphasis added

Provide a Greater Community

and Global Perspective

Pay Attention to

Presentation Design

Impact Primary Care

Be Accessible through

Multiple Channels

Utilize Outside/Visiting

Presenters

Highlight CCHMC Focus on Innovation

Include Topics for a

Diverse Audience

Cover Current Practice

A B A B

Visualizing hard art & maybe not so soft science But perhaps most of all, you may find

that some of the maps are simply

“pretty” and interesting to look at

(p 88).” 1

1 Trochim, WMK. 1989). Concept Mapping: Soft science or hard art? Evaluation and Program Planning. 12, 87-110. 2 Trochim, WMK. (1989). Outcome pattern matching and program theory. Evaluation and Program Planning. 12, 355-366.

“pattern matching approach implies a different

view of data … it treats relevant data about

programs, measures, participants, or outcomes

as patterns or as a whole rather than just as a

collection of individual measures or observations

(p 358).”2

Emphasis added

What goes around comes around,

visualizing program theory

1 Trochim, W. M. K. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and

Program Planning, 12(1), 1–16.

In 1989: “Group concept mapping is consistent

with the growing interest in the role of theory in

planning and evaluation (p 2).” 1

Logic Model for TTURC-I Evaluation accessed online at: https://www.teamsciencetoolkit.cancer.gov/public/TSResourceTool.aspx?tid=1&rid=762

See also: Trochim, WM, Marcus, SE, Masse, LC, Moser, RP, & Weld, PC. (2008). The evaluation of larger research initiatives: A participatory

integrative mixed –methods approach. American Journal of Evaluation, 29(1), 8-28.

Stokols D., Fuqua J., Gress J., Harvey R., Phillips

K., Baezconde-Garbanati L., Unger J., Palmer

P., Clark M.A., Colby S.M., Morgan G., Trochim

W. (2003). Evaluating transdisciplinary science.

Nicotine and Tobacco Research. 5(SUPPL. 1),

S21-S39.

Yampolskaya, S., Nexman, TM, Hernandez, M., &

Koch, D. (2004). Using concept mapping to

develop a logic model and articulate program

theory: A case example. American Journal of

Evaluation, 25(2), 191-207.

Trochim, W., Milstein, B., Wood, B. J., Jackson,

S., & Pressler, V.(2003). Setting objectives for

community and systems change: an application

of concept mapping for planning a statewide

health improvement initiative. Health Promotion

Practice, 8, 1-12.

In the new millennium, The logic

model connection to concept mapping:

What does a group of people think

about an issue, a problem, an

opportunity?

Concept mapping

How does a thought (or other

things) travel among a group of

people?

Social Network Analysis

The rise of network of analysis will influence concept mapping.

See for example, Christakis, NA & Fowler, JH. (2009). Connected: The amazing power of social networks and how they shape our

lives. New York, NY: Little, Brown & Co. Durland, M. M. & Fredricks, K. A. (Eds.), (2005a). New directions for evaluation: Social network analysis in program

evaluation: San Francisco, CA: Jossey-Bass. Gloor, P.A. (2006). Swarm Creativity. Oxford. Oxford University Press.

1

2

3

4

5

6

7

8

9

10

11

12

13

14 15

16

17

18 19

20

21

22

23

24

25 26

27

28

29 30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46 47

48

49

50

51

52

53

54

55 56

57

58

59

60

61

62 63 64

65

66 67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89 90

Practice change

Broad

Perspective

Multidisciplinary

Diverse audience

Access/format

Visiting presenters

Organization-centric

Innovation

Inspiration

This is a concept map

See for example, McLinden, D. (2013). Concept maps as network data: Analysis of a concept map using the methods of social network analysis. Evaluation and Program Planning, 36(1), 40-48.

This is a network analysis of a concept map

Finding additional meaning

* Dichotomized the

sorting matrix, GE 11=

1, LT 11=0

Practice change

Broad

Perspective

Multidisciplinary

Diverse audience

Access/format

Visiting

presenters

Organization-centric Innovation

Inspiration

A little more network analysis of a concept map.

What more could we do?

Practice change

Broad

Perspective

Diverse audience

Access/format

Visiting presenters

Organization-centric Innovation

Inspiration

Multidisciplinary

* Increasing point size

corresponds to increasing

degree centrality

See for example, McLinden, D. (2013). Concept maps as network data: Analysis of a concept map using the methods of social network analysis. Evaluation and Program Planning, 36(1), 40-48.

I speculate that visualization will increase

Also see, Borner, K. (2010). Atlas of science: Visualizing what we know. Cambridge, MA: MIT Press.

Check out Jove, the first

scientific video journal

http://www.jove.com/

This is a spherical plot of a concept map using SmacofShpere in R, Stress=0.045.*

Could 3D visualization make a comeback?

Should we emulate a trend in the physical sciences?

* R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical

Computing, Vienna, Austria. URL http://www.R-project.org/.

Back to the future

In the new millennium, concept mapping has provided an approach to:

• Address wicked problems

• Enable collaborative & participatory decision-making

• Embrace diverse thinking

• Visualize the issues of a community

What more can we do to further the art and science?

xkcd accessed online at: HTTP://XKCD.COM/209/