A Framework for Analysing Critical Thinking in Computer Conferencing EURO-CSCL, Maastricht March 22, 2001 Walter Archer.

A Framework for Analysing Critical Thinking

in Computer Conferencing

EURO-CSCL, Maastricht

March 22, 2001

Walter Archer

2

Outline of This Presentation

• The research team

• Background of the study

• The conceptual model and its elements– social presence, cognitive presence, teaching

presence

• Methodology, and issues of practicality

• Linguistic structure - a proxy for semantics?

• Towards automation and practicality

3

The Research Team

• Terry Anderson (principal investigator)

• Randy Garrison

• Walter Archer

• Liam Rourke (Ph.D. candidate in Education)

• Wolf Wikeley (Ph.D. candidate in Linguistics)

4

Rationale for the Study

• Distance delivery increasingly important in higher education

• Particularly Generation 3 (CMC, including computer conferencing)

• Promotion of critical thinking a central goal of higher education

• Our research question: How well does CMC support critical thinking?

5

The Conceptual Model and its Elements

• From a review of the literature and our own experience as educators, we conclude that critical thinking is usually embedded within a community of inquiry composed of students and teachers (or learners and facilitators)

• We propose a model of an educational experience within such a community

6

From Garrison, Anderson, & Archer (2000, p.88) with permission from Elsevier Science

7

Elements of the Model

• “Community” implies that participants interact with each other

• Social presence - project their personalities

• Teaching presence - the actions of (a) person(s) who structures the interaction

• Cognitive presence - the resulting collaborative construction of meaning

8

Social Presence

• Term originated by Short, Williams, & Christie (1976). The Social Psychology of Telecommunications. Toronto: John Wiley and Sons.

• A number of scholars have developed this concept and variously defined it

• Some observers of CMC have questioned whether this medium is capable of supporting social presence

9

Our Definition of Social Presence

“The ability of participants in the Community of Inquiry to project their personal characteristics into the community, thereby presenting themselves to the other participants as ‘real people.’”

Rourke, Anderson, Garrison & Archer (1999). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education/Revue de l'enseignement à distance: 14 (2), 50-71. Available online at http://cade.athabascau.ca/vol14.2/rourke_et_al.html

10

Content Analysis of Computer Conference Transcripts

• From the literature, we identified several broad categories of content that would indicate social presence

• Within each category, we identified more specific indicators of each category

• Employed 2 individuals to find and code (independently) occurrences of these indicators within the course transcripts

11

Category IndicatorsAffective Expression of emotions

Use of humourSelf-disclosure

Interactive Continuing a threadQuoting from others' messagesReferring explicitly to others'messagesAsking questionsComplimenting, expressingappreciationExpressing agreement

Cohesive VocativesPhatics, salutationsAddresses or refers to thegroup using inclusivepronouns

Social Presence Categories & Indicators

12

Interactive

a Social presence density = total number of social presence indicators coded in transcript/total number of words in transcript.b Transcript A: Number of words (n = 24 132), aggregate social presence density = 22.83. c Transcript B: Number of words (n = 6 260), aggregate social presence density = 33.54.

Social presence densitya of all indicators in transcript Ab and transcript Bc

From Rourke, Anderson, Garrison, & Archer (1999, p. 99) with permission of the editor.

13

Teaching Presence

A new term, defined as “the design, facilitation and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes.”

Anderson, Rourke, Garrison & Archer (unpublished), p. 7.

14

Teaching Presence (2)

• Distinguishes educational experiences from experiences of incidental learning

• Often provided by a single individual

• In CMC settings, particularly in higher education, parts of this function may be provided by other individuals such as instructional designers, program coordinators, and the students in the class

15

Teaching Presence Categories & Indicators

Categories IndicatorsDesign andOrganization

Setting curriculumDesigning methods(3 others)

FacilitatingDiscourse

Identifying areas ofagreement/ disagreementSetting climate for learning(4 others)

DirectInstruction

Present content/ questionsFocus the discussion onspecific issues(5 others)

16

Health Course Education Course

f % f %Instructional Design 31 22.3 12 37.5Facilitating Discourse 60 43.2 2475.0Direct Instruction 107 77.0 28 87.5

In the Graduate Health course, number of instructor messages =139.In the Graduate Education course, number of instructor messages = 32.

Frequencies of teaching presence categories in instructor messages

Teaching Presence in Two Graduate Courses

17

Cognitive Presence• This concept developed more fully in:

Garrison, D.R., Anderson, T., & Archer, W. (in press). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education.

• Grounded in the literature on critical thinking, operationalized within a model of practical inquiry

18

From Garrison, Anderson, & Archer (2000, p. 99) with permission from Elsevier Science

19

Cognitive Presence Defined

Cognitive presence is defined as the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry.

Garrison, Anderson, & Archer (2000)

20

Cognitive Presence Categories & IndicatorsCategory IndicatorsTriggering event Recognizing the problem

Sense of puzzlement

ExplorationInformation exchangeBrainstorming(4 others)

IntegrationConvergence among groupConvergence within a singlemessageConnecting ideas, synthesisCreating solutions

ResolutionVicarious application to realworldTesting solutionsDefending solutions

21

integration13%

resolution4%

other33%

trigger8%

exploration42%

Relative frequencies of categories of cognitive presence

From Garrison et al. (in press)

22

Methodological Issues in Content Analysis of Computer Conference

Transcripts

Explored in detail in Rourke, Anderson, Garrison, & Archer (in press (2001))

Issues include:

• unit of analysis

• objectivity, replicability, reliability

23

Content Analysis - ProblemsQuantitative content analysis is "a research technique for the objective, systematic, quantitative description of the manifest content of communication" (Berelson, 1952, p. 519). Despite the potential of this technique, researchers who have used it have described it as difficult, frustrating, and time-consuming. Very few have published results derived from a second content analysis.

From Rourke et al. (in press (2001))

Since many of the indicators used by our group and others researching computer conferences are latent, rather than manifest, this problem becomes even more acute.

24

Linguistic Structure - A Proxy for Semantics?

• Most of our work (and the work of others doing similar research) relies on semantic (meaning) analysis of content

• This can’t be automated, even though we are in the year 2001 (Sorry, Hal)

• But analysis of lexical and structural features of content CAN be automated

• So let’s look for structural/semantic correlations

25

The Language of CMC

• More like writing or more like speech?

• This is not a single continuum– both speech and writing encompass a wide

range of styles and registers

• Baselines for lexical/structural features of formal and informal speech and writing have been established in Chafe and Danielewicz (1987)

26

Chafe/Danielewicz chapter

Chafe, W., & Danielewicz, J. (1987). Properties of spoken and written language. In R. Horowitz & S.J. Samuels (Eds.), Comprehending oral and written language (pp. 83-113). San Diego: Academic Press.

27

What C & D Measured - Samples

• Type/token ratios (variety of vocabulary)

• Contractions

• Words per intonation unit (defined by them - more or less same as clause)

• Conjoining two elements into a compound phrase using “and”

• 15 other features

28

C & D’s Results - Sample Table

Conversations 8Lectures 12Letters 18Academic papers 24

Table 3.11 (p. 101): Conjoining

(Occurrences per 100 intonation units)

29

Towards Automation and Practicality

• We are now attempting to automate, to some extent, the analysis of computer conference transcripts – may make content analysis a more useful tool

• Use some of C & D’s measures - those that lend themselves to automation– also those that may correlate with degree or

category of social presence, cognitive presence, and teaching presence

30

Issues We Now Face

• Unit of analysis (again!)– C & D’s “intonation unit” a rather slippery

concept, and difficult to automate– we may have to go back to the punctuated

sentence as the unit of analysis

• Lexical/structural analysis may give only a very rough cut at cognitive presence, etc.– will this be good enough to be of use to educators?

31

Our First Conceptual Paper

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 1-19.

32

The Social Presence Paper

Rourke, L., Anderson, T., Garrison, D.R., & Archer, W. (1999). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education/Revue de l'enseignement à distance: 14 , 2. Available online at http://cade.athabascau.ca/vol14.2/rourke_et_al.html

33

The Cognitive Presence Paper

Garrison, D. R., Anderson, T., & Archer, W. (in press). Critical thinking and computer conferencing: A model and tool to assess cognitive presence. American Journal of Distance Education.

34

The Teaching Presence Paper

Anderson, T., Rourke, L., Garrison, D.R., & Archer, W. (submitted for publication). Assessing teaching presence in a computer conferencing context.

35

The Methodology Paper

Rourke, L., Anderson, T., Garrison, D.R., & Archer, W. (2001). Methodological issues in analyzing text-based computer conferencing transcripts. International Journal of Artificial Intelligence in Education (12, to appear).

36

Our website

• http://www.atl.ualberta.ca/cmc/

[email protected]

[email protected]

[email protected]

[email protected]