Social Network Analysis of Asynchronous Discussion in ... · B. Social Network Analysis Social network analysis (SNA) represents a “theoretical paradigm that emphasizes that the

Social Network Analysis of Asynchronous Discussion in Online Learning

by: Audrey Fried

A thesis submitted in conformity with the requirements for the degree of Master of Arts

Curriculum, Teaching and Learning Ontario Institute for Studies in Education

University of Toronto

© Copyright by Audrey Fried 2016

  ii  

Social Network Analysis of Asynchronous Discussion in Online Learning

Audrey Fried

Master of Arts Curriculum, Teaching and Learning

Ontario Institute for Studies in Education University of Toronto

2016 Abstract

Online learning is often informed by constructivist principles aimed at encouraging

collaborative knowledge building through asynchronous online discussion. Social network

analysis is uniquely suited to studying these socio-cognitive processes because it emphasizes

interactions rather than the attributes of individual participants. This paper uses social network

analysis to explore the networks formed by different types of interaction in online discussion and

to analyze the ways in which each type of interaction is related to the others. The resulting

insights will help instructors design and monitor online courses to facilitate knowledge building as

well as to assess both the quality and the quantity of student contributions.

  iii  

Acknowledgements

Many thanks to Clare Brett, my incredible supervisor, for her generous guidance

throughout this process. Thanks also to Jim Hewitt for being my second reader during a very

busy year. And thanks as well to Monique Herbert for her excellent assistance resolving

quantitative issues. Finally, thank you to my parents Evelyn and Peter Fried for their unfailing

confidence, and to my children Ellie and Daniel Grushcow for their enthusiastic encouragement.

  i  

Table of Contents

Chapter 1: Introduction ...................................................................................................... 1

A. Rationale .................................................................................................................... 1

B. Social Network Analysis ............................................................................................. 2

C. Overview .................................................................................................................... 2

Chapter 2: Background & Literature Review ..................................................................... 4

A. Social Constructivism & Knowledge Building in Online Discussion ......................... 4

B. Social Network Analysis ............................................................................................. 5

C. Social Network Analysis and Asynchronous Online Discussion ................................ 9

D. Limitations and Critiques ........................................................................................ 18

E. Research Questions .................................................................................................. 21

Chapter 3: Methods .......................................................................................................... 22

A. Pepper ...................................................................................................................... 22

B. Social Network Analysis ........................................................................................... 23

C. Participants ............................................................................................................... 24

Chapter 4: Results & Discussion ....................................................................................... 26

A. Degree Centrality ..................................................................................................... 26

B. Relationships between Reading, Replying, Liking, and Linking ............................. 34

Chapter 5: Conclusions & Implications for Future Research ........................................... 44

Works Cited ...................................................................................................................... 47

Appendix A: Subject Classes ............................................................................................. 56

  ii  

Appendix B: Adjacency Matrices ..................................................................................... 59

Appendix C: Spearman’s Rho Correlation Matrix .......................................................... 72

  1  

Chapter 1: Introduction

A. Rationale

Online discussion-based courses that are informed by constructivist principles build on

the insights of John Dewey and Lev Vygotsky that learning is an inherently social phenomenon

(Dewey, 1926; Vygotsky, 1930/1980). For this reason, social constructivists “focus on developing

activities that promote learner-to-learner interactions to support the co-construction of

knowledge and the sharing of information and resources” (Dawson, 2008, p.224). Online

discussion is particularly well suited to this task, affording students the opportunity to collaborate

in knowledge building processes that result in deep learning (Vrasidas, 2000). At the same time,

it’s relatively easy to create a comprehensive record of interactions, opening the process to

academic study with the aim of improving the design of learning environments and developing

best practices to support participants.

This paper focuses on understanding more about the nature of social interactions in

Pepper, a research-based online discussion platform developed at the Ontario Institute for

Studies in Education (OISE). Students who participate in asynchronous class discussions on

Pepper read the “notes” (or posts) of other students and reply to those notes to form a threaded

discussion. Pepper also allows students to embed hyperlinks (“links”) to other notes and to “like”

notes. Each of these interactions has been studied in isolation (e.g. Hewitt, Brett, & Peters, 2007

(reading); Wilton & Brett, 2014 (reading); Hewitt, 2005 (replying); Zingaro, Daniel and Oztok,

2012 (replying); Phirangee & Hewitt, 2014 (linking); and Makos, Zingaro, Oztok, & Hewitt, 2014

(liking)). This paper will add to the literature by using social network analysis to explore the four

types of interactions and the ways in which they are related.

  2  

B. Social Network Analysis

Social network analysis (SNA) represents a “theoretical paradigm that emphasizes that

the elemental unit of social life is the social relation” (Carolan, 2014, p.262). It is specifically

designed to “describe, understand, and model relationships among actors” (Assimakopoulos &

Yan, 2006, Background section). SNA’s emphasis on interactions rather than on the attributes of

individual participants allows researchers to focus on the social nature of learning. As a result,

SNA is particularly useful in “assessing [the] sociocognitive structures of participation in

computer-supported learning” (Palonen & Hakkarainen, 2013, p.338) and is an important tool in

the study of knowledge building processes in asynchronous online discussion.

A social network “consists of a set of actors, and a set of ties that define linkages between

pairs of actors” (Haya, Daems, Malzahn, Castellanos, & Hoppe, 2015, p.303). In studying the

social networks formed in online discussion-based classes, the actors (or nodes) are the

participants in the discussion and the ties are the interactions between the actors. In the case of

Pepper, ties can be either reads, replies, links, or likes. This study investigates the networks

formed by each type of interaction as well as the relationships between the different networks.

C. Overview

The next chapter provides a brief overview of social constructivism and knowledge

building in online learning, outlines some of the fundamental concepts in social network analysis,

and reviews the literature on social network analysis and asynchronous online discussion.

Chapter 3 outlines research methods, including information about the Pepper platform, a

description of participants, data collection and preparation, and the analysis of the data. Chapter

  3  

4 presents the research findings along with a discussion of their significance. Finally, Chapter 5

discusses conclusions and next steps for further research.

  4  

Chapter 2: Background & Literature Review

A. Social Constructivism & Knowledge Building in Online Discussion

Under “the constructivist paradigm … knowledge is constructed cooperatively through

social negotiation” (Aviv, Erlich, Ravid, & Geva, 2003, p.5). Knowledge building pedagogy

builds on this insight, seeking to emulate the social learning that takes place in the real world.

(Scardamalia, 2002). As Scardamalia explains, knowledge building requires the capacity to work

with ideas in a way that recognizes their interconnectedness; “one idea subsumes, contradicts,

constrains, or otherwise relates to a number of others [and] [t]o gain understanding is to explore

these interconnections, to drill deeper while rising-above to gain broader perspective”

(Scardamalia, 2002, p.6). In a knowledge building process, students collaborate to “negotiate and

share meanings relevant to the problem-solving task at hand” (Stahl, Koschmann, & Suthers,

2006, Cooperative Learning in Groups section). “[I]t is through discourse that knowledge or

ideas are constructed, negotiated and improved” (Sing & Khine, 2006, p.251). In turn, “the

results of social interactions are internalized in individuals’ cognition” (Aviv, Erlich, & Ravid,

2005, p.4).

In this view of learning, “participation and discourse become the key concepts”

(Lipponen, Rahikainen, Lallimo, & Hakkarainen, 2003, p.489-90). And a “necessary feature” of

this collaboration “is that the participants display for each other their understanding of the

meaning that is being constructed in the interaction” (Stahl et al., 2006, From Quantitative

Comparisons to Micro Case Studies section). This type of work is greatly facilitated by

technology that makes it easy for learners to communicate across time and space and to display

understanding and share information in a central repository. (Scardamalia, 2002; Stahl et al.,

  5  

2006). When combined with social practices that foster collective cognitive responsibility1 for the

improvement of ideas, online discussion forums “can prompt learners to structure, integrate, and

interconnect new ideas with pre-existing knowledge and prior experiences facilitated by tools that

enable them to rearrange, synthesize and restructure information” (Quinton, 2010, p.338-39).

For this reason, a well-designed online discussion forum is one of the most fertile environments

for the kind of “intellectual exchange in a pedagogical community” that is essential to knowledge

building (Xin & Feenberg, 2006, p.3).

Social constructivism shifts the focus from the individual learner to the social processes

that comprise knowledge construction. The “shared meaning making” of the knowledge building

process “is not assumed to be an expression of mental representations of individual participants,

but is an interactional achievement” (Stahl et al., 2006, From Mental Representations to

Interactional Meaning Making section). When this process is carried out via asynchronous online

discussion, the interactions are made visible. Together, these features make social network

analysis a natural fit for the study of knowledge building processes carried out through online

discussion.

B. Social Network Analysis

Although researchers have used quantitative analysis to study online learning (e.g.

Zingaro, Daniel and Oztok, 2012), social network analysis offers a unique vantage. By looking at

online discussion from the perspective of network data rather than more conventional data the

focus shifts from “actors and attributes” to “actors and relations” (Hanneman & Riddle, 2005,

                                                                                                               1 For a discussion of the concept of collective cognitive responsibility, see Scardamalia

(2002).

  6  

ch.1). Social network analysis “captures and visualizes how certain patterns of social relationships

(e.g. network structure, position, composition, etc.) among individuals influences their social

behavior or conduct” (Lee, 2014, p.455), making it a valuable tool for the study of knowledge

building processes in online discussion.

A social network is composed of a set of actors, or nodes, connected by relations, or ties.

Relationships can be either binary (e.g. presence or absence of a friendship tie) or weighted (e.g.

intensity of a friendship tie operationalized as frequency of communication). Ties can also be

directed (e.g. citation) or undirected (e.g. marriage). The network formed by these actors and ties

is characterized by both actor and group level metrics (Assimakopoulos & Yan, 2006). The rest of

this section will review some of the key metrics at both levels.

The most common actor-level metric is centrality, which “is used ... to identify important

nodes or those that occupy influential positions in a network” by identifying “the most connected

individuals” (Ruane, 2014, p.581). Centrality can be measured in a number of ways, each based

on a slightly different conceptual understanding of centrality (Carolan, 2014, p.155). The most

commonly used measure is degree centrality, which is simply the number of ties between an actor

and others in the network. When ties are directed, centrality can be measured by in-degree and

out-degree in addition to the more general measure of degree, which is the sum of in-degree and

out-degree (Hanneman & Riddle, ch.10). As a general rule, actors with a high in-degree are

believed to have a high level of prestige and actors with a high out-degree are considered to have

a high level of influence (Ruane, 2014, p.581). Finally, degree can be binary, indicating the

presence or absence of a tie, or weighted, indicating the intensity of a tie, typically in terms of the

number of interactions between two nodes.

Closeness centrality looks beyond the direct ties of an actor (Carolan, 2014, p.156).

Instead, it ranks actors “based on their position in the network – how fast they can spread the

  7  

information to the whole network” (Rabbany et al., 2014, p.6). Closeness centrality is calculated

by finding the “average distance [of] an actor from all other actors in the network and is a

function of an actor’s geodesic distance to others, which equals the length of the shortest path

connecting a pair of actors” (Carolan, 2014, p.156). Actors with a high closeness score “can

exchange something with many others relatively quickly” (Carolan, 2014, p.159).

Betweenness centrality “captures how actors control or mediate the relations between

pairs of actors that are not directly connected” (Carolan, 2014, p.157). For this reason,

“betweenness centrality represents the control of a node over communication within its community”

(Carolan, 2014, p.157). It is measured by “the number of shortest paths between any other nodes

that have to pass through this node” (Rabbany et al., 2014, p.6). Betweenness thus indicates who

has “control over information exchange or network flows within a network” (Carolan, 2014,

p.158).

At the group level, the most frequently used metrics are density and centralization.

Density “describes the general level of cohesion in a graph” (Ruane, 2014, p.581). It is

operationalized as “the number of total edges [or ties] present in the network relative to the

number of edges [or ties] in a fully-connected network” (Hernández-García et al., 2014, p.70). It

measures “how structured or unstructured, or how cohesive, a network is, with higher network

density values corresponding to more structured and cohesive networks” (Hernández-García et

al., 2014, p.70, internal citations omitted). It’s important to note that “as the size of the network

increases, network density tends to decrease” (Hernández-García et al., 2014, p.70). For this

reason, comparing network densities across networks of different sizes is not meaningful.

Centralization “describes how tightly interaction within a network is organized around a

particular focal point or points” (Lipponen et al., 2003, p.493). It does this by measuring how

variable or heterogeneous the actor centralities are as compared to the most centralized network

  8  

possible, a “star” network (Assimakopoulos & Yan, 2006, Background section). In a star network,

the central actor is connected to each of the other actors but the other actors are only connected

to the central actor. Freeman centralization is “the degree of inequality or variance in [a]

network as a percentage of that of a perfect star network of the same size” (Hanneman & Riddle,

2005, ch. 10). When centralization values are high, “the power of individual actors varies rather

substantially, and this means that, overall, positional advantages are rather unequally distributed

in th[e] network” (Hanneman & Riddle, 2005, ch. 10).

Beyond density and centralization, it’s helpful to understand the concept of reciprocity,

another group-level metric. Reciprocity “occurs when the existence of a link from one to another

triggers the creation of the reverse link” (García-Saiz, Palazuelos, & Zorrilla, 2014, p.416). The

presence of reciprocity may indicate the development of social norms within the network (Aviv et

al., 2005).

Viewed through the lens of social network analysis, reading, replying, linking, and liking

in a Pepper discussion forum create directed ties between participants. Each type of interaction

creates its own network. But each network is related to the others. With an emphasis on actor-

level metrics, this paper explores the patterns created by each type of interaction and analyzes

the relationships between different types of interaction. In particular, the paper focuses on the

power of out-degree ties, which are within the control of the student, to elicit engagement in the

form of in-degree ties. This investigation will add to a growing body of research using SNA to

analyze online learning that has gained momentum as the use of social network analysis has

soared over the past five to ten years (see Figure 1 below).

  9  

Figure 1. Proportion of all articles indexed in Google Scholar with “social network” in the title by year (Halgin & Labianca, 2013, slide 4).

C. Social Network Analysis and Asynchronous Online Discussion

Social network analysis has been used in educational research to analyze in-person

interactions (e.g. Grunspan et al. (2014)(study partners); Jostad et al. (2013)(outdoor education

groups); Oshima et al. (2012)(knowledge building network)), online interactions (e.g. Hernández-

García et al. (2014); Paredes, Shing, & Chung (2012); and Russo & Koesten (2005)), and

sometimes both (e.g. Enriquez (2010)(including face-to-face and telephone interactions along

with interaction via online forum, chat, and text message); and Cowan & Menchaca (2014)

(studying the networks that were created before, during, and after participation in a mostly online

master's program)). As Cela et al. (2014) conclude, SNA research in online learning is most often

descriptive or exploratory, focusing on a single case or other small sample (e.g. Lipponen et al.

(2003)(taking a descriptive approach to the study of 23 elementary school students participating

in computer-supported collaborative learning); Jimoyiannis & Angelaina (2012)(using a sample of

21 students to “develop an analytic framework for evaluating blog-based learning activities”)).

  10  

Researchers also frequently introduce or try to validate an SNA tool they’ve created (e.g. García-

Saiz et al. (2014)(E-learning Web Miner); Haya et al. (2015)(Social Learning Analytics toolkit);

Nistor et al. (2015)(ReaderBench); Oshima et al. (2012)(Knowledge Building Discourse Explorer);

Rabbany et al. (2014) and Rabbany, Takaffoli, & Za (2011)(Meerkat-ED); and Reffay & Chanier

(2003) and Reffay & Chanier (2002)(exploring cohesion metrics for the purpose of developing

software to monitor academic discussion and detect problems)). Many of the small studies

explicitly limit their generalizability (e.g. de Laat, Lally, Lipponen, & Simons (2007, p.90,

94)(n=8 including 7 students and a tutor, looking at whole-network metrics of cohesion for a

single workshop to demonstrate the utility of SNA, but noting that the “research [was] of a

qualitative nature … [and] that no inferential statistical tests were carried out on the data”);

Doran & Mazur (2011, p.11, 14)(n=11, demonstrating the utility of SNA for assessment but

characterizing the analysis as “exploratory” and “descriptive” and cautioning that “the

significance of th[e] study is limited by the small size of the data set examined”); Palonen &

Hakkarainen (2013, p.335, 339) (n=28, including 19 females and 9 males, concluding that

interactions of 5th and 6th grade students aimed at knowledge building in a single class were

dominated by high-achieving girls but noting that the data was from a single, gender-skewed

classroom); Russo & Koesten (2005, p.259-60) (n=21, finding significant correlation between in-

degree and final grade but noting the limitations of the small sample and dependence of

observations); Shea et al. (2014, p.12, 13, 15) (n=18, finding “significant correlations” between

centrality and contributions to community of inquiry presences but noting the “relatively small

sample for the type of statistical analysis carried out for this study” and allowing that a “larger

sample size would generate more reliable results”); Tirado, Hernando, & Aguaded (2012,

p.305)(studying the relationship between the group-level metrics of cohesion and centrality and

social and cognitive presence in 10 learning groups but acknowledging that “before generalizing,

  11  

the results of this analysis should be studied with larger sample sizes”). However, some attempt

inferential statistical analysis with small or incomplete samples that are of questionable utility (e.g.

Nurmela et al., (1999, Communication Structure section)(n=18, finding that "[n]one of these

[social network] measures show statistically significant correlation to the course achievement of

student pairs" despite analyzing a single class); Sing & Khine (2006, p.255)(n=11, finding a

moderate correlation between writing and reading notes in a professional development course

delivered on Knowledge Forum)). At their best, however, small studies demonstrate the

usefulness of SNA tools for practitioners and hint at productive areas for further research. More

recently, several larger studies have begun to move the field forward.

A number of studies look at whether there is a correlation between SNA metrics and

student achievement. Smaller studies have reached contradictory results. For example, with a

sample size of n=21, Russo & Koesten (2005) found a significant correlation between prestige (in-

degree) and cognitive learning operationalized as final grade. A slightly larger study of n=36 by

Paredes, Shing, & Chung (2012) found a significant correlation between social network metrics

and a “content richness score” devised by the authors. Unfortunately, this finding is of limited use

because the definition of content richness is somewhat circular. On the other side, Heo, Lim &

Kim (2010) studied project groups of n=7 and found that high levels of interaction within the

small groups did not necessarily correspond to high scores on the project outcome. Content

analysis showed that it was the quality of the interaction that differentiated the high and low

performing groups: “Compared to the team with the highest project performance, most

interaction among members of [the team with a relatively poor score] related only to information

sharing and accepting new ideas with little discussion. They engaged in lower levels of cognitive

processes and then ended up with less successful outcomes” (Heo, Lim, & Kim, 2010, p.1391).

While there is no doubt that high-quality interactions are essential to good learning outcomes, a

  12  

number of larger studies have found at least some evidence that high quality interactions can be

seen in the structure of the social network and can be captured by SNA metrics.

Romero, López, Luna, & Ventura (2013) analyzed data from a 114-student first-year

computer science course to see whether participation in an online forum could predict who

would pass the course. While the course was primarily delivered in-class, students were instructed

to use the forum to discuss “course content, [and to] solv[e] doubts and problems between

students” without the participation of the instructor (Romero et al., 2013, p.463). Although

participation in the forum was not mandatory, it counted in a student’s favour if they had a

“near-pass mark” on the final exam (68 out of 114 students passed the final exam) (Romero et al.,

2013, p.463). The researchers found that the most important attributes for predicting final

student performance were: “two quantitative measures (the number of messages sent and the

number of words written), together with the only qualitative measure (the average evaluation of

individual messages) and the two social network measures (the degree of centrality [normalized

out-degree] and the degree of prestige [normalized in-degree])” (Romero et al., 2013, p.470).

Interestingly, prestige (normalized in-degree) was not an important indicator in the first half of

the course but “bec[a]me relevant in the second half of the course” (Romero et al., 2013, p.464).

Also significant was the researchers’ finding that the number of messages read by a student did

not contribute to the prediction of who would pass or fail the course (Romero et al., 2013, p.464).

Similarly, Hernandez-Garcia et al. (2014) analyzed reading and replying interactions

from 10 sections of a one-semester online finance course at The Open University of Catalonia

(UOC). Each section had approximately 60-70 students and was taught by a different teacher.

Data from “the learning system’s activity log” was collected along with the students’ final grades

(Hernández-García et al., 2014, p.71). For the read network, the study found a “significant but

low overall positive relation between the different centrality measures and final grade,” except

  13  

that eigenvector centrality (a type of closeness centrality) was not significant and closeness

centrality was negatively correlated with student outcomes (Hernández-García et al., 2014, p.72).

The results were similar for the reply network except that the correlation between closeness

centrality and final grade was positive (Hernández-García et al., 2014, p.73). However, the study

showed “mixed results when analysis was performed on a per-classroom basis” with a correlation

between centrality measures and final grades found in some classes but not others (Hernández-

García et al., 2014, p.76). The researchers state that they used SPSS to analyze the correlation

between centrality measures and final grade but do not say whether they used specialized

methods to account for the lack of independence between observations.

One of the most interesting studies was conducted by Vaquero & Cebrian, who aimed to

determine whether there are “individual and group-level behavioral patterns that lead to low

scoring and possible dropout” (2013, p.2). They analyzed 80,000 interactions by 290 students

who took a 12-week freshman course in computer science for journalism students that included

both online and face-to-face components (Vaquero & Cebrian, 2013). The researchers observed

that “low performance students tend to initiate many transient interactions regardless of the

performance of the students they interact with [and that] [t]hese interactions … start late in the

course, allowing high performers to establish a closely knit group” (Vaquero & Cebrian, 2013,

p.3). In contrast, “the higher the score of the students, the higher the percentage of their

interactions that were persistent … [and] [a]s the score of the student increase[d], these persistent

interactions [were] initiated with a reduced number of similarly performing colleagues

(assortative interaction pattern)” (Vaquero & Cebrian, 2013, p.5). Transient interactions were

defined as “those not reciprocated within a week” and persistent interactions as “those sustained

over time” (Vaquero & Cebrian, 2013, p.5). While high performing students interacted with

students of all performance levels, “high scoring students [kept] persistent interactions between

  14  

themselves” (Vaquero & Cebrian, 2013, p.5). The authors concluded that the interaction

patterns “likely indicated the presence of a benefit maximization process by which students focus

their efforts on potentially more fruitful connections” (Vaquero & Cebrian, 2013, p.7). Finally,

the researchers found that the “number of connections (students that a student has interacted

with) and number of interactions (times a student has contacted or been contacted with/by other

students) were … [significantly and highly] positively correlated2 with the final score of the

student” (Vaquero & Cebrian, 2013, p.3). Stepanyan, Borau, & Ulrich (2010) had similar results.

However, their analysis suggested that “while participants with similar scores tend to have

greater interaction among each other, the predominant pattern within the studied network is the

popularity of participants with higher scores” (Stepanyan et al., 2010, p.72).

Toikkanen & Lipponen (2011) took a different approach to the study of SNA metrics and

course outcomes. They studied relationships between various SNA metrics and affective learning,

which they define as “that measured by student opinions in a questionnaire,” as opposed to

cognitive learning, defined as “that measured by course grades” (Toikkanen & Lipponen, 2011,

p.367). The study used data from 392 students in 17 classes from both elementary and secondary

schools in Helsinki. While the classes were quite heterogeneous in terms of length, class size, and

grade level, all the classes included a teacher-guided process of progressive inquiry using an

online learning environment. The authors found that student ratings of “meaningfulness [of the

learning experience] increase[d] when pupils repl[ied] to messages of many other pupils, and

they read messages from many other pupils” (Toikkanen & Lipponen, 2011, p.376). As the

researchers observed, “these two activities are of course tightly linked, since in order to reply to a

message, it has to be read first” (Toikkanen & Lipponen, 2011, p.376). They concluded that “the

                                                                                                               2 r=.81 (connections) and r=.85 (interactions), p<.01 (Vaquero & Cebrian, 2013, p.3).

  15  

act of writing many messages to several other pupils [out-degree] is the best correlate for

meaningfulness of [the] learning process” (Toikkanen & Lipponen, 2011, p.376).

Also in the affective sphere, Shane Dawson (2008) found a moderate correlation between

centrality measures and sense of community. The study analyzed 464 students in 25

undergraduate and graduate education courses with an online component, some of which were

fully online and some of which were blended. Data about student interactions came from

automatically generated communication logs from the online discussion forum while data about

sense of community was collected from student surveys based on Rovai’s Classroom Community

Scale (CCS). The CCS comprises a social community subscale, which “relate[s] to the students’

perceived levels of belonging, trust and cohesion,” and a learning community subscale, which

measures “the degree to which students share similar learning values and goals” (Dawson, 2008,

p.227). An ordinary least squares regression analysis showed that the adjusted r2=.253 for

betweenness, closeness, and degree centrality, indicating that a “significant but moderate

proportion of the variance in community was accounted for by the[se] … variables” (Dawson,

2008, p.231). Degree centrality was a positive predictor for community (Dawson, 2008, p.231).

Dawson concluded that “students engaged with a greater number of learners report a higher

level of sense of community than their less socially active peers” (2008, p.231). These findings

must be read with caution, however, because the response rate for the CCS survey was only 23%

(Dawson, 2008, p.230) and it’s unclear whether the lack of independence of observations was

taken into account in the regression analysis.

Aviv et al. (2003) analyzed social networks in online discussion from a different

perspective, looking primarily at network-level structures that promote knowledge creation. As

they explain, “[c]ohesion is a primary network structure that contributes to the creation of

knowledge, shared beliefs and behaviors” (Aviv et al., 2003, p.5). In their view, “[c]ohesion is

  16  

manifested by the existence of cliques of participants who are connected internally more than

externally” (Aviv et al., 2003, p.5). As a result, “[m]embers of a clique tend to create knowledge

by virtue of their strong intra-responsiveness relations [and] [t]hey drive the process of

constructing knowledge” (Aviv et al., 2003, p.5).

Aviv and his colleagues compared the network structure of two online discussion groups

that were part of a course in Business Ethics at the Open University of Israel. The first group

(with 18 participants and a total of 248 messages) was required to commit to participating in the

online discussion and to follow a structured multi-step process for moral decision-making that

included identifying and exploring the problem, debating the solution and proposing a synthesis,

and testing the solution against different sets of principles. As the researchers acknowledged, this

process “specifically direct[ed] the students to higher phases of critical thinking” (Aviv et al.,

2003, p.9). The second online discussion group (with 19 participants and a total of 70 messages)

was part of the course the following year and was open to any of the 300 students in the course.

In this second group, students were not required to register or commit to participate in any way.

The discussion was unstructured in that no specific process was set out and any of the

participants could raise an issue related to any course topic. Unsurprisingly, content analysis

based on Gunawardena’s five-phase Interaction Analysis Model revealed that the structured

discussion involved substantial participation in the first four (out of five) phases of knowledge

construction while the unstructured discussion was stalled at the first phase. Much of the

interaction in the unstructured discussion involved “simple Q&A, triggered by students’

assignments and course components” (Aviv et al., 2003, p.9). In looking at the structure of the

networks, Aviv and his colleagues found that the structured discussion network comprised 16

cliques of about 4-5 students, only one of which included the tutor. Many of the students

belonged to more than one clique, allowing information to flow between cliques. In contrast, the

  17  

unstructured discussion network comprised only two cliques, each with two students and the

tutor. The differences were also apparent in the power structures of the two networks as revealed

by degree centrality. The dominant feature of the structured discussion network was the broadly

distributed in-degree or “triggering power” across “a large number of actors” (Aviv et al., 2003,

p.14). The unstructured discussion network was a simple one where the tutor was the only

participant who responded to messages and so maintained a central position in terms of out-

degree (Aviv et al., 2003, p.14).

There is almost no literature about the relationship between different types of interactions

in asynchronous online learning. Sing & Khine (2006) found a moderate correlation between

writing and reading notes in a professional development course delivered on Knowledge Forum;

but with only n=11 participants, the generality of this finding is limited. Similarly, Haya et al.,

(2015) found that teams’ behaviour on discussion and voting/rating processes was correlated;

although, again, with only n=11 team-nodes this finding is not reliable. In addition, each team

was required to comment and vote on every other team’s work product (a video) which may have

created an artificial result. In a somewhat larger study (n=57), Xie, Yu, & Bradshaw (2014) found

that for both moderators and other students, interaction attractiveness (in-degree) was positively

correlated with participation diversity (out-degree). As they explain, a “student’s in-degree can be

considered as their interaction attractiveness in a communication network, an index of the extent

to which other students in the network have selected this student as a salient discussion partner”

(Xie et al., 2014, p.11). A student’s participation diversity, or unweighted out-degree, reflects “the

extent to which students engage in social interactions with a diverse range of other peers …

encounter[ing] information and ideas that are different from their own” (Xie et al., 2014, p.11-

12). The researchers concluded that “students may develop their reputation and establish their

interaction attractiveness through their own participation effort” and that “students tend to

  18  

develop and maintain their social interaction attractiveness and centrality by actively and

frequently engaging in posting participation and diversely interacting with others” (Xie et al.,

2014, p.18).

D. Limitations and Critiques

Caution is in order when it comes to studies about SNA and online discussion. One

difficulty results from the analysis of online courses animated by very different pedagogies. Some

courses are teacher-centered while others are student-centered. While some are unstructured,

others are highly structured and may include explicit scaffolding for knowledge building (e.g.

Palonen & Hakkarainen (2013)(analyzing interactions aimed at knowledge building in

Computer-Supported Intentional Learning Environment); Sing & Khine (2006)(analyzing

interactions aimed at knowledge building in Knowledge Forum); and Toikkanen & Lipponen

(2011)(students were instructed in “the progressive inquiry method”)) or require a particular

number of posts or responses (e.g. Doran & Mazur (2011)(requiring one initial post and two

response posts in each discussion forum); Haya et al. (2015)(requiring every team to comment on

the video of every other team); Jimoyiannis (2013, s.2.1)(requiring interaction "with peers in three

different blog groups at least"); Romero et al. (2013, p.470)(requiring students to "send at least

two new messages/questions and at least two replies to other messages/questions"); Stepanyan et

al. (2010, p.70)(requiring students "to post at least seven microblogging messages a week and to

read the incoming messages of their fellow students”)). Results from one context may not be

generalizable to other contexts. Nonetheless, SNA metrics can be helpful at a descriptive level,

revealing the pedagogical approach of an instructor. For example, Garcia-Saiz et. al. found that

the instructor in the subject course had the highest degree centrality by far and concluded that

  19  

this likely resulted from the fact “that the instructor answered the questions in the forum faster

than the students, preventing them from helping one another” (2014, p.428).

Other difficulties arise when SNA is used to analyze asynchronous online discussions in

courses that also include other types of interactions—a common situation (e.g. Haythornthwaite

& Gruzd (2012, p.3359-60)("The variety of media used included real-time audio for weekly,

synchronous, two-hour class sessions ... [and students had to] come to campus over a weekend

once per semester for one day sessions."); Heo et al. (2010, p.1385)(a "lecture-based classroom

with selected online learning activities ... [including] a three-week online project [which] was

chosen for study"); Jimoyiannis (2013, s.2.1)(The course "include[ed] classroom sessions and face-

to-face discussions ... and on-line collaborative work."); Nurmela, Lehtinen, & Palonen

(1999)("combin[ing] written communication with face-to-face communication"); Romero et al.

(2013)(primarily a face-to-face course with a supplemental online forum in which participation

was voluntary); Sing & Khine (2006, p.253)("Half the lessons were conducted face-to-face while

the other half were online."); and Xie et al. (2014, p.18)("[T]he data from the online discussion

forum could only partially reflect group interactions and did not consider social relations being

established through face-to-face meetings or offline communications.")). As Dawson explains, a

limitation of theses studies “resides in the unknown number of communication exchanges

undertaken by the sampled population external to the monitored online environment” (2008,

p.230). Enriquez puts it more strongly, arguing that using social network analysis to chart online

interactions “‘hides’ ties that are outside a message thread” (2010, p.56). She suggests that

“commonly identified relational ties in a network pattern of an online discussion are those

structural traces that can be recorded or logged in a file [but that] [t]hese are not necessarily the

ones we would like to record to do justice in representing the practice of networked learning”

(Enriquez, 2008, p.118). As she explains, “[o]ther social and material elements, not necessarily

  20  

online, ‘trigger’ computer-mediated connections” and social network research typically pays

“insufficient attention … [to] the manner in which links are established in the first place”

(Enriquez, 2010, p.56, 62). She concludes that “[w]e must also take into account the content of

the exchange, whether material or informational … [and] must not confine the context of the

exchange within the online environment nor think of the ties as a direct contact between human

actors” (Enriquez, 2010, p.62). Instead, she proposes an approach to SNA that includes

information about other interactions collected through surveys of the participants.

In her 2010 study, Enriquez therefore analyzed data that included “perceived social ties

of participants in six communication media alongside the response ties in discussion forums”

(Enriquez, 2010, p.56). The six communication media were “face-to-face, email, mobile phone

text messaging, land phone, discussion board (Blackboard), and online chats (e.g. MSN

Messenger)” (Enriquez, 2010, p.57). This approach has its own limitations as it relies on self-

reporting and can lead to incomplete or inaccurate data due to low response rates and poor

recollection. In Enriquez’s study, five out of 21 students did not participate in the self-reporting

surveys. Unfortunately, missing data is a particular problem for social network analysis. As

Enriquez acknowledges, this “is not just a matter of completeness, rather a different network may

be produced that may render the analysis invalid and the findings misleading” (2010, p.57).

Nonetheless, her analysis of a single class showed that “there are ties outside the virtual

environment that have relational effects on the threads that are tied online, and … there are ties

that are produced on the [discussion] board that would have not been established otherwise”

(Enriquez, 2010, p.65). She concluded that “[n]etwork links are established in the PGCE history

course through social relations maintained and sustained across different communication

networks and through a common goal” (Enriquez, 2010, p.65). Heo et al. make a similar point,

  21  

noting that “[g]iven that students were able to talk via face-to-face or synchronous chatting,

various methods to capture other forms of interaction should be considered” (2010, p.1391).

Although her research was conducted in the context of “a blended learning situation”

(Enriquez, 2010, p.64), Enriquez’s critique goes further than simply arguing that SNA must be

used with care in blended settings. Instead, she contends that “‘perceived’ ties are important for

studying social influences,” arguing that “[w]e should not solely rely on the ‘actual’ ties (i.e. by

clicking the ‘Reply’ button) captured and saved in a database” (Enriquez, 2008, p.125). Thus,

Enriquez and others argue for a nuanced approach to SNA that accounts for context and takes

account of other information such as perceived ties, qualitative interviews, and content analysis to

provide a full picture of the learning process and environment.

E. Research Questions

This paper seeks to add to the scant literature about the relationships between the

different types of interaction in asynchronous online discussion. It begins by exploring the relative

frequency and distribution of reading, replying, linking, and liking interactions and then analyzes

the relationships between the four types of interaction. Particular attention is paid to the

relationships between outbound interactions, which are within a participant’s control, and

inbound interactions. In order to maximize the utility of this research, the study analyzed the

interactions of 121 students enrolled in seven fully-online courses designed using social

constructivist pedagogical principles. While this phase of the study was quantitative in nature,

future research will include surveys and qualitative interviews to give further context and insight.

  22  

Chapter 3: Methods

A. Pepper

This research focuses on several fully-online classes taught on the Pepper learning

management system created at OISE. Pepper was developed as both a teaching and research

tool and has been used to teach hundreds of classes at OISE. Pepper was designed to facilitate

knowledge building primarily through asynchronous online discussion and is continually updated

and improved in response to feedback and research.

Students participate in asynchronous class discussions on Pepper by reading the “notes”

(or posts) of other participants and replying to those notes to form a threaded discussion.

Participants can also embed hyperlinks to other notes and “like” other notes. The online

discussion comprises four types of interactions: reads, replies, links, and likes.

Pepper automatically records all interactions and a suite of Pepper tools extracts data for

each type of interaction, generating reports on who reads who, who replies to who, who links to

who, and who likes who. Using these tools, a series of “edge lists” were created for each class,

recording the source and target for each interaction. Interactions for participants who dropped

the class were excluded. This resulted in a loss of less than 5% of the interactions in each

category for each class except when there was significant activity by a teaching or research

assistant after a class had ended (see Appendix A). Each edge list was saved in Excel format for

easy import into SPSS for conventional statistical analysis and UCINET for social network

analysis.

  23  

B. Social Network Analysis

UCINET, a specialized social network analysis tool, was used to transform the edge lists

generated from the Pepper reports into adjacency matrices. An adjacency matrix is a square

matrix that lists the actors in a network in both the rows and the columns of the matrix and

“record[s] information about the ties between each pair of actors” in the cells of the matrix

(Hanneman & Riddle, ch.5). Directed, weighted ties were calculated by counting each instance of

each type of interaction in order to measure the intensity of the relationships between

participants. For example, each time one participant replied to another the replying out-degree

of the source increased by one and the replying in-degree of the target increased by one. Multiple

replies to the same note were counted separately since they contributed to the intensity of the

relationship. The same procedure was applied to linking and liking interactions. While somewhat

less intuitive, reading interactions were treated the same way. Each time one participant read a

note written by another participant the reading out-degree of the source (reader) increased by

one and the reading in-degree of the target (author) increased by one. Multiple “reads” of a note

(revisiting) were counted as separate interactions since they contributed to the intensity of the

relationship. Adjacency matrices for each type of interaction (reads, replies, links, and likes) were

created for each class (see Appendix B).

Once the data was stored in matrices, UCINET was used to calculate the density,

centralization, and average in-degree and out-degree centrality for each network. UCINET was

also used to calculate the weighted in-degree and out-degree for each type of interaction for each

participant. Finally, to facilitate comparisons across the seven classes, normalized degrees were

calculated by dividing each participant’s tie count by the number of other participants (n-1) in

the network (Hanneman, 2005, ch.10). In-degree and out-degree (and their standardized

counterparts) for each type of interaction were then recorded in an attribute matrix more typical

  24  

of standard statistical analysis. SPSS was used to create histograms to visualize the frequency

distributions of students’ in-degree and out-degree for each type of interaction.

Inferential statistics present particular issues in social network analysis because “not only

are the data dependent among observations, but we are fundamentally interested in that

dependence as our core question” (Grunspan et al., 2014, Starting Analysis section). Therefore,

after using SPSS to check assumptions, Pearson correlation coefficients were calculated using a

permutation correlation test in UCINET to account for the dependence of the data. A

permutation correlation test works as follows:

The general idea is to create a distribution of correlations from [the] data by randomly

sampling values from one variable and matching them to another … This creates a null

distribution of correlation coefficients … We can then test the null hypothesis … using

this created distribution.

(Grunspan et al., 2014, Ties as Predictors of Performance section). Because many of the degree

distributions were not normal, I also performed a non-parametric Spearman’s rho analysis in

SPSS (see Hernández-García et al., 2014).

C. Participants

Data were collected from seven fully-online, asynchronous discussion-based classes taught

at the graduate level at the faculty of education of a large Canadian research university between

2011 and 2014. All seven classes were taught by the same instructor. The classes comprised four

sections of “Constructivism and the Design of Online Learning Environments” and three

sections of “Educational Applications of Computer-Mediated Communication.” Class

discussions were organized into weekly conferences that focused on the class readings. The

  25  

instructor or a student moderator posted several prompts (usually three) at the beginning of each

week and students logged in throughout the week to engage in the online discussion. Students

were encouraged to interact with each other and the instructor also participated in the discussion.

The instructor posted a brief video at the beginning of each week to clarify or summarize the

previous week’s discussion and to introduce the topic for the upcoming week.

Classes ranged in size from 14 to 23 students and sometimes included a teaching assistant.

A total of 131 participants, including 121 students, were studied across all seven classes. Students,

teaching assistants, and the instructor were counted as separate actors when they participated in

more than one class. While social network metrics were calculated using all 131 participants,

frequency distributions and correlations were generated using only student data in order to gain

clear insights into student behaviour.

  26  

Chapter 4: Results & Discussion

A. Degree Centrality

Directed ties emanate from a source and terminate at a target. However, messages posted

in an online discussion forum are broadcast to all participants, creating ambiguity as to the target

of the interaction, especially in the case of replies. Although a participant may be responding to a

post by a particular person, the reply is visible to all. As Rabbany et al. (2014, p.8) note, some

researchers treat messages as if they are directed to all participants (e.g. Tirado et al., 2012) and

some treat them as though they are directed to the author of the previous message (e.g. Russo &

Koesten, 2005). Others take a more complicated approach. For example, Haythornthwaite &

Gruzd classified a post as a reply only “when one actor’s post [was] the very next post on the

same topic,” where a topic was considered the same only when the subject line text was identical

to that of the previous post (2012, p.3360). Each of these models is necessarily an imperfect

representation of the interaction.

In this paper, a reply is deemed to be directed only to the author of the post to which it

responds because, as Aviv et al. explain, this “model concentrates on the trigger/response

mechanisms” of the interaction (2003, p.8). These mechanisms are central to the process of

knowledge building discourse. Moreover, a model that treats replies as directed to all other

participants may be too noisy to be illuminating. For example, when Tirado et al. treated all

messages as directed to all other students they found low centralization because all students

received approximately the same number of messages (2012, p.303). Just as a debater replies to

his or her opponent and not to the audience, those participants not directly involved in an

exchange on a discussion forum are more like observers than conversation partners. This

intuition makes even more sense with respect to the other types of visible interaction. Just because

  27  

others in the forum can see a like or a link does not somehow transform them into the target of

the interaction.

Nonetheless, the broadcast feature of posting in an online forum has significant

implications for the selection of centrality measure. While many researchers report closeness and

betweenness centrality for networks created by threaded online discussion, these measures make

little sense in an environment where most3 information is visible to everyone. Recall that

closeness centrality is concerned with “how fast [a node] can spread … information to the whole

network” (Rabbany et al., 2014, p.6), and betweenness centrality “represents the control of a node

over communication within its community” (Carolan, 2014, p.157). These concepts make little

sense in the context of online asynchronous discussion.

Degree centrality, on the other hand, provides information about who is engaging with

who in the social endeavour of knowledge building. A student’s out-degree indicates their efforts

to exert influence (Ruane, 2014). And a “student’s in-degree can be considered as their

interaction attractiveness in a communication network” (Xie et al., 2014, p.12). For these

reasons, this paper uses degree to measure centrality. The remainder of this section reports on

the frequency distributions and central tendencies of the weighted normalized in-degree and out-

degree for each type of interaction.

1. Reading. A total of 151,165 reading interactions were analyzed across all seven

courses. Normalized in-degree reading was positively skewed with Mdn=49.565, M=55.848, and

S.D.=31.715 (see Figure 2 below).

                                                                                                               3 Pepper allows students to send private messages and to post private notes that are visible

only to selected participants. Private messages and notes were not part of this study.

  28  

Figure 2. Normalized in-degree reading.

Normalized out-degree reading was approximately normally distributed, with Mdn=58.280,

M=59.222, and S.D.=26.448 (see Figure 3 below).

Figure 3. Normalized out-degree reading.

  29  

2. Replying. A total of 9,711 replying interactions were analyzed. Normalized in-degree

replying was positively skewed with Mdn=3.471, M=4.001, and S.D.=2.312 (see Figure 4 below).

Figure 4. Normalized in-degree replying. Normalized out-degree replying was also positively skewed, with Mdn=3.706, M=4.110, and

S.D.=2.622 (see Figure 5 below).

  30  

Figure 5. Normalized out-degree replying.

3. Liking. Normalized in-degree liking was quite positively skewed, with Mdn=2.235,

M=3.082, and S.D.=2.528 (see Figure 6 below).

Figure 6. Normalized in-degree liking.

  31  

Normalized out-degree liking was even more positively skewed, with Mdn=1.867, M=3.309, and

S.D.=3.808 (see Figure 7 below).

Figure 7. Normalized out-degree liking.

4. Linking. Normalized in-degree linking was very positively skewed with Mdn=.412,

M=.879, and S.D.=1.166 (see Figure 8 below).

  32  

Figure 8. Normalized in-degree linking.

Normalized out-degree linking was very positively skewed with Mdn=.250, M=.881, and

S.D.=1.616 (see Figure 9 below).

Figure 9. Normalized out-degree linking.

  33  

5. Degree distributions compared. A summary of the different types of interactions

is shown in Table 1 below.

Table 1

Degree Distributions Interaction (norm’d) Median Mean Std. Deviation

nReadIn 49.565 55.848 31.715

nReadOut 58.280 59.222 26.448

nReplyIn 3.471 4.001 2.312

nReplyOut 3.706 4.110 2.622

nLikeIn 2.235 3.082 2.528

nLikeOut 1.867 3.309 3.808

nLinkIn .412 .879 1.166

nLinkOut .250 .881 1.616

Recall that the normalized degree is the number of interactions divided by the number of

other students in the class, i.e. the number of possible interaction partners (degree/(n-1)).

Reading interactions are by far the most frequent type of interaction, with a normalized median

of approximately 50-58 inbound and outbound interactions per student. This relatively high

degree is to be expected because a reading interaction is the necessary pre-condition for all other

interactions. Replying interactions, which are more demanding than reading interactions, are still

quite a frequent occurrence, with a normalized median of approximately 3-4 inbound and

outbound replying interactions per student. Again, this is expected since replying interactions

form the core of the traditional asynchronous online discussion. Liking interactions are also quite

common, with a normalized median of 2 inbound and outbound interactions per student.

Finally, linking interactions are relatively infrequent, with fewer than one interaction per

potential interaction partner. Degree frequencies for all interactions are highly variable.

  34  

Degree distributions fall on a continuum from out-degree reading, which is

approximately normal, to in-degree reading and in- and out-degree replying, which are positively

skewed, to in- and out-degree liking and linking, which are very positively skewed.

B. Relationships between Reading, Replying, Liking, and Linking

1. Overall trends. Linear relationships and Pearson correlations between all

combinations of normalized degrees were explored. Because many of the degree distributions

were not normal, a Spearman’s rho correlation test was also done using SPSS. The results were

quite similar and the remainder of this section will refer only to Pearson correlations. A

comparison of the Spearman’s rho and Pearson correlations is shown in Appendix C.

Correlations between in-degree ties were moderate to strong while correlations between

out-degree ties were only weak to moderate or, in some cases, not significant. In other words,

participants employ a variety of strategies and choices when it comes to engaging in the academic

discourse of the class, the outbound interactions that are within their control. However, notes

that inspire other participants to engage tend to elicit a high degree of inbound interaction across

all four types (while less engaging contributions generate low levels of inbound interaction across

the board).

In every case, in-degree and out-degree for each type of interaction were moderately or

strongly correlated. This may suggest at least some amount of reciprocity or it may be that both

inbound and outbound interactions of a particular type have some other common cause. Hew et

al. found that their “interview data suggested that students in [their] studies received help from

the same individual they helped before” suggesting that contributing ideas “motivate[s] other

  35  

students to reciprocate by contributing in return” (2010, p.593). Stepanyan et al. found evidence

of a “tendency towards reciprocating … incoming ties” (2010, p.71). And Aviv et al. found that

“reciprocity is a real effect” after analyzing online discussions from 75 classes (2005, p.8).

Additional research is needed to determine the relative importance of reciprocity and/or

exogenous causes for different types of interaction.

Below at Table 2 is the correlation matrix for the inbound and outbound degrees for all

four types of interaction. The remainder of this paper will focus on the ways in which outbound

interactions, which are within the control of the participant, are related to inbound interactions.

Table 2

Correlation Matrix for In-Degree & Out-Degree nRead

In nReplyIn

nLike In

nLink In

nReadOut

nReplyOut

nLikeOut

nLinkOut

nReadIn 1 nReplyIn .868** 1 nLikeIn .597** .642** 1 nLinkIn .373** .362** .524** 1 nReadOut .401** .455** .277** .110 1 nReplyOut .969** .853** .569** .331** .454** 1 nLikeOut .412** .348** .545** .329** .298** .434** 1 nLinkOut .323** .304** .455** .754** .145 .289** .368** 1 ** Correlation is significant at the 0.01 level (2-tailed).

2. Outbound reading. Out-degree for reading was not strongly correlated with any

other type of interaction. More specifically, outbound reading was only moderately correlated

with inbound reading (r=.401, p<.01, Figure 10) and with inbound and outbound replying

(r=.455, p<.01, Figure 11, r=.454, p<.01), weakly correlated with inbound and outbound liking

(r=.277, p<.01, r=.298, p<.01) and not significantly correlated with either inbound or outbound

  36  

linking. In other words, the number of other people’s replies (or notes4) that a student reads is not

strongly related to their other interactions and possibly, therefore, not all that important for their

engagement in the class discussion. This is somewhat surprising given that, as I noted before,

reading is a precondition of all other inbound interactions. However, these findings support those

of Romero that the number of messages read by a student did not contribute to the prediction of

who would pass or fail the course (2013, p.464).

Figure 10. Normalized out-degree reading vs. normalized in-degree reading.

                                                                                                               4 Most notes are replies and the number of notes and outbound replies in this sample

were very highly correlated (r=.956, p<.01). For this reason, I sometimes use the terms “outbound replies” and “notes” interchangeably. For the sake of consistency, I use normalized outbound reply data for analysis and comparison with other normalized degree data.

  37  

Figure 11. Normalized out-degree reading vs. normalized in-degree replying.

The weak to moderate correlation between outbound reading and other inbound and

outbound interactions has a number of implications. It suggests that what you do once you’ve

read a reply or note is more important than how many notes you’ve read. A participant can be

highly engaged in the class discussion by actively interacting with a higher percentage of the

notes they read rather than by simply reading more notes. This finding supports research by

Wise et. al., who concluded that “listening deeply to some of the discussion is preferable to

listening shallowly to all of it” (2014, p.204). Second, it suggests that participants might be able to

maintain a high degree of engagement in the class discussion by employing a variety of strategies

to read efficiently in order to focus on the posts they are most likely to engage with. One such

strategy would be preferring (or avoiding) posts written by particular students. As discussed

above, Vaquero & Cebrian found evidence “of a benefit maximization process by which students

focus their efforts on potentially more fruitful connections” (2013, p.7). Another strategy may be

to preferentially read posts that have been liked or linked to by other participants. Research on

the characteristics of notes that include links and that have been liked suggests that such notes

  38  

have both cognitive and social value and so linking and liking interactions may indicate utility in

the knowledge building process (Makos et al., 2014; Phirangee & Hewitt, 2014). Finally, students

may simply choose to focus on fewer threads each week or to employ other strategies. It will be

important to explore whether (and which) efficient reading strategies enhance or diminish the

quality of the knowledge building process and to consider designing the online platform to

support or encourage these strategies and/or discourage others. For example, we may not want

the discussion to be dominated by a few students who are particularly engaging early in the class

so we may not emphasize authorship of posts. Conversely, we may want peer judgments of

content to help guide the knowledge building efforts of the class to the most productive avenues

of discussion and so might make linking and liking more prominent in the thread view, perhaps

bolding posts that have received a threshold number of links or likes. Further work is needed to

determine what strategies students employ in deciding what to read and what the effects of those

strategies are on both engagement and quality of discourse.

3. Outbound replying. The strongest correlations observed in this study were the very

strong positive correlations between the normalized out-degree for replying and the normalized

in-degrees for reading (r=.969, p<.01, Figure 12) and replying (r=.853, p<.01, Figure 13). Similar

correlations were shown between number of notes and normalized in-degrees for reading

(r=.957, p<.01) and replying (r=.825, p<.01), which is unsurprising given that most notes are

replies. The number of notes and normalized out-degree for replying are very highly correlated

(r=.956, p<.01). Outbound replies were also moderately associated with in-degree liking (r=569,

p<.01, Figure 14) and in-degree linking, although the correlation was much weaker for linking

(r=.331, p<.01, Figure 15). Interestingly, Figure 15 appears to show that participants may fall into

two distinct groups when it comes to the correlation between outbound replies and inbound

  39  

links. One group triggers many more inbound links when they reply more, while the other group

triggers relatively few inbound links when they reply more. Further investigation is needed to

determine whether these two groups are in fact distinct and what may be causing this dichotomy.

Figure 12. Normalized out-degree replying vs. normalized in-degree reading.

Figure 13. Normalized out-degree replying vs. normalized in-degree replying.

  40  

Figure 14. Normalized out-degree replying vs. normalized in-degree liking.

Figure 15. Normalized out-degree replying vs. normalized in-degree linking.

These correlations suggest that in-degree reading and replying, and to a lesser extent, in-

degree liking, are associated with prolific writing. Further research is needed to determine how

  41  

the volume of replies/notes relates to the quality of engagement in terms of knowledge building.

Again, it’s unclear whether the high correlation is the result of reciprocity or exogenous causes.

Answering this question may shed light on the value of prolific replying (and the highly correlated

inbound reading) for the knowledge building enterprise.

4. Outbound linking. Outbound linking was strongly correlated with inbound linking

(r=.754, p<.01, Figure 16), but only moderately correlated with inbound liking (r=.455, p<.01,

Figure 17) and weakly correlated with inbound reading (r=.323, p<.01) and replying (r=.304,

p<.01). Interestingly, out-degree linking is also moderately correlated with out-degree liking

(r=.368, p<.01). These correlations might suggest that the purpose served by linking interactions

is somewhat related to the purpose served by liking interactions but different from the functions

served by reading and replying interactions. Again, more research is needed to determine the

relative influence of reciprocity norms and exogenous factors.

Figure 16. Normalized out-degree linking vs. normalized in-degree linking.

  42  

Figure 17. Normalized out-degree linking vs. normalized in-degree liking.

5. Outbound liking. Outbound liking presents the most complicated picture. It is

moderately correlated with inbound liking (r=.545, p<.01, Figure 18), reading (r=.412, p<.01),

and replying (r=.348, p<.01) and weakly correlated with inbound linking (r=.329, p<.01). Unlike

outbound reading, outbound liking is not a necessary precondition of other interactions; yet it is

somewhat more highly correlated with other interactions than outbound reading. It may be that

reciprocity is at play, particularly since likes are much more visible than reads in the Pepper

interface. A thumbs up shows a like whereas participants need to make a point of checking to see

who has read a particular note. Nonetheless, it’s likely that there is more at work than simple

reciprocity since liked notes seem to be of higher quality (Makos et al., 2014). It may be that even

within a reciprocal interaction participants select the best notes of their target for likes. Further

research is needed to explore the relationship between liking and other types of interaction.

  43  

Figure 18. Normalized out-degree vs. in-degree liking.

6. Limitations. This study was limited to several small classes in a single department

taught by a single instructor on a single platform at the faculty of education of a large research

university. Consequently, the results may not be generalizable to other types of classes or settings.

In addition, the correlations discussed in this paper do not imply causality. Further research is

needed to investigate whether these relationships are causal. In particular, additional information

about the attributes and experiences of students should be gathered through surveys and

qualitative interviews.

  44  

Chapter 5: Conclusions & Implications for Future Research

This study yielded a number of interesting insights into the relationships between the

different types of interaction in asynchronous online discussion. One interesting finding was that

correlations between different types of in-degree interactions were moderate to strong while

correlations between different types of out-degree interactions were only weak to moderate, or

even not significant. This suggests that students use a variety of different strategies when they

choose to engage in online discussion and that students who establish “interaction attractiveness”

elicit inbound interactions of all types.

One surprising result was that outbound reading is not strongly correlated with other

types of interaction, despite being a prerequisite for all other interactions. The high frequency of

reading compared to other interactions may account for this phenomenon. The weak to

moderate correlations may also indicate that broad reading without further interaction is not

particularly useful. It may be that focused reading strategies can lead to high engagement in

online discussion and help students avoid overload. As discussed above, some efficient reading

strategies may be more salutary than others for the knowledge building process. For example,

preferential reading of notes written by participants who quickly establish themselves as highly

engaging at the beginning of the course may result in the exclusion of inexperienced students or

English language learners and the valuable ideas and diverse perspectives they may otherwise

contribute. Conversely, preferential reading of liked notes or notes with links may help move the

knowledge construction process forward. Additional research is needed to determine which

strategies students use and how they affect the engagement of others and the quality of the

knowledge building process.

  45  

Another interesting finding was that linking interactions seem to be sui generis. The

frequency distributions for both in- and out-degrees for linking are highly positively skewed and

look very different from the frequency distributions of reading and replying. The only interaction

strongly correlated with out-degree linking is in-degree linking and vice versa. More broadly, it

seems that reading and replying interactions may fall towards the social end of a social-cognitive

spectrum, linking may fall towards the cognitive end, and liking may fall in between (although

each type of interaction is a mixture of both social and cognitive).

Finally, some of the strongest correlations were between outbound replying and inbound

replying and reading, and between outbound and inbound linking. These results may indicate

that norms of reciprocity play a crucial role in shaping some interaction networks. However,

other dynamics are also likely at play. One possibility is that students who write more replies

simply have more notes for others to read and reply to. However, it’s also possible that the

correlations indicate assortative interaction patterns between similarly performing students.

Students who write more replies may tend to interact with other students who write more replies.

This may be even more true of linking interactions. It might be that students who work to

integrate their notes into the broader discussion by linking to other notes tend to interact more

with others who are doing the same thing. It may also be that the presence of links indicates a

higher level of cognitive work that makes students who include links attractive interaction

partners. Further research is needed to determine which dynamics shape the networks formed by

the different types of interaction. A better understanding of these processes will allow instructors

to use SNA metrics to assess both the quality and the quantity of student contributions and also

to ensure that the discussion is a productive one for all participants.

This study has focused on courses that were informed by social constructivism and

designed for collaborative knowledge building. Future research using larger and more diverse

  46  

groups should aim to clarify characteristic social network patterns for different types of online

courses. These patterns would allow instructors to monitor courses to ensure that they align with

the intended pedagogy and could also be used to test the effects of pedagogical innovations.

  47  

Works Cited

Assimakopoulos, D. &, & Yan, J. (2006). Social Network Analysis and Communities of Practice.

In Encyclopedia of Communities of Practice in Information and Knowledge Management. IGI Global.

Aviv, R., Erlich, Z., & Ravid, G. (2005). Reciprocity analysis of online learning networks. Journal

of Asynchronous Learning Networks, 9(4), 3–13.

Aviv, R., Erlich, Z., Ravid, G., & Geva, A. (2003). Network analysis of knowledge construction in

asynchronous learning networks. Journal of Asynchronous Learning Networks, 7(3), 1–23.

Bijedic, N., Hamulic, I., Junuz, E., Maksumic, I., & Radosav, D. (2012). Modeling SNA result to

improve learning community. Procedia - Social and Behavioral Sciences, 64, 52–57.

http://doi.org/10.1016/j.sbspro.2012.11.007

Carolan, C. B. V. (2014). Social Network Analysis Education: Theory, Methods & Applications. Thousand

Oaks: SAGE Publications, Inc.

Cela, K. L., Sicilia, M. Á., & Sánchez, S. (2014). Social network analysis in e-learning

environments: A preliminary systematic review. Educational Psychology Review, 27(1), 219–246.

http://doi.org/10.1007/s10648-014-9276-0

Cowan, J. E., & Menchaca, M. P. (2014). Investigating value creation in a community of practice

with social network analysis in a hybrid online graduate education program. Distance

Education, 35(1), 43–74. http://doi.org/10.1080/01587919.2014.893813

Dawson, S. (2008). A study of the relationship between student social networks and sense of

community. Educational Technology & Society, 11(3), 224–238.

  48  

De Laat, M., Lally, V., Lipponen, L., & Simons, R.-J. (2007). Investigating patterns of interaction

in networked learning and computer-supported collaborative learning: A role for Social

Network Analysis. International Journal of Computer-Supported Collaborative Learning, 2(1), 87–103.

http://doi.org/10.1007/s11412-007-9006-4

Dewey, J. (1926). My Pedagogic Creed. The Journal of Education, 104(21), 542.

Doran, P. R., & Mazur, A. (2011). Social network analysis as a method for analyzing interaction

in collaborative online learning environments. Systemics, Cybernetics & Informatics, 9(7), 10–16.

Enriquez, J. G. (2008). Translating networked learning: Un-tying relational ties. Journal of

Computer Assisted Learning, 24(2), 116–127. http://doi.org/10.1111/j.1365-

2729.2007.00273.x

Enriquez, J. G. (2010). Fluid centrality: A social network analysis of social–technical relations in

computer‐mediated communication. International Journal of Research & Method in Education,

33(1), 55–67. http://doi.org/10.1080/17437271003597915

García-Saiz, D., Palazuelos, C., & Zorrilla, M. (2014). Data mining and social network analysis

in the educational field: An application for non-expert users. In A. Peña-Ayala (Ed.),

Educational Data Mining: Applications and Trends (Vol. 524, pp. 411–439). Berlin: Springer

International Publishing. http://doi.org/10.1007/978-3-319-02738-8

Grunspan, D. Z., Wiggins, B. L., & Goodreau, S. M. (2014). Understanding classrooms through

social network analysis: A primer for social network analysis in education research. Cell

Biology Education, 13(2), 167–178. http://doi.org/10.1187/cbe.13-08-0162

  49  

Halgin, D., & Labianca, J. (2013). Introduction to social network analysis [PDF document]. Retrieved

from Links Center for Social Network Analysis, University of Kentucky:

http://danhalgin.com/yahoo_site_admin/assets/docs/AOM_Intro_to_SNA_PDW_2013_

Final.23591848.pdf

Hamulic, I., & Bijedic, N. (2009). Social network analysis in virtual learning community at

faculty of information technologies (fit), Mostar. Procedia - Social and Behavioral Sciences, 1(1),

2269–2273. http://doi.org/10.1016/j.sbspro.2009.01.399

Hanneman, R. A., & Riddle, M. (2005). Introduction to social network methods. Riverside, CA:

Published in digital form at http://faculty.ucr.edu/~hanneman/.

Haya, P. a., Daems, O., Malzahn, N., Castellanos, J., & Hoppe, H. U. (2015). Analysing content

and patterns of interaction for improving the learning design of networked learning

environments. British Journal of Educational Technology, 46(2), 300–316.

http://doi.org/10.1111/bjet.12264

Haythornthwaite, C., & Gruzd, A. (2012). Exploring patterns and configurations in networked

learning texts. In 45th Hawaii International Conference on System Sciences (pp. 3358–3367). Ieee.

http://doi.org/10.1109/HICSS.2012.268

Heo, H., Lim, K. Y., & Kim, Y. (2010). Exploratory study on the patterns of online interaction

and knowledge co-construction in project-based learning. Computers & Education, 55(3),

1383–1392. http://doi.org/10.1016/j.compedu.2010.06.012

  50  

Hernández-García, Á., González-González, I., Jiménez-Zarco, A. I., & Chaparro-Peláez, J.

(2014). Applying social learning analytics to message boards in online distance learning: A

case study. Computers in Human Behavior, 47, 68–80.

http://doi.org/10.1016/j.chb.2014.10.038

Hew, K. F., Cheung, W. S., & Ng, C. S. L. (2010). Student contribution in asynchronous online

discussion: A review of the research and empirical exploration. Instructional Science, 38(6),

571–606.

Hewitt, J. (2005). Toward an Understanding of How Threads Die in Asynchronous Computer

Conferences. Journal of the Learning Sciences, 14(4), 567–589.

http://doi.org/10.1207/s15327809jls1404_4

Hewitt, J., Brett, C., & Peters, V. (2007). Scan Rate: A New Metric for the Analysis of Reading

Behaviors in Asynchronous Computer Conferencing Environments. American Journal of

Distance Education, 21(4), 215–231. http://doi.org/10.1080/08923640701595373

Jimoyiannis, A., & Angelaina, S. (2012). Towards an analysis framework for investigating

students’ engagement and learning in educational blogs. Journal of Computer Assisted Learning,

28(3), 222–234. http://doi.org/10.1111/j.1365-2729.2011.00467.x

Jimoyiannis, Athanassios; Tsiotakis, Panagiotis; Roussinos, D. (2013). Social network analysis of

students’ participation and presence in a community of educational blogging. Interactive

Technology and Smart Education, 10(1), 15–30. Retrieved from

http://www.emeraldinsight.com/doi/pdf/10.1108/17415651311326428

  51  

Jostad, J., Sibthorp, J., & Paisley, K. (2013). Understanding groups in outdoor adventure

education through social network analysis. Australian Journal of Outdoor Education, 17(1), 17–

31.

Lee, M. (2014). Bringing the Best of Two Worlds Together for Social Capital Research in

Education: Social Network Analysis and Symbolic Interactionism. Educational Researcher,

43(9), 454–464. http://doi.org/10.3102/0013189X14557889

Lipponen, L., Rahikainen, M., Lallimo, J., & Hakkarainen, K. (2003). Patterns of participation

and discourse in elementary students’ computer-supported collaborative learning. Learning

and Instruction, 13(5), 487–509. http://doi.org/10.1016/S0959-4752(02)00042-7

Makos, A., Oztok, M., Zingaro, D., & Hewitt, J. (2013). Use of a Like Button in a Collaborative

Online Learning Environment Community of Inquiry Framework. In American Education

Research Association Annual Conference. San Francisco, CA.

Makos, A., Zingaro, D., Oztok, M., & Hewitt, J. (2014). Examining the qualities of liked notes

versus non-liked notes in a collaborative online learning environment. In American Education

Research Association Annual Conference (pp. 1–5). Philadelphia, PA.

Nistor, N., Trăuşan-Matu, Ş., Dascălu, M., Duttweiler, H., Chiru, C., Baltes, B., & Smeaton, G.

(2015). Finding student-centered open learning environments on the internet: Automated

dialogue assessment in academic virtual communities of practice. Computers in Human

Behavior, 47, 119–127. http://doi.org/10.1016/j.chb.2014.07.029

  52  

Nurmela, K., Lehtinen, E., & Palonen, T. (1999). Evaluating CSCL log files by social network

analysis. In Proceedings of the 1999 Conference on Computer Support for Collaborative Learning - CSCL

’99 (p. 54–). Morristown, NJ, USA: Association for Computational Linguistics.

http://doi.org/10.3115/1150240.1150294

Oshima, J., Oshima, R., & Matsuzawa, Y. (2012). Knowledge Building Discourse Explorer: a

social network analysis application for knowledge building discourse. Educational Technology

Research and Development, 60(5), 903–921. http://doi.org/10.1007/s11423-012-9265-2

Palonen, T., & Hakkarainen, K. (2013). Patterns of interaction in computer-supported learning:

A social network analysis. International Conference of the Learning Sciences, 334–39.

Paredes, W. C., Shing, K., & Chung, K. (2012). Modelling learning & performance: A social

networks perspective. In Proceedings of the 2nd International Conference on Learning Analytics and

Knowledge (pp. 34–42).

Phirangee, K., & Hewitt, J. (2014). Lively Discussions: Using Linking to Enrich Threaded

Discourse Krystle Phirangee and Jim Hewitt Ontario Institute for Studies in Education,

University of Toronto. In American Education Research Association Annual Conference.

Philadelphia, PA.

Quinton, S. R. (2010). Principles of effective learning environment design. In M. Ebner & M.

Schiefner (Eds.), Looking Toward the Future of Technology-Enhanced Education: Ubiquitous Learning

and the Digital Native (pp. 327–352). IGI Global. http://doi.org/10.4018/978-1-61520-678-0

  53  

Rabbany, R., Elatia, S., Takaffoli, M., & Zaïane, O. R. (2014). Collaborative Learning of

Students in Online Discussion Forums  : A Social Network Analysis Perspective. In A. Peña-

Ayala (Ed.), Educational Data Mining: Applications and Trends (pp. 1–25). Berlin: Springer.

Rabbany, R., Takaffoli, M., & Za, O. R. (2011). Social Network Analysis and Mining to Support

the Assessment of On-line Student Participation. SIGKDD Explorations, 13(2), 20–29.

Reffay, C., & Chanier, T. (2002). Social network analysis used for modelling collaboration in

Distance Learning groups. In Intelligent Tutoring System (pp. 31–40). Biarritz and San

Sebastian, France: Springer-Verlag.

Reffay, C., & Chanier, T. (2003). How social network analysis can help to measure cohesion in

collaborative distance-learning. In Computer Supported Collaborative Learning (pp. 343–352).

Bergen, Norway: Kluwer Academic Publishers. Retrieved from

http://www.intermedia.uib.no/csl/

Romero, C., López, M.-I., Luna, J.-M., & Ventura, S. (2013). Predicting students’ final

performance from participation in on-line discussion forums. Computers & Education, 68, 458–

472. http://doi.org/10.1016/j.compedu.2013.06.009

Ruane, R. (2014). Social network analysis of undergraduate education student interaction in

online peer mentoring settings. MERLOT Journal of Online Learning and Teaching, 10(4), 577–

89.

Russo, T. C., & Koesten, J. (2005). Prestige, centrality, and learning: A social network analysis of

an online class. Communication Education, 54(3), 254–261.

http://doi.org/10.1080/03634520500356394

  54  

Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge. In

B. Smith (Ed.), Liberal Education in a Knowledge Society (pp. 1–26). Chicago: Open Court.

Retrieved from http://ikit.org/fulltext/inpressCollectiveCog.pdf

Shea, P., Hayes, S., Uzuner-Smith, S., Gozza-Cohen, M., Vickers, J., & Bidjerano, T. (2014).

Reconceptualizing the community of inquiry framework  : An exploratory analysis. Internet

and Higher Education, 23, 9–17.

Sing, C. C., & Khine, M. S. (2006). An analysis of interaction and participation patterns in online

community. Educational Technology & Society, 9(1), 250–261.

Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning  :

An historical perspective. In R. K. Sawyer (Ed.), Cambridge Handbook of Learning Sciences (pp.

409–426). Cambridge, UK: Cambridge University Press.

Stepanyan, K., Borau, K., & Ullrich, C. (2010). A social network analysis perspective on student

interaction within the Twitter microblogging environment. In The 10th IEEE International

Conference on Advanced Learning Technologies (pp. 70–72). Sousse, Tunisia. Retrieved from

http://wrap.warwick.ac.uk/55804

Tirado, R., Hernando, Á., & Aguaded, J. I. (2012). The effect of centralization and cohesion on

the social construction of knowledge in discussion forums. Interactive Learning Environments,

23(3), 293–316. http://doi.org/10.1080/10494820.2012.745437

Toikkanen, T., & Lipponen, L. (2011). The applicability of social network analysis to the study of

networked learning. Interactive Learning Environments, 19(4), 365–379.

http://doi.org/10.1080/10494820903281999

  55  

Vaquero, L. M., & Cebrian, M. (2013). The rich club phenomenon in the classroom. Scientific

Reports, 3, 1174. http://doi.org/10.1038/srep01174

Vrasidas, C. (2000). Constructivism versus objectivism: Implications for interaction, course

design, and evaluation in distance education. International Journal of Educational

Telecommunications, 6(4), 339–362.

Vygotsky, L. (n.d.). Mind in Society: The Development of Higher Psychological Processes. Cumberland, RI:

Harvard University Press.

Wilton, L., & Brett, C. (2014). Exploring Revisiting in an Online Collaborative Learning

Environment. In Annual meeting of the American Educational Research Association. Philadelphia, PA.

Wise, A. F., Hausknecht, S. N., & Zhao, Y. (2014). Attending to others’ posts in asynchronous

discussions: Learners’ online “listening” and its relationship to speaking. International Journal

of Computer-Supported Collaborative Learning, 9(2), 185–209. http://doi.org/10.1007/s11412-

014-9192-9

Xie, K., Yu, C., & Bradshaw, A. C. (2014). Impacts of role assignment and participation in

asynchronous discussions in college-level online classes. The Internet and Higher Education, 20,

10–19. http://doi.org/10.1016/j.iheduc.2013.09.003

Xin, C., & Feenberg, A. (2006). Pedagogy in cyberspace: The dynamics of online discourse.

Journal of Distance Education, 21(2), 1–25.

Zingaro, Daniel and Oztok, M. (2012). Interaction in an asynchronous online course. Journal of

Asynchronous Learning Networks, 16(4), 71–82.

  56  

Appendix A: Subject Classes

The tables below summarize the raw data for each class as well as the data omitted for

students who dropped the class.

CTL 1609 (Fall 2011) [db12] ni=1 nta=1 ns=14 Total Remaining # Deleted % Deleted Reads 19,778 19,170 608 3.07%* Replies 1248 1248 0 0% Links 179 179 0 0% Likes 187 187 0 0%

* Note: All 608 reads that were deleted were by teachers/TA’s or visitors. CTL 1608 (Winter 2012) [db72] ni=1 ns=17 Total Remaining # Deleted % Deleted Reads 20,802 20,648 154 .74%* Replies 1253 1253 0 0% Links 102 102 0 0% Likes 474 474 0 0%

* Note: All 154 reads that were deleted were by teachers/TA’s or visitors. CTL 1608 (Winter 2013) [db190] ni=1 nta=1 ns=19 Total Remaining # Deleted % Deleted Reads 19,806 19,492 314 1.59% Replies 1051 1051 0 0% Links 696 696 0 0 % Likes 1724 1723 1 .06%

  57  

CTL 1609 (Fall 2012) [db191] ni=1 ns=15 Total Remaining # Deleted % Deleted Reads 16,685 15,696 989 5.93% Replies 1044 1018 26 2.49% Links 752 664 88 11.70%* Likes 831 820 11 1.32%

* Note: 77 of the links that were removed were created by a T.A. after the course was completed. CTL 1609 (Fall 2013) [db369] ni=1 nta=1 ns=17 Total Remaining # Deleted % Deleted Reads 20,102 18,131 1971 9.80% Replies 1358 1358 0 0% Links 211 167 44 20.85%*

Likes 880 880 0 0% * Note: All 44 links that were removed were created by a T.A. after the course was completed. CTL 1608 (Winter 2014) [db370] ni=1

ns=16 Total Remaining # Deleted % Deleted Reads 27,051 25,945 1106 4.09% Replies 1888 1830 58 3.07% Links 325 322 3 .92% Likes 2032 1976 56 2.76%

  58  

CTL 1608 (Fall 2014) [db718] ni=1

ns=23 Total Remaining # Deleted % Deleted Reads 32,669 32,083 586 1.79% Replies 1869 1843 26 1.39% Links 108 108 0 0% Likes 1721 1695 26 1.51%

  59  

Appendix B: Adjacency Matrices

Below are adjacency matrices for each type of interaction for each of the seven classes

included in the study. Participants, represented by an ID#, are listed in both the rows and the

columns of the matrix. The rows show the number of interactions initiated by the participant and

the columns show the number of interactions directed to the participant. For example, in Table

B1, participant #88 read 196 of participant #60’s notes and participant #60 read 25 of

participant #88’s notes.

Table B1 CTL 1609 (Fall 2011) [db12]: Who Reads Who ID# 24 25 26 27 60 65 88 89 105 159 438 457 476 556 624 665

24 200 18 88 130 217 59 54 75 48 66 58 32 97 46 34 58 25 120 31 70 106 161 60 49 59 44 51 84 28 62 40 23 72 26 198 31 118 183 239 95 77 103 84 83 108 40 132 86 46 106 27 168 18 105 184 186 67 55 76 39 60 63 26 98 45 31 81 60 116 16 53 61 241 27 25 50 11 23 37 16 32 3 4 7 65 189 28 113 159 217 95 58 86 47 72 71 34 118 38 32 70 88 166 19 87 126 196 46 78 71 34 69 88 35 81 29 24 40 89 196 30 114 172 238 93 74 103 82 82 104 39 126 61 41 91

105 59 6 22 13 37 7 10 18 84 13 15 8 18 36 10 8 159 152 11 72 74 157 36 60 60 21 83 90 28 69 19 19 34 438 149 20 74 91 157 58 54 62 18 61 108 29 67 40 22 49 457 181 27 94 139 218 76 73 81 47 80 93 40 109 19 37 72 476 193 28 115 181 225 92 60 93 48 70 77 35 132 36 42 96 556 198 30 115 182 239 95 75 102 84 83 108 40 132 88 45 106 624 79 4 47 71 75 29 13 28 11 17 16 11 34 1 47 52 665 177 24 97 154 157 81 45 73 36 48 54 24 105 23 36 107

  60  

Table B2 CTL 1609 (Fall 2011) [db12]: Who Replies to Who ID# 24 25 26 27 60 65 88 89 105 159 438 457 476 556 624 665

24 0 1 7 25 18 12 10 9 0 7 7 1 8 4 3 4 25 1 0 2 2 1 3 2 2 1 2 5 1 3 1 0 0 26 17 3 0 13 9 5 3 6 1 6 3 3 3 1 6 6 27 40 2 21 0 17 7 8 11 1 1 5 4 11 2 5 9 60 35 5 24 24 0 8 12 31 4 6 20 8 18 2 1 5 65 13 6 14 8 5 0 1 2 0 3 4 1 3 0 1 4 88 9 0 5 7 4 0 0 2 2 7 10 1 3 1 2 3 89 8 2 5 10 21 3 3 0 4 5 9 3 7 1 3 3

105 5 1 7 1 3 0 2 3 0 2 1 4 5 24 1 2 159 12 2 13 1 7 1 9 9 0 0 11 0 4 1 4 2 438 5 4 3 3 11 5 12 5 1 13 0 4 5 0 2 2 457 0 0 3 1 1 0 4 2 1 4 4 0 3 1 0 3 476 11 3 14 13 7 8 7 9 2 7 10 9 0 1 4 5 556 4 1 1 4 1 0 1 1 4 2 2 1 3 0 0 4 624 2 0 5 3 2 1 1 0 0 1 1 0 0 0 0 3 665 10 1 3 13 5 5 5 1 1 6 3 1 8 2 5 0

Table B3 CTL 1609 (Fall 2011) [db12]: Who Likes Who ID# 24 25 26 27 60 65 88 89 105 159 438 457 476 556 624 665

24 0 0 0 0 0 0 2 0 0 1 3 0 0 1 1 0 25 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 26 3 0 0 1 2 0 5 4 1 2 2 1 4 0 2 1 27 1 0 0 0 0 1 2 0 0 1 1 0 0 1 0 1 60 2 0 0 1 0 0 2 3 0 0 0 1 0 0 1 0 65 2 0 8 10 1 0 3 1 0 1 1 1 4 1 1 4 88 0 0 1 2 0 0 0 1 0 1 3 0 1 0 0 1 89 5 0 2 1 9 3 6 0 1 2 1 0 5 1 6 3

105 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 159 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 438 0 0 1 0 1 0 0 0 0 1 0 1 0 0 0 0 457 0 0 1 2 2 0 1 0 0 0 0 0 1 1 0 0 476 0 0 1 2 0 1 1 1 3 3 1 2 0 2 0 0 556 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 624 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 665 0 0 1 0 0 2 0 1 0 0 0 1 0 0 0 0

Table B4 CTL 1609 (Fall 2011) [db12]: Who Links to Who ID# 24 25 26 27 60 65 88 89 105 159 438 457 476 556 624 665

24 19 0 2 0 0 0 0 1 0 1 1 0 1 0 0 0 25 0 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 26 1 1 6 2 1 1 0 0 0 1 0 0 0 0 0 1 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 65 4 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 88 1 0 0 0 0 1 2 1 0 1 3 1 0 0 0 0 89 1 0 3 6 8 1 5 0 4 2 6 1 3 1 0 0

105 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 159 1 0 1 0 2 1 1 1 0 3 1 0 1 0 0 0 438 1 1 0 0 2 0 0 3 0 0 1 0 1 0 0 0 457 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 476 4 0 3 5 1 0 2 2 0 1 5 1 2 1 0 1 556 0 0 4 0 0 1 0 1 0 0 1 0 2 2 0 0 624 0 0 0 1 1 0 0 0 0 0 1 0 0 0 2 0 665 3 0 2 1 2 2 0 0 0 0 1 1 0 0 1 1

  61  

Table B5 CTL 1608 (Winter 2012) [db72]: Who Reads Who ID# 24 26 27 59 88 105 185 203 486 602 1308 1354 1357 1382 1407 1489 1490 1501

24 173 91 106 33 47 48 76 69 87 75 66 85 55 44 91 43 107 20 26 172 109 139 43 59 47 86 84 112 77 71 89 63 48 120 45 118 24 27 140 98 141 41 53 42 74 77 103 74 67 83 55 45 92 42 114 24 59 169 106 122 43 56 47 85 83 110 76 69 88 63 47 119 41 118 22 88 144 99 125 41 59 45 70 79 103 74 65 83 61 47 100 39 116 24

105 70 38 49 5 21 48 19 29 35 36 23 31 28 25 45 26 45 8 185 169 108 140 43 59 48 86 84 112 77 70 89 63 48 120 44 118 24 203 99 55 55 19 31 17 32 84 49 47 33 46 29 30 49 19 56 15 486 172 107 138 42 59 47 86 82 112 77 70 88 62 48 119 43 118 23 602 89 64 59 27 32 11 37 40 63 77 35 49 21 36 51 18 75 8

1308 125 76 88 11 32 19 42 49 67 73 71 54 28 39 38 27 86 13 1354 121 64 52 12 27 18 27 50 54 54 35 89 30 37 47 27 69 9 1357 104 57 71 27 39 25 44 49 59 51 42 53 63 29 53 26 74 11 1382 130 71 45 11 24 21 41 45 63 54 38 37 26 48 42 26 65 12 1407 148 100 137 40 53 47 79 79 97 71 67 79 60 47 121 38 105 24 1489 148 98 100 37 49 39 64 70 86 69 56 81 56 46 98 50 97 24 1490 154 101 101 38 51 43 70 77 102 70 66 88 54 46 101 40 118 21 1501 119 80 86 27 44 27 42 61 74 67 59 66 45 41 59 29 103 24 Table B6 CTL 1608 (Winter 2012) [db72]: Who Replies to Who ID# 24 26 27 59 88 105 185 203 486 602 1308 1354 1357 1382 1407 1489 1490 1501

24 0 7 10 1 8 1 8 6 15 13 10 2 1 2 3 6 5 2 26 14 0 8 3 7 0 1 7 9 6 5 8 2 3 7 6 7 2 27 13 12 0 6 9 2 4 7 15 7 10 6 4 3 6 6 4 1 59 1 6 7 0 1 0 1 2 5 1 0 5 2 2 2 0 2 1 88 2 4 7 2 0 2 3 3 4 2 3 3 4 3 3 1 3 1

105 1 3 3 1 3 0 0 4 1 0 1 1 4 4 6 0 4 0 185 13 5 8 1 4 0 0 5 1 3 6 1 1 1 9 3 10 0 203 8 9 11 1 2 0 4 0 8 6 5 2 4 2 6 1 4 1 486 16 16 14 4 7 0 3 9 0 6 3 4 6 4 4 2 5 1 602 9 5 6 0 2 0 1 4 6 0 5 2 5 3 5 3 6 1

1308 6 7 13 1 3 0 2 3 2 2 0 2 3 2 4 7 6 0 1354 14 5 8 1 3 0 0 6 5 8 5 0 4 3 8 4 7 0 1357 2 4 3 3 2 1 1 6 7 3 4 4 0 2 3 2 2 2 1382 6 4 5 1 3 1 1 2 1 1 1 0 1 0 2 2 2 0 1407 5 9 15 3 3 6 7 10 6 6 0 10 5 6 0 3 15 1 1489 2 5 6 0 1 0 1 3 4 3 1 6 4 1 1 0 2 1 1490 10 9 11 2 4 2 7 6 6 9 12 3 5 7 12 4 0 2 1501 2 4 0 0 0 0 1 0 0 1 1 0 1 2 1 0 4 0

  62  

Table B7 CTL 1608 (Winter 2012) [db72]: Who Likes Who ID# 24 26 27 59 88 105 185 203 486 602 1308 1354 1357 1382 1407 1489 1490 1501

24 0 0 0 0 1 0 0 0 2 0 0 0 0 0 1 0 0 1 26 2 0 6 1 4 0 1 2 6 2 2 4 1 1 1 0 6 2 27 3 1 0 0 8 1 4 3 4 3 4 3 0 2 5 3 2 2 59 0 0 6 0 0 2 0 0 1 2 1 0 0 1 0 0 1 0 88 2 1 1 0 0 0 2 0 1 0 1 0 0 0 1 0 0 0

105 1 3 4 0 1 0 0 0 2 0 0 0 2 5 4 1 2 1 185 4 2 5 0 4 0 0 3 2 1 1 3 1 2 6 2 2 2 203 0 1 1 0 2 0 3 0 3 2 0 2 1 3 2 1 1 0 486 2 3 4 0 3 0 0 1 0 2 2 0 1 1 0 0 0 1 602 0 0 0 0 0 1 0 2 0 0 0 0 0 0 1 0 1 0

1308 0 0 1 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 1354 1 1 0 0 0 0 0 1 0 0 1 0 0 1 0 0 0 0 1357 1 3 6 1 0 2 1 2 4 5 2 4 0 1 0 1 1 1 1382 7 0 2 0 3 2 1 0 2 5 0 1 0 0 2 0 1 1 1407 14 9 13 4 7 2 9 12 5 9 4 9 5 5 0 3 11 2 1489 1 3 3 0 5 2 2 2 3 2 1 1 0 0 4 0 0 2 1490 0 0 1 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 1501 2 3 0 0 0 0 0 0 0 1 5 1 0 0 0 0 1 0 Table B8 CTL 1608 (Winter 2012) [db72]: Who Links to Who ID# 24 26 27 59 88 105 185 203 486 602 1308 1354 1357 1382 1407 1489 1490 1501

24 5 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 26 5 1 1 0 2 2 0 1 3 1 1 1 0 1 1 0 2 1 27 3 0 2 0 0 0 0 1 0 1 1 2 0 1 2 0 0 0 59 1 0 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 88 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0

105 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 185 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 203 0 1 1 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 486 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 602 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

1308 1 1 1 0 1 0 0 3 1 1 0 0 0 0 0 0 3 0 1354 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1357 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1382 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1407 0 3 4 0 0 0 0 1 1 2 0 0 0 0 1 0 1 0 1489 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 2 1 0 1490 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1501 2 3 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0

  63  

  64  

  65  

Table B13 CTL 1609 (Fall 2012) [db191]: Who Reads Who ID# 24 185 452 480 486 1362 1374 1396 1649 2746 2962 3001 3074 3823 3967 4093

24 191 57 42 78 71 44 48 80 35 110 169 54 72 36 26 55 185 181 64 45 86 86 47 54 91 38 117 178 59 80 39 25 61 452 175 57 47 81 82 40 51 82 34 114 170 55 74 37 17 60 480 169 50 42 87 74 42 42 78 28 110 166 56 71 32 21 56 486 181 64 45 87 90 47 54 92 39 117 178 59 81 40 26 62

1362 183 64 45 87 87 49 54 92 39 117 178 59 81 40 26 62 1374 123 39 33 62 66 36 54 61 25 80 143 48 67 31 16 41 1396 183 64 45 87 87 47 54 95 39 117 178 59 81 40 26 62 1649 111 20 23 38 35 18 20 38 39 44 77 35 30 10 8 20 2746 187 64 45 87 87 47 54 92 39 124 178 59 81 40 26 62 2962 183 64 45 87 87 47 54 92 39 117 180 59 81 40 26 62 3001 183 64 45 87 87 47 54 92 39 117 178 60 81 40 26 62 3074 99 26 26 48 58 22 28 56 9 76 123 41 81 24 14 33 3823 73 5 12 20 28 14 12 17 7 20 38 10 14 40 7 17 3967 63 3 6 8 9 7 8 5 3 10 37 9 15 7 31 7 4093 123 18 20 27 47 12 27 20 9 38 83 20 34 15 10 71 Table B14 CTL 1609 (Fall 2012) [db191]: Who Replies to Who ID# 24 185 452 480 486 1362 1374 1396 1649 2746 2962 3001 3074 3823 3967 4093

24 0 7 3 8 11 4 6 5 4 9 18 9 4 3 4 3 185 7 0 1 7 9 0 4 2 1 5 6 2 4 2 0 5 452 7 0 0 3 1 5 0 4 0 3 7 1 3 1 0 1 480 14 4 2 0 6 4 5 7 0 12 9 4 1 3 0 6 486 10 5 2 3 0 3 6 6 0 10 11 6 5 2 1 8

1362 2 2 2 3 4 0 5 4 1 5 5 3 2 1 0 0 1374 4 0 1 4 6 2 0 4 0 5 6 2 7 1 1 2 1396 11 3 3 14 3 5 2 0 1 12 11 4 6 3 2 1 1649 11 0 3 1 2 2 2 4 0 2 3 1 0 0 0 2 2746 23 6 4 11 15 4 4 11 0 0 18 4 5 1 2 7 2962 28 5 8 15 10 6 5 13 1 18 0 8 14 6 2 11 3001 4 0 3 4 4 2 3 4 0 6 9 0 4 1 0 1 3074 10 3 1 2 6 3 6 4 0 9 15 5 0 3 2 3 3823 10 0 1 5 5 3 0 1 0 1 5 0 1 0 0 2 3967 4 0 0 0 0 0 0 1 0 2 2 0 2 0 0 0 4093 8 6 1 2 7 1 4 0 2 4 6 2 3 3 0 0

  66  

Table B15 CTL 1609 (Fall 2012) [db191]: Who Likes Who ID# 24 185 452 480 486 1362 1374 1396 1649 2746 2962 3001 3074 3823 3967 4093

24 0 2 1 3 2 0 1 1 1 2 4 1 0 2 2 1 185 5 0 3 4 8 1 5 1 2 3 3 4 3 2 1 8 452 1 0 0 3 2 0 2 0 0 5 7 1 1 1 0 0 480 2 3 0 0 2 0 0 1 0 2 4 1 4 0 0 1 486 8 5 2 9 0 2 7 1 2 4 6 2 1 1 0 4

1362 1 3 0 3 2 0 0 1 1 2 4 1 2 1 0 2 1374 3 8 2 3 7 3 0 2 3 3 4 3 3 0 1 5 1396 3 2 2 5 1 2 2 0 0 1 8 0 5 0 0 1 1649 3 2 3 2 4 1 2 1 1 5 1 2 1 1 2 3 2746 11 2 1 5 10 1 2 3 1 0 18 4 7 3 1 8 2962 11 12 7 19 14 1 7 5 5 5 0 9 14 3 6 15 3001 2 3 1 4 10 3 4 0 0 9 13 0 7 0 0 5 3074 7 8 4 7 10 1 3 4 0 11 18 6 0 1 1 7 3823 0 0 1 1 3 0 0 0 0 0 3 0 0 0 0 0 3967 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 4093 18 12 11 9 12 2 7 1 3 13 24 6 11 5 3 0 Table B16 CTL 1609 (Fall 2012) [db191]: Who Links to Who ID# 24 185 452 480 486 1362 1374 1396 1649 2746 2962 3001 3074 3823 3967 4093

24 7 0 0 0 1 1 1 0 0 2 0 1 0 0 0 0 185 1 0 2 3 1 1 3 4 0 2 7 2 2 4 1 1 452 3 1 0 3 4 1 0 7 0 5 17 0 5 1 0 7 480 5 2 4 6 6 0 1 3 0 6 8 3 5 1 0 3 486 6 5 5 4 1 2 9 5 1 7 17 4 5 2 1 11

1362 0 1 0 2 0 0 1 1 0 0 4 1 1 0 0 0 1374 1 0 3 3 5 0 2 0 1 3 10 1 1 2 0 5 1396 1 2 1 4 1 1 2 1 0 1 5 1 0 2 0 0 1649 4 4 3 2 4 1 2 1 0 3 5 2 4 0 2 2 2746 3 2 2 2 3 2 2 2 0 6 15 2 1 1 0 6 2962 11 7 11 12 6 4 5 13 1 21 6 12 3 7 2 9 3001 2 0 1 4 4 3 4 3 0 2 10 0 4 0 0 1 3074 4 2 4 0 1 3 2 4 0 3 11 3 0 0 1 0 3823 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 3967 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4093 6 5 1 1 9 3 4 2 0 6 14 3 3 1 0 0

  67  

  68  

  69  

Table B21 CTL 1608 (Winter 2014) [db370]: Who Reads Who

ID# 24 1361 1424 2675 2776 2810 6429 8286 8384 8531 9495 9737 9859 9975 10291 10717 10735 24 237 66 243 117 105 108 90 59 84 134 98 144 80 49 98 124 95

1361 173 68 162 65 95 57 57 21 30 117 69 138 68 49 86 137 100 1424 208 67 286 125 117 133 104 62 88 142 105 190 86 54 113 151 109 2675 120 29 157 125 69 60 76 27 58 75 53 96 48 28 31 69 51 2776 132 27 78 33 121 34 48 5 33 88 47 47 25 36 32 72 41 2810 208 67 285 125 117 134 104 62 88 142 105 190 85 54 113 151 109 6429 193 53 256 115 104 115 110 60 76 125 90 158 72 50 83 126 87 8286 114 12 105 45 37 44 52 75 47 54 39 56 20 12 36 47 18 8384 173 55 238 98 107 112 90 49 92 123 93 148 68 45 87 132 89 8531 179 56 234 100 109 99 92 56 74 145 95 130 72 51 93 122 87 9495 189 57 200 103 94 90 88 44 66 113 109 129 60 37 72 83 73 9737 209 67 283 118 116 132 100 62 85 142 101 203 85 53 113 150 109 9859 145 51 119 49 70 23 44 19 25 101 57 67 87 45 62 103 61 9975 131 22 48 36 53 18 32 13 21 79 34 41 33 54 45 83 40

10291 147 47 122 48 70 38 44 20 28 88 52 80 65 42 116 102 67 10717 177 51 139 64 97 53 56 38 39 113 55 128 74 49 91 158 85 10735 196 57 164 76 84 63 67 48 45 107 57 156 74 51 107 140 112 Table B22 CTL 1608 (Winter 2014) [db370]: Who Replies to Who

ID# 24 1361 1424 2675 2776 2810 6429 8286 8384 8531 9495 9737 9859 9975 10291 10717 10735 24 0 3 15 6 9 4 11 9 4 12 6 20 9 12 13 7 9

1361 3 0 4 1 10 1 4 1 0 6 2 4 7 4 2 6 3 1424 17 9 0 33 12 25 19 6 13 23 20 19 15 5 7 9 6 2675 2 3 18 0 6 4 12 1 5 9 6 11 11 4 1 6 2 2776 13 9 9 6 0 1 9 0 3 15 8 7 1 4 4 11 5 2810 7 1 30 12 3 0 8 2 10 10 5 12 3 4 3 7 1 6429 11 3 11 14 10 3 0 4 7 8 4 4 1 4 2 1 4 8286 10 1 10 4 0 0 4 0 5 3 5 7 2 0 0 8 1 8384 8 2 6 7 5 6 12 4 0 7 6 3 1 1 2 3 2 8531 13 7 11 10 20 4 8 2 6 0 7 7 2 11 6 13 5 9495 6 5 11 8 17 0 11 0 3 8 0 7 1 2 7 2 2 9737 30 8 13 20 7 6 11 9 3 13 4 0 9 3 2 19 10 9859 16 7 5 5 5 2 3 2 0 6 0 7 0 4 5 9 3 9975 2 2 1 2 4 1 3 0 0 6 2 1 4 0 2 5 2

10291 18 7 4 4 8 1 3 4 1 15 8 2 10 8 0 9 3 10717 13 4 8 7 15 4 5 2 2 16 3 17 10 8 9 0 4 10735 21 10 4 2 5 0 5 3 2 8 0 10 7 6 4 11 0

  70  

Table B23 CTL 1608 (Winter 2014) [db370]: Who Likes Who

ID# 24 1361 1424 2675 2776 2810 6429 8286 8384 8531 9495 9737 9859 9975 10291 10717 10735 24 0 6 11 9 12 4 7 5 3 13 2 7 6 5 6 5 8

1361 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1424 19 2 0 11 12 6 7 1 5 18 5 6 4 6 3 13 4 2675 22 7 34 0 21 13 21 7 12 26 7 23 15 9 6 15 5 2776 19 9 14 7 0 4 19 1 8 33 13 11 9 7 6 17 7 2810 65 9 68 26 11 0 20 13 19 29 10 36 6 4 14 20 13 6429 10 4 8 11 2 4 0 1 5 9 3 5 3 4 0 1 1 8286 0 1 3 1 0 0 0 1 2 0 0 1 0 0 0 1 0 8384 7 0 0 1 1 1 1 2 0 6 1 2 1 0 0 0 0 8531 8 1 1 3 11 2 4 0 1 0 1 0 0 7 4 5 3 9495 11 0 12 7 3 1 4 3 5 3 0 3 1 1 0 1 0 9737 43 7 7 21 15 5 15 15 5 13 9 0 10 4 4 20 18 9859 21 6 13 9 6 1 4 1 1 10 4 2 0 3 4 8 4 9975 3 2 3 2 1 0 2 0 0 6 1 0 0 0 0 5 2

10291 14 6 6 1 4 3 7 1 0 8 6 4 8 2 0 11 5 10717 23 9 19 15 17 7 5 3 3 19 3 14 11 6 9 0 12 10735 29 16 3 6 9 1 9 5 4 9 4 8 12 7 8 19 0

Table B24 CTL 1608 (Winter 2014) [db370]: Who Links to Who

ID# 24 1361 1424 2675 2776 2810 6429 8286 8384 8531 9495 9737 9859 9975 10291 10717 10735 24 8 4 5 4 3 3 4 4 0 6 3 3 2 2 2 4 1

1361 0 0 0 0 0 0 0 0 0 0 0 1 0 2 1 0 0 1424 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2675 0 1 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 2776 7 7 6 4 4 0 4 0 4 10 5 4 4 5 5 7 2 2810 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6429 1 0 0 3 1 0 1 0 0 2 0 0 1 1 0 0 0 8286 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8384 1 1 0 1 4 1 0 0 0 1 0 0 0 0 0 0 0 8531 1 0 0 1 5 2 1 0 0 0 3 0 0 1 0 0 0 9495 0 0 1 0 6 0 2 0 0 0 1 0 0 1 0 0 0 9737 3 0 0 1 2 0 0 0 0 0 0 2 0 0 0 0 0 9859 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 3 0 9975 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1

10291 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10717 3 4 0 1 0 0 0 0 0 4 0 2 2 3 1 7 1 10735 25 10 3 2 3 0 3 0 1 3 2 4 5 3 5 6 13

  71  

  72  

  73  

  74  

  75  

Appendix C: Spearman’s Rho Correlation Matrix Table C1 Spearman’s Rho Correlation Matrix nRead

In nReplyIn

nLike In

nLink In

nReadOut

nReplyOut

nLikeOut

nLinkOut

nReadIn 1

nReplyIn .904** .868**

1

nLikeIn .534** .597**

.542**

.642** 1

nLinkIn .437** .373**

.473**

.362** .555** .524**

1

nReadOut .443** .401**

.448**

.455** .172 .277**

.168

.110 1

nReplyOut .959** .969**

.878**

.853** .503** .569**

.416**

.331** .486** .454**

1

nLikeOut .440** .412**

.372**

.348** .610** .545**

.398**

.329** .268** .298**

.453**

.434** 1

nLinkOut .355** .323**

.400**

.304** .394** .455**

.622**

.754** .333** .145

.358**

.289** .396** .368**

1

Spearman’s rho in bold with Pearson correlation below. ** Correlation is significant at the 0.01 level (2-tailed).