NETWORK ANALYSIS OF KNOWLEDGE CONSTRUCTION IN ASYNCHRONOUS LEARNING NETWORKS

Download PDF version:

Zippy Erlich, Ph.D.
Department of Mathematics and Computer Science, Open University of Israel
Email: zippy@openu.ac.il

Gilad Ravid, M.A.
Center for Information Technology in Distance Education, Open University of Israel
Email: gilad@openu.ac.il

Aviva Geva, Ph.D.
Department of Economics and Management, Open University of Israel
Email: avivage@openu.ac.il

ABSTRACT
Asynchronous Learning Networks (ALNs) make the process of collaboration more transparent, because a transcript of conference messages can be used to assess individual roles and contributions and the collaborative process itself. This study considers three aspects of ALNs: the design; the quality of the resulting knowledge construction process; and cohesion, role and power network structures. The design is evaluated according to the Social Interdependence Theory of Cooperative Learning. The quality of the knowledge construction process is evaluated through Content Analysis; and the network structures are analyzed using Social Network Analysis of the response relations among participants during online discussions. In this research we analyze data from two three-month-long ALN academic university courses: a formal, structured, closed forum and an informal, non-structured, open forum. We found that in the structured ALN, the knowledge construction process reached a very high phase of critical thinking and developed cohesive cliques. The students took on bridging and triggering roles, while the tutor had relatively little power. In the non-structured ALN, the knowledge construction process reached a low phase of cognitive activity; few cliques were constructed; most of the students took on the passive role of teacher-followers; and the tutor was at the center of activity. These differences are statistically significant. We conclude that a well-designed ALN develops significant, distinct cohesion, and role and power structures lead the knowledge construction process to high phases of critical thinking.

KEYWORDS
Asynchronous Learning Networks, Learning Effectiveness, Social Network Analysis, Cohesion Analysis, Role Analysis, Power Analysis, Content Analysis

I. INTRODUCTION

Asynchronous Learning Networks (ALNs) offer new possibilities for study that were not available in traditional learning models. The pedagogical advantages of online collaborative learning are well known [1, 2]. Certain implementations of collaboration are more successful than others. According to Mason and Bacsich [3], the degree of integration of online collaborative learning within a course radically influences its acceptance by students. An ALN makes the collaboration process more transparent, because a transcript of conference messages can be used to assess both the collaborative process itself, and individual roles and contributions to the process [4].

ALN studies call for a greater research focus on the ways peer interaction determine learning outcomes [5, 6]. The Social Interdependence Theory of Cooperative Learning [7] suggests that interaction processes are determined by how social relations among members of the group are designed into the learning environment.

Johnson and Johnson [7] specified the required design characteristics for effective cooperation. Briefly, the requirements are that participants form promotive (i.e., face-to-face, close together interaction, not across the room), formal, cooperative groups, with positive interdependence between participants (including deliverable, task, resource, role and reward interdependence), inclusion of group reflection processes, and enforcement of individual accountability and interaction. In addition, certain social skills (trusting others, accurate communication, acceptance and support of others, conflict resolution) are required. Numerous studies have demonstrated that these design characteristics are necessary conditions for high achievement [7, 8]. Aviv [9] noted that in asynchronous (virtual) cooperation, the design should be extended to include bridging and triggering roles—bridging to silent students and triggering after silent periods. Geva [10] demonstrated that combining text-based learning with an asynchronous seminar, designed as a formal cooperative group with reward mechanisms and on a tight schedule, enriched both the individual and the group learning processes.

It was found [11] that it is not easy to move students through the phases of critical thinking, yet online learning would seem to favor cognitive tasks. One can quantify the quality of the knowledge construction process in online cooperative learning through Content Analysis of the transcript of communications among the participants. Several models have been proposed. Henri [6] suggested five parameters, two of which measure critical thinking (cognitive and meta-cognitive) phases. Using that model, Aviv [9] demonstrated that implementing an extended version of Johnson and Johnson’s design in an asynchronous online discussion led to the highest phases of cognitive communication, compatible with the goal of cooperation. The difficulties which arose in using Henri’s model led Gunawardena, Lowe and Anderson [12] to suggest a new Interaction Analysis Model, which we used in this research.

Once the design is implemented and the learning process starts, social relations develop among participants. These social relations control the learning outcomes; in particular, the knowledge construction process. Studying these relations is the focus of this research, using Social Network Analysis (SNA). SNA methods (explained in section II) quantify social relations in terms of network structure parameters which encode certain causal group forces. Burt [13] identified the causal force encoded in cohesion structures that control shared beliefs and behaviors. As mentioned above, certain roles need to be taken on to ensure that group function is uninterrupted and that no one is left behind. Roles affect the distribution of power among ALN participants. SNA can identify cohesion, role and power structures of the ALN.

We assume that cohesion, role and power distribution control the construction of knowledge. From this assumption, it follows that different design characteristics of online discussion groups result in significant differences in network structures leading to different phases of critical thinking. We test this assertion by analyzing the recorded data of two online cooperative learning discussion groups with a marked distinction in their designs. The proof of this assertion is the major contribution of this paper.

The paper is structured as follows: Section II provides background information regarding the two analytic schemes, Content Analysis and Social Network Analysis. Section III details research goals and questions. Section IV describes the two ALNs that served as a test-bed, the data sources and their pre-processing. Section V provides Content Analysis of the two ALNs. Sections VI, VII and VIII are devoted to cohesion analysis, role analysis, and power analysis of the two ALNs, respectively. In section IX we attempt to model the ALNs using simple symmetrical graphs, discussing the results and limitations of the analysis in section X. Section XI presents an outline of future work. The appendix gives a supplemental explanation with the technical details of the role analysis.

II. THEORETICAL BACKGROUND

A. Content Analysis

As early as 1990, Hiltz [14] and Mason [15] suggested that analyzing transcripts of asynchronous (text-based) communication could enable quality assessment of the learning process and its outcomes. Henri provided a detailed model for the analysis [6]. Henri’s model relies on breaking down the transcript into “units of meaning” (a message or a part of it), and classifying these units into categories and sub-categories according to expressions within the units. In particular, two of the categories express critical-thinking phases of knowledge construction: cognitive and meta-cognitive actions. This model has been used successfully in several transcript analyses to identify high phases of critical thinking [16]. Gunawardena and colleagues [12] noted that Henri’s model integrates the “participation” category within categories of critical thinking. They also emphasized difficulties in identifying “units of meaning.” Moreover, Henri’s model emphasizes critical thinking phases of individual students, but not of the group process. To overcome these difficulties and simplify practical analysis, Gunawardena and colleagues suggested a five-phase Interaction Analysis Model, geared towards answering two questions: “What degree of knowledge construction is achieved by the cooperative group?” and “What degree of evidence is there that the knowledge of individual participants changes?” In general, the first question is answered by the dominant cognitive phase observed in the transcript, while the second question is answered by individual expressions which directly relate to such changes (meta-cognition) or expressions which demonstrate the application of changed knowledge.

Specifically, the Interaction Analysis model specifies that Knowledge Construction Process proceeds through five phases; accordingly, messages (or parts thereof) should be classified into the associated phases of critical thinking. The model defines five phases:

1. Sharing/Comparing Knowledge
2. Discover/Explore disagreements
3. Synthesis via negotiating meaning
4. Testing/modifying proposed synthesis vs. schemas, theory, facts, beliefs
5. Proofs of reaching agreements or meta-cognitive admitting change of knowledge.

Thus, knowledge construction processes of different cooperative scenarios will differ in the phases they reach, as expressed in the associated transcripts.

Gunawardena and colleagues used their model to analyze a global online debate (which actually served as the basis for developing their model) and another social-like online ALN. The former reached phase 3, whereas the latter only reached phase 1. De Laat [17] used the same Interaction Analysis model to analyze another online ALN which did not go beyond phase 1. Both authors noted that, in retrospect, these results matched the designs of the ALNs. In general, in assessing the success or failure of an ALN, one should consider the aims of the ALN and its design. One should not expect the Knowledge Construction Process to reach phase 4 (testing) if it was not designed to do so.

We use the Interaction Analysis model to analyze the two ALNs described in section IV.

B. Social Network Analysis

Social Network Analysis (SNA) is a useful tool for studying relations. It is a collection of graph analysis methods that researchers developed to analyse networks in social sciences, communication studies, economics, political science, computer networks, and others. SNA methods provide precise mathematical definitions of five groups of characteristics of the actors and of the network itself [18, 19]: cohesion, equivalence (role-groups), power of actors, range of influence, and brokerage. These characteristics are expressed in terms of corresponding Network-Structure parameters derived from the relations among the actors. An introduction to SNA can be found in Scott [20] and Hanneman [21]. For a comprehensive text, see Wasserman and Faust [22]. Burt [13] elaborates on the insights that can be obtained from the various values of the network structures.

A “social network” is defined as a group of collaborating (and/or competing) entities that are related to each other. Mathematically, this is a graph (or a multi-graph); each participant in the collaboration is called an actor and depicted as a node in the graph. Valued relations between actors are depicted as links between the corresponding nodes. Actors can be persons, organizations, or groups—any set of related entities.

SNA is used in a wide range of applications, from analyzing relations within families [23] to analysis of a Military C4ISR (Command, Control, Communications, Computers and Intelligence, Surveillance, and Reconnaissance) network [24]; from analyzing the positions of adolescents in social networks and their sexual experience [25] to analyzing political power networks [26]; from analyzing management structures in multi-national corporations [27] to analyzing terrorist networks [28]. The website of the association of researchers, INSNA [29], provides links to journals, mailing lists and other resources.

Recently Garton, Haythornwaite and Wellman [30] suggested using SNA methods for analyzing online networks, in particular learner networks. Several authors have demonstrated the applicability of SNA to specific learning situations. In these studies, the collaborating persons (students, tutors, experts, and so) are the actors. Links between a pair of actors represent the amount of communication between them. Most researchers concentrated on analyzing the distribution of power (or centrality) in the resulting network. Martinez and colleagues [31] compared the centrality of inter-group and intra-group communication. Power distribution characteristics of multi-relation networks have been analyzed by Haythornthwaite [32]; she analyzed time variation as well as media communication-channel dependence of the network power distribution. Cho, Stefanone and Gay [33] found that powerful actors in web-based communication are also more influential than others as referrals. Reffay and Chanier [34] concentrated on the evolution of cohesion in learning groups. De Laat [17] combined SNA with Content Analysis and demonstrated that the interaction patterns in the analyzed course were centralized and that the Knowledge Construction Process focused on sharing and comparing information (that is, concentrated on phase one). This line of research is growing fast: a search for “Social Network Analysis” in a recent Computer Supported Collaborative Learning (CSCL) conference [35] revealed 91 papers.

III. THE RESEARCH HYPOTHESIS

This work stems from the constructivist paradigm [36], in which knowledge is constructed cooperatively through social negotiation. In online discussion groups, relations are created via messages. Simply exchanging messages is not enough, however. As noted by Rafaeli [37], the emphasis should be on the responsive nature of the communication. We therefore focus on analyzing structures of responsiveness relations between participants in the ALN.

Cohesion is a primary network structure that contributes to the creation of knowledge: shared beliefs and behaviors [13]. Cohesion is manifested by the existence of cliques of participants who are connected internally more than externally. Members of a clique tend to create knowledge by virtue of their strong intra-responsiveness relations. They drive the process of constructing knowledge.

Cohesion in itself does not guarantee that no member will be isolated, nor can it ensure uninterrupted communication. Certain tasks need to be performed for cohesion to occur. For example, members should communicate with otherwise isolated members. Others should make sure that the feed of responses does not stop. These roles may be pre-assigned, but in the absence of such assignments, various group members should take on these roles implicitly. Undertaking these roles during collaboration affects the distribution of power among the participants of the ALN. SNA provides procedures for identifying role and power structures within a given network.

We will examine the recorded data of two ALNs. Each of the ALNs is modeled as a Knowledge Constructing network: the actors are the students and the tutor, the participants in the ALNs; actors are related to each other by response messages. We hypothesize that network-cohesion forces, roles and power structures are major factors in determining the quality of the knowledge construction process. In other words we assert that:

A marked difference in the design of ALNs is associated with marked distinctions in the cohesion, role and power structures of the ALNs, which are associated with a marked distinction in the critical thinking phases of the knowledge construction processes.

IV. THE TEST-BED

The analysis in this research is based on recorded data from two ALNs that were part of the Open University of Israel course, Business Ethics. The first ALN (18 participants) ran during the fall 2000 semester. The other ALN (19 participants) ran during the spring 2002 semester. The designs of the ALNs were different. Neither of the ALNs fulfills all of the specifications of Social Interdependence Theory of Cooperative Learning, but the fall 2000 ALN was more structured than the spring 2002 ALN. We will refer to these ALNs as the structured ALN and the non-structured ALN, respectively.

The structured ALN was a three-month long, formal online seminar; in signing up for it, students committed themselves to active participation and other requirements. A reward mechanism for fulfilling the requirements (including active participation) was employed. 18 students opted to participate in this ALN. Geva [10] has described the course and the online seminar. Relevant details relating to the structured ALN follow.

Students were asked to simulate the role of an advisory committee to a high-tech company (“Cellularphone”). Their problem was stated as follows [10]:

“The subject of the seminar this semester is the safety of cellular phone emissions. Existing data and studies do not rule out the possibility that radiation from cellular telephones causes ill health effects. The cumulative balance against cellular phones today has not yet changed the industry policy.
You are required to determine what a morally responsible company (e.g. Cellularphone) should do, based on the assumption that it wants to fulfill its goal as a profit-oriented organization in a set of circumstances which is not entirely clear.”

The structured ALN was designed as a sequence of 5 steps, according to a well-known phase model of moral decision-making [38]. In the first step (four weeks), the students were to identify the various facets of the problem, debate solutions and propose a synthesis. In the next three steps (2-3 weeks each), the synthesized solution was to be tested against several sets of principles. In the last step (one week—if the resolution passed the last test), a summary of the proposal was to be reported. The sequencing was broadly controlled by the tutor, who began and ended each step according to a prescribed schedule.

The non-structured ALN was a three-month long online conference, open to all 300 students in the course, with no need to register or commit themselves in advance. No specific cooperative goal was defined for this ALN. Students and the tutor could raise a variety of issues related to the course topics (which were the same as in the fall 2000 course). No structure was designed and no schedule was imposed (though the deadlines for submitting assignments were reflected in the ALN), and no reward mechanism was implemented. 19 students opted to use this ALN. Table 1 summarizes the design of the two ALNs.

Table 1: Design of the two ALNs

A. Data Sources

The ALNs of Business Ethics are accessible by username and password from the TELEM website at http://telem.openu.ac.il/.

The ALNs are in Hebrew, maintained by the Open University of Israel Opus Learning Management System. Each ALN is a collection of threads. Each thread begins with an initial message sent by a member of the ALN. Other messages in the thread are responses to a predecessor message.

Figure 1. Thread Response Tree

The messages contained in a thread can be represented by a Thread Response Tree (Figure 1). In this figure, ALN members are represented as labeled nodes. A message is represented as an arc from the node of the sender of the message to the node of the sender of the predecessor message. The initial message is considered a response to an instruction from an “external entity”. Thus, the thread presented in Figure 1 includes 9 messages: Message 1 was sent by Z in response to the “external entity”. Messages 2 (from N), 3 (from A) and 6 (from R) in this thread are responses to Message 1. Message 4 (from Z) is a response to Message 3. Message 5 (from A) is a response to Message 4. Messages 7 (from L) and 9 (from A) are responses to Message 6, and Message 8 (from A) is a response to Message 7.

B. Preparation for Social Network Analysis

The Social Network Analysis was performed using Cyram NetMiner —a software tool for exploratory network data analysis and visualization [39]. We developed a Visual Basic conversion program, Opus2Ntf.exe, that scans the SQL database of the messages of a given ALN, thread after thread, constructing the response trees. From the response trees, it constructs the response matrix of the ALN. Rows and columns of a response matrix are labeled with numbers representing the members (including the external entity, which was labeled “-1”). The (i, j) entry—that is, row i, column j—is the number of messages sent by member i as responses to predecessor messages sent by j during the life of the ALN. The content of the response matrices was then written by Opus2Ntf.exe into data files in ntf format which is acceptable for analysis by NetMiner.

Actors are conceived as sending response messages among themselves. These exchanges define the responsiveness relation between the actors. The two values of this relation (one in each direction), for any pair of actors i and j, are the (i, j) and (j, i) elements of the response matrix of the online ALN. Mathematically, we have a valued directed network model. A more general network model consists of a set of actors and one or more directed (or undirected) relations called layers. Our models of the structured and non-structured ALNs contain a single layer: the responsiveness layer.

Note that the model does not capture all the details of the real ALN. For example, in the ALN, a response from member A to a previous message sent by B is actually broadcast to all members. Yet, in the model, this is represented by a message sent directly from the actor representing A to the actor representing B. The model concentrates on the trigger/response mechanisms.

Also note that the “-1” actor who represents the external entity (to whom all initial messages respond) cannot be analyzed in the same way as other actors in the network: It has a fixed location in the thread response trees (always at the root); it always has one child (there is only one initial message in every thread); and it never responds. The “-1” actor, then, has no outgoing arcs, and the number of its ingoing arcs is equal to the number of threads. These properties make the “-1” actor rather special. Including it in the analysis would almost certainly result in revealing its special roles. In general, this is desirable. But our first set of analyses aims to identify the relative positions of actors representing real ALN members in the network structure. This could be biased by including the “-1” actor. For this reason we filtered out “-1” and all its (received) messages. In the filtered response matrix, initial messages (messages that are not responses) are not counted. This does not effect the relations among all other actors.

V. CONTENT ANALYSIS

The transcript analysis procedure consisted of reading each message and classifying it as one or more of the five phases of the Interaction Analysis model [20]. Thus messages were not divided into “units of meaning” as in Henri’s model; however, occasionally a message was classified as belonging to more than one phase. The messages were coded by three researchers. Discrepancies were discussed and a single coding was agreed upon. There were 248 messages in the structured ALN, and 70 messages in the non-structured ALN. Table 2 summarizes the results.

Table 2 reflects the knowledge construction processes of the two ALNs. Recall that the structured ALN was an online seminar designed as a series of time-bounded steps, specifically directing the students to higher phases of critical thinking. The first step consisted of the elaboration of the facts and possible actions that were and were not relevant, gradually approaching a proposition for a solution. In this step, 100 messages were, roughly, equally divided between phase 1, 2, and 3 messages. The next three steps were rephrasing and/or testing the proposition against a set of predefined principles of the phase model of moral decision-making [19]. This is reflected in a wealth of phase 4 messages that were sent at this stage. Finally, in the last step, some students explicitly stated that they had learned. No application of new knowledge was reported as the entire seminar was a role-playing simulation. One message summarized the final proposition (which, by the way, was not agreed upon by all participants). We can conclude that the Knowledge Construction Process of the structured ALN functioned at all phases up to and including phase 4.

Table 2: Classification of messages in the two ALNs

These results are compatible with students’ evaluation of the online seminar: The fall 2000 ALN of Business Ethics received the highest scores on students’ assessment of learning in comparison with all 68 courses that were evaluated that semester. Students rated “sense of community” as 3.67 (out of 5), “knowledge construction” as 4.6, and understanding course materials as 4.33.

Students in the non-structured ALN were not part of a formal team, so they used the ALN according to their own needs. Inspection of the messages reveals that many of the transactions were simple Q&A, triggered by students’ assignments and other course components. None of these questions developed into critical thinking. Here the knowledge construction process functioned at the lowest phase, 1. One should note that the non-structured ALN was not a failure. The design of the spring 2002 Business Ethics course specified knowledge construction through other learning activities (self-study). The associated ALN was designed as a simple Q &A to support learning.

VI. COHESION ANALYSIS

Networks can be thought of as developing from smaller sub-networks. The smallest sub-network is a dyad (two actors exchanging messages). A larger network is a triad, and so on. Cohesion analysis of a network consists of identifying the sub-structure architecture of the network.

The idea is that “similar actors are tied together by socializing bonds of interaction through which they come to share beliefs and behavioral tendencies. Causal force lies in the strength of the communication ties” [13]. There are several methods for performing cohesion analysis. Here we will use the simplest one, the clique method [40]. A clique is a maximal connected sub-network. Table 3 shows the result of performing clique analysis on the two ALNs. The fictitious names in the table refer to the participants.

Note that a clique is not isolated: there might be external actors linked to some members of the clique, but not to all members of the clique. The Cohesion Index is a measure of the degree to which there are strong links within the clique rather than outside of it (where the strength of a link reflects the number of responses exchanged along the link). If the cohesion is greater than 1, then the links within the clique are stronger on average than the links with the outside. A precise definition is given by Bock and Husain [40]. Table 3 shows the result of preparing clique analysis on the two ALNs. We use fictitious names: N1-N19 for the non-structured ALN, and P1-P18 for the structured ALN.

Table 3: Clique Analysis Reports

For reference, the tutors were P1 in the structured and N18 in the non-structured ALNs. The cliques in the two ALNs have similar cohesion indices. But there is a marked distinction between the structures of the cliques. First, the number of cliques: 2 in the non-structured ALN, 16 in the structured ALN. Second, in the structured ALN, the students formed relatively large cliques—4 or 5 students in each clique (except K16); that is, they maintained response relations with several others. Moreover, the tutor (P1) participated in one clique only (K16). On the other hand, the cliques in the non-structured ALN included only 2 students and the tutor (N18).

Even more striking is the difference between the inter-clique connectivity, maintained by actors who belong to more than one clique. Table 3 shows that many students in the structured ALN belong to more than one clique. This bridging phenomenon provides for a wealth of information flowing to all members.

The inter-clique connectivity of the two ALNs is visualized in Figures 2 and 3. These bipartite graphs include clique nodes (Ki) and actor nodes (Pj). Membership is represented by links. Note the relative (non)membership of the tutor (P1) in the structured ALN (Figure 2).

Figure 2. Clique bipartite graph: Structured ALN

Figure 3. Clique bipartite graph: Non-structured ALN

To assess the difference between the cohesion structures, we constructed the co-membership matrices of the cliques in both ALNs: An entry in a clique co-membership matrix, CCM(i, j), is the number of cliques in which both actors i and j are members. We then tested the null hypothesis that there is no difference between the mean clique co-membership in the ALNs, using a t-test According to the results, the null hypothesis is rejected; the difference is significant at p < 0.001. Table 4 depicts the mean and standard error of the two structures.

Table 4: Mean clique co-membership in the two ALNs

VII. ROLE ANALYSIS

Role Analysis of networks aims at identifying, on the basis of relations between actors, classes of actors that implement certain social roles in the network. Any two individuals in any one class are equivalent in the sense that as far as the social roles that these individual implement, they can replace one another. The classes are also called role groups. Mathematicians refer to them as equivalence classes. Note that social roles (or social positions) may be revealed through the procedure of identifying the role groups. These are not pre-determined. Hence, the role description (and its name) can be understood only after the analysis has been performed.

The analysis consists of embedding the actors in a certain “role space” and subsequent cluster analysis. Actors in a resulting cluster have a certain role (yet to be identified) and they can replace one another. Actors in different clusters have different roles. Details of the analysis are provided in the appendix. A visual presentation of the role groups is provided through cluster maps (Figs. 4 and 5).

Figure 4. Role groups: Structured ALN

Figure 5. Role groups: Non-structured ALN

Consider the four role groups of the structured ALN. The roles of two of these groups are easily recognized: Role group C2 consists of a single actor— P2— this is the actor who connects to 15 cliques (see Fig. 2). Similarly, role group C4 consists of all the other strong bridges—those that connect to at least four cliques. Note that C2 and C4 are relatively close on the MDS plane. Next, consider the two role groups of the non-structured ALN (Fig. 5). Group C2 (left) consists only of the tutor (N18). C1 (right) consists of all the students. Further analysis is needed in order to identify the actual roles of C1 and C3 groups of the structured ALN and the roles of C1 and C2 in the non-structured ALN. This will be accomplished through power analysis, described below.

To assess the difference between the role structures we constructed the role equivalence matrices of the ALNs: An entry in a role equivalence matrix, REM(i, j), is the distance in role space between the two actors, i and j. We then tested the null hypothesis that there is no difference between the mean role distances of the structured and non-structured ALNs, using a t-test. According to the results, the null hypothesis is rejected; the difference is significant at p < 0.001. Table 5 shows the mean and standard error of the two structures.

Table 5: Means of the role distances of the two ALNs

VIII. POWER ANALYSIS

Power (or Centrality) analysis places the actors hierarchically in some “power space”. Leaders are at the center; the led are at the periphery. The meaning of “leadership”, depends on the causal force embedded in the relation being studied. In our analysis, the causal forces are trigger and response: if an actor gets responses from many others, who in turn get many responses, then that actor is very successful at triggering others. That actor is a powerful trigger. Similarly, if an actor sends more responses than others, then this actor is more easily triggered. He or she is a better responder. The power distributions among the actors are usually presented in topographic-like “centrality maps,” with powerful actors at the centers. Figures 6 and 7 present centrality maps of the structured ALN, and Figure 8 presents a centrality map for the non-structured ALN. It is clear that the distributions of power in the two ALNs are quite different: In the unstructured ALN, power is centered in one participant (N18); in the structured ALN, power is embedded in several actors.

There are several ways to measure power or centrality. In the Bonacich Algorithm [41], the eigenvector centrality of an actor is (recursively) proportional to the sum of the eigenvector centralities of the actors it is connected to. It is computed by the principal eigenvector of the response matrix. We use this algorithm to calculate the distribution of the combined trigger/response power of the structured ALN (Figure 6).

Figure 6. Eigenvector Centrality map: Structured ALN

Power is not evenly distributed—some of the actors are on the fringe. These are lurkers. Close inspection reveal that the actors who are on the fringe are in fact the members of role group C1 (shown in Figure 4). In addition, the single most powerful member of the structured ALN is P2. This actor, in addition to being a major bridge, is a prominent trigger and responder.

Another way of measuring centrality is by using Freeman’s Degree Centralities [42]. Here an actor’s centrality is measured simply by determining the proportion of actors that send or receive responses to or from that actor. Figure 7 presents the degree centrality map of the structured ALN considering only incoming responses. That is, members in the center were those who attracted more responses than others. Triggering power is embedded in a large number of actors. In fact, the central triggers include P2—who has a unique role, and the members of role group C3 (shown in Figure 4).

Figure 7. In degree Centrality map: Structured ALN

Figure 8 presents the centrality map of outgoing responses of the non-structured ALN. It is clear that the ALN included one responder—the tutor (N18); Students (Role group C2 in Figure 5) did not send responses. They just read messages or asked questions. Clearly, the non-structured ALN consisted mainly of Q &A, triggered by the students.

Figure 8. Out Degree Centrality Map: Non-structured ALN

The degree centralities are directly derived from the response matrices of the ALNs. To assess the difference between the power structures, we tested the null hypothesis that there is no difference between the mean densities of the ALNs, using a t-test. According to the results the null hypothesis is rejected; the difference is significant at p< 0.001. Table 6 depicts the results.

Table 6: Mean densities of the two ALNs

IX. MODELING THE ALNS USING SYMMETRICAL GRAPHS

Network observations are not deterministic. The measured response relation does not accurately reflect the “real” state of the network; the possibility that “it could have been different”—due to measurement errors or other “noise”—cannot be ignored.

As a simple example, one might surmise that the two ALNs can be modeled using the two extreme, symmetrical digraphs: the structured ALN in a fully connected digraph, and the non-structured ALN in a star-shaped digraph, with the tutor in the center; observed response relations incorporate some random “noise”. These digraphs relate to teaching/learning styles [43]; the fully connected digraph model reflects a collective of equally contributing cooperators, and the star-shape digraph model reflects a strict information-transmission style of learning. These conjectures follow the null hypotheses: there are no differences between the densities (the average number of links between actors) in the observed ALNs and in the corresponding symmetrical digraphs. The appropriate t-tests for the densities of the two pairs of networks were conducted and the results are presented in Table 7.

Table 7: Mean density of the observed ALNs and the theoretical ALNs

We see that the null hypotheses are rejected; the differences in the densities are significant. The two ALNs discussed in this paper cannot be modeled using simple symmetrical digraphs. Actual teaching/learning styles are not that simple. This conclusion is stronger for the structured ALN (mean difference=0.6405, p < 0.001), than for the unstructured ALN (mean difference=0.0527, p < 0.05), which probably reflects the fact that the participants in the structured ALN are certainly not equivalent—some of them tend to send and/or receive responses more than others. A symmetrical graph cannot model non-equivalence. Similarly, students in the unstructured ALN experienced some collaboration, which is not captured in the star network. These negative results exhibit the limitations of simple models and call for more complex ones. We will return to the limitations and the modeling approach in Section X.

X. DISCUSSION

We know that the structured ALN was designed to achieve high-phase knowledge construction; it established critical thinking goals for each step in the construction; students committed themselves to the project, were instructed about the rules, and were appropriately rewarded. The construction was closely monitored by the tutor and the students had a feeling of “togetherness.” It is not surprising, therefore, that the interaction analysis of this ALN revealed high phases of critical thinking. From this analysis, however, we cannot tell which of the design characteristics is the primary factor in the dynamics of the ALN. Is it the goals? The strict “rules of the game”? The reward? These questions can be dealt with through a comparative evaluation of a set of ALNs with various designs.

What we have learned in this research is that the structured design was associated with a high degree of cohesion, encoded by a dense inter-linked set of cliques. Maintaining such a dense network of cliques requires effort on the part of the participants, and yet the students felt that the effort was worthwhile. Note that cohesion could have both a beneficial or debilitating influence on discourse and reflection. Too cohesive a group could stifle criticism and, therefore, open discourse. What is the optimal degree of cohesion? How should the cohesion be “tuned”?

The structured design also led to embedding of triggering and responsiveness into a certain set of students. These students took on bridging and triggering roles, without which the operation might have led to split groups or gaps in the construction schedule. This role-taking significantly shifted the triggering and response power distribution from the tutor to the students.

There are several limitations to our rather qualitative, descriptive analysis. Consider, for example, the non-structured ALN: Is the difference between the centrality of the tutor and the students significant? This raises the question of the quantitative accuracy of the conclusions. Second, descriptive conclusions cannot easily be generalized to other situations which are, in some sense, similar. We cannot make inferences. Third, the qualitative analysis does not easily provide clues to the underlying, hidden factors that explain what is happening. What attributes of the participants in an ALN (if any) are dominant in creating a particular pattern of connectivity? The qualitative analysis limits the deeper understanding of the processes underlying the development of networks.

One remedy might be to set up parametric stochastic models for the forums. The fixed parameters define the model; they govern a set of stochastic processes that are responsible for creating the observed network. These processes could have created other networks, due to the random portion of the model. A model provides a probability distribution for a large (usually infinite) set of networks. The distribution is the basis for inferences, generalizations and hidden factors identifications. We will explore such models in future research.

XI. FURTHER RESEARCH

It seems that Social Network Analysis can be a useful research tool for revealing network structures of cooperative learning groups. Some directions for future research that come to mind are the following:

Position Analysis: Several studies, e.g. Lipponen and colleagues [44], revealed that certain participants take on the roles of influencers (who trigger responses) or of celebrities (who attract responses). Others are isolated—no-one responds to them or is triggered by them. The question is whether this behavior depends on the individuals’ attributes or whether this is more universal and can be found across ALNs.

Network Dynamics: One fascinating direction is an inquiry into the time development of network structures. When do cliques develop? Are they stable? What are the network structures that determine this behavior?

Large Group Information Overload: It is well known that the dynamics of large groups lead to boundary effects that occur when either the group size and/or the thread size increases [45]. How are these manifested in learning groups?

Effective Construction of Network: Burt [19, 46] noted that increasing the density of a network may lead to redundant connectivity, and thus to less effective connectivity. Non-redundancy appears as holes around actors. It might be worthwhile to examine whether a well-designed cooperative network develops into a more or a less effective network.

Stochastic modeling of ALNs: Multivariate statistics for modeling the behavior of social networks by stochastic processes are detailed by Wasserman and Faust [22, part V]. Using this analysis, one can test hypotheses about, and provide quantitative estimates for, actor effects such as tendency to send and tendency to receive, and to network effects such as tendency for reciprocity. A more advanced program is to incorporate actor attributes (such as status, seniority or average grade) as covariates for the actor and network effects. Software tools for performing this analysis is available (e.g., StOCNET [47]).

Stability of Results: One might consider embedding SNA tools into an ALN support environment, enabling the tutor to monitor group dynamics closely. But a word of caution is necessary: There are various definitions of network structures. Experience shows that applying different definitions may lead to different, even contradictory, results. Further research is needed to determine the stability of network structures under such redefinitions.

XII. APPENDIX: ROLE ANALYSIS TECHNICAL SUPPLEMENT

Role analysis proceeds in two stages. First, actors are embedded in a role space. A point in this space is a possible social role. Distance in this space measures the dis-equivalence between actors. Next, actors are partitioned into equivalence classes, or clusters, using cluster analysis. Actors in one cluster have a certain role (yet to be identified) and can replace one another. Actors in different clusters have different roles.

There are several methods for embedding actors in a role space. We shall use the Hummel Sodeur method [48], simply called the “role equivalence method”. Burt [13] provides a detailed explanation of this method. The basis for constructing the role space is triads. A triad is a 3-node graph in which one node is denoted as ego and the two others are alters. When listing all possible connectivity patterns of triads (considering if directed links exist or not), one can identify 36 triad types. One type is, for example, when ego is connected to the two alters with bi-directional links, but in which the alters are not connected. Another triad type is when all three are connected in a clockwise direction.

Each actor is assigned a 36-long role-vector; a coordinate in the role-vector is the frequency of triads of a given type in which that actor plays the role of ego; the other two actors are any of the other actors in the network. A role-vector of an actor is thus the frequency distribution of triad types for that actor. By assigning role-vectors in this fashion, all actors are embedded in a 36-dimensional role space.

Cluster analysis is then performed on the actors in the role space, where dis-equivalence is measured by the Euclidean distance between role vectors. For an introduction to cluster analysis, see [49]. The cluster analysis algorithm constructs clusters in a hierarchical fashion: smaller clusters are combined into larger clusters, at the cost of increasing the intra-cluster distance. In our analysis each of the identified clusters is a role group.

Figures 9a and 9b show the clustering algorithms. The algorithm proceeds from the top of the figures to the bottom. The first actors identified as role-equivalent are (P14, P15) in the structured ALN (Figure 9a), and (N1, N2, N4, N6, N8, N10, N12, N14, N15, N16) and (N5, N13) in the non-structured ALN (Figure 9b). Members in these clusters are strictly equivalent; their intra-distance to other members in the same cluster is 0 (intra-distance is labeled as “Level” in the figures). At this stage, there are 17 clusters in the structured ALN: one size-2 (P14 and P15) and 16 size-1 (all the rest), and 9 clusters in the non-structured ALN (the two listed above and 7 size-1 clusters). During clustering, other actors are added to already established (and new) clusters, increasing the intra-distance and the number of size-1 clusters decreases, until, when the final clusters join into one big cluster, the intra-distance grows to 0.404 and 1.050, respectively.

When do we stop the clustering algorithms? This point comes when the “slope”—the change in the intra-distance required for decreasing the number of clusters by 1—increases abruptly. From the figures, we can see that these points are at intra-distances 0.235 in the structured ALN, and 0.180 in the non-structured ALN. At these points, the structured ALN reveals 4 clusters, i.e., four roles, one of which is a single actor (actor P2) cluster. The non-structured ALN reveals two roles, one of which is a single-actor (actor N18).

The 36-dimensional role groups of the structured ALN are reduced into 2-dimensional cluster maps (Figures 4 and 5 in section VII) using Classical Multidimensional Scaling (C-MDS). See, for example, Van Deun and Delbeke [50].

Figure 9a. Clustering algorithm: Structured ALN

Figure 9b. Clustering algorithm: Non-structured ALN

XIII. REFERENCES

1. Kaye, A. R., (Ed) Collaborative Learning Through Computer Conferencing. The Najaden Papers. NATO ASI Series F, 1992.
2. McConnell, D., Implementing Computer Supported Co-Operative Learning. London: Kogan Page, 1994.
3. Mason, R., and Bacsich, P., Embedding Computer Conferencing into University Teaching. Computers & Education 30(3/4): 249-258, 1998.
4. Macdonald, J., Assessing Online Collaborative Learning: Process and Product. Computers & Education 40: 377-391, 2003.
5. McCreary E., Eliciting more Rigorous Cognitive Outcomes Through Analysis of Computer-Mediated Discussion, 15th International Conference on Improving Teaching, Vancouver, 1989.
6. Henri, F., Computer Conferencing and Content Analysis. In: Kaye, A. (Ed.), Collaborative Learning through Computer Conferencing: The Najaden Papers, Berlin: Springer-Verlag, 117-136, 1992.
7. Johnson, D. W., and Johnson, R. T., Learning Together and Alone. Cooperative, Competitive and Individualistic Learning. Needham Heights, MA: Allyn and Bacon, 1999.
8. Skon, L., Johnson, D. W., and Johnson, R., Cooperative Peer Interaction versus Individual Competition and Individual Efforts: Effects on the Acquisition of Cognitive Reasoning Strategies. Journal of Educational Psychology 73(1): 83-92 (1981).
9. Aviv, R., Educational Performance of ALN via Content Analysis. Journal of Asynchronous Learning Networks 4(2): 53-72, ISSN 1092-8235 (2000).
10. Geva, A., The Internet and the Book: Media and Messages in Teaching Business Ethics. Teaching Business Ethics 4: 85-106 (2000).
11. Garrison, D. R., Anderson, T., & Archer, W., Critical thinking, cognitive presence and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23 (2001).
12. Gunawardena, C. N., Lowe, C. A., and Anderson T. A., Analysis of a Global Online Debate and the Development of an Interaction Analysis Model for Examining Social Construction of Knowledge in Computer Conferencing. J. Educational Computing Research 17(4): 397-431 (1997).
13. Burt, R. S., Structure, A General Purpose Network Analysis Program. Reference Manual, New York: Columbia University, 1991.
14. Hiltz, S., Evaluating the Virtual Classroom. In: Harasim, L. (Ed.), Online Education, New York: Praeger, 134-184, 1990.
15. Mason, R., Methodologies for Evaluating Applications of Computer Conferencing. In: Kay, A. R. (Ed.), Collaborative Learning Through Computer Conferencing, Berlin: Springer Verlag, 1991.
16. Newman, D. R., Webb, B., and Cochrane, C. A., Content Analysis Method to Measure Critical Thinking in Face to Face and Computer Supported Group Learning. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century 3(2): 56-77, 1995. http://www.qub.ac.uk/mgt/papers/methods/contpap.html.
17. de Laat, M., Network and Content Analysis in an Online Community Discourse. In: G. Stahl (Ed.), Proceedings of Computer Support for Collaborative Learning (CSCL) 2002 Conference, Jan. 7-11, Boulder, CO. Mahwah, NJ: Lawrence Erlbaum, 625-626, http://newmedia.colorado.edu/cscl/62.pdf
18. Bonacich, P., Power and Centrality. American Journal of Sociology 92: 1170-1182 (1987).
19. Burt, R. S., Structural Holes, Cambridge, MA: Harvard University Press, 1992.
20. Scott, J., Social Network Analysis: A Handbook, 2nd ed., London: Sage, 2001.
21. Hanneman, R. E., Introduction to Social Network Methods. Online Textbook Supporting Sociology 157. Riverside, CA: University of California, 2000.
22. Wasserman, S., and Faust, K., Social Network Analysis: Methods and Applications, Cambridge, UK: Cambridge University Press, 1999.
23. Widmer, E. D., and La Farga, L., Boundedness and Connectivity of Contemporary Families: A Case Study. Connections 22(2): 30-36 (1999).