THE EBB AND FLOW OF ONLINE DISCUSSIONS:
WHAT BLOOM CAN TELL US ABOUT OUR STUDENTS’ CONVERSATIONS

Katrina A. Meyer
Associate Professor of Higher and Adult Education
The University of Memphis

ABSTRACT

This study looks at online discussions within the context of a group endeavor and attempts to evaluate three assertions: 1) students in an online discussion proceed through higher levels of thinking; 2) Online conversations follow an “ebb and flow” pattern; 3) The level of the first posting to the discussion influences the level of subsequent postings. Postings were classified by the Bloom’s taxonomy of educational objectives and were graphed to help analyze the assertions. There was only modest support for the first assertion and better support for the second and third assertions. Analyzing online discussions as a group endeavor may well prove an interesting area of research for those interested in applying the existing body of literature and theory on group discussions in the face-to-face arena to the online environment. Researchers need to evaluate whether this literature is applicable to online discussions and what distinctions can legitimately be made between group discussions in the face-to-face or online mode.

KEYWORDS

Online learning, evaluation of online discussions, group discussion, Bloom’s taxonomy

I. Introduction

An earlier study by the author [1] used several frames—including the Bloom Taxonomy of Educational Objectives—to analyze several online discussions of graduate students in educational leadership classes. As a result of this study, it was determined that a further analysis was needed to assess the discussions in terms of how each contribution related to contributions that preceded or followed it, and not in terms of individual contributions. In other words, the issue was to capture the progress of an online conversation by using Bloom’s Taxonomy in order to assess the level at which the entire discussion took place and/or the levels the discussion passed through to its conclusion.

II. LITERATURE

Online discussions have several advantages to researchers and instructors. Not only is there a printed record of the discussion that is available long after the discussion has concluded, but it can be analyzed and reanalyzed by faculty using several methods and looking for a variety of student-learning outcomes. Several research studies have looked at these online discussions in an attempt to ascertain the student’s level of critical thinking, or cognitive processing [1, 2, 3, 4, 5, 6], or other learning outcomes of interest. Some researchers of online discussions have also focused on the group, more specifically the group’s social construction of knowledge in online debates [7], computer-supported cooperative work [6], or social interdependence theory of cooperative learning [8, 9]. These studies clarify how individuals interact and learn in the online discussion, but there remains a need for studies on how a group discussion proceeds online.

Group discussion has long been an area of study within psychology and has been a mainstay of educational psychology, especially in terms of helping teachers use discussion effectively in the classroom [10, 11, 12]. Thus, there is a substantial body of literature and theory from researchers focusing on face-to-face discussions. It is not clear at this stage whether this body of work may be applicable to the online, asynchronous discussion, but perhaps it is a good place to look for explanations of what occurs in online discussions. As research progresses, it will be useful to identify exceptions to this theory for the online context.

There is a long tradition in this literature on the problem-solving process, captured by Dewey’s [13] concept of “reflective thinking.” In this work, Dewey proposed that reflective thinking followed a pattern that began with a problem, proceeded to analysis, then suggested solutions, reflected on the solutions and weighed their advantages and disadvantages, and then selected the best solution. This has been described as the “discussional pattern or the steps in the discussional process” [13, p.16]. Not surprisingly, one finds a very similar process outlined in authors that followed Dewey [10, 12]. In other authors, additional stages may be added, such as a first stage of “pooling information” and a final stage of “mutual adjustment of opinions” [14]. Other areas of research on group dynamics have looked at the individual’s contribution to the operation of the group, development of roles, life cycles of groups, etc. Current researchers into small groups [15] stress the complexity and adaptability of these groups as well as their dynamic qualities. In fact, this view of groups as complex systems will add richness as well as tentativeness to the analysis of online groups, since interpretations will require a thorough understanding of the context for the discussion, as well as the internal states of group members.

The earlier literature provides evidence that researchers used a variety of tools for analyzing and evaluating discussions, including graphing contributions, preparing charts or diagrams of communication flows, and comparing contributions to time, negativity, and reference to persons within or outside of the group [16, 17]. More current researchers have also used a variety of graphing or diagrams to display analysis of online discussions [7, 9, 18].

Given this foundation, is there a tool that might be seen to capture the stages of discussion as well as an individual’s contribution to an online discussion? Certainly, there is a long history with Bloom’s taxonomy of educational objectives. Originally published in 1956 by Bloom [19], the taxonomy has recently been updated by Anderson and Krathwohl [20] to incorporate recent findings in knowledge creation. In other words, Anderson and Krathwohl [20] argued that the highest two levels of the taxonomy should be reversed, with “create” at the highest level and “evaluate” at the second highest level. Labels for all six levels have been somewhat simplified, and are captured by the terms (in order of level) as:

Drops [21] suggested that the Bloom taxonomy might be a useful tool for evaluating online discussions, which prompted its use in an earlier study by the author [1]. That study suggested that the level of a posting to an online discussion may reflect that student’s “primary or most comfortable level of a response or a response that the group must pass through to develop higher-level analyses.” Rather than analyzing online discussions in terms of individual contributions (or the contributions of specific individuals), it may be worthwhile to examine the entire conversation as a “group effort.”

At issue are three beliefs or perceptions that require examination. The first and perhaps idealistic expectation is the hope that students’ conversations progress through higher levels of thought, and through the stages proposed by Dewey and others. In other words, faculty may expect an online conversation to climb through levels to reach the higher levels at its closure. The second, but perhaps more realistic expectation, is that online discussions are like face-to-face conversations, and surge up and down much like the waves of the ocean which ebb back toward the sea and flow upwards to the shore. In this view, a conversation online operates in an up-and-down cadence rather than an ever-upwards climb toward higher and higher levels of thought. The third assertion is that the level of the initial posting tends to influence the level of subsequent postings, something that was asserted in [1], when three of the four frames used to analyze online discussions indicated that the initial posting may have influenced the level of subsequent contributions; however, this insight requires further analysis. These assertions are testable by investigating a number of online discussions conducted by students. While firm answers may need to be held in abeyance until further research is done, tentative insights into the ebb and flow of online discussions may be both interesting and a challenge to our presumptions.

III. METHODOLOGY

Online discussions that occurred in two doctoral-level classes in educational leadership held over two semesters in 2002–03 were the subject of the analyses. Each discussion was printed, dated, and individual student names blacked out. A total of 17 different online discussions were analyzed, comprising 278 total postings by the same ten students in each class. Students had become relatively comfortable with participating in online discussions, having used this method in several classes. This was deemed an advantage, as interactions might be more natural or be less inhibited by the online setting. In other words, these students are not likely to be responding to the novelty of online discussions per se.

Online discussions that occurred in two doctoral-level classes in educational leadership held over two semesters in 2002–03 were the subject of the analyses. Each discussion was printed, dated, and individual student names blacked out. A total of 17 different online discussions were analyzed, comprising 278 total postings by the same ten students in each class. Students had become relatively comfortable with participating in online discussions, having used this method in several classes. This was deemed an advantage, as interactions might be more natural or be less inhibited by the online setting. In other words, these students are not likely to be responding to the novelty of online discussions per se.

All of the online discussions were led by a student who selected the question or topic they wished to discuss with their class peers related to that week’s class reading. Students were not charged by the instructor with accomplishing a particular goal in the discussion, only with discussing the reading material. Therefore, the discussions took on the purpose and interests of the different students, which added variety to the discussions and varied the goals of each discussion. Making the online discussions student-led also ensured that each student could practice leadership of a discussion as well as pursue a discussion that addressed personal interests with the assistance of his/her peers.

Each discussion was analyzed using the updated version of Bloom’s taxonomy of educational objectives by Anderson and Krathwohl [20]. Using Bloom’s taxonomy had three advantages. First, it focuses the analysis on the level of students’ thinking, not their knowledge of the topic being discussed (e.g., higher education finance, governance models). Second, Bloom was well-known to the researcher, and its classifications and distinctions were clear and understandable, which would help ensure more consistent ratings across discussions and more reliable results. Third, Bloom proposes various levels of thinking, from a lower level of knowing to a higher level of creating; this characteristic would be helpful in graphing the online discussions and evaluating the assertion that discussions would proceed from lower to higher levels of thought.

The researcher used content analysis of the online discussions, a technique that has been used in several of the previous studies already noted [1, 3, 6, 7]. Content analysis is not without its problems and critics [22], because coding can be prone to subjective interpretations. In this research, two precautions were taken to ensure more reliable analyses. First, coding was done by the researcher and used the key words or verbs for Bloom’s taxonomy found in Krumme [23], reproduced in Table 1. This aid was helpful in ensuring more accurate classifications of postings. Second, if an individual posting within a discussion could not be categorized, it was deemed “not categorized” and dropped from further analysis. However, one more decision is important to understand this research. The unit of analysis chosen was a single “posting” to the discussion. Although Henri [24] encourages researchers to use the “unit of meaning” for analysis, this was found to be unwieldy by other researchers [7]. Choosing the posting as the unit of analysis meant some postings that were quite lengthy could be categorized at multiple levels; in these cases, the level or category most consistent with the entire posting was used.

Table 1:Taxonomy of Educational Objectives: Bloom (1956)

Analysis of the discussions involved identifying sequences between postings, focusing first on paired postings that stayed at the same level, second on pairs that have moved one level (up or down), third on pairs that skipped one level (up or down). The next analyses focused on consistencies, including the frequency of each level of posting and the number of subsequent postings that were consistent with the level of the first posting. The final analyses attempted to graph each online discussion, as was done by [9, 10] and to categorize the conversation into general patterns.

IV. FINDINGS

A. Pairs

Tables 2 and 3 focus the analysis on adjacent pairs of postings. Pairs were determined to be any two postings that were contiguous in time; in other words, one posting preceded (or followed) the other in the discussion. Total number of adjacent pairs possible was 245.

Table 2: Adjacent Pairs at Same Bloom Level

Of the 245 pairs in the 17 discussions analyzed, 27.8% were at the same level. If we assume that the level of any posting could be randomly allocated among the six levels of Bloom’s taxonomy, then each level might garner approximately 16% of the total postings. A goodness of fit test resulted in a chi-square of 30.82, df=5, p<0.001. This seems to imply that there are more pairs at the same Bloom level than might reasonably occur by chance, with pairs at one level—analysis—being the most frequent in number.

Table 3: Adjacent Pairs with One-Step Changes

In Table 3, the analysis takes a different turn. First, the total number of pairs moving up one level or down one level on Bloom’s taxonomy are equal. This may imply that conversations are being conducting in an “ebb and flow” fashion. Second, it appears that the distribution of the pairings are not occurring by chance; chi-square for the pairs moving up one level is 17.23, df=4, p<0.01; chi-square for the pairs moving down one level is 19.9, df=4, p<0.001.

Table 4: Postings That Skip a Step (Two-Step Changes)

Table 4 might imply two insights into these conversations. First, instances where one posting elicited a second posting that was at a Bloom level two levels higher are more frequent, and thus more likely, than instances where the movement was downward. Chi-square for the movement up two levels is 25.2, df=3, p<0.001; chi-square for movement down two levels is 9.42, df=3, p<0.05. This means that the distribution of the various steps is not occurring by chance. Second, analysis was the one level that comprised a common denominator to the two categories that occurred most frequently (analyze to create, understand to analyze). Analyze, as will be seen in the following section, was the most frequent type of posting.

B. Consistency

Table 5: Levels with Most Frequent Occurrence

Table 6: Subsequent Postings Consistent with First Posting

Table 6 is an attempt to assess the assertion that the first posting heavily influences the level at which the remainder of the discussion occurs. Two things are worth noting from the results in Table 6. First, no discussion began with the highest level (Create), which may be interpreted to mean several things, including the lower likelihood of conversations beginning at that level or the lack of a requirement by the faculty person to begin at this level (which may be difficult to do if creation must happen as the result of a process that requires first proceeding through the other levels). Second, there appears to be modest support for the assertion that subsequent postings (or 36.8%) are consistent with the level of the first posting. Chi-square for the number of postings consistent with the first posting is 95.11, df=4, p<0.001, which confirms that this distribution would not occur by chance. Given that students may choose to respond at any of six levels, the preponderance of postings that are consistent with the level of posting that began the discussion is remarkable. On the other hand, the majority of postings (63.2%) were inconsistent with the first posting, but again given that students had six levels at which to contribute might argue for a less-than-random response. Chi-square for the number of postings inconsistent with the first posting is 115.22, df=4, p<0.001, which again confirms that this is not a chance distribution.

C. Patterns of Conversation

For each of the 17 online discussions under analysis, a simple graph was plotted for the conversation using the number of the level of the posting as the plotting point (1=Know; 2=Understand; 3=Apply; 4=Analyze; 5=Evaluate; 6=Create). Each point in the graph represents one person’s posting to the discussion, from the first person to post to the last person who participated in the discussion. Each graph was evaluated independently by the author and two graduate students and placed into a category that described the general or overall progress of the conversation. Interrater consistency was over 90% for this grouping of graphs into categories. Each rater came up with a descriptor for the category; these descriptors have been used in Table 7. Four categories became evident: conversations that tended to generally climb upwards, labeled “uphill;” conversations whose predominant characteristic was its wavering up and down; conversations that were relatively steady at one or two levels; and conversations that tended downwards, or “downhill.” The average number of postings per discussion was 16.3, and discussions ranged from a low of 12 postings to a high of 27 postings. Therefore, given the number of postings in each online discussion, no conversation was a perfect example of the category. Table 7 presents the classifications of the 17 online discussions held by students.

Table 7: Patterns of Online Discussions

Figures 1 through 4 capture representative conversations that were classified in each of the above categories. The X axis is the level of the posting; the Y axis approximates the passage of time (which in reality did not occur in consistent blocks of time).

Figure 1: Discussion # 1 — “Uphill”

Figure 2: Discussion # 5 — “Wavering”

Figure 3: Discussion #15 — “Steady”

Figure 4: Discussion #11 — “Downhill”

Because only 3 of the 17 discussions could be classified as “uphill,” these figures tend to provide only modest support for the faculty’s hope that student discussions are striving upwards towards higher levels of thinking and the progression through the stages of discussion proposed by Dewey. On the other hand, 12 of the 17 conversations provide better support for the belief that most conversations track up-and-down on the Bloom taxonomy, capturing the majority of these online discussions’ tendency to ebb and flow.

V. IMPLICATIONS

There are four insights from this modest analysis that may be worth further exploration and tentative use in online class discussions. First, because many pairs of postings occurred at the same level and the first posting did seem to increase the likelihood that subsequent postings occurred at the same level, it is clear that initially setting a high level is important. This may even argue for faculty to either begin the discussion or to require that students who lead discussions to begin with a posting that is at a high Bloom level.

Second, despite this tendency to remain at the same level, “leaps” (over one or two steps in the Bloom taxonomy) occurred nearly as frequently as movement up or down in pairs. In other words, even though there is some tendency to keep a discussion at one level, individuals do appear to make ample contributions that are at higher (or lower) levels. Thus, the level of an online conversation is not determined by the first posting, but people do come in and change the level (either up or down about evenly).

Third, given the number of responses that were one step higher or lower and the graphing of conversations that captured wavering or whipsawing activity, there does seem to be some support for the characterization of online conversations as an ebb and flow, the classic colloquial “give and take” of conversation. Perhaps conversations online proceed much as other conversations do: participants analyze a situation, surge upwards into evaluation, only to descend to check facts and make sure of the group’s understanding. Unfortunately, this is an interpretation that may make sense from the external view but would require verification by the group. It will be the group, not the outsider, who can declare whether the group is conscious of its strategies, or whether it is operating unconsciously, or without any strategy at all, or simply conversing as they would were they face-to-face.

Fourth, if the instructor’s goal were to have online group discussions operate in the best manner as face-to-face group discussions charged with solving a problem, then there might be a need to train or remind students of good practice for discussions, i.e., Dewey’s “reflective thinking” or another model. In addition, instructors may find a wealth of issues and characteristics in the literature on face-to-face discussion that should be evaluated in the online setting.

This analysis added a focus on the operations of a group as it held discussions online. Clearly, there is a need to analyze whether group dynamics and behavior are similar in the online setting, and whether the theories and research that have been done in face-to-face groups translate to the online discussion. While this is a sensible assumption, it may be that dynamics are different in online discussions, so this assumption requires further research so that the distinctions between online and face-to-face discussions made more clear.

VI. REFERENCES

1. Meyer, K. A. Evaluating Online Discussions: Four Different Frames of Analysis. The Journal of Asynchronous Learning Networks 8 (2): April 2004. Available online: http://www.sloan-c.org/publications/jaln/v8n2/v8n2_meyer.asp.
2. Meyer, K. A. Face-to-Face Versus Threaded Discussions: The Role of Time and Higher-Order Thinking. The J ournal of Asynchronous Learning Networks 7 (3): 2003.Available online: http://www.sloan-c.org/publications/jaln/v7n3/v7n3_meyer.asp.
3. Garrison, D. R., T. Anderson, and W. Archer. Critical Thinking, Cognitive Presence, and Computer Conferencing in Distance Education. The American Journal of Distance Education 15 (1): 7–23, 2001.
4. Jeong, A. C. The Sequential Analysis of Group Interaction and Critical Thinking in Online Threaded Discussions. The American Journal of Distance Education 17 (1): 25–43, 2003.
5. Hiltz, S. R. Evaluating the Virtual Classroom. In Harasim, L. (Ed.), Online Education, 134–184. New York: Praeger, 1990.
6. Newman, G., B. Webb, and C. Cochrane. A Content Analysis Method to Measure Critical Thinking in Face-to-Face and Computer Supported Group Learning. Interpersonal Computing and Technology 3 (2): 56–77, 1995.
7. Gunawardena, C. N., C. A. Lowe, and T. A. Anderson. Analysis of a Global Online Debate and the Development of an Interaction Analysis Model for Examining Social Construction of Knowledge in Computer Conferencing. Journal of Educational Computing Research 17 (4): 397–431, 1997.
8. Aviv, R. Educational Performance of ALN via Content Analysis. The Journal of Asynchronous Learning Networks 4 (2): 53–72, 2000. Available online: http://www.sloan-c.org/publications/jaln/v4n2/v4n2_aviv.asp.
9. Aviv, R., Z. Erlich, G. Ravid, and A. Geva. Network Analysis of Knowledge Construction in Asynchronous Learning Networks. The Journal of Asynchronous Learning Networks 7 (3): 2003. Available online: http://www.sloan-c.org/publications/jaln/v7n3/v7n3_aviv.asp.
10. Borman, E. G. Discussion and Group Methods: Theory and Practice. New York: Harper and Row, 1975.
11. Potter, D., and M. P. Andersen. Discussion: A Guide to Effective Practice. Belmont, CA: Wadsworth Publishing Company, Inc., 1970.
12. Smith, W. S. Group Problem Solving Through Discussion. New York: Bobbs-Merrill Co., 1965.
13. Dewey, J. How We Think. Boston, MA: D. C. Heath & Co., 1910.
14. Bridges, D. Education, Democracy and Discussion. Windsor, England: NFER Publishing Co., Ltd., 1979.
15. Arrow, H., J. E. McGrath, and J. L. Berdahl. Small Groups as Complex Systems. New York: Sage Publications, 2000.
16. Mills, T. M. Group Transformation: An Analysis of a Learning Group. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1964.
17. Gulley, H. E. Discussion, Conference and Group Process. New York: Holt, Rinehart & Winston, Inc., 1968.
18. Levin, J. A., H. Kim, and M. M. Riel. Analyzing Instructional Interactions on Electronic Message Networks. In Harasim, L.M. (Ed.), Online Education, 185–213. New York: Praeger, 1990.
19. Bloom, B. S., and D. R. Krathwohl. Taxonomy of Educational Objectives: The Classification of Educational Goals. New York: Longmans, Green, 1956.
20. Anderson, L. W., and D. R. Krathwohl (Editors). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman, 2001.
21. Drops, G. Assessing Online Chat Sessions. Online Cl@ssroom, University of Calgary: April 2003. Available online: https://www.ucalgary.ca/~commons/oc/0304OC.pdf.
22. Rourke, L., T. Anderson, D. R. Garrison, D. R., W. Archer. Methdological Issues in the Content Analysis of Computer Conference Transcripts. International Journal of Artificial Intelligence in Education 12: 18–22, 2001.
23. Krumme, G. Major Categories in the Taxonomy of Educational Objectives (Bloom 1956), n.d. Available online: http://faculty.washington.edu/krumme/guides/bloom.html.
24. Henri, F. Computer Conferencing and Content Analysis. In A. Kaye (Ed.), Collaborative Learning Through Computer Conferencing, 117–136. London: Springer-Verlag, 1991.

VII. About the Author

Katrina Meyer is currently associate professor of higher and adult education at The University of Memphis specializing in online learning and higher education. Dr. Meyer is the author of Quality of Distance Education: Focus on On-Line Learning, a 2002 publication of the ASHE-ERIC Higher Education Report Series. For over three years, she was Director of Distance Learning and Technology for the University and Community College System of Nevada. Prior to this, she served over 8 years as Associate Director of Academic Affairs for the Higher Education Coordinating Board in the state of Washington and was responsible for technology planning and policy related to online learning.