FACE-TO-FACE VERSUS THREADED DISCUSSIONS:
THE ROLE OF TIME AND HIGHER-ORDER THINKING

Download PDF version:

ABSTRACT
This study compares the experiences of students in face-to-face (in class) discussions with threaded discussions and also evaluates the threaded discussions for evidence of higher-order thinking. Students were enrolled in graduate-level classes that used both modes (face-to-face and online) for course-related discussions; their end-of-course evaluations of both experiences were grouped for analysis and themes constructed based on their comments. Themes included the “expansion of time,” “experience of time,” “quality of the discussion,” “needs of the student,” and “faculty expertise.” While there are advantages to holding discussions in either setting, students most frequently noted that using threaded discussions increased the amount of time they spent on class objectives and that they appreciated the extra time for reflection on course issues. The face-to-face format also had value as a result of its immediacy and energy, and some students found one mode a better “fit” with their preferred learning mode. The analysis of higher-order thinking was based on a content analysis of the threaded discussions only. Each posting was coded as one of the four cognitive-processing categories described by Garrison and colleagues [1]: 18% were triggering questions, 51% were exploration, 22% were integration, and 7% resolution. A fifth category – social – was appropriate for 3% of the responses and only 12% of the postings included a writing error. This framework provides some support for the assertion that higher-order thinking can and does occur in online discussions; strategies for increasing the number of responses in the integration and resolution categories are discussed.

KEYWORDS
Online Learning, Evaluation, Content Analysis, Higher-Order Thinking

I. INTRODUCTION

A. Purpose

As faculty members struggle to determine how to use new technologies appropriately, they must grapple with understanding the advantages (and disadvantages) of traditional, in-classroom activities versus taking those activities online. This is especially true for the conduct of class discussions, where faculty can choose between in-class discussions or online threaded discussions (the issue of chat discussions is not included in this analysis). At least two questions need to be answered for faculty to make good decisions on which tactic to use and when. First, what are the differences between face-to-face versus online discussions and which setting might be better for which learning objective? Second, what evidence exists that higher-level thinking occurs in online discussions? This study begins to answer these questions and proposes some tentative conclusions for further testing and revision.

II. LITERATURE

A. The Role of Time

The literature contains several excellent guides to online learning, especially for building electronic learning communities [2] and creating online coursework [3]. However, there seems to have been few thorough analyses conducted on the differences between classroom and online discussions. Critics of online learning bemoan the loss of face-to-face interactions, which have energy and immediacy that is important to some faculty and students. Weinberger [4] contends that there are “elements of real-world conversations . . . in threaded discussions, [but] there is nothing quite like threaded discussions in the real world” (p. 110). Threaded discussions are focused on one speaker at a time, create semi-permanent records of a discussion (at least for the duration of the class), and do not require everyone to participate at the same time or to be in the same room (or time zone, geographic region, or nation). One area of difference that deserves some analysis is the use and impact of time in the two modes or styles of discussion. Time may be a crucial area of difference because of the way the “time is a resource” metaphor [5] shapes our consciousness of time and those activities that require time to unfold.

The reason that the “time is a resource” metaphor is important for distinguishing between face-to-face and online discussions is its message that time is valuable and limited and that subjective experience of time may be different from one individual to another. First, because time is a valuable resource, individuals use time efficiently or wisely, weighing other demands on their time. This is most consistent with the reasoning behind higher education institutions’ increasing their offering of online courses, since many students today have multiple demands placed on their time, including work and family obligations. This is not to say that on-campus courses should not use class time wisely, only that the time of day chosen for offering classes is likely to be different (for example, during the day for on-campus students, evening for off-campus students, and asynchronously for online students). Dutton and colleagues [6] found that online students are different: 84% work on average 37.7 hours per week (only 54.8% of on-campus lecture students were working an average of 20.5 hour per week). The importance placed on controlling time is a function of its value. Since the use of time is also discretionary (except for that time controlled by one’s employer), the choice of how it is used reflects the values of the individual who controls it [4].

Second, time is limited. On campus, class time is limited to the 50 minutes or limited hours allocated to the course in the college schedule. Furthermore, because class time is limited, students must compete for time in class to display knowledge to the teacher and to get their ideas before the group or, in other words, to get their share of “air time.” Online, time spent on the course can be greater than or less than what on-campus students must endure, although no research has yet determined what the difference in time spent on class objectives may be.

Third, our experience of time is often at odds with the objective measurement of time, seeming fast or slow depending on our fascination with an activity. Clearly, this sense of time is subjective and personal and is a unique experience for different learners in the classroom. Whether online discussions are more prone (or less) to this subjectivity is an important future research question.

Fourth, different educational models support a different use of time. The instructivist model of teaching and learning, where information is passed from one mind to another through the medium of talk or writing, is an efficient educational model, that is, it takes less time. On the other hand, constructivist learning, where knowledge is constructed from experience, is not only famously messy, but time-consuming as well, as students learn at different rates and through different experiences. However, taking the time necessary to construct knowledge has several known advantages, including improved retention of learned material and better learning capability on the part of the student. Therefore, a constructivist educational model may take more time, but provide greater benefits.

Fifth, time has become synonymous with learning, or a particular amount of learning. This assumption has been criticized by those who disparage higher education’s continuing reliance on “seat time” and “clock hours” to stand in for an assessment or quantification of learning. While learning does take time, it may require a long period or be instantaneous. In any case, time does not provide a sensible way to measure the quantity or quality of learning.

These aspects of time may be helpful in illuminating the experience of online discussions. On the other hand, the attraction of online discussions may derive from the “pull of the web” ([4], p. 143). This pull may be due to several factors, not least of which is the web’s ability to convey the voices of others and allow us to respond to those voices at a time that is convenient for us. And there is little doubt that some participants can convey their “social presence” (the degree to which a person is perceived as real in an online conversation), and that such presence is a strong predictor of satisfaction with computer-mediated communications [7]. These “immediacy behaviors” reduce the “social distance” between teachers and students and have been found to be positive predictors of student learning and course satisfaction [8]. Therefore, the attraction of using threaded discussions will likely depend on the individual’s ability to create a realistic “self” in written responses.

Obviously, there are critics of threaded discussions [9] [10]. These authors do not, however, criticize holding class discussions online per se, but they criticize current software programs and current uses or practices for threaded discussions.

It is fundamental that interaction between the student and course content, the faculty member, and other students contributes to learning. Appropriate interaction has been tied to higher student satisfaction and achievement. Proponents of online education support its growth based in part on its ability to facilitate such interactions. Picciano [11] looked specifically at levels of online interactions (postings that were grouped into low, moderate, and high interaction categories) and achievement. While there were no significant differences between the three groups on scores of the final exam, the high interaction group did markedly better on the written assignment. Spiceland and colleagues [12] conclude that increasing interaction through online communications is a form of active learning, and students view such coursework more favorably and deem these communication tools (email, bulletin boards) highly. Many more such studies are needed.

In any case, the use and experience of time may well be an important difference in the experience of in-class versus online discussions. Understanding time may help faculty make better decisions of which setting to use for certain class discussions.

B. The Role of Thinking

It is obvious that thinking and time are related, since thinking occurs in time and for some, the passage of time is crucial to improving one’s thinking. However, for the purpose of this review, we will focus on the research that has been done on critical or higher-level thinking in online, threaded discussions.

The field of research on threaded discussions has benefited from the development of a number of rubrics and analytical structures to analyze the printouts of threaded discussions. One rubric for assessing student online learning [13] established five categories for communications: promptness and initiative (timely and consistent engagement), delivery of post (grammatical correctness), relevance of post (to the current discussion), expression within the post (how well ideas are presented), and contribution to the learning community (contributions to the group). Such an assessment also helps faculty evaluate their own involvement and contributions to the discussion.

Another research project [14] found evidence derived from the literature on successful collaborative learning in face-to-face situations and applied it to online discussions. Curtis and Lawson found substantial evidence of collaboration in the online setting. Approximately equal proportions (of around 25% each) of three behaviors (planning, contributing, and seeking input) occurred in the online setting, with fewer occurrences of reflection (15%) and only 5% comments classified as “social” [14].

The report on a study at Athabasca University [15] found online students “experienced greater cognitive and explanatory learning” as a result of greater participation in course communications where students exchanged “between 80 and 100 messages, which is far richer than the classroom.” This seems to imply that online discussions generate more messages and more involvement in learning.

Another useful question is whether online work can improve critical thinking. Newman and colleagues [16] used content analysis of online messages to look for critical thinking indicators in computer conferences. Students were more likely to make important statements and link ideas, although they contributed fewer novel ideas than the face-to-face comparison group. This may indicate that online conversations are less suited to operations like brainstorming, or that working online encourages respondents to work in a linear fashion, linking comments to earlier statements and bringing in outside knowledge. Shapley [17] also looked at complex reasoning in Chemistry courses, and found the online students scored slightly higher, generating a similar score on an American Chemical Society exam (an exam that requires complex reasoning skills) as that of other graduate students. In another case study of online courses [18], students felt they learned to think more critically and that they could not get through the course by working hard only at exam times.

In a study of critical thinking in computer-mediated communications, Garrison and colleagues [1] proposed a four-stage process: (1) triggering (posing the problem), (2) exploration (search for information), (3) integration (construction of possible solution), and (4) resolution (critical assessment of solution). Transcripts of online discussions were coded, resulting in 8% of the responses coded as triggers, 42% as exploration, 13% as integration, and 4% as resolution. The authors hypothesize that the low numbers for integration and resolution may be due to the need for more time to reflect on the problem and that individuals would hesitate to offer inadequate solutions in a public setting in order to avoid rejection. Garrison and colleagues [1] also coined the term “cognitive presence,” as the “extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry” (p. 11), which is a useful concept to add to the language about online discussions.

Clearly, these are promising areas of research into the effectiveness of online discussions. However, much additional work must be done.

III. METHODOLOGY

The research in this study is primarily ethnographic due to its small sample size, input drawn from student evaluations of two courses, and a lack of statistical testing. It was designed to begin development of an answer to the two questions: (Q1) what are the differences between face-to-face versus online discussions and which setting might be better for which learning objective? (Q2) what evidence exists that higher-level thinking occurs in online discussions?

The first study would develop an answer to Q1 by asking students to compare discussions that were face-to-face or online. This comparison of face-to-face and threaded discussions is drawn heavily from student (N=22) evaluations of two graduate-level courses in educational leadership, each of which experienced both face-to-face and online discussions. Each course used Blackboard’s Discussion Board function, which will be referred to throughout as a threaded discussion. Students were asked at the end of the course to evaluate the threaded discussions as well as the in-class discussions and delineate similarities, differences, pros and cons for each discussion method. This method required students to remember and evaluate discussions that may have occurred several weeks earlier, and thus may be inaccurate or contain memories that are flawed in some fashion. In any case, such drawbacks afflict all research that depends upon the accuracy of human memory. Comments were made anonymously, grouped into similar themes, and analyzed for connections among the themes. The comments fell into several themes focused on different uses and perceptions of time.

The second study would develop an answer to Q2 by analyzing the online threaded discussions for higher-order thinking skills. For this study, the analysis used Garrison’s model [1] and placed each student contribution (or “posting”) into one of four categories: (1) triggering, (2) exploration, (3) integration, and (4) resolution. Further examples of these categories, their indicators, and the sociocognitive processes captured by the indicators, are in Table 1, taken in toto from Garrison and colleagues [1]. A fifth category became necessary, which in the current study was called “social;” Garrison and colleagues [1] had a similar category (called “other”) for remarks deemed “not cognitive presence” (p. 19). The unit of analysis was the complete posting of the student, as in Garrison [1]. However, given the length and complexity of some student postings, this was problematic and was resolved by assessing the contribution’s main, or predominant, quality, which may introduce a more subjective aspect to the analysis. Names of the contributing students were deleted from the transcripts of the online discussions; unfortunately, this means that faculty contributions are also included in the analysis. Total contributions numbered 751 postings across 25 threaded discussions. One additional analysis was whether the posting included writing errors (such as grammatical, spelling, or punctuation errors).

Table 1 Categories of Contributions (based on Garrison and colleagues).

This study is necessarily limited in several ways. First, it depends upon the input of a small number of students. Second, it requires those students to remember and evaluate discussions held over the course of a term of study. Third, faculty comments could not be excluded from the analysis of the contributions to the threaded discussions. Finally, the codification of each posting to the threaded discussions depended upon the understanding of the researcher.

IV. FINDINGS

A. Comparison of F2F vs. Threaded Discussions

In answer to Q1—what are the differences between face-to-face and online discussions, and which setting might be better for which learning objective?—four themes emerged from the student comments. Perhaps the most consistent finding was the experience that time expanded in the online discussions. Instead of a discussion being limited to the time set aside for the face-to-face class in the university’s class schedule, discussions occurred over several hours and several days after the prescribed class time was over or between established class meeting times. “It takes a lot of time” was not an uncommon comment from students. The online discussion requires a marked expansion of the time devoted to a particular class and its material and an indicator of students’ commitment to learning and to using the new medium. Almost every student mentioned how much time it took to read others’ postings, think about a response, prepare a response, and check back later to see others’ contributions to the discussion. And while many students recognized this expansion as a drain on their time, many balanced this criticism with an appreciation that they got more from the discussion because it took time for them to recognize connections, understand others’ ideas, and develop and convey a detailed response or posting.

Another set of comments had to do with the experience of time in the two discussion modes: face-to-face versus threaded discussion. In favor of face-to-face discussions, students said they enjoyed its “speed,” “spark,” or “energy,” the way they could build upon each others’ comments, collaborate on the spot, and benefit from the enthusiasm of others. The threaded discussions were “slow,” took more time to read, and sometimes were made even more slow (or impossible) by problematic connections to the Blackboard site.

Some comments described the quality of the discussion in each setting. The face-to-face discussions were often felt to be “off the hip” (or “off the lip” as one student put it), and its speed and the competition for time made it difficult to ask for clarification or research to back up an opinion. (In fact, several mentioned how often a class conversation would swerve in a new direction leaving the student wanting to comment about a topic that was passed. Timing, in this case, is important and the “window” for speaking is rapidly missed.) Several students greatly appreciated the opportunity to participate more fully in class discussions afforded by the online setting, and indeed the online discussions included contributions from every student and several contributions from each student. “I finally got to have my say” was one student’s comment. The threaded discussions were often more “thoughtful,” more reasoned, and drew evidence from other sources (either other writers or studies pertinent to the discussion). Students were able to share current articles or reports they were reading with the group, rather than waiting for class time to do so, and to relate their readings or current events to course topics.

Another important set of comments related to the needs of the student, more particularly what they felt they needed to participate in the discussion. One student specifically missed the “facial and hand gestures” of the face-to-face discussions in the online setting, thinking they derived important cues to the meaning and feelings of the speaker. Clearly, these are lost in the threaded discussion setting, although several postings did make some use of emoticons (which should not be taken to imply that an emoticon is an equivalent experience to gestures). However, the importance of being a good writer became of supreme importance in the online setting, which not only included the ability to express ideas clearly, but taking the time to reread one’s postings before sending them to the class site to avoid making embarrassing mistakes.

An important need of many students was the opportunity to reflect before “speaking.” Many students spoke in favor of the threaded discussions as they allowed the student to reflect on what was said and to take their time to develop a useful response. “Time to reflect” was mentioned as an advantage by almost all of the students, who felt the constrained time of the classroom meant they could not contribute well or regularly to the discussions. This is an important insight into the value of online discussions, and one that is amply supported by learning theory.

And if the instructor may be accorded an insight, the use of faculty expertise in the two settings is also different. Where the face-to-face classroom exchanges are restricted to the faculty person’s ability to remember and respond to issues on the spot, the threaded discussions also allowed the instructor to reflect on a question and develop better, more detailed responses. Questions from students in the online setting could be handled one-on-one or one-to-all, depending on the content of the question, but ensured that the answer for one could be the same for all. Taking additional time to prepare a response ensured a better answer for the student.

B. Higher Order Thinking

An answer to Q2—what evidence exists that higher-level thinking occurs in online discussions?—is more mixed, providing some reassurance that higher-level thinking occurs, but perhaps not as much as desired. Table 2 presents the distribution of students’ online postings across the four categories used in Garrison and colleagues. [1]: (1) triggering (asking questions), (2) exploration (search for and offering of information), (3) integration (construction of possible solution), and (4) resolution (critical assessment of solution). A fifth category, social, included postings that were purely social (agreeing to a prior posting without additional commentary, requesting personal information, assistance on a non-course-related issue, or commenting on issues not related to the topic). The final column compares the subjects’ responses to those in Garrison and colleagues [1]. The table also includes the number of postings that had grammatical or spelling errors.

Table 2 Postings by Four Categories of Higher-Order Thinking

V. IMPLICATIONS

A. Use of Time

The first question asked in this study was what are the differences between face-to-face versus online discussions and which setting might be better for which learning objective? The comments from students participating in both face-to-face and online discussions provide the basis for three assertions that require further research to determine if they are valid beyond this study. First, the use of threaded discussions expands the amount of time spent by students on class objectives. Second, students may use this additional time to reflect on the material, to ask questions, and to participate more equitably in class discussions. Third, different students with different strengths respond differently to the online setting. The student who learns or processes information by talking and who enjoys the give-and-take of discussion may feel disadvantaged in the online setting; the student who requires reflection to learn or construct an answer may be advantaged. Therefore, offering a mix of ways to be involved in discussion may well improve the likelihood that most students find an avenue for contributing that satisfies their learning needs.

While these findings are surely tentative and somewhat speculative, they do suggest some recommendations for faculty desiring to use face-to-face and online discussions appropriately. This presumes that faculty can choose to use the setting that works best, using both types of discussion only at different times, or for different topics, or to achieve different ends. First, if at all possible, faculty must know the predominant learning preferences of students as well as their own interactional styles. While there are several online tools for assessing learning styles (http://www.humanmetrics.com), it may not be necessary to have the class undergo such assessments, although they may be valuable tools for encouraging students to understand their learning preferences. This is because even though some students and faculty prefer discussions in one setting or the other, it is much more likely that each class has a mix of student preferences and thus using both settings will satisfy more individuals. Second, it appears that some activities may work best face-to-face: brainstorming, visual demonstrations, and topics where energy and enthusiasm can contribute to the success of the discussion. This may not be an exhaustive list and it should not be taken to mean the activity should always occur face-to-face (some visual demonstrations may actually be clearer in an online setting). Third, time for reflection is key to the learning of many students, and a few seconds in class may not be sufficient time to reflect and think about a course topic; thus, topics that require additional reflection would probably work best in the threaded discussion setting.

In time, it may be recognized that although differences exist between discussions held face-to-face or online, the difference in their value for learning is the result of individual preferences or preparation. In other words, the search for hard-and-fast rules or guidelines may need to allow for the faculty’s knowledge of student learning preferences. It will also be interesting to watch for evidence that students participating in online discussions will develop differently in order to determine whether “new worlds create new people” ([4], p. 9). Will the shy person find his or her voice? Will reflective thought become more common? Will the ability to express voice in writing become as valuable online as a reasoned expression of opinion? Only time will tell.

B. Higher-Order Thinking

The second question asked, “What evidence exists that higher-level thinking occurs in online discussions?” It is important to remember that the analysis of evidence for higher-order thinking skills is not a comparison to face-to-face discussions, but focuses only on the threaded discussions. While higher-order thinking may (or may not) occur in the classroom, the evidence does appear to support earlier studies that students involved in threaded discussions are exhibiting higher-order thinking, especially by contributing comments that are exploratory (51%), integrative (22%), or resolution (7%). Integration and resolution especially require “time for reflection” ([1], p. 20), which may be more likely to occur in the extended time period of the threaded discussion. The lack of comments in the resolution category was also noted by Garrison and colleagues [1], and may be due to a number of factors, including the complexity or difficulty of issues raised, a lack of student skill or information to propose or test a resolution to the problem at hand, or a missed opportunity on the part of faculty who could have pressed for a resolution to questions being raised by the discussion.

It is also clear that students stayed mostly on task, generating only 3% of the postings with comments deemed “social,” although this is not to say that taking care of personal relationships online is not a valuable and useful role for online discussions. Also, because these students also met face-to-face, they may have had less need to maintain personal relationships online. In addition, it also appears that students took extra care with their postings, as only 12% of the comments posted to the threaded discussion contained some type of writing error (such as grammatical, spelling, or punctuation error). This seems to support the contention that the greater public nature of the medium may encourage students to take more care in preparing written responses, lest their peers see and judge them on their writing skill.

In any case, there are some interesting directions for faculty in these findings. Faculty may need to be more directive in their assignments for threaded discussions, charging the participants to resolve a particular problem, and pressing the group to integrate their ideas or prepare a resolution of the matters under discussion. It may also be important to analyze threaded discussion against a variety of critical-thinking frameworks. In any case, the written records of threaded discussions may be a boon to researchers desiring to study the online environment for evidence that students gain the intellectual skills that higher education values.

VI. REFERENCES

1. Garrison, D. R., Anderson, T., and Archer, W. Critical Thinking, Cognitive Presence, and Computer Conferencing in Distance Education. The American Journal of Distance Education, Vol. 15, No. 1, pp 7-23, 2001.
2. Palloff, R. M., and Pratt, K. Building Learning Communities in Cyberspace. San Francisco, CA: Jossey-Bass, 1999.
3. Palloff, R. M., and Pratt, K. Lessons from the Cyberspace Classroom: Realities of Online Teaching. San Francisco, CA: Jossey-Bass, 2001.
4. Weinberger, D. Small pieces loosely joined. Cambridge, MA: Perseus Publishing, 2002.
5. Lakoff, G., and Johnson, M. Metaphors We Live By. Chicago: University of Chicago Press, 1980.
6. Dutton, J., Dutton, M., and Perry, J. How Do Online Students Differ from Lecture Students? JALN, Vol. 6, No. 1, 2002.
   http://www.sloan-c.org/publications/jaln/v6n1/index.asp
7. Gunawardena, C. N., and Zittle, F. J. (1997). Social Presence as a Predictor of Satisfaction within a Computer-Mediated Conferencing Environment. The American Journal of Distance Education, Vol. 11, No. 3, 1997.
8. Arbaugh, J. B. How Instructor Immediacy Behaviors Affect Student Satisfaction and Learning in Web-Based Courses, Business Communication Quarterly, Vol. 64, No. 4, pp 42-54, 2001. http://www.alnresearch.org/data_files/articles/full_text/arbaugh01.pdf
9. Klemm, W. R. Extending the Pedagogy of Threaded-Topic Discussions. The Technology Source, September/October. 2002. http://ts.mivu.org/default.asp?show=article&id=1015
10. Turoff, M., and Hiltz, S. R. Software Design and the Future of the Virtual Classroom. Journal of Information Technology for Teacher Education, Vol. 4, 197-215, 1995.
11. Picciano, A. G. Beyond Student Perceptions: Issues of Interaction, Presence, and Performance in an Online Course. JALN, Vol. 6, No. 1, 2002.
    http://www.sloan-c.org/publications/jaln/v6n1/v6n1_picciano.asp
12. Spiceland, J.D., and Hawkins, C. P. The Impact on Learning of an Asynchronous Course Format. JALN, Vol 6, No. 1, 2002.
13. Edelstein, S., and Edwards, J. If You Build It, They Will Come: Building Learning Communities through Threaded Discussions. The Online Journal of Distance Learning Administration, Vol. 5, No. 1, 2002. http://www.westga.edu/~distance/ojdla/spring51/edelstein51.html
14. Curtis, D. D., and Lawson, M. J. Exploring Collaborative Online Learning, JALN, Vol. 5, No. 1, pp 21-34, 2001.
15. Paskey, J. A Survey Compares 2 Canadian MBA Programs, One Online and One Traditional. The Chronicle of Higher Education, April 26, 2001. http://chronicle.com/free/2001/04/2001042601u.htm
16. Newman, D. R., Webb, B., and Cochrane, C. A Content Analysis Method to Measure Critical Thinking in Face-to-Face and Computer Supported Group Learning, 1999. http://www.qub.ac.uk/mgt/papers/methods/contpap.html
17. Shapley, P. On-line Education to Develop Complex Reasoning Skills in Organic Chemistry. JALN, Vol. 4, No. 2, 2000.
    http://sloan-c.org/publications/jaln/v4n2/v4n2_shapley.asp
18. Eklund, J., and Eklund, P. Integrating the Web and the Teaching of Technology: Cases across Two Universities, 1996. http://ausweb.scu.edu.au/aw96/educn/eklund2/paper.htm

VII. ABOUT THE AUTHOR

Dr. Meyer is currently assistant professor of educational leadership at the University of North Dakota specializing in online learning and higher education. For over three years, she was Director of Distance Learning and Technology for the University and Community College System of Nevada. Prior to this, she served over 8 years as Associate Director of Academic Affairs for the Higher Education Coordinating Board in the state of Washington and was responsible for technology planning and policy related to online learning. She can be reached at the University of North Dakota, P.O. Box 7189, Grand Forks, ND 58202; phone: (701) 777-3452; or email: katrina_meyer@und.nodak.edu.