An Evaluation of Online Syllabi in The University of Tennessee College of Communications
by kiralyse
printer friendly version I. INTRODUCTION
As more academic courses develop an online presence, there is an increasing need to evaluate form and content in order to increase quality and usability. An evaluation instrument was developed that synthesizes website design criteria with syllabus development criteria. This case study used the instrument to evaluate online syllabi in five departments within the University of Tennessee College of Communications: Advertising, Broadcasting, Communications, Journalism, and Speech Communications, plus the allied School of Information Sciences. This case study indicates that the higher the percentage of department courses with online syllabi, the higher the quality of those syllabi. Shared factors of the highest rated online syllabi include connectivity to relevant online resources and a high degree of interactivity. Within The University of Tennessee College of Communications, the Public Relations Department and the School of Information Sciences have the highest percentage of courses with online syllabi and the highest average evaluation score for online syllabi. One shared factor of these two departments is the offering of at least one course in webpage development.
The purpose of this case study is to determine how many College of Communications classes listed in Tennessee 101: A Guide to Courses at the University of Tennessee [1] have online syllabi, and to evaluate the content of those syllabi. An evaluation instrument is proposed that synthesizes syllabus development criteria and webpage design criteria.
II. SYLLABUS DEFINITION AND FUNCTIONS
The word "syllabus" is from the Greek sillybos, literally "parchment label." Originally it meant the summary label of a parchment scroll, but it came to mean a summary or outline of a course of academic study.
According to Appleby [2] and Rubin [3] the course syllabus serves at least seven functions:
- Helps to plan and strengthen the course. This is what Gabennesch [4] calls the enriched syllabus.
- Reveals the implicit assumptions about the course.
- Introduces the instructor to students.
- Explains why students should take the course - the purpose or value of the course.
- Explains the various aspects of the course including assignments, exams, and grading policies.
- Explains how students will change as a result of the course, including knowledge and/or skills they will acquire.
- Communicates the nature and content of the course to colleagues and administration.
- Provides a documented record of an instructor's teaching career.
Greg Chan [5] lists possible content areas of a syllabus, which he suggests could be modified to suit individual needs:
- Course information: institution, term, course title & number, section number, location, class and lab times.
- Instructor information: name, title, contact information plus teaching assistants and their contact information.
- Texts: cites title, author, preferred edition, publisher, date of publication, cost, where available. May be divided into "Required" and "Recommended."
- Course description: provides an overview of course content and objectives; information about instructional methods, course prerequisite(s).
- Schedule of activities: provides a daily/weekly listing of topics, readings, and assignments. Includes due dates for all exams, papers and other means of assessment; holidays should also be noted.
- Course policies: makes it clear where you stand on attendance, lateness, classroom conduct, participation, missed exams, late assignments, and plagiarism.
- Grading system: clearly details the type, number, and % value of all course assignments; explains evaluation scheme/marking criteria.
- Disclaimer: states that syllabus is tentative, and subject to change according to the needs and interests of the class. This will increase flexibility.
Chan adds [5] "a growing number of instructors are making their syllabi available to students (both graduate and undergraduate) before the first day of class, through mail-outs, pick-ups or even online access."
III. THE ONLINE ACADEMIC SYLLABUS
"When technology extends our senses, a new translation of culture occurs
as swiftly as the new technology is interiorized." Marshall McLuhan [6]
An online course syllabus adds a dynamic dimension beyond the scope of a printed syllabus - online interactivity.
According to Dr. Rodney P. Riegle [7], "With the WWW, instructors now have a tool which allows them to do what everyone has always known is the most efficient and effective way to facilitate learning - learner-controlled interactivity."
According to a survey of Stanford instructors, [8] those who had introduced online technology into their courses believed that both teaching and learning had improved. They reported that students were better prepared and more willing to enter into class discussions. Those instructors who required students to review online material and send feedback via e-mail, listserv, or class forum or bulletin board were able to summarize student responses at the beginning of class and begin discussion from a more advanced point.
Brians et al. [9] suggest that an online environment can become a knowledge-making, dialogic environment, one in which students and faculty interact with each other, one in which the idea of learning communities, active learning, and engagement can evolve. The learning environments we construct online must encourage synthesis, inquiry, analysis, etc.--students must move beyond memorization to actually work with class materials.
Thus, the technologies we develop and employ must enable students to learn more and to learn it better. In other words, we need to promote what we already know are the most effective learning strategies - the two most significant factors in persistence to degree are (1) interaction with peers and (2) interaction with faculty. Other studies demonstrate that effective courses offer frequent opportunities for students to show what they know and to get feedback on their progress. So, for sound educational reasons, asynchronous, online education can and should incorporate such features.
IV. HYPERMEDIA LEARNING THEORY
Some understanding of hypermedia learning theory is useful in the development and evaluation of online course syllabi.
The effectiveness of hyperlinks and hypermedia on the World Wide Web has been related to several learning theories. Jonassen [10] suggests that hypertext reflects a model of learning based on schemas, the accumulation and organization of knowledge as semantic memory objects connected by attributes.
Kearsley [11] writes that "hypertext matches human cognition; in particular the organization of memory as a semantic network in which concepts are linked together by associations."
A similar model also based on nodes and links by Chan et al. [12] is OCTR:
- Orientation, or relating prior knowledge
- Coaching, or apprenticeship
- Tuning
- Routinization
The learning process in this model is explained as the tentative connection of old knowledge with new knowledge, the accretion of more links, the articulation whereby some links are strengthened and some deleted, and the solidification and reinforcement of surviving links.
"By engaging the associative structure of memory," Gingher [13] writes, the new medium "compels a traditional linear, synchronous, sequential and exclusive pedagogy to adapt to a new paradigm - one associative, asynchronous, contextual, and collaborative."
V. EVALUATION CRITERIA OF ONLINE RESOURCES
An effective online syllabus adds additional elements of interactivity and connectivity to online learning resources and requires an awareness of web design principles and evaluation criteria.
James Rettig [14] of the College of William and Mary reviews general or academic reference sites. Rettig [15] offers "Comparative Criteria for Reviewing Reference Books and Web Sites" in which he adapts criteria for evaluating print resources to online resources:
- Accuracy - Is the information presented accurate?
- Appropriateness - Is it appropriate for the intended audience?
- Arrangement - Is the arrangement appropriate and useful for the topic?
- Authority - What are the author's qualifications, and are they stated?
- Bibliography - Are the links "hot," useful, and appropriate?
- Comparability - How does it compare to others similar in scope and purpose?
- Completeness - Does it cover its subject adequately?
- Content - What is the overall value of content? Does it contribute to its field?
- Distinction - Does it have a distinctive character or attribute?
- Ease-of-use - Is it easy to use, well-organized with site map or index?
- Illustrations - What is the quality, how well are they integrated, and captioned?
- Index - How well is it indexed (internal links, search engine)?
- Level - What is the level of topic treatment and who is the intended audience?
- Reliability - Is the information reliable, backed up by citations or links?
- Revisions - Are revisions/updates done frequently?
- Uniqueness - Does it make a unique contribution?
Paul Groves [16] lists four levels of online educational interactivity, with increasing complexity:
- Virtual lite - use of e-mail and/or discussion lists
- Virtual medium - above plus online lecture notes
- Virtual heavy - above plus interactive tutorials
- Virtual expert - above plus virtual environment (MUD, Multi-User Domain or MOO, Multi-user Object Oriented domain)
Dr Nancy Everhart of Duke University has a detailed "Web Page Evaluation Worksheet" [17] which details nine main categories on a 100-point scale: Currency, Content/information, Authority, Navigation, Experience, Multimedia, Treatment, Access, and Miscellaneous.
The WWW Cyber Guide contains a "WWW CyberGuide Ratings Guide for Content Evaluation" by Karen McLachlan [18]. The evaluation form uses a 5-point Likert scale to rate: Speed; First impression; Ease of navigation; Use of images, audio, video; Content; Currency; Contact person.
Dr. Gretchen Whitney has an online web page design workshop, "Caught in the Web: Web Page Design" [19] and "Webpage Evaluation Criteria." This details criteria and provides additional links for evaluating: Access, Aesthetics, Content, and Technical aspects on a 5-point Likert scale. General criteria for web sites are presented in the "Additional Resources" section following the References section.
VI. METHODOLOGY: IDENTIFICATION OF ONLINE SYLLABI
The UT College of Communications is composed of six main departments: Advertising, Broadcasting, Communications, Journalism, Public Relations, and Speech Communications, plus an allied unit, the School of Information Sciences.
According to Tennessee 101: A Guide to Courses at the University of Tennessee, Spring 1998 [1], there are one hundred nineteen classes in the College of Communications, ninety-one of which are undergraduate courses and twenty-eight of which are graduate courses.
To identify courses with online syllabi, the UTK web server was searched for relevant course names and instructor names. Out of twelve Advertising classes, three, or 25%, have online syllabi; four of fourteen Broadcasting classes, or 29%, have online syllabi; nine of twenty-two Communications classes, or 41%, have online syllabi; six of fourteen Journalism classes, or 43%, have online syllabi; four of six Public Relations classes, or 67%, have online syllabi; and, seven of thirty Speech Communications classes, or 23%, have online syllabi; and, fourteen of twenty-one Information Sciences classes, or 67%, have online syllabi.
Public Relations classes and Information Sciences classes have the highest percentage of courses with online syllabi (67%), and Speech Communications classes have the lowest percentage of courses with online syllabi (25%). (See Figure 1.)
Figure 1. UT College of Communications: Number of Courses by Department.
VII. METHODOLOGY: EVALUATION OF ONLINE SYLLABI
The usefulness of an academic course syllabus depends upon the value of its content and design. While there are no established standards for online syllabi, the established criteria for course syllabi content may be combined with the established criteria for webpage design to create an instrument to evaluate online syllabi.
Chan's "Criteria for an Inclusive Syllabus" [5] was synthesized with Whitney's "Web Evaluation" criteria [19] into a detailed checklist under ten main topics, with each topic worth ten points.
Seven of the main evaluation categories address syllabus content: Course Information, Instructor Information, Texts, Course Description, Course Policies, Grading, and Schedule. Three evaluation categories address technical and design aesthetics of an online syllabus: Access, Aesthetics, Technical. In addition, a bonus category was included with a point added for each additional interactive feature not listed in the above categories such as an online forum, downloadable software, or an interactive quiz. The online syllabi were graded based on the 100-point scale (see Table 1) and averaged according to sub-discipline (see Figure 2).
|
| Criteria | None/Poor 0 points | Adequate 1 point | Excellent |
| Access | Table of contents |
|
|
|
|
| Internal links |
|
|
|
|
| Legible fonts |
|
|
|
|
| Uniform style, layout |
|
|
|
|
| Logical title |
|
|
|
| Aesthetics | Well-organized |
|
|
|
|
| Logical flow |
|
|
|
|
| Visual appeal, wise use of color & graphics |
|
|
|
|
| Balance of color & white space |
|
|
|
|
| Variety, balance of font sizes |
|
|
|
| Technical | Correct spelling, grammar |
|
|
|
|
| Correct HTML code (not browser specific) |
|
|
|
|
| Link to instructor |
|
|
|
|
| Current update |
|
|
|
|
| Hot, usable links |
|
|
|
| Course Information | Parent institution |
|
|
|
|
| Course title & number |
|
|
|
|
| Section number |
|
|
|
|
| Class location(s) |
|
|
|
|
| Class/lab hours |
|
|
|
| Instructor Information | Full name |
|
|
|
|
| Title, education |
|
|
|
|
| Office location & hours |
|
|
|
|
| Office and/or home phone |
|
|
|
|
| E-mail address |
|
|
|
| Text Information | Required text(s) |
|
|
|
|
| Recommended text(s) |
|
|
|
|
| Full citation(s) |
|
|
|
|
| Cost |
|
|
|
|
| Where available |
|
|
|
| Course Information | Course description |
|
|
|
|
| Course objectives |
|
|
|
|
| Instruction methods |
|
|
|
|
| Pre-requisites |
|
|
|
|
| E-mail requirement |
|
|
|
| Course Policies | Attendance |
|
|
|
|
| Participation |
|
|
|
|
| Missed exams |
|
|
|
|
| Late assignments |
|
|
|
|
| Plagerism |
|
|
|
| Grading | Course assignments |
|
|
|
|
| Grading criteria |
|
|
|
|
| Grading scale |
|
|
|
|
| Number value of each item |
|
|
|
|
| % value of each item |
|
|
|
| Schedule | List of topics by week |
|
|
|
|
| Resources linked to topics |
|
|
|
|
| Assignment due dates |
|
|
|
|
| Exam dates |
|
|
|
|
| Holidays, off days |
|
|
|
| Bonus Points | Downloadable software |
|
|
|
|
| Interactive assignments |
|
|
|
|
| Class forum or listserv |
|
|
|
|
| Student work posted on website |
|
|
|
3 point rating scale, 100 points total ,
0 = none or very poor quality; 1 = adequate or medium quality; 2 = excellent or very good quality
Table 1. Academic Syllabi Evaluation Instrument.
Figure 2. UT College of Communications: Online Syllabi Evaluation Score by Department.
VIII. FINDINGS
Using the evaluation instrument that synthesizes syllabus content criteria and webpage design critera (see Table 1), Public Relations online syllabi rated the highest average score, 82%. Information Sciences averaged the next highest score, 80%; then, Journalism with an average score of 77%. Communications and Speech Communication each had an average score of 72%; Broadcasting, an average score of 68%; and Advertising, an average score of 53% (see Figure 2).
It appears from this case study that the higher the percentage of department courses with online syllabi, the higher the average quality of those syllabi. Two departments, Public Relations and Information Sciences, have the highest percentage of courses with online syllabi and the highest average online syllabi evaluation score (see Figure 3).
Figure 3. UT College of Communications Plot of Online Syllabi Evaluation Score by Percent of Courses with Online Syllabi.
IX. CONCLUSION
The evaluation of online course syllabi is another form of peer review within the academic community. Peer review of course syllabi recognizes quality of content and design and promotes new and innovative uses of course syllabi for curriculum and instruction in an online environment.
This case study indicates that the higher the percentage of department courses with online syllabi, the higher the quality of those syllabi. Within The University of Tennessee College of Communications, the Public Relations Department and the School of Information Sciences have the highest percentage of courses with online syllabi and the highest average evaluation score for online syllabi. Shared factors of the highest rated online syllabi include connectivity to relevant online resources and a high degree of interactivity.
The Public Relations Department and the School of Information Sciences were early adopters of webpage technology and both departments offer webpage development courses, which may have led to heightened awareness and expectations within their departments to develop online syllabi.
As more academic courses develop an online presence, there is an increasing need to evaluate form and content in order to increase quality and usability. An effective online syllabus includes standard syllabus information that students require and online resources such as tutorials, relevant links and online interactivity. An evaluation instrument such as the one developed for this study can be useful as a guide in developing and evaluating effective online syllabi.
X. REFERENCES
- The University of Tennessee, Tennessee 101: A Guide to Courses at the University of Tennessee, Spring 1998. Available: http://www.it.utk.edu/~tenn101/
- Appleby, D.C., How to Improve Your Teaching with the Course Syllabus, APS Observer, 18-19, 26, 1994.
- Rubin, S., Professors, Students, and the Syllabus, The Chronicle of Higher Education, 56, August 7, 1985.
- Gabennesch, H., The Enriched Syllabus: To Convey a Larger Vision, The National Teaching & Learning Forum, 4-5, 1992.
- Chan, G., Getting Started: Designing the Inclusive Syllabus, 1997. Available: http://www.cstudies.ubc.ca/facdev/services/newsletter/97/dessyllabus.html
- McLuhan, M., The Medium is the Message: An Inventory of Effects, New York, Bantam Press, 1967.
- Riegle, R.P., Lessons from a CyberGuide, 1998. Available: http://coe.ilstu.edu/rpriegle/eaf228/lessons.htm
- Stanford University Center for Teaching and Learning, Technology and Teaching: Using Online Technology to Break Classroom Boundaries, Speaking of Teaching: Stanford University Newsletter on Teaching, 8, Fall 1996.
- Brians, P., Wyche, S. and Condon, B., (Eds.), Vision of Virtual Washington State University: An Alternative, 1998.
- Jonassen D., Designing Hypertext for Learning, New Directions in Educational Technology, Berlin, Springer-Verlag, 1992.
- Kearsley, G., Authoring Considerations for Hypertext, Educational Technology, 21-24, November 1988.
- Chan, T., Lin, C., Lin, S., and Kuo, H., OCTR: A Model of Learning Stages, Proceedings of AI-ED 93, August 1993.
- Gingher, R.S., IT's Challenge to Literature and "Cognitive Dissonance," Computers & Texts, April 1998. Available: http://info.ox.ac.uk/ctitext/publish/comtxt/ct16-17/gingher.html
- Rettig, J., Rettig on Reference, 1997. Available: http://www.hwwilson.com/retintro.html
- Rettig, J., Beyond "Cool:" Analog Models for Reviewing Digital Resources, Online, September 1996. Available: http://www.onlineinc.com/onlinemag/SeptOL/rettig9.html
- Groves, P., Internet Teaching: Some Basic Concepts, 1997. Available: http://info.ox.ac.uk/jtap/reports/teaching/basic.html
- Everhart, N., Web Page Evaluation Worksheet, 1996. Available: http://www.duke.edu/~de1/evaluate.html
- McLachlan, K., WWW CyberGuide Ratings for Content Evaluation, WWW CyberGuide, 1996. Available: http://www.cyberbee.com/guide1.html
- Whitney, G., Caught in the Web: Web Page Design, 1998. Available: http://web.utk.edu/~gwhitney/design.html
XI. ADDITIONAL RESOURCES
- Berkeley Digital Library's Guidelines for Web Document Style & Design
http://sunsite.berkeley.edu/Web/guidelines.html - How Users Use the Web by Jakob Nielsen. Excellent article on how to write for the Web. http://www.useit.com/alertbox/9710a.html
- Multi-User Domains (MUDs). Educational Virtual Reality Sites.
http://tecfa.unige.ch/edu-comp/WWW-VL/eduVR-page.html - World Lecture Hall. Links to academic syllabi by subject.
http://www.utexas.edu/world/lecture/ - WWW Teach. A list of online resources for academic web development.
http://www.massey.ac.nz/~wwcues/teaching/www-teach.htm#papers
XII. ABOUT THE AUTHOR
Teresa Welsh is a doctoral candidate and a Graduate Teaching Associate in The University of Tennessee College of Communications. She has a Master of Library and Information Science Degree from The University of Tennessee and is currently working in the UT School of Information Sciences Distance Education Program. Research interests include visualization of information, telemedicine, and the development and evaluation of educational web sites. Professional memberships include American Library Association, American Society for Information Science, Association of Library and Information Science Education, American Telemedicine Association, Visual Resources Association.