An Evaluation of Online Syllabi in The University of Tennessee College of
Communications
Teresa Welsh
School of Information Sciences
The University of Tennessee
Knoxville, TN 37996-0003
URL: http://web.utk.edu/~twelsh/
Phone: 865-573-5030
Fax: 865-974-4967
I. INTRODUCTION
As more academic courses develop an online presence, there is an increasing
need to evaluate form and content in order to increase quality and usability. An
evaluation instrument was developed that synthesizes website design criteria
with syllabus development criteria. This case study used the instrument to
evaluate online syllabi in five departments within the University of Tennessee
College of Communications: Advertising, Broadcasting, Communications,
Journalism, and Speech Communications, plus the allied School of Information
Sciences. This case study indicates that the higher the percentage of department
courses with online syllabi, the higher the quality of those syllabi. Shared
factors of the highest rated online syllabi include connectivity to relevant
online resources and a high degree of interactivity. Within The University of
Tennessee College of Communications, the Public Relations Department and the
School of Information Sciences have the highest percentage of courses with
online syllabi and the highest average evaluation score for online syllabi. One
shared factor of these two departments is the offering of at least one course in
webpage development.
The purpose of this case study is to determine how many College of
Communications classes listed in Tennessee 101: A Guide to Courses at the
University of Tennessee [1] have online syllabi, and to evaluate the content of
those syllabi. An evaluation instrument is proposed that synthesizes syllabus
development criteria and webpage design criteria.
II. SYLLABUS DEFINITION AND FUNCTIONS
The word "syllabus" is from the Greek sillybos, literally
"parchment label." Originally it meant the summary label of a
parchment scroll, but it came to mean a summary or outline of a course of
academic study.
According to Appleby [2] and Rubin [3] the course syllabus serves at least seven
functions:
- Helps to plan and strengthen the course. This is what Gabennesch [4] calls
the enriched syllabus.
- Reveals the implicit assumptions about the course.
- Introduces the instructor to students.
- Explains why students should take the course - the purpose or value of the
course.
- Explains the various aspects of the course including assignments, exams,
and grading policies.
- Explains how students will change as a result of the course, including
knowledge and/or skills they will acquire.
- Communicates the nature and content of the course to colleagues and
administration.
- Provides a documented record of an instructor's teaching career.
Greg Chan [5] lists possible content areas of a syllabus, which he suggests
could be modified to suit individual needs:
- Course information: institution, term, course title & number, section
number, location, class and lab times.
- Instructor information: name, title, contact information plus teaching
assistants and their contact information.
- Texts: cites title, author, preferred edition, publisher, date of
publication, cost, where available. May be divided into "Required"
and "Recommended."
- Course description: provides an overview of course content and objectives;
information about instructional methods, course prerequisite(s).
- Schedule of activities: provides a daily/weekly listing of topics,
readings, and assignments. Includes due dates for all exams, papers and
other means of assessment; holidays should also be noted.
- Course policies: makes it clear where you stand on attendance, lateness,
classroom conduct, participation, missed exams, late assignments, and
plagiarism.
- Grading system: clearly details the type, number, and % value of all
course assignments; explains evaluation scheme/marking criteria.
- Disclaimer: states that syllabus is tentative, and subject to change
according to the needs and interests of the class. This will increase
flexibility.
Chan adds [5] "a growing number of instructors are making their syllabi
available to students (both graduate and undergraduate) before the first day of
class, through mail-outs, pick-ups or even online access."
III. THE ONLINE ACADEMIC SYLLABUS
"When technology extends our senses, a new translation of culture
occurs
as swiftly as the new technology is interiorized."
Marshall McLuhan [6]
An online course syllabus adds a dynamic dimension beyond the scope of a
printed syllabus - online interactivity.
According to Dr. Rodney P. Riegle [7], "With the WWW, instructors now have
a tool which allows them to do what everyone has always known is the most
efficient and effective way to facilitate learning - learner-controlled
interactivity."
According to a survey of Stanford instructors, [8] those who had introduced
online technology into their courses believed that both teaching and learning
had improved. They reported that students were better prepared and more willing
to enter into class discussions. Those instructors who required students to
review online material and send feedback via e-mail, listserv, or class forum or
bulletin board were able to summarize student responses at the beginning of
class and begin discussion from a more advanced point.
Brians et al. [9] suggest that an online environment can become a
knowledge-making, dialogic environment, one in which students and faculty
interact with each other, one in which the idea of learning communities, active
learning, and engagement can evolve. The learning environments we construct
online must encourage synthesis, inquiry, analysis, etc.--students must move
beyond memorization to actually work with class materials.
Thus, the technologies we develop and employ must enable students to learn more
and to learn it better. In other words, we need to promote what we already know
are the most effective learning strategies - the two most significant factors in
persistence to degree are (1) interaction with peers and (2) interaction with
faculty. Other studies demonstrate that effective courses offer frequent
opportunities for students to show what they know and to get feedback on their
progress. So, for sound educational reasons, asynchronous, online education can
and should incorporate such features.
IV. HYPERMEDIA LEARNING THEORY
Some understanding of hypermedia learning theory is useful in the development
and evaluation of online course syllabi.
The effectiveness of hyperlinks and hypermedia on the World Wide Web has been
related to several learning theories. Jonassen [10] suggests that hypertext
reflects a model of learning based on schemas, the accumulation and organization
of knowledge as semantic memory objects connected by attributes.
Kearsley [11] writes that "hypertext matches human cognition; in particular
the organization of memory as a semantic network in which concepts are linked
together by associations."
A similar model also based on nodes and links by Chan et al. [12] is OCTR:
- Orientation, or relating prior knowledge
- Coaching, or apprenticeship
- Tuning
- Routinization
The learning process in this model is explained as the tentative connection
of old knowledge with new knowledge, the accretion of more links, the
articulation whereby some links are strengthened and some deleted, and the
solidification and reinforcement of surviving links.
"By engaging the associative structure of memory," Gingher [13]
writes, the new medium "compels a traditional linear, synchronous,
sequential and exclusive pedagogy to adapt to a new paradigm - one associative,
asynchronous, contextual, and collaborative."
V. EVALUATION CRITERIA OF ONLINE RESOURCES
An effective online syllabus adds additional elements of interactivity and
connectivity to online learning resources and requires an awareness of web
design principles and evaluation criteria.
James Rettig [14] of the College of William and Mary reviews general or academic
reference sites. Rettig [15] offers "Comparative Criteria for Reviewing
Reference Books and Web Sites" in which he adapts criteria for evaluating
print resources to online resources:
- Accuracy - Is the information presented accurate?
- Appropriateness - Is it appropriate for the intended audience?
- Arrangement - Is the arrangement appropriate and useful for the topic?
- Authority - What are the author's qualifications, and are they stated?
- Bibliography - Are the links "hot," useful, and appropriate?
- Comparability - How does it compare to others similar in scope and
purpose?
- Completeness - Does it cover its subject adequately?
- Content - What is the overall value of content? Does it contribute to its
field?
- Distinction - Does it have a distinctive character or attribute?
- Ease-of-use - Is it easy to use, well-organized with site map or index?
- Illustrations - What is the quality, how well are they integrated, and
captioned?
- Index - How well is it indexed (internal links, search engine)?
- Level - What is the level of topic treatment and who is the intended
audience?
- Reliability - Is the information reliable, backed up by citations or
links?
- Revisions - Are revisions/updates done frequently?
- Uniqueness - Does it make a unique contribution?
Paul Groves [16] lists four levels of online educational interactivity, with
increasing complexity:
- Virtual lite - use of e-mail and/or discussion lists
- Virtual medium - above plus online lecture notes
- Virtual heavy - above plus interactive tutorials
- Virtual expert - above plus virtual environment (MUD, Multi-User Domain or
MOO, Multi-user Object Oriented domain)
Dr Nancy Everhart of Duke University has a detailed "Web Page Evaluation
Worksheet" [17] which details nine main categories on a 100-point scale:
Currency, Content/information, Authority, Navigation, Experience, Multimedia,
Treatment, Access, and Miscellaneous.
The WWW Cyber Guide contains a "WWW CyberGuide Ratings Guide for Content
Evaluation" by Karen McLachlan [18]. The evaluation form uses a 5-point
Likert scale to rate: Speed; First impression; Ease of navigation; Use of
images, audio, video; Content; Currency; Contact person.
Dr. Gretchen Whitney has an online web page design workshop, "Caught in the
Web: Web Page Design" [19] and "Webpage Evaluation Criteria."
This details criteria and provides additional links for evaluating: Access,
Aesthetics, Content, and Technical aspects on a 5-point Likert scale. General
criteria for web sites are presented in the "Additional Resources"
section following the References section.
VI. METHODOLOGY: IDENTIFICATION OF ONLINE SYLLABI
The UT College of Communications is composed of six main departments:
Advertising, Broadcasting, Communications, Journalism, Public Relations, and
Speech Communications, plus an allied unit, the School of Information Sciences.
According to Tennessee 101: A Guide to Courses at the University of
Tennessee, Spring 1998 [1], there are one hundred nineteen classes in the
College of Communications, ninety-one of which are undergraduate courses and
twenty-eight of which are graduate courses.
To identify courses with online syllabi, the UTK web server was searched for
relevant course names and instructor names. Out of twelve Advertising
classes, three, or 25%, have online syllabi; four of fourteen Broadcasting
classes, or 29%, have online syllabi; nine of twenty-two Communications classes,
or 41%, have online syllabi; six of fourteen Journalism classes, or 43%, have
online syllabi; four of six Public Relations classes, or 67%, have online
syllabi; and, seven of thirty Speech Communications classes, or 23%, have online
syllabi; and, fourteen of twenty-one Information Sciences classes, or 67%, have
online syllabi.
Public Relations classes and Information Sciences classes have the highest percentage
of courses with online syllabi (67%), and Speech Communications classes have
the lowest percentage of courses with online syllabi (25%). (See Figure 1.)
Figure 1. UT College of Communications: Number of Courses by Department.
VII. METHODOLOGY: EVALUATION OF ONLINE SYLLABI
The usefulness of an academic course syllabus depends upon the value of its
content and design. While there are no established standards for online syllabi,
the established criteria for course syllabi content may be combined with the
established criteria for webpage design to create an instrument to evaluate
online syllabi.
Chan's "Criteria for an Inclusive Syllabus" [5] was synthesized with
Whitney's "Web Evaluation" criteria [19] into a detailed checklist
under ten main topics, with each topic worth ten points.
Seven of the main evaluation categories address syllabus content: Course
Information, Instructor Information, Texts, Course Description, Course Policies,
Grading, and Schedule. Three evaluation categories address technical and
design aesthetics of an online syllabus: Access, Aesthetics, Technical. In
addition, a bonus category was included with a point added for each additional
interactive feature not listed in the above categories such as an online forum,
downloadable software, or an interactive quiz. The online syllabi were graded
based on the 100-point scale (see Table 1) and averaged according to
sub-discipline (see Figure 2).
| |
Criteria |
None/Poor
0 points
|
Adequate
1 point
|
Excellent
2 points |
|
Access |
Table of contents
|
|
|
|
|
|
Internal links |
|
|
|
|
|
Legible fonts |
|
|
|
|
|
Uniform style, layout
|
|
|
|
|
|
Logical title |
|
|
|
|
Aesthetics |
Well-organized |
|
|
|
|
|
Logical flow |
|
|
|
|
|
Visual appeal, wise use of color
& graphics |
|
|
|
|
|
Balance of color & white
space |
|
|
|
|
|
Variety, balance of font sizes
|
|
|
|
|
Technical |
Correct spelling, grammar
|
|
|
|
|
|
Correct HTML code (not browser
specific) |
|
|
|
|
|
Link to instructor
|
|
|
|
|
|
Current update |
|
|
|
|
|
Hot, usable links
|
|
|
|
|
Course Information |
Parent institution
|
|
|
|
|
|
Course title & number
|
|
|
|
|
|
Section number |
|
|
|
|
|
Class location(s)
|
|
|
|
|
|
Class/lab hours
|
|
|
|
|
Instructor Information |
Full name |
|
|
|
|
|
Title, education
|
|
|
|
|
|
Office location & hours
|
|
|
|
|
|
Office and/or home phone
|
|
|
|
|
|
E-mail address |
|
|
|
|
Text Information |
Required text(s)
|
|
|
|
|
|
Recommended text(s)
|
|
|
|
|
|
Full citation(s)
|
|
|
|
|
|
Cost |
|
|
|
|
|
Where available
|
|
|
|
|
Course Information |
Course description
|
|
|
|
|
|
Course objectives
|
|
|
|
|
|
Instruction methods
|
|
|
|
|
|
Pre-requisites |
|
|
|
|
|
E-mail requirement
|
|
|
|
|
Course Policies |
Attendance |
|
|
|
|
|
Participation |
|
|
|
|
|
Missed exams |
|
|
|
|
|
Late assignments
|
|
|
|
|
|
Plagerism |
|
|
|
|
Grading |
Course assignments
|
|
|
|
|
|
Grading criteria
|
|
|
|
|
|
Grading scale |
|
|
|
|
|
Number value of each item
|
|
|
|
|
|
% value of each item
|
|
|
|
|
Schedule |
List of topics by week
|
|
|
|
|
|
Resources linked to topics
|
|
|
|
|
|
Assignment due dates
|
|
|
|
|
|
Exam dates |
|
|
|
|
|
Holidays, off days
|
|
|
|
|
Bonus Points |
Downloadable software
|
|
|
|
|
|
Interactive assignments
|
|
|
|
|
|
Class forum or listserv
|
|
|
|
|
|
Student work posted on website
|
|
|
|
3 point rating scale, 100 points total ,
0 = none or very poor quality; 1 = adequate or medium quality; 2 = excellent
or very good quality
Table 1. Academic Syllabi Evaluation Instrument.
Figure 2. UT College of Communications: Online Syllabi Evaluation Score by Department.
VIII. FINDINGS
Using the evaluation instrument that synthesizes syllabus
content criteria and webpage design critera (see Table 1), Public Relations
online syllabi rated the highest average score, 82%. Information Sciences
averaged the next highest score, 80%; then, Journalism with an average score of
77%. Communications and Speech Communication each had an average score of 72%;
Broadcasting, an average score of 68%; and Advertising, an average score of 53%
(see Figure 2).
It appears from this case study that the higher the percentage of department
courses with online syllabi, the higher the average quality of those syllabi.
Two departments, Public Relations and Information Sciences, have the highest
percentage of courses with online syllabi and the highest average online syllabi
evaluation score (see Figure 3).
Figure 3. UT College of Communications Plot of Online Syllabi Evaluation Score
by Percent of Courses with Online Syllabi.
IX. CONCLUSION
The evaluation of online course syllabi is another form of peer review within
the academic community. Peer review of course syllabi recognizes quality of
content and design and promotes new and innovative uses of course syllabi for
curriculum and instruction in an online environment.
This case study indicates that the higher the percentage of department courses
with online syllabi, the higher the quality of those syllabi. Within The
University of Tennessee College of Communications, the Public Relations
Department and the School of Information Sciences have the highest percentage of
courses with online syllabi and the highest average evaluation score for online
syllabi. Shared factors of the highest rated online syllabi include connectivity
to relevant online resources and a high degree of interactivity.
The Public Relations Department and the School of Information Sciences were
early adopters of webpage technology and both departments offer webpage
development courses, which may have led to heightened awareness and expectations
within their departments to develop online syllabi.
As more academic courses develop an online presence, there is an increasing need
to evaluate form and content in order to increase quality and usability. An
effective online syllabus includes standard syllabus information that students
require and online resources such as tutorials, relevant links and online
interactivity. An evaluation instrument such as the one developed for this study
can be useful as a guide in developing and evaluating effective online syllabi.
REFERENCES
- The University of Tennessee, Tennessee 101: A Guide to Courses
at the University of Tennessee, Spring 1998. Available: http://www.it.utk.edu/~tenn101/
- Appleby, D.C., How to Improve Your
Teaching with the Course Syllabus, APS Observer, 18-19, 26, 1994.
- Rubin, S., Professors, Students,
and the Syllabus, The Chronicle of Higher Education, 56, August 7, 1985.
- Gabennesch, H., The Enriched Syllabus:
To Convey a Larger Vision, The National Teaching & Learning Forum, 4-5,
1992.
- Chan, G., Getting Started: Designing
the Inclusive Syllabus, 1997. Available: http://www.cstudies.ubc.ca/facdev/services/newsletter/97/dessyllabus.html
- McLuhan, M., The Medium is the Message: An Inventory
of Effects, New York, Bantam Press, 1967.
- Riegle, R.P., Lessons from a CyberGuide,
1998. Available: http://coe.ilstu.edu/rpriegle/eaf228/lessons.htm
- Stanford University Center for Teaching and Learning, Technology
and Teaching: Using Online Technology to Break Classroom Boundaries, Speaking
of Teaching: Stanford University Newsletter on Teaching, 8, Fall 1996.
- Brians, P., Wyche, S. and Condon, B.,
(Eds.), Vision of Virtual Washington State University: An Alternative, 1998.
- Jonassen D., Designing Hypertext for Learning, New
Directions in Educational Technology, Berlin, Springer-Verlag, 1992.
- Kearsley, G., Authoring Considerations for Hypertext,
Educational Technology, 21-24, November 1988.
- Chan, T., Lin, C., Lin, S., and Kuo, H.,
OCTR: A Model of Learning Stages, Proceedings of AI-ED 93, August 1993.
- Gingher, R.S., IT's Challenge
to Literature and "Cognitive Dissonance," Computers & Texts,
April 1998. Available: http://info.ox.ac.uk/ctitext/publish/comtxt/ct16-17/gingher.html
- Rettig, J., Rettig on Reference,
1997. Available: http://www.hwwilson.com/retintro.html
- Rettig, J., Beyond "Cool:"
Analog Models for Reviewing Digital Resources, Online, September 1996.
Available: http://www.onlineinc.com/onlinemag/SeptOL/rettig9.html
- Groves, P., Internet Teaching:
Some Basic Concepts, 1997. Available: http://info.ox.ac.uk/jtap/reports/teaching/basic.html
- Everhart, N., Web Page Evaluation
Worksheet, 1996. Available: http://www.duke.edu/~de1/evaluate.html
- McLachlan, K., WWW CyberGuide Ratings
for Content Evaluation, WWW CyberGuide, 1996. Available: http://www.cyberbee.com/guide1.html
- Whitney, G., Caught in the Web:
Web Page Design, 1998. Available: http://web.utk.edu/~gwhitney/design.html
ADDITIONAL RESOURCES
- Berkeley Digital Library's Guidelines for Web Document Style & Design
http://sunsite.berkeley.edu/Web/guidelines.html
- How Users Use the Web by Jakob Nielsen. Excellent article on how to write
for the Web. http://www.useit.com/alertbox/9710a.html
- Multi-User Domains (MUDs). Educational Virtual Reality Sites.
http://tecfa.unige.ch/edu-comp/WWW-VL/eduVR-page.html
- World Lecture Hall. Links to academic syllabi by subject.
http://www.utexas.edu/world/lecture/
- WWW Teach. A list of online resources for academic web development.
http://www.massey.ac.nz/~wwcues/teaching/www-teach.htm#papers
ABOUT THE AUTHOR
Teresa Welsh is a doctoral candidate and a Graduate Teaching Associate
in The University of Tennessee College of Communications. She has a Master of
Library and Information Science Degree from The University of Tennessee and is
currently working in the UT School of Information Sciences Distance Education
Program. Research interests include visualization of information, telemedicine,
and the development and evaluation of educational web sites. Professional
memberships include American Library Association, American Society for
Information Science, Association of Library and Information Science Education,
American Telemedicine Association, Visual Resources Association.
|
