THE ILLINOIS ONLINE NETWORK IS MAKING THE VIRTUAL CLASSROOM A REALITY: STUDY OF AN EXEMPLARY FACULTY DEVELOPMENT PROGRAM

Download PDF version:

Michael Lindeman
mlindema@uillinois.edu
Computer Assisted Instruction Specialist
Illinois Online Network, University of Illinois
510 Devonshire Dr., Suite H
Champaign, IL 61820
Telephone: (217) 333-3746
Fax: (217) 333-5581

Iris K. Stovall
istovall@uillinois.edu
Director
Illinois Online Network, University of Illinois
510 Devonshire Dr., Suite H
Champaign, IL 61820
Telephone: (217) 333-4393
Fax: (217) 333-5581

ABSTRACT
The Illinois Online Network (ION) is a faculty development partnership between all forty-eight community colleges in the state of Illinois and the University of Illinois. The goals of the ION program are to help faculty to develop and deliver courses in a completely online format, and also to produce online courses that incorporate best practices for engaging students in discussion and critical thinking. ION accomplishes its goals with a variety of programs, such as the Making the Virtual Classroom a Reality series of online faculty development courses. Evaluation surveys indicate that ION activities have had an impact on the satisfaction and confidence of faculty teaching online courses. This paper examines the programs and resources that ION provides, the effectiveness of the program as a whole, and the lessons ION has learned about providing a large-scale faculty development program.

KEY WORDS
Illinois Online Network, Faculty Satisfaction, Virtual Classroom, Faculty Development, Learning Effectiveness, Course Design

I. INTRODUCTION

The explosive growth of the Internet, permitting access to information and improved interpersonal communication, has fundamentally altered the face of higher education in America. The number of traditional, face-to-face college courses that use online learning materials and network communications tools continues to grow substantially each year. The Internet has a profound impact in the realm of distance education and lifelong learning. In just a few years, the Internet has become the predominant distance education medium, outpacing growth in other delivery modes. At the same time, the demand for distance education, especially among adult learners, is growing rapidly. A publication from the National Center for Education Statistics (NCES), The Condition of Education 2003 [1], reports “participation in adult education among those age 16 and above increased to 47 percent in 2001 from 34 percent in 1991 and from 42 percent in 1995.” Another publication from the NCES, A Profile of Participation in Distance Education: 1999–2000 [2], found that “During the 1990s, distance education availability, course offerings, and enrollments increased rapidly. The percentage of 2-and 4-year degree-granting institutions offering distance education courses rose from 33 to 44 percent between 1995 and 1997, and the number of such courses nearly doubled.”

Within the state of Illinois, dramatic growth trends are evident in distance education data gathered by the Illinois Virtual Campus (IVC) [3]. For example, the number of distance education courses offered in Illinois increased from 962 in Fall 1999 to 5,740 in Fall 2002, and course enrollments rose sharply from 14,689 to 69,213 during the same period. More than half of the courses in the IVC catalog are delivered over the Internet, accounting for almost 60% of the distance education enrollments in Illinois. In the Fall 2002 term, online enrollments statewide increased 74% from the previous fall term. It is clearly no longer a question of whether the Internet has a role within the higher education community, but rather how to use Internet-based technologies most effectively and most efficiently and how to ensure that online courses and programs are of the highest possible quality. If Illinois is to prepare its citizens to thrive in the coming years, its college and university faculty must learn how to be effective online educators. To this end, the University of Illinois conceived and manages the Illinois Online Network (ION) [4], a statewide faculty development program.

ION is a partnership between all forty-eight community colleges in the state of Illinois and the University of Illinois. Funding for the program came initially from the state of Illinois Board of Higher Education through a Higher Education Cooperation Act grant. In 2002, funding became part of the University of Illinois budget.

Each community college partner has selected a Steering Committee member who provides guidance to ION at three annual face-to-face meetings, and in one conference call. The Steering Committee member is involved with online learning at his/her campus and able to provide input on the direction and scope of ION projects. In addition, most colleges have appointed a liaison with whom the ION staff interact. Because ION staff members do not work on the campuses they serve, it is important to have direct contact with campus representatives to ensure that ION’s programs are appropriate and effective.

The goals of the ION program are not only to help faculty to develop and deliver courses in a completely online format, but also to produce online courses that incorporate best practices for engaging students in discussion and critical thinking.

Additional goals of the ION initiative include:

• Increasing faculty interest in and receptiveness to the benefits of online learning as a means of achieving improved learning outcomes
• Producing faculty members skilled in and knowledgeable about facilitating high quality online courses that incorporate active and collaborative learning
• Assisting member institutions in the support of online academic programming
• Identifying and sharing best practices related to online teaching and learning

ION accomplishes its goals through the following activities:

• The Making the Virtual Classroom a Reality series of online courses for faculty, aimed at developing an understanding of online pedagogies and technologies
• A Master Online Teacher certificate program
• On-site campus visits, involving a combination of faculty workshops, private consulting, and meetings with faculty, administrators, and technical personnel
• An annual faculty development institute, involving over 170 faculty from ION member institutions
• A Resource-rich website

This paper examines the programs and resources that ION provides, the effectiveness of the program as a whole, and the lessons ION has learned about providing a large-scale faculty development program.

II. OVERVIEW OF THE ION PROGRAM

The fastest growing component of the ION program, and the component with the potential to expose the largest number of faculty, staff and administrators to pedagogically sound principles of teaching and learning, is the Making the Virtual Classroom a Reality (MVCR) series of online faculty development courses [5]. The MVCR courses are designed to help faculty members acquire the skills and knowledge needed to teach online. There are ten courses that cover many aspects of online education. All MVCR courses are delivered entirely online and rely on asynchronous discussion and collaboration as the principal form of communication. MVCR courses are taught by the ION staff, and by instructors who have taken MVCR courses and demonstrated exceptional skill at facilitating student-centered online courses. Instructors model the strategies that participants will eventually use in their own online courses, and the online format gives faculty new to online teaching an opportunity to experience the virtual classroom from students’ perspectives. Students interact with one another and with the instructor extensively via a web-based conferencing system. Because MVCR courses are delivered asynchronously over an extended period, instructors and participants can reflect on course content in far greater depth and more thoroughly than they can accomplish in a 3-hour face-to-face workshop. MVCR courses are available for Continuing Education Units (CEUs) through the University of Illinois at Urbana-Champaign, or for college course credit through the University of Illinois at Springfield [6].

Currently, MVCR offers the following courses:

Core Courses:
Online Learning: An Overview
Technology Tools for Online Learning
Instructional Design for Online Course Development
Student Assessment in Online Courses
Encouraging Communication in Online Courses

Elective Courses:
Copyright and Intellectual Property Issues for Online Courses
Issues and Strategies for Faculty Training
Multimedia Principles for Online Educators
Web Design Principles for Online Educators

The Master Online Teacher Certificate (MOT) [7] program recognizes and certifies faculty, staff, and administrators who have achieved a measurable level of knowledge related to online course design, online instruction, and other issues related to online teaching and learning. The Master Online Teacher Certificate is designed for faculty, administrators, and staff members interested in making online teaching an important part of their academic careers, or who wish to take a leadership role in online learning at their institutions. To earn the certificate, participants must take four core courses, an elective course, complete all assigned work, receive a minimum score of 80% in each course and successfully complete the Practicum. There are currently 391 faculty members from around the world enrolled in ION’s MOT program, and eighty-one faculty have completed the MOT certificate. Faculty have indicated support and enthusiasm for the certificate program that enables them to be certified as master online teachers without paying large fees for courses.

As the capstone in the Master Online Teacher certification program, the Practicum is a project-based workshop in which participants work closely with an MVCR instructor to develop or enhance an online course. Individuals demonstrate their practical knowledge of online course design and creation by incorporating appropriate roles for faculty and students, choosing technology suitable to the audience, creating and assigning effective summative and formative online assessments, demonstrating the communicative nature of online courses, and following sound online design principles. In addition, participants practice facilitating online discussions under the mentorship of the MVCR instructor team. Enrollment is limited to a small number of participants in order to maximize the benefits of a collaborative learning environment in which group members rigorously cross-critique each other’s course designs. All of the MOT candidates are already experienced teachers in the traditional classroom setting, and they bring a wealth of practical experience and pedagogical expertise to the group. Their experience combined with their MVCR coursework, leads to evaluative and formative feedback from a much greater range and number of perspectives than the instructor team alone could provide.

In the process of completing the MOT certificate, participants must have demonstrated knowledge in the following areas related to online teaching and learning:

• Roles of faculty in online courses
• Roles of students in online courses
• Communications in online courses
• Collaborative nature of online courses
• Technologies available for delivering online courses
• Methods of assessment for online courses
• Learning outcomes in online courses
• Learning activities in online courses
• Evaluation of online courses

ION has developed a personalized web portal that MVCR course participants use to track their progress in the courses they take. Participants login to the MVCR server to access their MyMVCR account. They can then register for courses, complete course orientations, view a history of the courses they have taken, and edit their profiles. ION administrators can view statistics and enrollment data, and ION instructors can view class rosters and a history of the courses they have taught.

Two of ION’s primary face-to-face activities are presentations and workshops (mostly delivered on-site, but some are delivered via tele- and video-conferencing), and the Faculty Summer Institute (FSI). The ION staff delivers presentations and workshops at individual community colleges, and at regional gatherings. Over 300 presentations from the past five years are listed on its website [8] on topics ranging from educational copyright issues and web programming to pedagogy and instructional design.

One of the largest ION activities is the yearly FSI [9]. The 2.5-day FSI brings together over 170 community college faculty and staff from Illinois to discuss and learn about a wide-range of topics on the integration of networked information technologies into the curriculum. Registration at the FSI is free to Illinois community colleges faculty and staff. It is aimed at faculty who are new to online learning, introduces faculty members to the use of learning technologies, and provides an opportunity for faculty teaching in similar disciplines to share best practices.

In addition to offering MVCR courses and face-to-face presentations, the ION staff provides assistance and advice through e-mail and by telephone to faculty interested in specific information or in need of advice with the design of their courses.

ION has a well established Website that supplements face-to-face activities [4]. Like most websites, ION provides an extensive list of links to other resources which are reviewed for quality, categorized, and have useful descriptions [10]. The ION website also includes resources created by the ION staff. The resources include tutorials on a wide range of computer applications, a library of presentations and workshop files [8], pedagogical and andragogical resources, instructional design topics, intellectual property resources, and many others. A primary resource is the Pointer and Clicker Technology Tip of the Month, which presents original information on topics of current interest in online education [11]. ION case studies also provide a selection of original research in the area of online or technology-enhanced education [12].

ION has recently begun a yearly Train the Trainers meeting based on feedback received at the FSI that indicated support for trainers in Illinois was an important component of ION’s mission. The meetings work synergistically to bring together people from across the state of Illinois who are willing to share training ideas and tips. A listserv allows the technology trainers to share ideas during the year.

III. THE EFFECTIVENESS OF THE ION PROGRAM

There are established methods to evaluate training and online learning programs [13–19] and to evaluate online courses [20-22] which ION considered to evaluate its programs. Online and paper survey forms are used to validate the value of MVCR, FSI, and individual workshops. In addition to these single event sources, ION periodically performs overall program evaluations. A program impact study evaluates the long-term effects the program has had on participants, and the extent to which participants have been able to transfer what they have learned to their own work context. Web log analysis and peer feedback on the quality of web resources provide information about the usefulness of web resources.

A. Making the Virtual Classroom a Reality and the Master Online Teacher Certificate Program

Participants are encouraged to complete anonymous post-course evaluation surveys that provide feedback on the course design and organization, and also on the instructor performance in the course. Compiled course evaluations are available online [23]. A sample of the results for Online Learning: An Overview follows:

• 94% of respondents indicated that the course overall was EXCELLENT or GOOD.
• 83% of respondents indicated that they would STRONGLY AGREE or AGREE that there was an appropriate amount of participation among classmates.
• 96% of respondents STRONGLY AGREED or AGREED that the course required them to engage in analysis, synthesis and evaluation.
• 91% of respondents STRONGLY AGREED or AGREED that the course gave them skills and techniques directly applicable to their jobs.
• 96% of respondents indicated that they would take and MVCR course again.

In the spring of 2003 all MVCR courses underwent peer review and usability testing. To improve quality, all courses have since been altered to reflect the feedback from the reviews.

In July of 2002, ION conducted a survey to measure the satisfaction and long term effects of MVCR training on faculty who registered for an MVCR course [24]. The survey focused on three areas in which MVCR training might have affected instructors’ online courses.

First, faculty satisfaction with online teaching was surveyed. A t-test conducted on the differences between before training and after training responses showed that the MVCR program had a highly significant impact on increasing faculty satisfaction with online instruction. (Figure I and Table I)

The second set of questions examined the confidence faculty felt when teaching online to determine whether the level of knowledge gained was sufficient to instill confidence. A t-test conducted on the differences between before training and after training responses was highly significant, showing an increase in faculty confidence after participating in the program. (Figure II and Table II)

The level of interaction in an online course is generally considered to be a key indicator of quality. As Palloff writes “… it is the interaction and connections that students will remember as the keys to learning in an online course” [25]. The third focus area of the survey included questions to determine whether or not the level of interactivity in online courses taught by MVCR participants had increased as a result of MVCR training. The responses indicated that MVCR courses had a positive influence. (Table 3)

Participants were also asked to provide additional comments about how MVCR courses had influenced the quality of their online courses, addressing the results level using Kirkpatrick’s model [13]. Qualitative analysis of the responses showed that respondents cited improved skills, changes in attitudes and beliefs, positive experiences, and suggestions for improvements as summarized in Figure III. Table 4 provides example comments for each category.

Forty-eight percent of MVCR course registrants register for more than one course. Of 1,314 participants who registered for courses, 88% successfully completed the course with an 80% or better grade average. Only 5.5% dropped, and only 6.5% did not successfully complete the course. Anecdotal reasons for non-completion range from lack of time to a death in the family. The retention and return rates suggest a high student satisfaction in the MVCR program.

B. Faculty Summer Institutes

ION evaluates the Faculty Summer Institute with paper surveys collected on the last day of the institute. Highlights of the 2002 and 2003 evaluation are presented below. All FSI evaluations are available on the ION website [26].

A total of 380 community college administrators, faculty, and staff attended the 2002 and 2003 FSIs, and 205 of them (54%) returned their evaluation forms. On a Likert scale with 1 = poor to 4 = excellent, the overall quality of the FSI experience was rated 3.2, with only 1 individual returning a value of poor [27].

2002 and 2003 FSI participants report an increase in understanding in several areas based on the sessions they attended. (Table 5). In general, the institute increased the participant’s knowledge somewhat, except in the area of technologies, where the increase was more substantial.

While the question options above were based on Kirkpatrick’s Reaction/Response and Learning levels of evaluation, other questions on the survey questioned the third level of evaluation, Performance/Behavior changes [13]. Question 7 of the evaluation (Table 6) asked whether participants could take what they had learned at the FSI and apply it. 65% found the potential for application to be extremely likely, while only 2 respondents felt that it was not at all likely.

Qualitative data obtained from the open ended area of each evaluation question mirrored those from the survey numbers. Additional information is available on the ION website [4].

C. The ION Website

The ION website receives an average of 19-20,000 successful hits per day, with over 85,000 web page views each day. Over 2,000 visitors per month revisit the site multiple times each month, showing that those who locate the site often return. The average site visit length is approximately 15 minutes, suggesting that visitors are spending time viewing what they find. ION has optimized its website for search engine indexing. Each month, top referring keywords are used to identify key content within the site to list as spotlight resources, highlighting information that others have found to be important that month. The ability to find the ION materials under the expected keyword searches suggest that ION is reaching its target audience.

The ION website has also won several awards for content. It was selected as a WebCT Exemplary Site in 2000 (WebCT has since discontinued this award to our knowledge in favor of their exemplary course program) [28]. In 2001, it was chosen as a Spotlight Site by the Technology Source [29].

D. Long Term Effects

In 2002, ION conducted a study to measure the impact of MVCR and FSI. Five hundred and sixteen e-mail requests were sent to faculty and staff who had completed at least one MVCR course or who had attended the 2000 or 2001 FSI. One hundred thirty-eight web-based forms were completed during the two weeks that the survey was open. Comprehensive results can be found on the ION website, including all open-ended responses [30]. Factors indicating ION’s long term effects are summarized in Tables 7.

Of those completing the survey, 74% were currently developing or teaching online or web-enhanced courses. The 74% that were developing or teaching online or web-enhanced courses were then asked about the program’s influence on their approach to (1) facilitating online discussion, (2) incorporating active learning activities, (3) selecting appropriate technologies, (4) assessing learning outcomes, (5) building online community, (6) applying instructional design principles, (7) applying web design and usability principles, and (8) using copyrighted materials appropriately, as these were main goals in many MVCR courses and FSI presentations. All categories were positively influenced (Table 8). The least influenced was the use of copyrighted materials at 66%. Other areas ranged in effect from 79% to 91%, with 20% to 32% responding that the program had exerted tremendous influence on their long-term behavior.

The timeliness of the ION training was determined by asking respondents how soon after participating in an MVCR course or an FSI they were able to put the knowledge to use. 57% of MVCR participants and 51% of FSI participants were able to make immediate use of the information. Only 10% of each group had not yet had an opportunity or a reason to use the information. Some survey participants were taking MVCR courses at the time they completed the survey. Others had participated in MVCR or FSI as much as two years prior to the survey.

IV. SUMMARY / CONCLUSIONS

When ION was first created in the fall of 1997, only ten of the forty-eight Illinois community colleges were ION partners. The initial goal was to work with ten colleges, determine their needs, and then expand the partnership to the remaining colleges, using what was learned from working with the initial ten partners. ION staff members met with college representatives to determine which colleges had already begun to offer online courses, and how ION could provide help. Among ION’s first activities in 1997 were creation of a website and delivery of face-to-face workshops. The subject of the workshops was how to use software such as web page editors, or image editing software. With only ten partners, it was relatively easy to focus attention on the needs of the partner institutions. However, by 1999, all forty-eight community colleges were partners. The drive-and-present model of faculty development was not viable in a state as large as Illinois. It was inefficient and expensive to drive six hours to deliver a three-hour workshop or presentation to a relatively small number of people. ION began offering regional one-day workshops held on the campus of a sponsoring community college where as many as six workshops or presentations could be offered. This model is much more efficient, and allows ION to work with more faculty per visit.

Although ION’s workshop focus in 1997 was on creating online course elements, such as web pages and graphics, the focus quickly shifted to designing and creating interactive online courses, and to providing information on the effective use of technology. Those topics are more difficult to address effectively in workshops or presentations. The MVCR program was initiated to provide participants an in-depth exposure to topics they need to successfully teach online, and as a way to reach large numbers of instructors. ION has reached many more faculty and provided them with opportunities to learn about online learning than would have been possible otherwise. That large numbers of faculty have chosen to use their own free time to take MVCR courses indicates that they have found value in doing so. MVCR program and individual course evaluation results indicate that faculty members are more confident teaching online and more satisfied with their teaching experiences.

With the success of the first few MVCR course sections, it was apparent that there was more demand for course sections than the ION staff could teach. ION recruited community college faculty who had taken MVCR courses, and had demonstrated exemplary skills in asynchronous discussion, to teach MVCR course sections. ION currently employs ten Adjunct Instructors. They make a significant and positive contribution to ION, and without them, it would be impossible for an organization with few full-time staff members to offer such a large program.

The Faculty Summer Institute is a very popular faculty development activity. Faculty like attending a conference where other community college faculty members are present, and networking and discussing common issues. We have learned that despite the success of MVCR, there is a real need for face-to-face interactions and instruction. Sessions that fill first continue to be those scheduled in computer labs where participants experiment with new software or explore web resources related to online learning.

The faculty development climate has changed considerably in Illinois. In 1997 there were very few faculty development organizations that specifically focused on online or technology-enhanced learning. The face-to-face training ION provided was valuable and a good way to begin. Most colleges now have at least one person who works directly with faculty, and there is less need for a state wide faculty development program to provide face-to-face training. The online, in-depth, MVCR training goes beyond what most colleges can offer, and gives ION the opportunity to continue delivering much needed faculty development. ION is expanding the MVCR program beyond eight-week courses. Beginning in fall 2003, an advanced online seminar series will address the specific needs of administrators and training staff.

V. ABOUT THE AUTHORS

Virgil E. Varvel Jr. is a Computer Assisted Instruction Specialist for the Illinois Online Network at the University of Illinois in Urbana-Champaign. Concurrently, he is a graduate student in the Department of Curriculum and Instruction at the University of Illinois at Urbana-Champaign. His research projects include the use of wireless networks in educational settings and the influence of online components to offline courses. He has a B.S. in Biochemistry, an M.S. in Biomolecular Chemistry, and an M.Ed. in Curriculum and Instruction.

Michael W. Lindeman is an instructional designer, online instructor, and web developer for the Illinois Online Network at the University of Illinois at Urbana-Champaign. He has developed and taught online courses for faculty members on instructional design, student assessment, and web design. He has served as a consultant for Rush University College of Nursing, IDX.com and UNESCO, helping these organizations move towards online instruction. He has conducted many workshops and delivered presentations at national conferences focusing on online course development, assessment of student learning, and online course evaluation.

Dr. Iris K. Stovall is the Program Coordinator for Illinois OnLine Network.

VI. REFERENCES

1. National Center for Educational Statistics (NCES). The Condition of Education 2003, Report, 2003. http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003067
2. NCES. A Profile of Participation in Distance Education: 1999-2000, Report, 2001. http://nces.ed.gov/das/epubs/2003154/index.asp
3. Illinois Virtual Campus. Reports and Resources, 2002. http://www.ivc.illinois.edu/AboutIVC/reports.htm
4. Illinois Online Network (ION). 2003. http://www.ion.illinois.edu/
5. ION. Making the Virtual Classroom a Reality, 2003. http://www.mvcr.org
6. ION. MVCR. Springfield, 2003. http://online.uis.edu/mvcr/
7. ION. MVCR. Master Online Teacher Certificate, 2003. http://www.mvcr.org/courses/MOT.asp?textonly=false
8. ION. Presentations Archive, 2003. http://www.ion.uillinois.edu/present/presentations/completelist.asp?sort=1
9. ION. Faculty Summer Institutes, 2003. http://www.ion.uillinois.edu/present/fsi/default.asp
10. ION. Online Education Resources, 2003. http://www.ion.uillinois.edu/resources/default.asp
11. ION. Pointers and Clickers – Technology Tip of the Month, 2003. http://www.ion.uillinois.edu/pointers/default.asp
12. ION Research, 2003. http://www.ion.uillinois.edu/research/default.asp
13. Kirkpatrick, Donald L. Evaluating Training Programs, 2nd ed., Berrett-Koehler Publishers, Inc.: San Francisco, CA, USA, 1998.
14. Phipps, R., and Merisotis, J. Quality on the Line – Benchmarks for Success in Internet-Based Distance Education, Study released at the Blackboard Summit, Prepared by the Institute for Higher Education Policy, April 2000. http://www.ihep.org/Pubs/PDF/Quality.pdf
15. Lorenzo, George and Moore, Janet. The Sloan Consortium Report to the Nation – Five Pillars of Quality Online Education, Report sponsored by the Alfred P. Sloan Foundation, November 2002. http://www.sloan-c.org/effective/pillarreport1.pdf
16. Phillips, Jack; Phillips, Patricia Pulliam; and Zuniga, Lizette. Evaluating the Effectiveness and the Return on Investment of E-Learning, Online Report for the American Society for Training and Development, 2000. http://www.astd.org/virtual_community/research/What_Works/e-learning/e-learning_main.html
17. Broadbent, Brooke and Cotter, Craig. Evaluating e-Learning, 2002. http://www.e-learninghub.com/articles/evaluating_e-learning.html
18. Treacy, Barbara; Kleiman, Glenn; and Peterson, Kirsten. Successful Online Professional Development. Learning and Leading with Technology 30(1): 42-47 (September 2002). http://www.edtechleaders.org/Resources/articles/SuccessfulOPD.pdf
19. Horton, William. Evaluating E-Learning, American Society for Training and Development: Alexandra, VA, USA, 2001.
20. Donovan, Sherri and Adams, Mark. Empowering Teachers to Soar in Online Classes, Paper presented at MACUL 2003 Technology Tools for Living and Learning Conference, Detroit MI, USA, 2003.
21. Graham, Charles; Cagiltay, Kursat; Lim, Byung-Ro; Craner, Joni; and Duffy, Thomas M., Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses. Technology Source, (March/April 2001). http://ts.mivu.org/default.asp?show=article&id=839
22. Sonwalkar, Nishikant. A New Methodology for Evaluation: The Pedagogical Rating of Online Courses. Syllabus 15(6): 18-21 (January 2002)
23. ION. MVCR Evaluations, 2002. http://www.mvcr.org/about/Evaluations/2001/default.asp
24. ION. MVCR Experience 2002 Survey, 2002. http://www.mvcr.org/about/Evaluations/2002/default.asp
25. Pallof, Rena M. and Pratt, Keith. Lessons from the Cyberspace Classroom, Josey-Bass Publishers: San Francisco, CA, USA, 2001.
26. ION. FSI Evaluations, 2003. http://www.ion.illinois.edu/research/fsi/fsiimpact.asp
27. Likert, Rensis. A technique for the measurement of attributes. Archives of Psychology: New York, NY, USA. 140 (June 1932).
28. WebCT, Inc. WebCT Exemplary Course Project, 2003. http://www.webct.com/exemplary
29. Michigan Virtual University. Past Spotlight Site Articles, The Technology Source, 2001. http://ts.mivu.org/default.asp?show=section&id=7
30. ION. FSI and MVCR impact on online teaching, 2001. http://www.ion.illinois.edu/research/impact/results.asp