Faculty Satisfaction in ALNs: A Dependent or Independent Variable?

Download PDF version:

Abstract
The authors describe relationships among infrastructure, student outcomes, and faculty satisfaction at the University of Central Florida (UCF). The model focuses on a developmental process that progresses from courses with some Web presence to those that are driven by ALN. Faculty receive support for on-line teaching in the form of release time for training and development, upgraded hardware, and complete course development services. Students receive assistance in the form of orientation, around-the-clock help services, a Pegasus Connections CD-ROM, and a learning on-line Web site. The results of the impact evaluation at UCF indicate that faculty feel that their teaching is more flexible and that interaction increases in the ALN environment. On the other hand, they are concerned that on-line teaching may not fit into the academy culture. The authors argue that faculty satisfaction and student outcomes are strongly related and that their interaction is the most important outcome. Finally, the authors conclude that faculty satisfaction is both a dependent and independent variable that is nested within colleges, departments, and program areas.

I. INTRODUCTION

UCF is a member institution of Florida's State University System. The university was established in 1963 and began offering classes in 1968 with an enrollment of 1,948 students. UCF has developed rapidly by working cooperatively with surrounding mid-Florida communities to deliver programs of instruction, research, and service that stimulate and sustain progress throughout the region. The University offers degree programs in five Colleges: Arts and Sciences, Business Administration, Education, Engineering, and Health and Public Affairs. In addition to the main Orlando campus, UCF operates branch campuses in Daytona Beach, and Cocoa, centers in downtown and south Orlando, and a Virtual Campus.

One of the defining characteristics of UCF is rapid growth. The University's current student population of nearly 32,000 is projected to reach 52,000 by the year 2010. Likewise, the University's metropolitan setting in Orlando is experiencing rapid expansion, and is recognized as a premiere tourist destination and center for high technology and aerospace development. Accordingly, UCF is responding to the diverse needs of its expanding region in part by developing Web-based and media-enhanced programs (http://pegasus.cc.ucf.edu/~outreach).

ALN activity at UCF is characterized as one of three types. Web-enhanced E courses are those using the Web in a useful but incidental manner. Typically, on-line materials are provided as supplemental or enrichment resources. All E courses use WebCT as a repository for course materials, resource links, or for conferencing and E-mail. Instructors delivering E courses have participated in WebCT Academy, a faculty development program targeted toward the use of that course management tool. Media-enhanced M courses make significant use of the Web as an integral element of course delivery. Currently, some M courses have a reduced number of class meetings (reduced seat time), using ALN activities to replace traditional class meetings. Fully Web-based courses are designated W, and have no required class meetings. Some have an optional pre-course orientation and/or required final examination. This continuum of ALN activity is illustrated in Figure 1.

Figure 1. ALN Continuum

The initial complement of fully Web-based degree programs began in fall 1996, with a total enrollment of 125 students. The fully Web-based (W designation) programs are intended to promote access to higher education for individuals in the state who are placebound due to family or work responsibilities. Currently, five fully Web-based on-line (W model) degree programs are offered, and others are in development:

• B.A. in Liberal Studies
• B.A. or B.S. in Vocational Education
• RN to BSN in Nursing
• M.S. in Vocational Education
• M.S. in Forensic Science

Soon after launching its ALN initiative, the University observed that 75% of the students in these courses were also enrolled in traditional on-campus courses. The primary reason for those students' participation in ALN was convenience. Students also reported increased satisfaction, based in part on increased interaction with the instructor. Thus, in 1997, a second mode of ALN course distribution was initiated. The M mode courses are targeted toward one or more of three specific goals: increase access, improve student retention through increased performance, and improve the quality of large enrollment courses.

Over the past three years, the growth of both W and M ALN modalities has been significant, as shown in Figure 2.

. 
Figure 2. Growth in On-line Course Enrollments

The University will expand the complement of fully on-line degree programs, adding additional programs in areas of high demand. However, going forward, our major emphasis will be on the M delivery model. There are several reasons for this. First, the M model has the potential for significantly greater impact because it is directed toward the University's 32,000 on-campus students. Second, the reduced seat-time feature of the M model will help alleviate the university's current shortage of classroom space. The institution's growth has been so rapid that despite an aggressive capital construction program, there is currently a 40% shortage of classrooms, and off-campus instructional space must be rented at a significant annual cost. Each M class that can be structured to reduce seat time will utilize up to 66% less classroom time than a traditional course, e.g., three course sections can be scheduled into one classroom slot. A third benefit is the increased information literacy that accompanies ALN-based learning. The M model combines the best elements of both face-to-face and ALN-based learning.

Once having completed their first ALN course, both faculty and students have mastered important information management and communication skills that benefit them throughout the remainder of their careers. All UCF ALN courses emphasize student-centered, active learning. The on-line environment is utilized for information access and, through computer-mediated communications (CMC), the development of learning communities. Our research has shown that student success, (as defined by the achievement of a grade of A, B, or C) is approximately 5% higher in M sections than the comparable traditional or W sections. More will be said about these outcomes later.

II. RATIONALE

Many institutions engaging in ALN activity seek to make courses more conveniently available or reduce time constraints for students [1-3]. UCF recognizes distributed learning as a strategic initiative to expand educational opportunities within and beyond our service area. The University has turned to ALN as its primary vehicle for expanding access while maintaining quality and reducing the costs of educational delivery. After more than three years of ALN activity, the University has established the model for Web-based instruction depicted in Figure 1. Seen from a different perspective, this model can be viewed as an instructional Web space, as shown in Figure 3.

 

Figure 3. Distribution of Web Use by UCF Faculty

Within this space are all UCF faculty who are using the Web for course delivery, support, or enhancement. The Web Presence layer includes faculty who are beginning to experiment with use of the Web to supplement their traditional courses with on-line resources. These faculty generally work individually, creating Web-based resources for their courses by themselves, with the assistance of a student aide, or with their college-based technical support unit.

The Enhanced stratum represents faculty who are using the University's standard course management tool, WebCT. This faculty use WebCT's advanced course management features and take advantage of regular training offered year-round. (See WebCT Academy at http://reach.ucf.edu/~webct411.) Faculty who use WebCT have free access to University-provided server space for their courses, as well as access to assistance from the Course Development and Web Services unit.

At the top of the hierarchy are the W and M course designations. These courses, described earlier, are fully transformed, both technically and pedagogically, into the ALN environment. Faculty teaching M and W courses are selected through a competitive request for proposal (RFP) process and receive intensive faculty development training, a new computer, and student support, as required.

Administrative leadership and institutionalized models for faculty development are essential for creating high-quality Web-based programs. Daigle and Jarmon [4] assert that technology-focused faculty development programs should strive to become woven into the fabric of the institution and agents of institutional transformation. The faculty development program should be connected to the University's strategic plan and have a business plan for measuring effectiveness [4]. The authors further suggest that faculty development programs should strive to develop information literacy.

Creating consistency within the instructional Web space is unlikely if faculty are working in isolation and without commonly shared tools and standards. This is often referred to as the boutique or craft approach [5]. A quality, scalable, institution-wide approach requires the development of a distributed model in which a central course development unit, perhaps supplemented by distributed, college-based support units, helps colleges and departments support faculty at all levels of ALN activity.

Within UCF's ALN space the institution has standardized tools and approaches so that faculty can more easily move upward within the pyramid-that is, make the transition from experimentation with the Web through use of a formally supported course delivery environment and on to the transformative M and W models. To date, more than 180 faculty members have participated in the University's IDL6543 (Interactive Distributed Learning for Technology-Mediated Course Delivery, http://reach.ucf.edu/~idl6543) faculty development program for ALN, resulting in the development of more than 400 courses. In the WebCT environment, there are more than 750 registered accounts, encompassing more than 25,000 participants (not all are ALN courses). The assessment data discussed in the Results section of this paper relate exclusively to the faculty participating in the M and W models. Please see Appendix A for a list of the courses evaluated.

The faculty development program, IDL6543 is organized as an eight-week M course in which the faculty members participate as "students."  IDL provides a realistic simulation of the ALN experience from the student's perspective, with Course Development staff and faculty mentors serving as the "instructors."  IDL6543 facilitators use the Web, E-mail and conferencing to create an experiential learning experience for participants. UCF Web vets-faculty who are experienced in teaching on-line-give presentations as peer teachers, sharing what they have learned. As the IDL6543 learning community develops, materials turned in by the faculty are produced by the Course Development team to build the on-line courses. Some materials are drawn from vaults of best practice examples. Instructional designers consult with faculty to assess course needs and give feedback on developing curriculum materials for on-line delivery.

 
Figure 4. Components of IDL6543 Faculty Development Program

Figure 4 shows the structure of the IDL6543 curriculum, which is delivered as a 60-hour program: five classes (15 hours), five labs (7.5 hours), ten on-line modules (30 hours), and consultations with instructional designers for needs assessment and development (7.5 hours). This time span allows faculty to develop course materials as homework for the course. Higher level learning and integration is most effectively achieved through dialogue, which occurs in computer conferencing [6]. Our experience has shown that this is equally true when computer conferencing is used for faculty development. Faculty have a difficulty at first adjusting to the flow of computer conferencing due to the disjointed nature of conversation. Ocker, Hiltz, Turoff and Fjermestad [7] reported on similar findings: "The actual sequence of talk is rearranged as communication occurs over time. This can lead to a disjointed flow of information. Many topics are also active at the same time requiring the ability to sort through topics to coordinate communication."

Noam [8] states that true teaching and learning are about more than transmitting information. Education is based on mentoring, internalization, identification, role modeling, guidance, socialization, interaction, and group activity. Using a combination of face-to-face and technology-mediated communication increases the formation of learning communities over using face-to-face meetings or pure mediated systems [9].

To date, IDL6543 participants have included nearly 180 faculty from all five UCF colleges, a few doctoral students, faculty from the Florida (on-line) High School, and representatives from the Naval Air Warfare Command Training Systems Division and the Army Simulation, Training, and Instrumentation Command.

Institutionalizing faculty development for ALN delivery has proven beneficial because it

• Provides experiential learning for faculty participants

• Fosters cross-discipline sharing of teaching techniques

• Builds learning communities among faculty

• Creates life-long learners among faculty

• Creates discussion of the teaching and learning process

• Allows peer evaluation of successes and failures

• Exposes faculty to tools and instructional best practices

• Models a combination of delivery techniques

• Uses cooperative and collaborative learning techniques

• Provides greater flexibility for busy faculty

• Transforms all teaching for more active student-centered learning

Instructional designers from course development and Web services facilitate cultural change across disciplines for faculty with varying levels of technical ability and experience. Faculty attending IDL6543 report being rejuvenated in their teaching. Each semester approximately 30 additional faculty from various disciplines become involved. This development cycle creates an upward spiral that allows both the faculty development and instructional models to continuously improve.

Faculty willing to experiment are essential to developing successful Web?based learning environments. Some factors that prevent faculty from pursuing technological innovations are: fear of change, fear of time involved, fear of appearing incompetent, fear of techno-babble, fear of failure, not knowing where to start, fear of making bad choices, fear of typing, and fear of reprisals and rejection [10]. In addition, in some environments, the efforts of faculty to develop high-tech classes often are unappreciated and even frowned upon [11].

The change process must be fostered by planning, designing, and evolving the technical infrastructure; providing administrative support and leadership; providing systematic faculty development; and assessing performance for continuous improvement of course delivery and services. Achieving a multifaceted response to consumer demand for increased access, improved quality, and reduced cost of higher education is the core concept of a virtual university [12].

III. TECHNOLOGY AND INFRASTRUCTURE

A. Organizational Infrastructure
In 1995, the Division of Information Technologies and Resources was formed and headed by a new Vice-Provost. The formation of this division brought together the Library, Computer Services (academic and administrative), Telecommunications, and Instructional Resources into a single administrative unit reporting to the Provost. In 1997, the Course Development unit was created to support ALN faculty and course development. The unit's role was later expanded to include campus-wide Web services.

Figure 5. Organizational Structure

In 1997, the Vice-Provost for Academic Programs created the Center for Distributed Learning and assigned it with responsibility for planning and administering interactive television, video, and Web-based programs, serving as an information clearinghouse, providing support, and marketing off-campus and distributed learning credit programs. This office also coordinates accreditation changes that may result from Web-based instruction. Within this division, the Faculty Center for Teaching and Learning, also created in 1997, offers campus-wide support for instructional improvement.

In addition to these units, each college has a Distributed Learning Coordinator, typically a faculty member who coordinates the respective college's ALN programs with the central units. The Student Academic Resource Center provides student support, including an on-line mentoring program called Peer Assisted Learning Online (PALS Online). Figure 6 shows the range of UCF resources supporting teaching and learning in the ALN environment.

 
Figure 6. Institution-Wide Collaboration

Among the benefits of centralized faculty ALN support is that faculty are freed from many of the mechanical tasks of ALN development, such as HTML programming. Another is that faculty come into contact with instructional designers who facilitate the transformation process [13]. The Course Development and Web Services unit fulfills this role at UCF. Instructional designers, programmers, and digital media specialists team with faculty to create on-line course materials. The cohorts of faculty who spend a semester participating in the IDL6543 program remain intact, meeting periodically to share ideas, problems, successes, and failures. This facilitates a cultural shift, transforming the teaching and learning process through modeling on-line learning communities and collaborative, experiential learning. Figure 7 depicts the UCF faculty development model, while Figure 8 describes typical on-line course components.

Figure 7. UCF Faculty Development Model	Figure 8. On-line Course Components

 
A major issue affecting faculty satisfaction is the support of students in the ALN environment. Without an institutional response to this issue, faculty must take valuable course and personal time to grapple with the myriad technical challenges and information gaps that face students taking an on-line course.

To remove as much of this burden from faculty as possible, UCF has put in place a wide range of ALN student support resources. For example, all students who register for ALN courses are invited to attend advance orientation sessions, where requirements and protocols are explained and students are walked through logging on to the network and navigating to the course home page.

The Computer Services Help Desk and a new CyberKnights program offer 7 X 24 telephone and walk-in assistance to supplement course development and Web services' support for ALN. To enhance on-line learner support, Course Development has developed a Learning On-line Web site (http://reach.ucf.edu/~coursdev/learning), and the Pegasus Connections CD-ROM (http://reach.ucf.edu/~coursdev/cdrom), a disc containing self-assessment tools, tutorials on personal productivity software and WebCT, campus information, and links to campus Web resources. The disc also contains software for configuring the student's personal computer to dial into the campus network. The purpose of the Pegasus CD is to give students taking ALN courses a just-in-time tool for self help.

Additionally, students are encouraged to form learning communities to help support each other regarding course content and technology problems. The campus-wide learner support model is illustrated in Figure 9.

 
Figure 9. Learner Support Model

B. Technology Infrastructure
Over the past five years, UCF has made significant investments in technology infrastructure, faculty and student support services, and organizational development to support both regular campus instruction and the asynchronous learning initiative.

Networking has become a core strategy for the University as it strives to serve a growing student population over an increasing geographic area and expand into new areas of research. This means more, however, than simply being a wired campus. UCF is using network technology to create a sense of community, extending 7 X 24 access to information, services, and people to all students and employees.

In 1996, UCF completed a campus connectivity project through which all faculty and staff offices not previously connected were provided with access to the campus backbone network. New premise wiring, optical fiber, and network components were implemented throughout the campus to establish universal network connectivity. At the same time, the network backbone was upgraded from Ethernet to ATM operating at OC-12 (622 Mbit/second) with distributed 100 Mbit/second Ethernet switches, providing high-speed connections to the desktop.

A series of core network services has been established, consisting of electronic mail, World Wide Web, the full range of Internet utilities, e.g., telnet, ftp, etc., extensive on-line library information, image processing, and on-line course delivery. Access to UCF information and services is made available through the University's main Web site (http://www.ucf.edu), and services such as POLARIS (https://polaris.ucf.edu), a Web interface to many administrative services for students, faculty and staff. These services are available to all UCF network users. To support these services, all central servers have been continuously upgraded, and new servers added to accommodate the growth in both users and on-line information.

The UCF Library, a unit of the Division of Information Technologies and Resources, has been given a prominent role in making available a wide array of electronic information services. More than 4,000 databases and full text resources are available on-line to UCF students (http://library.ucf.edu).

A system of Universal Access was implemented in 1995, whereby all UCF students are provided with network and E-mail accounts from the point they are admitted. These accounts are maintained past graduation. Likewise all faculty and staff are assigned network and E-mail accounts from the point of hiring. The pool of supported, user accounts now exceeds 50,000 users.

Access to on-line courses is viewed as one of an array of standard network services. A central server cluster hosts all ALN courses and provides convenient and consistent access to all available courses.

Information technology is integrated within the institution's strategic plan. (The current plan makes more than 60 specific references to IT. See http://pegasus.cc.ucf.edu/~ucfdist/strategic.html), and senior university administration have publicly identified IT as a means of achieving the institution's goals.

IV. CONTENT DELIVERY

The assessment results reported in the Results section of this paper involve a number of faculty and courses. All faculty included in the analysis have in common the IDL6543 faculty development experience and the use of WebCT as a course management environment. Faculty and courses represent all five colleges and a wide range of mainstream faculty-new and experienced, tenured, and untenured.

Although individual courses exhibit some structural differences, the majority pace the flow of content to assist students in keeping on task. Some faculty place the entire semester's assignments on-line at the beginning of the course; others release content throughout the semester to keep students at approximately the same point in the syllabus. UCF's on-line courses emphasize learning communities and asynchronous computer-mediated communication. The primary CMC tools are electronic mail, conferencing, and occasionally, synchronous chat, all of which are supported in the WebCT environment. An Oracle-based student directory containing photos, bios, and E-mail addresses has been integrated with WebCT, enabling students and faculty to better get to know one another on-line.

In most ALN courses, student performance is assessed with regular examinations. M courses are able to use traditional forms of testing during the live class meeting periods, while the W courses tend to rely on evaluations of student participation and writing, along with grades from on-line quizzes. Most W courses hold a formal final examination at which attendance is required.

V. ORGANIZATION AND EVOLUTION

ALN activity has been highly institutionalized at UCF. Accordingly, the significant level of institutional direction and support is perhaps the primary reason for the rapid growth of ALN and the high degree of overall faculty satisfaction. Academic priorities and policy are set through collaboration between central administration, the College deans, and department chairpersons.

The Office of Academic Affairs issues course development RFPs twice each year. Approximately 60 faculty are selected to participate in the IDL6543 faculty development program-30 in the fall, and 30 in the spring. Each faculty member is expected to develop a minimum of one new ALN course while participating in IDL6543. The RFP provides one course release or equivalent overload payment for each faculty member while participating in training, a new computer if needed, and additional funding for student assistance.

All M and W courses are created in the WebCT environment by the Course Development unit, whose instructional designers team with faculty to transform teaching and learning in the ALN environment. Programmers and digital media specialists create graphics, special programming, java applets, and other course-specific content as needed. All courses are installed on a centrally maintained, high-performance server that is administered by Computer Services.

Although UCF's ALN initiatives have been underway only a little more than three years, we believe we have determined a number of factors that, in our environment, contribute to faculty satisfaction. These include

• Reliable Infrastructure - By maintaining a fast, reliable, well-maintained course server, faculty are relieved of the burden of finding a machine to host their courses, and from the duties of system maintenance and backup.

• High-Quality Faculty Development - The IDL6543 faculty development program is continually being refined and extended. The program was recognized in 1999 as a Best Practice in a SHEEO-APQC benchmarking study of institutions in North America. The IDL program truly helps faculty transform to a student-centered, active learning mode of learning suitable to the ALN environment.

• Extensive Faculty Support - In addition to IDL6543, faculty can rely on the Course Development unit to help create and maintain their courses. Although eventual faculty independence is encouraged, this need not come before the faculty member is ready and willing to assume greater course maintenance responsibility. Also, faculty have a central unit to call upon in the event of a technical anomaly.

• Faculty Recognition and Incentives - Faculty are recognized for their ALN work. For example, at the completion of each semester's IDL6543 program, an event is held in which each faculty member presents the course materials he or she has developed, and discusses the course strategy and goals. Deans and senior administrators attend these presentations and, thereby, become familiar with the ALN work of the faculty. Incentives include a new computer and the opportunity to apply ALN work when competing for Teaching Improvement Program awards ($5,000 base salary increment).

• Interdisciplinary Approach - The IDL6543 program involves faculty from all Colleges and a wide variety of disciplines. Faculty report that this is one of the few times they have had the opportunity to discuss teaching and learning with not only their own colleagues but also those from other disciplines. Often, spirited discussions occur. This interdisciplinary interaction is rated as one of the most beneficial features of the program.

• Web Vets - Faculty who have successfully taught ALN courses in both the M and W formats are heavily involved as guest presenters and mentors in the IDL6543 program. The ability of faculty just beginning to explore ALN to "hear it straight" from others more experienced plays a significant role in giving ALN credibility, as well as fostering best practices. Many of the faculty involved in ALN are senior faculty who report enjoyment from learning new ways to enhance student learning.

• Student Support - Student success is closely linked with faculty satisfaction. By providing enhanced support for students taking on-line courses, faculty are relieved of much of that burden and can focus on course content and flow.

• Assessment - As will be described in the next section, UCF has an extensive assessment effort underway. The results of this effort yield significant insights into how the ALN process works and for whom. Armed with this information, faculty can be more confident about how to deal with individual students, and are aware that not all individuals succeed equally in the ALN environment. The results of the assessment effort are available to all and the positive results being reported further add to the credibility of ALN as a quality-learning environment. Several faculty have worked with the assessment staff to conduct publication-quality research studies. In this manner, innovative teaching supports the faculty's research roles, as well. An interesting sidelight of the ALN assessment effort is the number of comments received suggesting that we now know much more about the dynamics of ALN than we do of traditional classroom instruction. For additional information on UCF's ALN assessment effort, please see http://reach.ucf.edu/~research.

• Institutionalization - We approach ALN not as a collection of unrelated, individual courses, but rather as a comprehensive, coordinated institutional initiative. Significant effort has been expended on putting all of the necessary elements in place, including infrastructure, coordination, program development, faculty development support, and assessment. Further, the University's ALN effort is linked with specific institutional goals, which can be traced directly to the University's Strategic Plan. The University thus knows not only how it is engaging in ALN, but also why.

• Continuous Improvement - All aspects of the ALN effort are continually being reassessed and improved. This, of course, must be balanced with the need for stability. By providing ongoing training for both faculty and staff, new tools and techniques can be quickly adopted.

The ultimate goal is for ALN to become so commonplace that it becomes invisible. By emphasizing a quality, systematic, institutionalized approach, we believe that ALN is slowly (or perhaps not so slowly) creating an institution-wide transformation of teaching and learning. We believe the day is not far away when courses that do not use such technology will be the exception.

This could not occur without a high degree of faculty participation and satisfaction, and we will continue to emphasize that outcome as one of the most important in our ALN endeavor.

VI. RESULTS

Our study of faculty satisfaction with ALN is one component of a more comprehensive evaluation of distributed learning at UCF. The assessment is entering its third year and includes several dimensions focusing on students and faculty:

• Student participation, success rates, and withdrawal rates in fully on-line and media-enhanced courses

• Impact of learning styles on the ALN environment

• Satisfaction of students and faculty with the ALN environment

• Demographic trends for students and faculty participating in ALN courses

• Developing strategies for faculty and student success in the ALN environment, and

• Faculty initiated research projects including

  • Teaching and assessing critical thinking on-line

  • Real time approaches to data collection

  • Quasi-experimental approaches to satisfaction and outcome assessment

  • The impact of ALN environments on the accreditation process

  • The impact of teaching large section ALN classes

  • Changes in personal theorizing (regarding approaches to teaching) that result from ALN instruction

  • Effectiveness of various assessment techniques in the ALN environment

Our satisfaction results are derived from a survey conducted during the spring semester of 1998. An advisory panel of students, faculty, and support staff assisted the investigators in developing a faculty questionnaire. The final instrument, containing questions in Likert, nominal, and free-response formats, was validated and refined through a series of pilot studies. The content of the survey included demographics, workload specification, assessment techniques, amount and quality of student interaction, satisfaction with the ALN experience, willingness to teach another course on-line, and finally, preferences and difficulties with ALN teaching.

The questionnaire was distributed during the spring semester of 1998 to the 48 faculty members who had taught fully on-line or media-enhanced courses since the inception of the program (fall 1996). Respondents were guaranteed anonymity, but we asked them to provide their names should they be willing to participate in follow-up interviews or focus groups. Responses were received from 38 instructors for a return rate of 79%. Some individuals, however, had experience with both fully on-line and media-enhanced courses. Accordingly, the instructors were given the opportunity to complete questionnaires for both formats. Several faculty members chose this option so that the final number of useable surveys totaled 48. The responses were treated as independent.

A. Faculty Satisfaction
1. Faculty Survey
Thirty-six courses (see Appendix A) representing all five colleges (Arts and Sciences, Business Administration, Education, Engineering, and Health and Public Affairs) were evaluated by faculty. Three course modalities were represented: fully on-line (n=32, 67%), media-enhanced with reduced seat time (n=8, 16.5%) and media-enhanced with no reduced seat time (n=8, 16.5%).

Appendix B contains further demographic information on the responding faculty. Sixty-seven percent were male and 33% were female with a median age of 49 years bounded by 55 years at the 75th percentile and 42 years at the 25th percentile. The median number of years teaching experience was 23.5 years with an upper value of 26 (75th percentile) and lower bound of 10 (25th percentile). The rank distribution of the respondents was as follows:  11% were instructors, 17% assistant professors, 56% associate professors, 10% full professors, and 6% were adjuncts.

Table 2 contains the results of faculty responses to the workload associated with the ALN environment. Ninety percent of the instructors felt that on-line courses required more work than traditional versions of the same courses. Approximately 88% felt the same for reduced seat-time courses and 63% of the faculty saw more work in full seat-time media-enhanced courses. The exact probability associated with these proportions is zero.

Table 2. Faculty Workload Compared with Face-to-face for On-line and Media-enhanced Courses, Spring 1998

Table 3 contains the summary of faculty responses regarding the amount of interaction in their courses. Ninety-seven percent of the respondents indicated the amount of interaction in their on-line classes was greater than in the same courses taught in a traditional mode. Seventy-five percent of the instructors in reduced seat-time courses reported increased interaction, while 50% of those teaching media-enhanced classes saw an increase. Once again, the exact probability associated with this contingency table is zero.

Table 3. Faculty Assessment of the Amount of Interaction in On-line and Media-enhanced Courses, Spring 1998

Faculty perceptions for interaction quality in their media-enhanced and on-line classes are presented in Table 4. Clearly, instructors in all three modes saw improvement over traditional courses. Ninety-four percent of the teachers in on-line courses indicated that the quality of teacher-to-student and student-to-student interaction had improved. Seventy-one percent of those teaching reduced seat-time, media-enhanced courses felt a similar improvement while 75% in the media-enhanced courses with no reduced seat time saw similar gains. The exact probability associated with this table is .07.

Table 4. Faculty Assessment of the Interaction Quality in On-line and
 Media-enhanced Courses, Spring 1998

Table 5 portrays faculty satisfaction with their media ALN experience. Eighty-three percent of those representing on-line courses showed satisfaction while 63% of the teachers of reduced seat- time courses exhibited a similar perception. The highest satisfaction levels (88%) were found in the media-enhanced courses (p=.32).

Table 5. Faculty Overall Satisfaction with On-line and
 Media-enhanced Courses, Spring 1998

Faculty disposition toward teaching another on-line or media-enhanced course is presented in Table 6. Ninety-four percent of the faculty participating in on-line classes indicated their willingness to continue. Eighty-eight percent of the reduced seat-time instructors indicated their favorableness toward continuing and 100% of the media-enhanced instructors wished to continue their involvement (p=.12).

Table 6. Faculty Willingness to Continue Teaching On-line and
 Media-enhanced Courses, Spring 1998

2. Faculty Interviews
Interviews and focus groups with on-line faculty are producing continuous updates for the original survey. The present survey, interviews, and focus groups produce these positive components for teaching in an ALN environment:

• Student and teacher interaction is enhanced.

• The teaching and learning environment is much more flexible.

• The environment forces continuous improvement.

• The teacher's role changes to that of a facilitator.

• Students are more actively and responsibly involved in their learning.

• Courses are expanded and more far reaching because of increasing resources.

• The teacher is able to use time more effectively and efficiently.

• The teacher is able to incorporate instructional resources created by others.

• Instructional design and theorizing are enhanced.

• The teacher is forced to be less ambiguous and more organized.

On the other hand, teachers working in the ALN environment see less positive components:

• Time demands are very severe.

• Technology problems will occur.

• There is decreased face-to-face student contact.

• Faculty control is lost.

• Students can be overwhelmed with a bottomless pit of information, the quality of which is unknown.

• Testing and assessment is a problem-particularly the honesty issue.

• Students' evaluations of teachers are perceived to be lower.

• Faculty feel an uneasiness about how this fits into the University culture of teaching, research, and service.

• Faculty are not sure they have departmental support.

B. Faculty Satisfaction and Impact Results
Figure 10 depicts our conception of intersecting faculty satisfaction (satisfied/dissatisfied) and student experience (positive/negative). The model portrays the two variables-teacher satisfaction and student impact-scaled and orthogonal. The quadrants SP and DN exemplify the expected interaction of outcomes and satisfaction. For instance, in programs where students perform as well or better than traditional sections a climate of faculty satisfaction is much more likely (SP). On the other hand, quadrant DN reflects negatives in terms of both student outcomes and faculty satisfaction. Quadrants SN and DP are likely to prove more transitional. When faculty members encounter continuing negative student performance, satisfaction will be more difficult to achieve (SN). In the opposite situation, however, a group of dissatisfied teachers who encounter improved student results will be much more likely to change their attitudes (DP).

 
Figure 10. A Model of Interacting Faculty Satisfaction and Student Outcomes

While this model is oversimplified, it does present some interesting suppositions. First, instructors are likely to adjust their satisfaction levels according to the impact of the process on students. Second, faculty satisfaction cannot sustain itself independently of student outcomes. Third, the probability is low that an entire faculty in one institution will reside in one quadrant. Our experience at UCF suggests that faculty may be found in different combinations of satisfaction and student impact, distributing themselves along various points of these continua.

C. Student Outcomes
1. American National Government
During the fall 1998 semester an extant comparison emerged in the Political Science Department. Five sections of American National Government were taught using Web-enhancement (M model) and ten sections were conducted in the face-to-face format with little or no enhancement. Figures 11, 12, and 13 present the results of comparing those sections for success (at least a C grade), withdrawal, and class size. Since these courses were self-selected by students, initial comparisons were made between the section modes for gender and ethnic distribution, comparative ability test scores, high school grade point average, and overall grade point average at UCF. None of these comparisons yielded exact probabilities of less than .05.

Figure 11 shows a median success rate of 85.1% for the media-enhanced courses and 78.1% for the traditional sections. This difference yielded an exact probability of .099. Figure 11, however, contains another noteworthy result. The interquartile range for the traditional sections (68.9%-85.1%) was considerably greater than that for the media-enhanced section (81.6%-88.5%).

 
Figure 11. Success Rate: 25th, 50th, and 75th Percentile Points for Five
 Media-enhanced (M) and Ten Traditional (T) Sections
 of POS 2041 American National Government

The results of the analysis of withdrawal rates in the sections (Figure 12) shows a median withdrawal of 2.1% in the media-enhanced sections and 3.1% in the traditional classes (p=.22). Once again, however, there was a large discrepancy in interquartile ranges, 3% for the media classes and 5% for the traditional classes.

 
Figure 12. Withdrawal Rate: 25th, 50th, and 75th Percentile Points for Five
Media-enhanced (M) and Ten Traditional (T) Sections
 of POS 2041 American National Government

Figure 13 contains the class-size comparison for the two modalities. The median class size for the media-enhanced sections was 85 compared to 71.5 for the traditional approach (p=.055). The Q-ranges showed substantial differences-12.5 students for the media-enhanced courses and 71 for the traditional sections.

 

Figure 13. Class Size: 25th, 50th, and 75th Percentile Points for Five
 Media-enhanced (M) and Ten Traditional (T) Sections
 of POS 2041 American National Government

2. Demographic Trends
Table 8 contains success rates (as defined by an A, B, or C grade) by ethnic groups for those who participated in M courses, on-line W courses, and those traditional sections that correspond to either one or both modalities (Fall 1998). White students appear roughly comparable across modes, but are somewhat more successful in the M sections (88%). Asian students, although a small sample, succeeded somewhat less in W sections than in either of the other two modes (61%). Black students exhibited their highest success rates in the on-line sections (80%), but Hispanic success rates are lower in W sections (72%). Generally, these data show comparable results. This information is based on a fractured data set because not each course in any one mode has matching sections in the other two. For instance, a media-enhanced course may not have a corresponding traditional section with which it can be compared. Unfortunately, this will be characteristic of large institutional data sets.

Table 8. Ethnicity of Students Succeeding in M, W, and Corresponding
Traditional Sections, Fall 1998, Percentages Rounded

Table 9 presents success rates of males and females in the three modalities for the fall 1998 semester, indicating that women succeed at a higher rate than men for all three modalities.

Table 9. Gender of Students Succeeding in M, W, and Corresponding
 Traditional Sections, Fall 1998, Percentages Rounded

Figure 14 presents a segmentation model of the fractured data set for the spring 1999 semester using success as the dependent measure and gender and course modality as predictor variables. At the top level, the model shows that media-enhanced courses outperformed the combination of traditional and on-line courses by approximately five percentage points. The next level of nodes indicates that females outperform males in all course modalities. The bottom level shows males in traditional courses outperform males in Web courses while females are equivalent in Web and traditional courses. This model used as the design for a logistic regression predicts students who succeed with 80% accuracy using gender and class modality as independent variables.

Figure 14. Segmentation Model for Predicting Success Rates
Using Class Modality and Gender, Spring 1991

3. Success and Withdrawal in the ALN Environment
Figures 14 and 15 present comparative success and withdrawal rates in on-line and media-enhanced courses with corresponding traditional sections in a non-fractured data set. This means that each course has been matched with its traditional counterpart. Figure 15 shows media-enhanced courses produce higher success rates (85.5%) than their traditional counterparts (78.8%, p = .00). On-line courses, however, produce a lower success rate (75.7%) when matched with traditional sections (88.6%, p = .00). Figure 16 indicates that media-enhanced courses yield lower withdrawal rates (3.4%) when compared to their matched traditional sections (5.9%, p= .00). Fully on-line courses, however, show a higher withdrawal rate (9.0%) than do the traditional matches (4.5%, p= .00).

 
Figure 15. Success Rates for Matched M and Corresponding Traditional and
 W and Corresponding Traditional Sections for Fall 1998


 

Figure 16. Withdrawal Rates for Matched M and Corresponding Traditional and
 W and Corresponding Traditional Sections for Fall 1998

VII. RESEARCH SUMMARY

Uniformly, faculty using the ALN environment indicate that their workload increases along with the amount and quality of the interaction with and between students. Generally, our instructors rate their experience as satisfactory with the large majority wishing to continue in the new modality. They see their students as more active and responsible for their own learning and these teachers feel more flexible and facilitative. Because of the conversion to ALN teaching, instructors are reexamining their approaches to pedagogy and personal theories of instruction. They see the need for reducing ambiguity in their teaching on-line, and have to pay considerably more attention to instructional design components. Teachers feel that both of these elements spill over into their traditional classes, making them more effective in both environments.

Faculty members do, however, express concerns over this approach to teaching. These instructors worry about the decline in student ratings in their on-line courses. They are concerned about how these activities fit into the university culture, especially at tenure and promotion time. These concerns, however, have not dissuaded significant numbers from continuing with ALN teaching at UCF.

The first concern-lower student ratings-requires further investigation, especially over the long term. Maintaining respondent anonymity, however, makes this a fairly daunting task. To date, diminished student ratings for instructors who participate in ALN teaching exist in the perceptions of faculty. Such findings have not been empirically verified, however. We plan to undertake this study but it depends on the cooperation of instructors who are willing to provide data for their ALN and traditional courses.
The second worry expressed by faculty-how is ALN teaching weighed in the tenure and promotion formula-must be answered at the department and college level. Faculty are virtually unanimous in claiming additional workloads associated with the ALN experience. Given finite time resources, the additional energies faculty devote to on-line teaching will detract from time spent performing research and service. Currently, we have no test cases, so this remains a phenomenological finding.

Our study of demographic trends shows that course modality (Web-based, Web-enhanced, or traditional) has no adverse impact on minority populations; however, women participate in fully on-line courses at a much higher rate then men do. Further, women succeed across all course modalities at a higher rate when compared to male students.

Segmented models show media-enhanced courses having higher success and lower withdrawal rates than traditional or fully on-line courses. When media-enhanced and fully on-line classes are matched with traditional sections, media-enhanced versions are superior in having greater numbers of students succeeding with an A, B, or C grade, and fewer withdrawals. A similar comparison of traditional and Web classes favors the traditional mode.

VIII. CONCLUSIONS

Regarding the title of this paper, we believe that faculty satisfaction and student outcomes co-vary when predicting success in ALN programs. However, the institutional underpinnings of satisfaction form components that provide infrastructure, development, and support for faculty in an environment where they are able to mentor and network with their colleagues. Faculty must be rewarded for their innovative activities. Possibilities include shared recognition, a program of research support where faculty can freely explore their hypotheses and present their results, and more tangible rewards such as money and equipment. Finally, teachers must see this as an institutional and cultural initiative where all share mutual benefits in a self-sustaining environment.

Faculty satisfaction and student impact are often considered independently. However, we suggest that they are not independent but rather co-linear. In the absence of positive learning outcomes, teachers will encounter great difficulty succeeding, even with superior support and resources. Students at UCF give three positive components for learning in an ALN environment. First, they see the institution, a large metropolitan university, responding to their needs in a meaningful way. Second, they see themselves as increasingly active in their learning. Finally, their experience empowers them to expand their learning beyond the normal boundaries of traditional classes.

We are continually pressed, however, for bottom line data. How do success and withdrawal rates compare between ALN courses and their traditional face-to-face counterparts?  Which departments are most and least successful?  How do ALNs impact our underrepresented populations and on the faculty side, how will this impact tenure and promotion decisions?  Will departments scale the increased workload and how will faculty sustain in this environment of continuous change?  Finally, are ALN classes as effective as their traditional counterparts?

After three years of research into the effects of ALNs on faculty satisfaction at UCF, we believe that it is, simultaneously, an independent and dependent variable, both impacting student outcomes and being impacted by them. Further, we suggest that the effect of the interaction between these two factors is greater than either one taken independently. Like the mystic no-significant-difference phenomenon, faculty satisfaction is nested within colleges, departments, and program areas yielding a complex pattern of interactions.

REFERENCES

1. Lewis, L., Alexander, D., and Farris, E. Distance education in higher education institutions. National Center for Education Statistics, NCES 98-062. U.S. Department of Education, Washington, DC, 1997.

2. Moore, M. G., and Kearsley, G. Distance Education: A Systems View. Belmont, CA: Wadsworth Publishing Company, 1996.

3. Oblinger, D. G., and Maruyama, M. K. Distributed learning. CAUSE Professional Paper Series #14 [PUB3014]. Available on-line at http://www.educause.edu/pub/profess.html, 1996.

4. Daigle, S., and Jarmon, C. Building the campus infrastructure that really counts. Educom Review, Vol. 32, Issue 4, pp. 35?38, 1997.

5. Sorg, S., and Truman, B. Learning about teaching through the Internet: lessons learned, technology and teacher education annual. In Proceedings of the Society for Information Technology and Teacher Education. Association for the Advancement of Computing in Education, Orlando, 1997.

6. Lauzon, A. C. Integrating computer-based instruction with computer conferencing: an evaluation of a model for designing on-line education. American Journal of Distance Education, Vol. 6, Issue 2, pp. 32-46, 1992.

7. Ocker, R., Hiltz, H. R., Turoff, M., and Fjermestad, J. The effects of distributed group support and process structuring on software requirements development teams: results on creativity and quality. Journal of Management Information Systems, Vol. 12, Issue 3, p. 127, 1995.

8. Noam, E. Electronics and the dim future of the university. Science, Vol. 270, pp. 247?249, 1995.

9. Etzioni, A., and Etzioni, 0. Communities: virtual vs. real. Science, Vol. 277, p. 295, 1997.

10. Williams, A. E. Integrating courses with the Internet: preparing the teacher as well as the learner. 1996 ASCUE Proceedings, ERIC Document ED405839, 1996.

11. Auter, P. J., and Hannah, M. S. The Challenge of Developing On-line Courses, ERIC Document ED406701, 1996.

12. Twigg, C., and Oblinger, D. The virtual university. A report from a joint Educom/IMB Roundtable, Washington, DC, November 5?6, 1996.

13. Willis, B. Enhancing faculty effectiveness in distance education. In Distance Education:  Strategies and Tools, Englewood Cliffs, New Jersey: Educational Technology Publications, 1994.

ABOUT THE AUTHORS

Joel L. Hartman is Vice Provost for Information Technologies and Resources at UCF. In this capacity, he has overall responsibility for the University's directions in computing, networking, library, telecommunications, and media resources. He came to UCF in 1995 from Bradley University where he was also CIO. Hartman has been an information technology consultant to several organizations in both the public and private sectors and has played an active role in the development of state networking in Illinois and Florida. He graduated from the University of Illinois, Urbana-Champaign, with bachelor's and master's degrees in Journalism and Communications, and is completing doctoral work at the University of Central Florida.

Contact: Information Technologies and Resources, University of Central Florida, Orlando, Florida 32816-2800; Telephone: 407-823-6778; Fax: 407-823-6710; E-mail: joel@mail.ucf.edu.

Charles D. Dziuban is Director of the Research Initiative for Teaching Effectiveness at UCF. He was founding director of the Faculty Center for Teaching and Learning and currently is conducting the impact evaluation of the distributed learning program at the university. Dziuban received his Ph.D. from the University of Wisconsin and specializes in applied multivariate and latent trait analysis. He has conducted over 100 project evaluations for educational, industrial, and governmental agencies. Currently, he is researching the application of learning style theory to asynchronous learning networks.

Contact: Research Initiative for Teaching Effectiveness, University of Central Florida, Orlando, Florida 32816-1250; Telephone: 407-823-5478; Fax: 407-823-0052; E-mail: dziuban@pegasus.cc.ucf.edu.

Patsy D. Moskal is the Faculty Research Associate for the Research Initiative for Teaching Effectiveness at UCF. She received an Ed.D. from UCF and holds B.S. and M.S. degrees in computer science. She specializes in statistics, graphics, and applied data analysis. She has extensive experience in research methods, including survey development, interviewing, and conducting focus groups and frequently serves as a consultant to school districts, industry, and government organizations.

Contact: Faculty Research Associate, Research Initiative for Teaching Effectiveness, University of Central Florida, Orlando, Florida 32816-1250; Telephone: 407-823-0283; Fax: 407-823-0052; E-mail: pdmoskal@pegasus.cc.ucf.edu.

APPENDIX A

Courses Represented by Faculty Survey

APPENDIX B

Some Demographics for Responding Faculty (N=48)