THE IMPACT OF INCREASING ENROLLMENT ON FACULTY WORKLOAD AND STUDENT SATISFACTION OVER TIME

Download PDF version:

ABSTRACT

Detailed daily records of instructor effort in an established asynchronous online course over a three and one-half year period are analyzed. Student satisfaction data acquired from course evaluation surveys over the same period are also examined. In response to a three-fold increase in enrollment over the period, instructors realized a twelve percent gain in efficiency. Contrary to expectations, a modest economy of scale was achieved with no discernible decrease in student satisfaction.

KEYWORDS

Faculty workload, Efficiency, Scalability, Student satisfaction, Asynchronous online course, Distance education

I. INTRODUCTION

A considerable body of literature reflects the belief that asynchronous online courses are necessarily labor-intensive for instructors [e.g., 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. With few exceptions, however [11, 12, 13], previous studies of faculty workload in distance education were based on the recollections of instructors who did not methodically record the time they actually spent teaching. Such recollections are unreliable; research by cognitive psychologists demonstrates that the less subjects attend to the passage of time, the less accurate their estimates of the duration of events are likely to be [14]. Although instructors’ impressions about the amount of time they spend teaching at a distance are relevant, it is also important to know how much time on task they actually spend.

Beliefs about faculty workload in asynchronous teaching are consequential. Since faculty participation is often voluntary [1], the ability to recruit outstanding faculty members to the practice of asynchronous teaching and learning depends in part on the extent to which the task is perceived as desirable. The belief that asynchronous teaching necessarily requires more work is one of the most frequently cited disincentives to faculty participation [4, 5, 15]. Junior faculty members may be actively discouraged from participation due to concerns about the inadequate value of distance teaching in the context of promotion and tenure decisions [6]. One response to such concerns is to limit enrollments in online classes to ensure smaller class sizes relative to face-to-face classes [9, 16]. Limiting enrollment may undermine the potential of asynchronous learning networks to realize their most important benefits, however, since the “small-class model limits the ability of programs both to scale (i.e., produce more cost-effective courses) and to serve more students (i.e., increase access)” [17].

The research reported here investigates the issue of scalability by analyzing detailed records of instructor time spent conducting an asynchronous online course over a three and one-half year period. Specifically, I test a hypothesis implied in an influential report of a year-long faculty seminar held at the University of Illinois, which concluded that “teaching the same number of students online at the same level of quality as in the classroom requires more time and money” [16, p. 2]. The report observes that instructors realize economies of scale in classroom teaching by reducing the attention paid to individual students as class size increases. Distance teaching, by contrast, is said to require a fixed cost of individual attention that cannot be reduced without undermining the success of the class. Implicit in this argument is the hypothesis that the time required to teach a high-quality asynchronous online university course can be predicted as a linear increasing function of course enrollment (Figure 1).

Figure 1: Hypothesized linear increasing relationship between course enrollment and instructor workload in an asynchronous distance course.

Is there indeed a fixed cost that prevents the realization of economies of scale in asynchronous teaching? There are no previously published sustained time studies that allow one to examine how faculty workload changes in response to the evolution of a course and the expansion of its enrollment. To explore this question, my teaching assistants and I maintained detailed daily records of time spent teaching an asynchronous online course (Geography 5121: The Nature of Geographic Information) through the Pennsylvania State University’s World Campus during three time periods: from July 1999 through June 2000 [11], from July through December 2001 [18], and from July through December 2002. We also collected student satisfaction data during the same periods. Description, analysis and interpretation of these time and satisfaction data are reported later in this article. First, a description of the course.

II. THE COURSE

Geography 5121: The Nature of Geographic Information is the first in a sequence of four courses that make up the Penn State Department of Geography’s Certificate Program in Geographic Information Systems (GIS). The Department has offered the program online through the University’s World Campus since January 1999. Seven hundred and sixty-six students enrolled in 18 offerings of Geography 5121 conducted through July 2003 (Students who enrolled but did not participate are excluded for the purposes of this study). Most students are adult professionals who use GIS (or wish to use it) in their jobs with government agencies, oil and gas utilities, environmental engineering consultancies, and many other fields. The average age of students who completed the Certificate Program through summer 2002 was 40 years.

Like other courses in the certificate program, Geography 5121 lasts ten weeks, and requires eight to twelve hours of student activity per week. Courses have scheduled, quarterly start dates and end dates. Students are required to complete a weekly schedule of assignments. Until January 2003 the commercial course management system WebCT was used to deliver course materials and to mediate communications. Students are free to study whenever and wherever they choose, but are provided with strong incentives to complete assignments on time.

A. Design Objectives

Counting the first day of the first course offering as the midpoint of the process, Geography 5121 was designed, developed and implemented over a period of about one year. Throughout this process I was fortunate to work with an experienced instructional designer provided by Penn State’s World Campus who helped me articulate design objectives for the course. In retrospect, it is apparent that our design objectives align neatly with the “five pillars” of quality online education identified by participants in the Sloan Foundation’s Asynchronous Learning Networks initiative [19]—learning effectiveness, student satisfaction, faculty satisfaction, cost effectiveness and access.

1. Learning Effectiveness: The course should provide an appropriately challenging, college-level introduction to geographic information science. Students who successfully complete the course should possess sufficient literacy in the concepts and terminology of the field, as well as the requisite skills in computing and Web publishing, to be able to succeed in subsequent courses.
2. Student Satisfaction: Students should rate the course high for quality and value. Instructors should earn high ratings for their knowledge and helpfulness. The level of student effort required should seem reasonable and worthwhile. At least 70 percent of enrollments in this introductory should be converted into enrollments in the following course (a rate suggested by the World Campus marketing team as a reasonable target).
3. Faculty Satisfaction: Instructors should consider their participation in the course to be enriching personally and professionally. Their workload associated with the course should be reasonable and sustainable.
4. Cost Effectiveness: The course and program should generate sufficient gross tuition revenue to offset expenses of course development and delivery while returning substantial net revenue to the host department. To generate sufficient tuition, the course should be able to support up to 100 students each quarter with one faculty instructor and one graduate teaching assistant.
5. Access: The course and program should serve students across the U.S. and around the world who are not well served by conventional on-campus offerings.

Assessing learning effectiveness: As a prelude to the skills-oriented courses that follow, Geography 5121 aims to ensure students’ familiarity with the fundamental concepts and specialized vocabulary that GIS professionals are expected to know. Students demonstrate success in fulfilling this objective in two ways—through their performance in weekly automated quizzes and a cumulative final examination, as well as in three project reports they publish in personal e-portfolios.

The format of the exam and the eight graded quizzes is “open book.” The primary value of quizzing in this course is to incentivize and direct students’ engagement of the 182 XHTML pages of original text, graphics, GIS software experiments, and many “site visits” to mapping-related Web sites that make up the course’s weekly lessons. The quizzes and exams also provide immediate feedback that explains why answers are judged correct or incorrect. Lessons are punctuated by 35 other low stakes quizzes that enable students to focus their study and self-assess their progress. The fact that two-thirds of students’ course grades are determined by their performance on quizzes and the exam reflects the value that I associate with students’ close engagement with the lessons. While acknowledging that time on task is a necessary but insufficient condition for effective learning [20], there is strong evidence that time on task and learning do go hand in hand [21]. Indeed, as Carol Twigg observes:

Because previous research has demonstrated that time on task may be the single most important predictor of increased learning, an explicit goal of course design should be to identify effective ways to increase students’ time on task [22].

Students’ performance in three projects accounts for the most important one-third of their course grades. Geography 5121 requires students to create and publish project reports on the course Web site, and to link their reports to personalized e-portfolios. e-Portfolios are “personalized, Web-based collections of selected coursework, artifacts of co-curricular activities, and (ideally) students’ reflective commentary” [23]. To the extent that it fosters a reflective approach to learning, e-portfolio development encourages students to become more actively engaged in planning, and more responsible for achieving, their educational goals. And, as Yancey argues, “the engaged learner, one who records and interprets and evaluates his or her own learning, is the best learner” [24].

Linn and Gronlund observe that portfolios tend to be “labor intensive for the teacher—requiring considerable time in planning, monitoring, and providing feedback to students” [25]. Geography 5121 instructors provide individual feedback to every student in response to every project report. Our experience was consistent with Linn and Gronlund’s observation: as revealed in the analysis of the faculty activity data reported below, we devoted between 62 percent and 89 percent of our total instructional effort to e-portfolio assessment, feedback, and communications with students during three study periods between 1999 and 2002. Recognizing that it is difficult to evaluate portfolios reliably [25], we provided students with extensive guidelines and developed detailed rubrics to standardize our project scoring efforts.

In addition to promoting student engagement, e-portfolios also allow students to represent their backgrounds, goals, and accomplishments to fellow students and instructors. The “gallery” of student e-portfolios developed by each Geography 5121 class is intended to contribute to the development of cohesive learning communities. Students continue to publish project reports in their e-portfolios throughout the Certificate Program in GIS. Successful students complete the program with extensive demonstrations of the knowledge and skills they have gained.

Assessing student satisfaction: We gauge students’ satisfaction with Geography 5121, and with the Certificate Program as a whole, by monitoring the prevailing tone of daily interactions (which are mediated primarily by threaded discussion and email), by asking students to evaluate the course (through a survey administered at the end of each course), and by monitoring enrollment retention rates. Survey data are analyzed in detail below. Retention rates from Geography 5121 into Geography 5222 averaged 75 percent across the four course offerings in 2002. Of the 642 students who enrolled in Geography 5121 through the Fall quarter 2002 (plus the additional 16 who enrolled in alternative orientation course, Geography 5021), 341 earned certificates of achievement at the conclusion of the Summer 2003 quarter. Our graduation rate of approximately 52 percent reflects the fact that the Certificate Program in GIS is challenging. In response to a survey of program graduates conducted in May 2002 (response rate 63 percent), over 96 percent of respondents agreed that “challenging” is an appropriate adjective. Encouragingly, over 97 percent characterized the program as “time well spent,” while over 88 percent characterized it as “a good value” [26].

Assessing faculty satisfaction: Also reported below is an analysis of the detailed records that my graduate assistant and I maintained during the three study periods. These records extend an earlier study which demonstrated that, on a per capita basis, my assistants and I spent less time teaching Geography 5121 than we did teaching Geography 121, a classroom version of the same course [11]. The data reported here confirm that the relationship between instructor effort and enrollment is very strong in asynchronous teaching and learning. Evidence of a modest economy of scale is also demonstrated, however. Most important, perhaps, the data demonstrate that a mature asynchronous course like Geography 5121 enables instructors to focus most of their efforts on the kinds of tasks that add the most value to students’ educational experiences—assessment, feedback and personal and group communications.

With regard to the extent to which instructors find their roles in the course to be enriching and sustainable, some years ago I commented that “the task of inventing the [Certificate Program in GIS] had been more challenging and interesting than anything I’d worked on before” [27]. Having now led 20 consecutive offerings of Geography 5121 over the past five years, I still find the experience of learning with adult professionals via the Web to be a worthy challenge. The professional rewards, in my case and for many colleagues here, have been considerable. The graduate assistants who have worked with me along the way have also consistently expressed satisfaction with their experience.

Assessing cost effectiveness: Like many other distance learning programs, ours is expected to be successful financially as well as educationally. A necessary condition for our success is that we be self-supporting; the sufficient condition is that we generate net revenue for our host department. To satisfy these criteria we designed Geography 5121 to be scalable to as many as 100 students per offering with one instructor and one graduate teaching assistant. We also devised a 10-week format that allows four enrollment opportunities each year. Thanks in part to an aggressive marketing effort, course enrollments have increased from an average of 18 in the first study period (Summer 1999–Spring 2000) to an average of 56 per class offering in the 2002 study period; 77 students enrolled in the most recent offering (Fall 2003). The Certificate Program in GIS began to generate net revenue after its fourth year of operation, and is now the Department of Geography’s largest source of discretionary income. The program’s financial success was an important factor in the Department’s successful proposal to Penn State’s Graduate School to create a new, online Professional Master of GIS degree program, which is expected to launch in Fall 2004.

Assessing accessibility: The Certificate Program in GIS aims to serve adult professionals whose commitments to family, career and community prevent them from participating in place-bound, synchronous educational opportunities. Within the constraints imposed by other program design criteria, we seek to be accessible financially, technologically and pedagogically to audiences that are as geographically and demographically diverse as possible.

To assess our performance relative to these objectives, we relay primarily upon demographic data collected routinely by the World Campus. Our own periodic student surveys provide additional information. For example, our 2002 survey of program graduates [26] revealed that 99 percent of respondents were aged 25 years or more; their average age was 40 years. Reflecting a regrettable characteristic of our field (and other science, technology, engineering and mathematics disciplines), the gender diversity of our graduates is somewhat less than that of the World Campus students as a whole; 29 percent of our respondents were female, in comparison with almost 45 percent of all World Campus students in Spring 2002 [28]. Reflecting in part the persistent socioeconomic barriers that obstruct Internet accessibility nationally and internationally, the racial and ethnic diversity of our program remains very low; over 90 percent of our students and over 80 percent of all World Campus students are white [28]. The geographic distribution of our students characterizes a program with a strong national market but a weak one internationally: through Summer 2003, Geography 5121 has attracted students from 49 U.S. states and four Canadian provinces but only nine countries on four continents. Over 90 percent of our enrollments come from the U.S. residents. The fact that our courses have so far been offered only in English, and that they are fairly expensive (over $1,000 per course except for Geography 5121, which is offered for about $600), has undermined our competitiveness overseas. We hope that our participation in the Worldwide Universities Network (http://www.wun.ac.uk) will lead to articulation agreements with other university-based programs like ours that will enable us to attract more international students. At present, however, it is clear that accessibility is the design criterion that presents our greatest challenges.

B. Delivery Unit Support Services

Instructors enjoy extensive support from Penn State’s World Campus. The World Campus Student Services team fields inquiries from prospective students, processes enrollments, collects and accounts for fees, and maintains student records. A marketing and sales team manages print and Web advertising, direct mail, and trade show promotions. A technical support group provides invaluable help to students whose system or network configurations interfere with their studies or who need tutoring on basic computing skills. An instructional design and development team assisted the program faculty in developing the courses and provides continuing support in coordinating the delivery of ancillary materials (such as readings and course CD-ROMs containing software and data used in project assignments). Results presented in the following sections represent an extensive support infrastructure that is in many cases superior to that provided to faculty members who teach traditional classroom courses at the University.

III. INSTRUCTOR TIME DATA AND ANALYSIS

This research is intended to counterbalance the impressionistic data analyzed in most previous studies of faculty workload in asynchronous online teaching with a reliable accounting of instructors’ actual behavior during three time periods: from July 1999 through June 2000, from July through December of 2001, and from July through December of 2002. (Gaps between the study periods—January through June 2001 and 2002—provided needed respites from the burden of daily time logging.) During these study periods my assistants and I maintained daily time logs through eight complete offerings of Geography 5121 over a three-year period: the third, fourth, fifth, sixth, eleventh, twelfth, fifteenth and sixteenth offerings since the course debuted in January 1999. In all, we maintained precise daily records for a total of 730 days. (Not every day included time on task, of course.)

It is important to bear in mind that the course had been offered twice before the earliest study period. Time spent on course development during the study periods (reported below) reflects only routine maintenance, updates, and improvements of course content in response to ongoing changes in the field and student comments (students were provided incentives to find and report substantive errors). I intentionally designed the study to look beyond the inevitable rigors associated with initial course development and to focus instead on longer-term concerns about sustainability and scalability.

During the first study period (1999–2000) I was the sole instructor of Geography 5121. During this period I recorded in a logbook (specifically, a “day planner”) any time I spent working on any aspect of the Geography 5121 course. I noted the start and end time of all work episodes lasting approximately five minutes or longer. I also briefly described each task.

By the second study period (2001), course enrollment had grown enough to warrant the appointment of a graduate teaching assistant, Hank Rademacher, a Ph.D. candidate in Geography who had also served as my assistant in the classroom version of the course that is offered to undergraduates on campus. Hank’s role was to help students develop the Web publishing skills they need to create and maintain e-portfolios, to evaluate portfolio projects (using rubrics we created collaboratively), and to provide individual feedback. I provided Hank with the same kind of logbook I used. At the outset of his appointment we discussed the objectives of my study and the importance of his daily participation. Fortunately, he was conscientious. I asked about his progress regularly. Although he was aware of my findings from the first study [11], I did not suggest that a particular outcome was expected this time. Furthermore, to minimize the risk of biasing our record keeping, we did not tally our data until the end of the study period.

I have described elsewhere [11] the qualitative method by which we identified four categories of teaching tasks. The categories are:

1. Setup and Coordination, including tasks associated with readying new implementations of the course within the course management system, as well as ongoing tasks related to administration and coordination;
2. Content Development and Maintenance, including all changes to course content, such as corrections, revisions, and additions to text and graphics;
3. Communication, including daily communications with students via threaded discussion, email, and occasional telephone calls; and
4. Assessment and Feedback, including development and revision of rubrics, evaluation of projects published in e-portfolios, and grade reports and comments sent to individual students by email.

Table 1 shows the amounts of time that Hank and I spent teaching, administering, and maintaining Geography 5121 during eight thirteen-week periods, each of which encompassed one ten-week course offering. (Where two numbers appear divided by a plus sign, the first boldface number denotes the hours spent by the faculty instructor, while the second number denotes the hours spent by the graduate teaching assistant.)

Table 1: Instructor time + teaching assistant time recorded during three study periods, 1999–2000, 2001, and 2002.

Setup and coordination, in cooperation with World Campus support personnel, accounted for between six and ten percent of total instructor effort per study period.

Content development and maintenance declined substantially as the course matured. During the 1999–2000 study period, when the course was only in its third through sixth iterations, content development (specifically, refining content developed for two earlier course offerings) accounted for about one-third of the total teaching effort. During the 2001 study period, however, only about five percent of total teaching time was devoted to content development. The proportion rebounded somewhat, to fifteen percent, in 2002, due to revisions of lessons and projects that had grown somewhat dated. (Might this decline/rebound represent the first wave in a cycle of periodic revisions?)

Communications with students by threaded discussion and email (not to mention reading their exchanges) accounted for a consistently large share of instructor effort throughout the three study periods. Although it oscillated from 45 percent of total effort in 1999–2000 to 55 percent in 2001 and 40 percent in 2002, communications accounted for the largest proportion of total time in all three periods.

Assessment and feedback occupied only sixteen percent of my time when I was teaching the course alone. The addition of an energetic graduate teaching assistant allowed us to increase our emphasis on assessment and feedback from 16 percent of total effort in 1999–2000 to 34 percent in 2001. By 2002, assessing student work and providing written feedback accounted for nearly as large a share of total instructor time (36 percent) as routine communications.

Together, communications and assessment accounted for the lion’s share of instructor effort (62 percent in 1999–2000, 89 percent in 2001, 76 percent in 2002).

In Figure 2 (below), total instructor hours per offering are plotted against numbers of students enrolled for each of the eight study periods. Black diamond-shaped points in the lower left quadrant represent course offerings in 1999–2000, when it was relatively new and attracted relatively few students. Similar points in the upper right quadrant represent later course offerings, which attracted larger enrollments and employed the teaching assistant. The curved line labeled “observed trend” represents a second-order polynomial least squares regression of the eight data points. The regression indicates that a remarkable 96 percent of the variation in instructor effort is explained by variations in student enrollment (r2 = 0.9637).

Figure 2: Instructor time plotted as a function of enrollment in course offerings between 1999 and 2002. The “observed trend” line represents a second-order polynomial least squares regression of the eight data points. The dashed “predicted trend” represents the hypothesized linear relationship between instructor effort and enrollment.

Three additional point symbols (white squares) are plotted in Figure 2. Two denote average observed enrollments and instructor hours for the 1999–2000 period (18 students and 48 total hours per course, an average rate per student of 2.7 hours) and the averages for 2001 and 2002 periods combined (52 students and 123 hours per course, 2.4 hours per student). A third white square symbol represents predicted average instructor hours for 2001 and 2002 combined (52 students and 140 hours, 2.7 hours per student). The prediction follows from the argument of the Illinois Faculty Seminar that there is a fixed cost per student associated with asynchronous teaching, which implies that the average number of hours of instructor effort per student should be expected to remain constant regardless of class size. Thus, an average enrollment of 52 in 2001 and 2002 at the average rate of 2.7 hours per student observed in 1999–2000 produces a predicted average instructor effort of 140 hours. The dashed line labeled “predicted trend” illustrates the hypothesized linear increasing relationship. In fact, however, the efficiency (i.e., total instructor effort per student) with which the course was delivered improved by about twelve percent as the course matured (and the instructors gained experience with asynchronous teaching) over the three study periods. The efficiency gain appears in the vertical gap between the predicted and observed averages for 2001 and 2002 combined.

IV. STUDENT SATISFACTION DATA AND ANALYSIS

If the hypothesized linear increasing relationship between enrollment and instructor effort were true, it would follow that the increased instructional efficiency demonstrated above would be likely to result in a corresponding decrease in student satisfaction. To test this assumption I compared satisfaction data acquired from Geography 5121 students in each of the eight course offerings during the three study periods: July 1999 through June 2000, July through December 2001, and July through December 2002.

Penn State instructors are expected to conduct surveys that gauge student satisfaction at the conclusion of every course. Student ratings are taken into account in annual faculty performance reviews and promotion and tenure decisions. A standardized instrument called the “Student Ratings of Teaching Effectiveness” (SRTE) is used to solicit student ratings. The SRTE consists of four or more seven-step Likert scales by which students are asked to rank the quality and effectiveness of courses and the instructors. Responses are anonymous. While not indicative of the effectiveness of a course in achieving its learning objectives, the SRTE has been shown to be a reliable measure of student satisfaction [29].

During the three study periods, the SRTE was implemented in Geography 5121 as an online survey in WebCT. Students in Geography 5121 and other courses in the Certificate Program in GIS are asked to rate the course on the following nine criteria:

1. Overall quality of the course
2. Overall quality of the instructor
3. Clarity of the instructor’s presentation
4. Instructor’s willingness to help students make progress
5. Instructor's skill creating a climate conducive to learning
6. Adequacy of the instructor’s knowledge of the subject matter
7. Instructor’s preparation for class
8. Organization of the course in terms of its logical arrangement of material and activities
9. Instructor’s skill in encouraging students to think

Obviously the criteria were designed with a synchronous classroom course, rather than an asynchronous online course, in mind. They were retained for use in Certificate Program courses to ensure comparability with the Department of Geography’s resident courses. (The online courses compare favorably.) Instructors who teach online Certificate Program courses meet semi-annually to review ratings and to discuss strategies for continuing course improvement.

Figure 3: Student ratings of course quality, instructor quality, instructors’ willingness to help, and course learning climate during eight offerings of Geography 5121 during three study periods: 1999–2000, 2001, and 2002.

The graphs in Figure 3 above summarize student ratings on the four most salient criteria (criteria 1, 2, 4, and 5) through the three study periods. From left to right, the columns of graphs report data from 1999–2000, 2001, and 2002. Each graph depicts the distribution of ratings on a single criterion during a single study period. Graphs are annotated with the number of survey respondents for each period; response rates were 66 percent of students in 1999–2000, 74 percent in 2001, and 68 percent in 2002. Average ratings are also reported. Each graph’s horizontal axis represents rating levels, ranging from “1” (the least favorable rating) to “7” (most favorable). Vertical axes represent frequencies of responses. The height of the bars is proportional to the number of responses associated with each rating level.

If there is a fixed cost of attention required to conduct a successful asynchronous online class, it is reasonable to expect that the data distributions revealed in the graphs ought to differ. Specifically, given that instructional efficiency (instructor time per student) increased as the course grew and matured, the fixed-cost hypothesis suggests that the graphs associated with the later periods can be expected to exhibit downward shifts in student satisfaction.

To test this expectation I used the Kolmogorov-Smirnov test, a non-parametric statistic that evaluates the differences between two ordinal-level data distributions. The Kolmogorov-Smirnov statistic (an alternative to the Mann Whitney “U” test that is recommended for data sets that include a high frequency of tied values) tests the null hypothesis that two samples have been drawn from identical populations by comparing cumulative frequency distributions of the samples. A difference in frequencies associated with any category in the two distributions that is greater than would be expected by chance under the null hypothesis suggests that the samples are in fact not identical [30]. For this study, I assume that statistically significant differences between ratings distributions associated with two different study periods would suggest a difference (positive or negative) in student satisfaction. Distributions that are not statistically different suggest similar states of satisfaction.

I first applied the Kolmogorov-Smirnov test to compare the nine 1999–2000 distributions to the corresponding 2001 distributions. No statistically significant differences were found. I then repeated the test to compare the 1999–2000 distributions with the 2002 distributions. Again, all nine pairs of distributions were shown to be statistically similar. These results confirm what is apparent in the Figure 3: although the heights of the bars vary in response to increasing enrollments over time, the shapes of the distributions (and their central tendencies) are quite similar among the three graphs in any one row. Clearly the ratings do not provide evidence in support of the expectation that the increase in instructional efficiency exhibited in Figure 2 was associated with any decrease in student satisfaction. In fact, as student comments attest, satisfaction remained consistently high throughout the three and one-half year study.

V. ELABORATING THE HYPOTHESIS

The time-on-task analysis reported above confirms that there is a very strong relationship between instructor effort and enrollment in asynchronous teaching and learning. The form of the relationship is positive and increasing, but not necessarily linear. Geography 5121 instructors realized a twelve percent decrease in effort per student despite a three-fold increase in enrollment over a three and one-half year period. This gain in instructional efficiency may be explained by a combination of factors including pedagogy, support, maturation of course content and instructors’ increasing experience. In addition, given that most instructors are busy people, a “ceiling effect” probably limits the total amount of time that an instructor can devote to any one course, regardless of enrollment or other factors.

Pedagogical approaches that increased the scalability of the course included encouraging, but not requiring, student participation in class discussions, and developing detailed rubrics that streamlined portfolio assessment and feedback. The support services provided to students and instructors by Penn State’s World Campus relieved the instructors of some of the burdens associated with setting up and administering the course, and with responding to student inquiries about administrative and technical problems. Although the study period purposefully did not include time and effort associated with initial development of course content, the data suggest that the amplitude of the cyclic process of keeping content up-to-date did decrease over time. Furthermore, the effort associated with “teaching” Geography 5121 scaled to larger enrollments as the instructors gained experience with asynchronous teaching and learning. (Workload management strategies for online teaching are suggested in [31].)

An important related question concerns the relationship between scalability and course quality (“course quality” here defined as learning effectiveness plus student satisfaction). The fixed-cost hypothesis seems to assume that high quality asynchronous courses must necessarily embody the least efficient and least scalable instructional approaches (that is, those that require the most communication, assessment, and individual feedback). In this study, student satisfaction was adopted as a measure of the quality of Geography 5121 in fulfilling (or exceeding) students’ expectations. (The study does not refer to student performance measures because there was insufficient variation in learning outcomes in all three study periods; very few of the students failed to fulfill course objectives.) To the extent that the satisfaction of the adult professionals enrolled in Geography 5121 represents valid evidence of the course’s quality, the results reported above demonstrate that it is possible to realize a certain economy of scale without sacrificing the perceived quality of the learning experience.

To what extent can these findings be generalized to other subjects and other instructional milieus? I believe that the results reported here set the stage for a more nuanced accounting of the relationship between instructor effort and enrollment. Instead of assuming a necessary “fixed cost” of instructional effort, one can expect that the efficiency and scalability of an asynchronous online course will vary depending on the pedagogical approach employed, the level of support provided, the stability of the subject matter, and the knowledge and experience of the instructor(s). Figure 4 illustrates the following hypothesized relationship between efficiency and scalability: courses that require the most instructor effort for a given enrollment of n students will also be the least scalable. The most labor-intensive courses will involve a fixed cost of instructor effort that results in a linear increasing relationship between enrollment and effort (illustrated by the diagonal trend line in Figure 4). Courses that require the least instructor effort (e.g., those that do not require or even encourage communication, and in which students are not provided with individual feedback) will be most efficient and most scalable, but may not be sufficiently effective or satisfying. Courses designed to support communication and provide individual assessment and feedback efficiently ought to be able to achieve reasonable efficiency and scalability, as illustrated in the middle trend line, without unduly sacrificing either student satisfaction and learning effectiveness. The Geography 5121 case study demonstrates that high levels of satisfaction at least need not be sacrificed to achieve a modest increase in efficiency.

Figure 4: Elaborated hypothesized relationship between course enrollment and instructor workload in an asynchronous distance course. How sensitive is course quality to decreases in instructor effort?

An important research question that arises from this study concerns the sensitivity of learning effectiveness and student satisfaction to changes in instructor effort. Should a decrease in instructional effort by one unit be expected to result in a decrease in effectiveness or satisfaction of an equal amount? Or are effectiveness and satisfaction less sensitive to changes in instructor effort? I hypothesize the latter; modest changes in instructor effort seem unlikely to have a demonstrable effect on learning effectiveness or student satisfaction. Indeed, the absence of impact of increased efficiency upon student satisfaction was demonstrated empirically in the case of Geography 5121. Future research that investigates the impact of changes in the magnitude (and frequency) of instructor effort is needed before we can say how much efficiency and scalability can be achieved without adversely affecting course quality. (It might also be interesting to discover the extent to which efficiency must be increased before instructors notice a difference in their workloads!)

VI. CONCLUSION

Analysis of instructor time recorded through eight course offerings over a three and one-half year period confirms the strong relationship between enrollment and instructor effort. The form of the relationship revealed in this analysis is increasing but curved rather than linear, however; a twelve percent decrease in instructor effort per student was achieved in response to a three-fold increase in enrollments. Analysis of course evaluation survey data demonstrated that the increased enrollments and increased efficiency resulted in no discernable decrease in student satisfaction. It is observed that the efficiency and scalability of an instructor-led, asynchronous online course varies with the pedagogical approach employed, the level of support provided, the stability of the subject matter, and the knowledge and experience of the instructor(s). The magnitude of efficiencies that can be realized without incurring unacceptable decreases in course quality is unknown. Methodical, sustained studies of instructor time-on-task, student performance, and student satisfaction are needed to address this question.

VII. ACKNOWLEDGEMENT

Thanks to Hank Rademacher for his assistance throughout the study, to the Alfred P. Sloan Foundation and the Penn State World Campus for their financial support, and to Melody Thompson, Ann Luck, three anonymous reviewers, and the editors, whose suggestions strengthened the manuscript.

VIII. REFERENCES

1. National Education Association. A Survey of Traditional and Distance Learning Higher Education Members, 2000. Accessed 19 May 2002: http://www.nea.org/he/abouthe/dlstudy.pdf.
2. SchWeber, C., Kelly, K. B., and Orr, G. J. Training, and Retaining, Faculty for Online Courses: Challenges and Strategies. Distance Learning '98. Proceedings of the Annual Conference on Distance Teaching and Learning, pp. 347-352. Madison WI, August 5–7, 1998.
3. Palloff, R. M. and Pratt, K. Building Learning Communities in Cyberspace: Effective Strategies for the Online Classroom. San Francisco: Jossey-Bass, 1999.
4. Rockwell, S. K., Schauer, J., Fritz, S. M. and Marx, D. B. Incentives and Obstacles Influencing Higher Education Faculty and Administrators to Teach via Distance. Online Journal of Distance Learning Administration 2(4): Winter 1999. Accessed 19 May 2002: http://www.westga.edu/~distance/rockwell24.html.
5. Wolcott, L. L. and Betts, K. S. What’s In It for Me? Incentives for Faculty Participation in Distance Education. Journal of Distance Education 14(2): 34–49, 1999.
6. Ellis, E.E. Faculty Participation in the Pennsylvania State University World Campus: Identifying Barriers to Success. Open Learning 15(3): 2000.
7. Johnston, T. C., Alexander, L., and Fieser, J. Compensation Models for Online/Distance Education. Proceedings of the Mid-South Instructional Technology Conference, April 9–11 2000.
8. Johnston, T., Alexander, L. Conrad, C. and Fieser, J. Faculty Compensation Models for Online/Distance Education. Panel discussion at the April 2000 Mid-south Instructional Technology Conference: Extending the Frontiers of Teaching and Learning.
9. Weiner, R. S. Instructors Say Online Courses Involve More Work at Same Pay. New York Times on the Web, June 21, 2000. Accessed 27 May 2002: http://www.nytimes.com/library/tech/00/06/cyber/education/21education.html.
10. Berge, Z. L. The Context of Distance Training: Predicting Change. In Z.L. Berge (Ed.) Sustaining Distance Training: Integrating Learning Technologies into the Fabric of the Enterprise, 3-12. San Francisco: Jossey-Bass, 2000.
11. DiBiase, D. Is Distance Teaching More Work or Less Work? American Journal of Distance Education 14(3):6–20, 2000.
12. Visser, J. A. Faculty Work in Developing and Teaching Web-based Distance Courses: A Case Study of Time and Effort. American Journal of Distance Education 14(3): 21–32, 2000.
13. Hislop, G. W. Does Teaching Online Take More Time? Proceedings of the 31st ASEE/IEEE Frontiers in Education Conference, pp. T1F-23-27, 2001. Accessed 20 June 2002: http://fie.engrng.pitt.edu/fie2001/papers/1198.pdf.
14. Glicksohn, J. Temporal Cognition and the Phenomenology of Time: a Multiplicative Function for Apparent Duration. Consciousness and Cognition 10(1): 1–25, 2001.
15. Schifter, C. C. Faculty Motivators and Inhibitors for Participation in Distance Education. Educational Technology XL(2): 43–46, 2000.
16. University of Illinois Faculty Seminar. Teaching at an Internet Distance: the Pedagogy of Online Teaching and Learning, 1999. Accessed 19 May 2002: http://www.vpaa.uillinois.edu/tid/report/.
17. Twigg, C. A. Can Online Education Scale? The Learning Marketspace, April, 2003. Accessed 30 April 2003: http://www.center.rpi.edu/LForum/LM/Apr03.html.
18. DiBiase, D. and Rademacher, H. Scaling Up: Faculty Workload, Class Size, and Student Satisfaction in a Distance Learning Course on Geographic Information Science. Forthcoming in Journal of Geography in Higher Education.
19. Lorenzo, G. and Moore, J. The Sloan Consortium Report to the Nation: Five Pillars of Quality Online Education. Alfred P. Sloan Foundation, 2002. Accessed 8 August 2003: http://www.sloan-c.org/effective/pillarreport1.pdf.
20. Bransford, J. D. and others (Eds). How People Learn: Brain, Mind, Experience and School. Washington DC: National Research Council, 2000.
21. National Survey of Student Engagement, 2003. Accessed 31 March 2003: http://www.indiana.edu/~nsse/.
22. Twigg, C. A. It’s Not How Fast You Run. The Learning Marketspace, March, 2002. Accessed 31 March 2003: http://www.center.rpi.edu/Lforum/LdfLM.html .
23. DiBiase, D. and colleagues. Using e-Portfolios at Penn State to Enhance Student Learning: Status, Prospects, and Strategies, February 2002. Accessed 3 January 2004: http://www.e-education.psu.edu/portfolio/e-port_report.html.
24. Yancey, K. B. General Patterns and the Future. In B. Cambridge and others (Eds), Electronic Portfolios: Emerging Practices in Student, Faculty, and Institutional Learning, 83-87. Washington, DC: American Association of Higher Education, 2001.
25. Linn, R. L. and Gronlund, N.E. Chapter 12: Portfolios. In Measurement and Assessment in Teaching, 8th Ed. Upper Saddle River, NJ: Prentice Hall, 2000.
26. Outreach Office of Marketing Research and Planning. World Campus GIS Certificate Graduates Survey Report. July 2002. The Pennsylvania State University, Outreach and Cooperative Extension.
27. DiBiase, D. Is Distance Education a Faustian Bargain? Journal of Geography in Higher Education 24(1): 130-135, 2000.
28. Outreach Office of Marketing Research and Planning. World Campus Student Surveys—Demographics Survey—Longitudinal Study. The Pennsylvania State University, Outreach and Cooperative Extension. March, 2003.
29. Dorris, M. J. An Analysis of the Penn State Student Rating of Teaching Effectiveness, Report to the Penn State University Faculty Senate, 1997. Accessed 19 May 2002: http://www.psu.edu/president/cqi/cqi/srte.
30. Blalock, H. M. Social Statistics, 2nd Ed. New York: McGraw-Hill. 1979.
31. Ragan, L. C. and Terheggen, S. L. Effective Workload Management Strategies for the Online Environment. The Pennsylvania State University World Campus, 2003. Accessed 4 January 2004: http://www.worldcampus.psu.edu/pub/home/fac/workload_strat.pdf.