Instructor Attitudes within the SCALE Efficiency Projects

Download PDF version:

Abstract
This paper presents the results of interviews with the principal investigators of the current Sloan Center for Asynchronous Learning Environments (SCALE) Efficiency Projects. There are six such projects: Spanish, Microbiology, Economics, Math, Chemistry, and Physics. The paper reviews each project individually, summarizes the results, and then discusses some common lessons learned as well as some still open issues. The paper considers satisfaction both from the perspective of the course director/designer and from the perspective of other instructors and graduate teaching assistants. The evidence appears to show that all of these groups are satisfied with ALN, relative to the prior situation. Nonetheless, it is not clear whether these results would translate to other high enrollment courses.

I. INTRODUCTION

The SCALE Efficiency Projects began in fall 1997 and are ongoing. The genesis of the projects was presented at the Third International ALN conference [1]. The SCALE evaluation team performed an independent evaluation of the costs and learning outcomes associated with the first semester of the Efficiency Projects. The results were published in the Journal of ALN (JALN) [2]. Below, we summarize some of the major points from these sources.

By fall 1997 substantial ALN activity had occurred at the University of Illinois at Urbana-Champaign (UIUC). SCALE was supporting more than 7,000 students in more than 70 courses. However, many instructors reported that while ALN enhanced the quality of instruction, especially with regard to increasing the amount of interaction with students, in reality instructors were spending more time on teaching, not less. As SCALE had promised at the outset of its initial grant that ALN would produce efficiency in instruction along a variety of dimensions, including increased retention, decreased time to degree, and lower cost of instruction, SCALE initiated the Efficiency Projects in the third year of the original grant.

In the first iteration, the Efficiency Projects covered a relatively small number of courses. Most SCALE-supported courses were not involved and SCALE continued its job of promoting ALN on the UIUC campus. Moreover, each initial Efficiency Project was in a high-enrollment course and that was one of the selection characteristics for choosing an Efficiency Project. Indeed, the total number of students involved in the first set of Efficiency Projects was in the thousands. In many of these courses, as in other high-enrollment courses on campus, graduate students provided a significant amount of the instruction. In the JALN paper, we differentiated productivity gains based on whether it was faculty or graduate student productivity that was increasing. Several of these courses also relied on undergraduate peer tutors as a way to maintain a substantial amount of human contact in a high enrollment course (something that is often lacking in the traditional approach) without incurring large cost increases from doing so.

In almost all of the original Efficiency Projects, some form of automated, Web-based quizzing and grading software was used. The two most popular packages, CyberProf and Mallard, were developed on the UIUC campus; one course used the commercial package WebCT. Attitudinal data from the students in the form of surveys and interviews revealed that they generally enjoyed learning this way. In particular, they liked the immediate feedback, the ability to re-try quizzes for credit, and the freedom to work at their convenience that ALN afforded. Despite their overall satisfaction, they also expressed their dissatisfaction with other aspects of the on-line quizzes; namely, server congestion, on-line hints that were not sufficiently useful, and multiple choice questions for which they could guess the correct answer without understanding why the answer was right. Happily, two features that contributed to learner satisfaction, Web-based delivery and computerized grading of assignments, also allowed for a more efficient use of instructors' time and effort.

In addition to an ALN component, most of the Efficiency Projects retained a face-to-face component to the course. Only two of the projects, Differential Equations taught with Mathematica and advanced Organic Chemistry, entirely abandoned the in-class lecture component.(1)

Based primarily on the success of the original Efficiency Projects, the SCALE grant was renewed in the summer of 1998 for an additional two years. Six Efficiency Projects were featured in the renewal proposal:  Spanish, Microbiology, Economics, Mathematics, Chemistry, and Physics. With the exception of Physics, a new project, all were expansions of Efficiency Projects started in fall 1997. The other five projects were expansions of the original Efficiency Projects. Two projects, Introductory Statistics for non-quantitative students and Introduction to Circuit Analysis, were not included in the renewal grant because they had already achieved a degree of maturity and it did not appear that additional funding would produce further gains.

Section II provides case studies of each Efficiency Project that SCALE has supported in the renewal.

II. CASE STUDIES

A. The Spanish Project
1. Rationale for the Project
This was the most ambitious project. Over the past 15 years, the campus has experienced a chronic, excess demand for the introductory Spanish language courses. Indeed, the problem exists at universities nationwide, as current resources simply cannot meet students' desire to study Spanish. Locally, graduate assistants accomplish the teaching in the basic language curriculum in Spanish, as in most high enrollment, introductory courses on campus. The number of graduate student teaching assistants available to staff these courses is restricted, in part, by the ability of the faculty to offer a quality graduate program and, in part, by the availability of suitable placement upon completion of students' graduate degrees. In short, increasing the size of the instructional staff on Spanish was not a viable solution.

To further exacerbate the problem, in 1998 the campus senate mandated a general education requirement of three semesters in foreign language that all students would have to satisfy for graduation. (Previously, only students in the colleges of Liberal Arts and Sciences and Commerce had a foreign language requirement.)  The campus administration, anticipating further increased demand in Spanish as a result of this new requirement, was willing to invest significantly to resolve the problem. Given the concomitant developments with ALN on campus, it was determined that an ALN approach might provide the best solution.

2. Project Description 
Three courses in the introductory Spanish sequence were reconfigured to use technology to deliver instruction in vocabulary, grammar, and reading. Face-to-face class meetings were reduced from four to two per week. During these sessions instruction focuses entirely on the development of oral communication skills. The courses in the introductory Spanish sequence covered by this project each had enrollments of around 640 per semester, divided into sections of 20 students each. The bulk of these students had some Spanish in high school, but not enough to place out of the foreign language requirement entirely. When the campus-wide foreign language requirement goes into effect in fall 2000, it is projected that these enrollments will increase to about 1000 per course per semester.

3. Technology 
The Spanish Project uses Mallard for the delivery and grading of course assignments and FirstClass for asynchronous conferencing.

4. Personnel 
Diane Musumeci, Associate Professor of Italian, is the project director. Professor Musumeci has won numerous teaching awards on campus. We consider her to be a SCALE early adopter. Among the SCALE early adopters there are disproportionately many in the category of instructors who have exceptional aptitude for teaching. In some sense, the Spanish Project represented the next logical step in ALN course development for Professor Musumeci. In 1996-97, she had first developed her own ALN approach in the introductory Italian sequence with support from Burks Oakley. Then she spearheaded the development of ALN in Spanish 210, a fifth-semester grammar course that was one of the first SCALE Efficiency Projects. Professor Musumeci encouraged the course coordinator, Professor Anna Maria Escobar, who had just received tenure, to adopt an approach similar to the one in the Italian sequence. Professor Escobar had no experience teaching with ALN and indeed could be characterized as a computer novice at that time. The success of the Spanish 210 project was pivotal in getting the second, larger Spanish project underway. For the latter, Professor Musumeci partnered extensively with Professor Giuli Dussias, the director of the introductory Spanish sequence and Assistant Professor in the department. While Professor Giuli was an attendee of a Faculty Summer Institute in 1996 that included presentations on ALN, Professor Giuli, like Professor Escobar, had no prior teaching experience with ALN.

In addition to the overall supervision provided by Professor Dussias as the Spanish language program director, the plan entailed the hiring of three new course coordinators. These are adjunct faculty whose sole province will be teaching, administration, and supervision of graduate student instructors in these introductory Spanish courses. However, their hiring was delayed and the first year of implementation proceeded without the benefit of having these staff in place. What has been accomplished to date has been with faculty and graduate students for whom the introductory sequence is not their long-term charge. The new adjunct faculty joined the project in fall 1999.

5. Faculty/Instructor Satisfaction 
Professor Musumeci derives a great deal of satisfaction in that her model for teaching the high enrollment, introductory Spanish language courses produced substantial and replicable productivity improvements. The department is now teaching almost twice as many students in the introductory Spanish sequence with no increase in instructional staff. Based on these results, the new campus-wide foreign language requirement will go into effect in fall 2000.
 
Professor Musumeci was only slightly less enthusiastic with regard to learning outcomes, as they seemed to depend on the way learning was measured. Pre-course and post-course performance on the Spanish placement exam favored the ALN group over the non-ALN group. Mid-term exam scores, performance on tests of listening comprehension, and final grades in the course were not significantly different between the two groups. Although performance on the final exam suggests that the non-ALN group were the higher performers, preliminary analyses suggest that intervening factors may have compromised the results.
 
In spite of these somewhat mixed results on learning outcomes, Professor Musumeci was clearly satisfied with the work she had done. The revision of the Spanish language sequence to include ALN has permitted the campus foreign language requirement to be implemented; it has allowed almost twice as many students who want Spanish to take it and the management of multi-section, high enrollment courses has been streamlined. In addition to allowing increased student access to Spanish language courses, the ALN component ensures that all students receive equal access to all course material and timely, consistent feedback on their assignments. Moreover, the model permits human resources-graduate student teaching assistants-to be used more appropriately, for interaction and communication purposes in the classroom and for consultation.

Professor Musumeci did express some displeasure with how such achievements seemingly fall out of the range of the traditional rewards system. This is a comment that SCALE has heard quite frequently from ALN faculty not involved with the Efficiency Projects. These comments reflect a degree of irony-the rhetoric from the campus administration strongly encourages the adoption of ALN but the rewards are set at the department level, and their determination reflects a more conservative view.
Meetings with the graduate student teaching assistants in Spanish reveal that they, too, are satisfied with ALN. They particularly appreciate the elimination of both routine grading and the responsibility for maintaining deadlines for the completion of assignments. Although they are in class for the same number of meetings each week-the difference being that instead of meeting one group of students four times per week they meet two different groups twice per week-they like the flexibility of scheduling that the new system allows. They are able to schedule their teaching to allow at least one and as many as three days per week without face-to face-instructional responsibilities.

With the foreign languages curriculum, diffusion of ALN throughout the Spanish is evidenced by a number of projects to introduce ALN into more advanced courses in the Spanish major. Similar initiatives are currently underway in French, German, and Chinese.

Whether or not the success of the Spanish Project at UIUC generalizes to resolve the national demand for Spanish language instruction remains yet unanswered. In academic year 1999-2000, the University of Minnesota will be testing the UIUC model in its basic Spanish language sequence. Other major universities have expressed similar interest.

B. Microbiology
1. Rationale for the Project
This has also been an ambitious project. The primary rationale for this redesign of the first course in Microbiology was pedagogic. Namely, the labs in their traditional format were not achieving the purpose for which they were designed. Due to high enrollments and finite laboratory facilities, students have limited time in the lab (three hours per week). Moreover, some of the laboratory assignments require such extensive setup that the students barely have time to do the data analysis from the experiments they have performed. Because time constraints prevent students from repeating their experiments, students never see variation in experimental outcomes. Virtual labs, on the other hand, allow for repeated trials and provide students the chance to conduct the data analysis more carefully. They also economize both on expensive reagents and on graduate assistant time, as the graduate assistants neither supervise the students nor grade the lab homework.

2. Project Description
Biology 122 enrolls roughly 400 undergraduates in the fall semester and 800 in the spring semester. A large percentage of these are pre-med students. Initially, ALN was used only for student self-assessment; now it is for course credit. Extensive animations in the on-line materials illustrate complex dynamic microbiological processes, something that textbook presentation simply cannot do. Automatically graded lab exercises measure student comprehension of the underlying microbiology. (There are still paper-based lab notebooks for further assessment of student comprehension.)

3. Technology
Delivery of course materials and automated grading rely heavily on CyberProf. Animations were produced with Macromind's Director.

4. Personnel
Deanna Raineri is the project director. Professor Raineri is an Assistant Professor in the department of Microbiology in a tenure-track position, but her contract is unlike other assistant professors in that she devotes her efforts to educational technology development rather than to research in basic science. Around 1993, Professor Raineri began using PacerForum, a now defunct Mac-based conferencing system. When SCALE came into being, she was already beginning to explore Web-based approaches. She uses ALN to teach the first half of the Biology 122 course. Professor William Daniel teaches the second half using a traditional approach. Currently, Professor Raineri is converting the remaining part of the course to ALN with Professor Daniel's supervision. Indeed, Professor Raineri is involved in adapting subsequent courses in Microbiology to ALN.

One goal of this project was to lessen the teaching burden of the graduate assistants who should be devoting more of their time to research. The project has been successful in achieving that goal. As it turns out, the approach might also allow the department to better confront recent turnovers.

5. Faculty/Instructor Satisfaction
Professor Raineri relishes designing on-line material for use in microbiology courses. And she enjoys teaching the course when students use the materials she has created. Her enthusiasm when talking about what she has done conveys her passion. Hence, her satisfaction may not be an appropriate measure to gauge the attitudes that her colleagues (who are not in the true-believer category) would have if they taught their courses via ALN. Perhaps a better indicator of general satisfaction with ALN is its diffusion throughout the department. Professor Daniel has volunteered to begin teaching his half of the course with ALN, using materials he is having Professor Raineri design. He sees the benefit from teaching in this mode, the students have been demanding it, and he is happy to oblige them-as long as he does not have to design the materials himself. Instructors in the upper division courses in Microbiology express similar sentiments. As long as someone else constructs the virtual labs and the CyberProf homework and the faculty are confident of the quality of the on-line materials, they are delighted to teach this way. This has been the diffusion strategy in Microbiology and it seems to be working.

C. Economics
1. Rationale for the Project
This project was motivated by two interrelated factors. First, the placements of recent Ph.D.s from the department of Economics have not been of the quality that the College and the campus would prefer. To upgrade the quality of the program, it was determined that the graduate program should downsize. Second, it has been increasingly difficult for the department to recruit students into the doctoral program, especially students whom the department feels comfortable about placing into the classroom. Both the quality of the graduate program and recruitment problems have contributed to a diminishing pool of graduate teaching assistants, intensifying the need to deliver instruction more efficiently.

2. Project Description
Two distinct courses are covered in this project-introductory microeconomics and the second semester course of the economics statistics sequence. The Microeconomic Principles course enrolls at least 1,500 students a semester. Professor Larry DeBrock delivers one large lecture to 750 students each spring. After trying an approach that relied on commercial software used in the basic Agricultural Economics course, Professor DeBrock put publisher-owned quiz material into Mallard and then developed a comprehensive database of related questions. He also had graduate assistants offer on-line help for students in the ALN sections through an internally developed, Web-based conferencing program. Initially, the graduate assistants also met with the ALN students in conventional recitation sections. Ultimately, the recitation component was dropped, although face-to-face office hours were retained. In spring 1998 the number of students assigned to the graduate assistants in the ALN sections doubled and the recitation sections were abandoned. In spring 1999, ALN was implemented in all sections that were affiliated with DeBrock's lecture. Because graduate teaching assistants are responsible for more students in the ALN format than they were in the traditionally taught sections, the delivery of instruction became more efficient.

The second Efficiency Project in Economics is Econ 173, the Introduction to Regression Analysis. This course enrolls about 525 students in the fall semester and twice that number in the spring. Prior to ALN implementation, the course was taught in straight lecture mode. Each teaching assistant would be assigned one section to teach in the fall and two in the spring, meeting every section for three hours per week. Following the approach taken by Lanny Arvan in Intermediate Microeconomics, the lecture component of the course was de-emphasized through the introduction of ALN. Professor Pin Ng authored exercises in which Mallard was integrated with Excel such that Mallard would randomly choose among a set of pre-selected data sets. Students would then run Excel to compute various statistics on the data. Ultimately, they would report the appropriate ones back into Mallard for grading. Professor Ng also made extensive use of conferencing, to give students rapid feedback on their performance.

In addition to the ALN component, the students began meeting twice per week in a combination lecture-discussion with Professor Ng, followed by one meeting per week with a graduate teaching assistant. The combination of ALN and the shift from TA-taught sections to a combination of faculty and TA-led discussions has allowed the number of graduate assistant discussion leaders in the course to be reduced substantially, realizing a savings of five TAs over the previous approach.

3. Technology
In spring 1996 the Microeconomics course used dedicated software from Agricultural Economics that forced adoption of the Ag Econ textbook that accompanied it. The approach was marred both by a lack of convergence between the topics in the two disciplines and by a failure of the software to work smoothly at the campus residence hall computing sites. In spring 1997 Mallard was employed for the first time and the course returned to the traditional textbook. The latter approach was well received, much better than in the previous offering. In spring 1998, FrontPage was used to develop the course Web site. The course had been using an internally developed Web-based conferencing system. In spring 1999, the course switched to WebBoard. The Economic Statistics course uses Excel in conjunction with Mallard and uses WebBoard, primarily for students to post queries about how to do the Mallard assignments.

4. Personnel 
Professor Larry DeBrock is the project director for the Microeconomic Principles ALN course. Professor DeBrock is in the Department of Economics and known to be the best teacher in the College of Commerce, a winner of teaching awards at the department, the college, and the campus levels. Professor DeBrock was an initial SCALE grantee in a project where he was co-principal investigator with Lanny Arvan. Professor DeBrock could be characterized as an expert computer user.

Professor Ng, the project director for the Economics Statistics course, is a lecturer in the department of Economics and was hired specifically to develop ALN in this course. He is also quite knowledgeable about computers. Professor Ng spent a year co-teaching Intermediate Microeconomics with Professor Arvan, using materials that Professor Arvan had developed. This served as preparation for the ALN component of the Economics Statistics course that Professor Ng developed and implemented in fall 1998.

5. Faculty/Instructor Satisfaction 
Much of Professor DeBrock's satisfaction with ALN might be attributed to his having managed the course development without the enormous personal up-front investment that others have faced. By using commercially available materials and assembling a team that included an ace programmer to put the materials on-line as well as excellent graduate assistants to serve as TAs for the ALN sections, he was able to delegate much of the responsibility for course creation to others. His role was that of supervisor and manager. In spring 1998, he took more direct control of the course by creating a Web site containing lecture material.

While the use of an available test bank may have limited the pedagogic effectiveness of the materials to some degree, it allowed Professor DeBrock to quickly create a functional set of assignments. As the students, in general, have reacted favorably to the approach, Professor DeBrock is quite satisfied with the effort. He is less satisfied with the lecture material he created in FrontPage. A significant fraction of the class appears to have not accessed that material at all. Apparently, for students at that level (mostly first-year undergraduates) presentation of material, without immediate assessment of the comprehension of that material, is inadequately motivating, especially when that presentation is on-line. Since Professor DeBrock had gone to some effort in constructing those materials, the result was demoralizing.

TAs for the course reported the need for revision of the instructions they received on how to respond to students. Originally, they were told to get on-line in the Economics department lab, set up for on-line interaction with students, and to respond to student queries immediately after the posts were made. However, this meant that they really could not do anything else, even when there was little activity, because they had to maintain a degree of accessibility. The TAs soon moved to an approach where they logged in from home via modem once an hour. This change reduced the instructional workload and let them focus on their graduate work, when not occupied with a student. With this single modification, they found ALN teaching more enjoyable than the traditional mode.
 
The problems in Economic Statistics were somewhat different. The students met in Professor Ng's large lecture twice a week, and then in small recitation sections with a graduate student TA once a week. Each TA was responsible for three recitation sections in the fall and five sections in the spring. This high workload occurred because the demand for the course in the spring is almost double the demand for the fall. While TA contact hours were actually reduced, when averaged over the entire academic year, the TAs non-prep out-of-class time increased dramatically because of the much higher student/TA ratio. Especially in the spring semester, this led to a marked degree of TA dissatisfaction with the course. A redesign will have the TAs teach three sections each semester, split between the first course in the sequence and the second course, in accordance with where the bulk of the demand currently lies.

D. Mathematics
1. Rationale for the Project
This project was also motivated by two factors. First, the department already had a distance-learning program in place, known as NetMath, which is based on the Calculus and Mathematica (C&M) approach. In the Differential Equation course, it seemed obvious that the distance learning approach would appeal to some on-campus students, since in the on-campus version some of the students rarely, if ever, came to class. Second, it was apparent that certain engineering students, especially those carrying a high number of credit hours, would benefit from taking their non-engineering required courses in a self-paced mode.

2. Project Description 
The Math Department maintains its longstanding commitment to offering required math courses in multiple modes and letting students select the instructional approach that is appropriate for them. Three approaches are offered in the high enrollment courses-a traditional approach, a Harvard Calculator approach, and a C&M approach. The Differential Equations course for engineering students enrolls about 400 students each semester with two sections being taught in the C&M mode. A totally on-line, self-paced version of the C&M approach was offered as an additional section of about 30 students. It has been easier to offer the self-paced option in the spring and allow the students to complete the course in the summer. In this way, the C&M staff can recruit students who have dropped the traditional course and yet are trying to maintain a high number of credit hours.

3. Technology  
The course relies extensively on Mathematica and most of the course content is in the form of Mathematica notebooks designed to illustrate the material. Homework assignments are notebooks that the students complete. They are uploaded to the course server via a tool internally designed for that purpose in conjunction with a record-keeping function. Much of the communication done in the course occurs via E-mail and occasionally via telephone. Owing to the self-pacing, asynchronous conferencing is not particularly useful in this context. A small number of exercises have been developed in Mallard to complement the Mathematica material.

4. Personnel 
Professor Jerry Uhl is the project director. He has been promoting the teaching of C&M for at least 10 years prior to SCALE's inception. He also runs the NetMath project, an initiative that advocates using the C&M approach as a distance learning activity.

Given that the Math Department was already involved in C&M as a distance learning activity, the extension of the approach to the Differential Equations course only entailed hiring additional undergraduate students who served as graders and mentors.

5. Faculty/Instructor Satisfaction 
Professor Uhl, like Professor Raineri, is a true believer in ALN. As such, he derives tremendous satisfaction from his work. He especially enjoys challenging the establishment in ways that might pay off for the students. A peculiar feature of his asynchronous offering is that all the students in the class receive "incomplete" grades at the end of the spring semester. They then finish the course during the summer session, as true distance students, when they have gone home. While this is a perfectly sensible solution for this type of course, it runs counter to conventional institutional practice which states that undergraduates should avoid grades of incomplete to ensure they are making good progress towards their degrees. Having achieved institutional accommodation on the incomplete grades in the ALN course appeared in the interview to be as satisfying to Professor Uhl as the fact that the students performed well in this totally self-paced arrangement.

A second source of satisfaction comes from the fact that Professor Uhl has built the bulk of the C&M operation relying on undergraduate student labor. He is very proud of this-his assistants have experienced C&M offerings themselves so there is no need to train them in the approach. They work for pay, to be sure, but also because they believe in this style of learning. With undergraduate assistants who are responsible for providing the bulk of general support for the students, Professor Uhl is able to give detailed, personal feedback to students currently taking the course without swamping either himself or his professional staff. But he has run up against the following barrier:  His department (and many other departments) recruit graduate students and assign them teaching assistantships as part of the recruiting package. As long as TA-ships are viewed as a recruiting lure rather than as a wage for teaching, the Math Department considers the graduate students a less expensive labor pool than the undergraduate hourlies, in as much as the graduate students would need to be compensated even if they did not teach. Professor Uhl counters this reasoning with the observation that not all of the incoming graduate students possess the requisite skills to render them successful in this instructional approach, and many are admitted primarily because there is a need to put them in the classroom. Especially in the latter case, the use of undergraduate hourlies in the C&M approach as a substitute for more costly graduate students is cost effective. However, it is so outside the norm to use a substantial chunk of department funds to finance undergraduate teaching assistants that Professor Uhl's arrangement is in the long run a financial jeopardy. In general, the mismatch between institutional culture and innovative solutions to staffing issues appears to be the major downside to his approach. In some instances the institutional constraints may prove too great a barrier, resulting in frustration or even dissolution of the innovation.

E. Chemistry
1. Rationale for the Project 
As with Microbiology, the rationale for ALN in the Introductory Chemistry program was also primarily pedagogic. Chemistry has enjoyed some form of computer-aided instruction since the days of Plato and from this tradition ALN initiatives gradually evolved. In addition, a need to lessen the graduate assistant teaching burden, to keep the Ph.D. program competitive, has emerged in recent years.

2. Project Description
The entire introductory sequence in Chemistry enrolls nearly 4000 students each semester. Chemistry 102, the second course in the sequence, was one of the original SCALE projects. It relied on quizzes written in CyberProf. Ultimately, the approach spread to the first course in the sequence and the other introductory chemistry courses as well. In fall 1997, some sections of the first and second course became part of the SCALE Efficiency Projects. Because Web-based assignments relieved graduate students from a substantial grading burden, instructors were given additional sections to teach. However, the implementation suffered from severe server problems and much of the intended innovation had to be abandoned. With resolution of the server problems, a successful implementation did occur in spring 1998. Under the renewal grant, the department has focused on further development and expansion of the approach to the entire introductory sequence.

3. Technology 
The ALN components of the course rely heavily on CyberProf and in one of the labs they use WebCT. The labs, which are not accessible over the Internet due to bandwidth and security limitations, make extensive use of video.

4. Personnel
Professor Steven Zumdahl is the project director and is also the director of the Introductory Chemistry sequence. He is the author of three textbooks in chemistry, each of which is used somewhere in the sequence. Much of the authoring of the on-line content written in CyberProf as well as the supervision of the servers that the department supports is done by Christopher Jones, an adjunct in the department. Many of the instructors in this sequence are relatively new and there has been high turnover historically. Some of these instructors are tenured faculty in the department while others are adjuncts.

5. Faculty/Instruction Satisfaction
As with Microbiology, the evidence on faculty satisfaction is probably best garnered from information about diffusion of the approach within the introductory sequence. Two years ago many instructors in the sequence resisted teaching with ALN. The reasons for such resistance are at least threefold. First, there was a general distrust of the approach, despite the departmental history. Second, the servers did not appear to be stable. Third, the content that was written into CyberProf did not necessarily match the content that particular instructors wanted to emphasize. This past year all sections in the sequence were taught with ALN.
 
The stability problems seem to have been resolved, which allowed everyone to gain confidence in the approach. The course coordinators continued to author questions for CyberProf so that individual instructors could choose from a menu the topics they would cover, rather than be forced into a prescribed curriculum. Indeed, if instructors wanted to author additional questions they received training in how to add them to the database. Finally, the program experienced substantial instructor turnover. The new instructors are younger and more comfortable using instructional technology.
 
Though at the outset this was not the intent of the project, ultimately, the graduate assistants have captured many of the efficiencies produced in this sequence. Most importantly, their workload has decreased, thus increasing the desirability of a teaching assistantship in the department as a tool for recruitment.

F. Physics
1. Rationale for the Project
This project is distinct from the others both in the way efficiencies are obtained and in its underlying rationale. In all the other projects, efficiency has been considered from the point of view of increasing student/instructor ratios. This project, on the other hand, focuses on the retention problem in the Engineering-Physics sequence. Upwards of 20% of the students in the first Calculus-based course either drop the course or earn Ds or Fs and must repeat the course. The rationale for the project was to use ALN to provide proactive intervention for students who seem most at risk. The project is motivated by a desire to overcome deficiencies in the secondary school education of some of the students in the Engineering/Physics curriculum, in general, and to encourage more minority students to major in Engineering, in particular. About half of the students identified at risk for unsatisfactory performance in the introductory Engineering-Physics course offered in the spring voluntarily enroll in Physics 100, an ALN course offered in the second half of the fall semester, aimed at compensating for their weak prior preparation in Physics.

2. Project Description 
All first-year Engineering students are pre-screened for their Physics knowledge via an exam given in a required one-credit Engineering orientation course. Students who score in the lower 20% receive an invitation to participate in Physics 100, a course offered in the second half of the fall semester, which is aimed at providing remediation for students with weak prior preparation in Physics. (Other students also receive these invitations, based on demographic information that puts them in the at-risk category for success in Physics.)  In the past year, it has been decided to continue with further follow-up work with the students. This follow up is offered concurrently to their taking the first course in the sequence. The bulk of those enrolled in Physics 100 have voluntarily assumed this additional work.

3. Technology
Physics 100, along with the rest of the Engineering-Physics sequence, relies on some internally developed software affectionately known as Tycho, after physicist Tycho Brahe.

4. Personnel
Professor Gary Gladding is the project director for the Efficiency Project in Physics 100. He is Associate Head of the department. A major part of his charge has been the course redesign of the Introductory Physics sequence for Engineering students. The staffing of the Physics 100 course has come exclusively from those who are also involved in the introductory Engineering-Physics sequence.

5. Faculty/Instructor Satisfaction
All the instructors who are involved in this project participate for altruistic reasons. They want to improve the retention and the understanding of Physics for at-risk students enrolled in this course. During the first year of implementation, it was not apparent that what the instructors were doing mattered, a cause of considerable frustration and anxiety. It was hypothesized that some of students were having problems because their general reasoning skills were weak. So they were given exercises in general reasoning that were unrelated to Physics. The teaching assistants thought it was a bad idea and apparently so did the students. The students could not see the connection between the more general reasoning problems and the content in the Physics course. In the next iteration of the course, there were improvements in both the teaching approach and the learning outcomes, with a parallel increase in instructor satisfaction. The general reasoning problems were eliminated and replaced by additional Physics materials. Instructors and students alike were happier.

III. SUMMARY OF THE CASE STUDIES

Below is a table that provides information about each Efficiency Project. We list the automated software that was used and to what purpose that software was applied. Most of the projects also had written work submitted on-line that was not machine-graded. We list those and describe the use. We also summarize how the project attained increased efficiency.

IV. FACTORS CONTRIBUTING TO INSTRUCTOR SATISFACTION

Professor Arvan interviewed each of the project directors for approximately two hours per interview. In some instances, there were follow-up E-mail threads and in a couple of cases the project directors supplied their own written documentation of their projects. As it was clear that graduate students do much of the teaching in these Efficiency Projects, attempts were made to interview them, as well; however, that was possible only in Economics. Professor Musumeci met separately with the graduate assistants involved in the Spanish Project and she has also received reports of instructor satisfaction from Professor Dussias, Director of the Spanish language program. For the other projects, these interviews are still pending. The interviews with the project directors covered several topics - a brief history of the project; a discussion of the efficiencies component and a review of any measure of learning outcomes. Only then did we turn to the third topic, the faculty satisfaction question. In some cases, there were also software demonstrations as part of the interviews.

It became apparent from the interviews that faculty satisfaction can be divided into two separate components. The first and simplest has to do with workload. The second and a bit harder to determine is the perceived quality of the teaching experience, abstracting from the time commitment. We begin this section with some general lessons garnered from these interviews and then proceed to some more project-specific information.

The role of automated Web-based grading software as a device for giving students informed, consistent feedback and keeping them current in the course has already been documented in our earlier JALN paper [2]. The effect on the graduate students and their relationship with the undergraduates they teach has not been commented on to the same extent. The most obvious effect, freeing the TAs from some of their historical grading obligation, seems to impact most the workload component. However, there are other effects that are perhaps equally significant. Many of the interactions that TAs have with their students are either in a clerical role, "The grade on my exam should be an 80 instead of a 76" or in a role of police officer, "I missed the exam because I overslept, but I still think I should be allowed to take a make-up test."  The automated software eliminates the great bulk of these transactions. In some instances, it frees class time to be used for other purposes, ones that are more directly related to course content. In other cases, it eliminates the need for class time entirely. Secondly, the power relationship between the TA and the undergraduates changes as a result of computerized grading and maintenance of assignment deadlines. Because it is the computer that tells the student that an answer is incorrect (or that an assignment is late), the TA is no longer the bearer of bad news and the students can accept the feedback as objective rather than as personal and vindictive. This makes the student more comfortable in seeking out the TA for additional tutoring. In general, the relationship between graduate student instructor and undergraduate learner is less adversarial. Not surprisingly, that seems to really improve TA satisfaction.

All of the courses described in the Efficiency Project case studies are based to a large extent on students doing homework. Because the students drive many of the interactions with their instructors-as they seek help on homework assignments-a disproportionate number of the interactions come from those students who are relatively weak. The strong students tend to proceed on his or her own without this sort of help. This is the opposite of what happens in a face-to-face class-where it is usually the better students who volunteer to answer questions, work problems, and who may dominate discussion. These outspoken students often overwhelm those who are shy or who have a less firm grasp on the course material. Increased contact with students who are experiencing difficulty with the subject matter and diminished contact with those who are excelling, produces mixed effects on instructor satisfaction. On the one hand, the instructors see the utility of interactions of the type, "How do you do x?"  Through these threads, the instructors help the students gain understanding of the material. In the Math and Physics cases, in particular, the TAs have self-selected into the job to fulfill this pragmatic mission. On the other hand, some of the instructors miss the banter with the better students. The instructors recognize that the latter probably enjoy this type of interaction in some of their other courses, so the regret does not stem from a feeling that somehow these students are missing an educational opportunity. Rather, it comes from the loss of a joyful part of teaching in the conventional approach.

On a related note, some of the instructors who are now doing less presentation of materials in class and more coaching or tutoring with students who are having trouble express mixed feelings about their change in role. Although they acknowledge that the presentations themselves may have little learning benefit for the students, some of these instructors like to perform in front of an audience and they miss the opportunity to use a lecture to do so.

While these Efficiency Projects have obviously been encouraged by Sloan Foundation dollars, each was selected on its potential to resolve some issue within the unit, the campus or, in the case of the Spanish Project, across the nation. It is not surprising, therefore, that all the project directors showed immense pride in what they have accomplished. Their projects improved instruction in concrete ways and each expressed tremendous satisfaction with how their design solved the learning problems that were specific to their course. However, because this group consists of dedicated teachers, this finding may be better attributable to project director characteristics than it is to ALN course design per se. Moreover, it should be emphasized that many of the instructors in these courses are graduate assistants who, for all intents and purposes, can be viewed as transients to ALN teaching. In our data gathering, we tried to be sensitive to their perceptions as well as to the perceptions of the course coordinators. The results of these and future interviews with graduate student instructors, a group that might include novice or less enthusiastic teachers, may prove more insightful in this regard.

One significant problem that several course coordinators commented upon is the abundance of E-mail traffic that is directed to the course coordinator from the students and that is not related to course content per se but instead to some course management/technology issues. Living under a flood of E-mail has a deleterious affect on faculty satisfaction. Other institutions appear to have solved this problem through the development of impressive campus support networks. In the UIUC Efficiency Projects, one tactic to mitigate the problem has been to obtain a course E-mail address and strongly encourage students to use it rather than to use the faculty member's personal E-mail address. Although this strategy has no affect on the volume of the traffic generated, it does allow the instructor to "shut out" the class when engaged in other activities. Another tactic is to make responding to the E-mail the primary responsibility for a member of the course staff, other than the course coordinator. In the Spanish Project, an undergraduate was hired on an hourly basis to respond to student E-mail inquiries about technological problems. It should be noted that conferencing does not solve this problem. Even if there is a message board set up for this type of post and FAQ's, our experience has been that students send E-mail instead.

It is noteworthy that in most, if not all, Efficiency Projects, the vast majority of the students are taking these courses to fulfill a campus or college requirement with no intention of majoring in the subject. It seems that a critical component in determining faculty satisfaction when teaching these courses is whether the instructors have come to grips with the student motivation for being in the course. The level of satisfaction achieved in these high enrollment, introductory level, undergraduate courses that constitute the Efficiency Projects is perhaps even more impressive when one considers that they are not graduate or professional certification courses in which one might expect a highly motivated audience.

A final factor that may have contributed to the high level of satisfaction among project participants is that the Efficiency Projects were constructed either from already mature ALN course development or by faculty who served in an apprentice relationship with others who had extensive experience in ALN development. In no case did we start from scratch and attempt to produce an ALN course that immediately delivered some kind of efficiency outcome.

Ancillary Support and 'Hidden' Subsidies
In the September 1998 JALN paper [2], a detailed costing of the in-kind support was done only for the Econ 300 project. There is an issue of how representative that project is of the other projects and how these projects compare to other courses offered on campus. All of the Efficiency Projects were in high-enrollment courses. This is an important point to consider in its own right. There are relatively few high-enrollment courses on campus. Many of the faculty do not have experience teaching such courses and, it seems reasonable to conjecture, most have no desire to teach such a course. High-enrollment courses are likely to entail less personal interaction with the students and much more administrative overhead in the form of supervising TAs and dealing with student complaints. To avoid the apples and oranges comparison of ALN in high enrollment courses versus traditional teaching in a seminar-like environment, we think the right benchmark for comparison is the same course prior to ALN, not traditional teaching in general. One question to be addressed by looking at the data on faculty satisfaction is whether conversion of other such high-enrollment courses to an ALN approach can be implemented willingly by the participants (with concomitant learning benefits for the students). A related question, relevant for the Efficiencies Projects in particular, is whether the cost savings that have been identified are real or, if instead, there are hidden subsidies in terms of increased effort from the course coordinators that render any cost savings on paper artificial. If those subsidies exist, they should show up in the satisfaction data.

The Spanish Project, Economics, and Mathematics use the software Mallard, developed at UIUC by Professor Donna Brown in the Department of Electrical and Computer Engineering. SCALE has a staff person with the title Mallard Administrator who assists faculty in the design of their Mallard-based materials and who also performs a lot of trouble-shooting functions. SCALE sub-contracts out for the administration of the Mallard server from the campus computer support organization. The Mallard Administrator manages that relationship. This service is provided for all instructors who use Mallard, not just those involved in the SCALE Efficiency Projects. However the bulk of the courses that use Mallard are high-enrollment courses. Taking advantage of Mallard's sophisticated quizzing features requires extensive authoring and it is hard to justify that authoring effort in a small course, especially one not taught repeatedly.

The Spanish Project and Economics also use some dedicated conferencing software, either FirstClass or WebBoard. SCALE administers the servers for these products, creates class conferences and student accounts, and provides training for new faculty users and for students. Again, this is a service that SCALE provides for all campus users. Indeed, conferencing is extremely popular across the board and in the main providing conferencing support is no different for the Efficiencies Projects than it is for the other courses SCALE supports. However, in the case of the Spanish Project, SCALE took the exceptional step of creating a FirstClass server dedicated exclusively to Spanish and Italian. This allowed SCALE to give the instructors administrative access to the server without compromising the privacy of other instructors and courses.

Several of the projects administered software themselves within their own units. Both Microbiology and Chemistry use CyberProf (which has a built in conferencing component) and, while SCALE has had a hand in the administration of the large Solaris server that the campus purchased to support both CyberProf and Mallard, SCALE has had no direct involvement in CyberProf support. Mathematics relies on E-mail and telephone for communication between students and mentors and provides its own support for Mathematica and the support of the software used to upload Mathematica notebooks that students submit for grading. This is the same software utilized for both their on-campus operation and their distance-learning program. Similarly, Physics offers extensive support of its own software, which it has developed in-house.

Does the support that SCALE supplies to the Efficiency Projects constitute a hidden subsidy for these projects? Likewise, what about the support that comes from within the units that are sustaining their own software?  These are tricky questions. The faculty who direct the Efficiency Projects go through a lot of administrative overhead as a matter of course. They have to deal with both departmental administration and campus administration. Some of this is quite bureaucratic. If the ALN support component seems efficient, by comparison, does that mean it is subsidizing the teaching activity?  On the other hand, once the proof of the concept phase is passed and we experience a scaling up of the ALN support, will the ALN support component continue to seem efficient to the faculty consumers?  It seems too early to tell the answers to these questions.

Despite the fact that increased faculty workload did not come up in the interviews with the project directors, and may, in fact, be a non-issue within this particular faculty profile, it is obviously another serious concern about whether large-course ALN development is sustainable, as the up-front development in these courses is even greater than for the more standard-sized course. Indeed, to emphasize the point, Professor Musumeci relayed how Professor Escobar felt that there was an enormous amount of work in the course development component of designing materials for Spanish 210 in Mallard. Regardless of faculty effort, the course redesign in Spanish 210 clearly was laborsaving for the graduate assistant instructors, as evidenced by their preference to teach it over their other alternatives. This is in spite of the fact that they are teaching twice the number of students as they have historically and that they have gone to two class meeting times per week (down from the historical norm of three but not as low as the one meeting hour per week in the first year of the ALN implementation.)  The workload issue should perhaps be evaluated both longitudinally with the hypothesis that it might decrease dramatically over time/experience and at various levels of course administration: project director, materials developer, course director/supervisor, and instructor/graduate TA. We contend that the conflagration of too many of these roles in one person might contribute to the tremendous increase in workload (real or perceived) for some ALN faculty.

V. CONCLUSIONS

TAs in high enrollment courses generally seem to like the ALN approach. It improves their work environment. The directors of the Efficiency Projects are extremely upbeat about their ALN experiences. Where they express dissatisfaction, it is with various institutional rigidities that they must confront, not with the teaching approach itself. In several of these courses, we are seeing more broad-based diffusion of the teaching approach across curricular, departmental, and disciplinary boundaries. This is happening through a de-coupling of the on-line content design and the course delivery. That diffusion is an indication the ALN approach is as satisfying for mainstream faculty as the traditional approach

REFERENCES

1. Arvan, L. Bottom up or top down? Using ALN to attain efficiencies in instruction. Presented at the Third International Conference on Asynchronous Learning Networks, October, 1997. http://www.aln.org/conf97/slide/arvan/arvan/index.htm.
2. Arvan, L., Ory, J. C., Bullock, C. D., Burnaska, K. K., and Hanson, M. The SCALE efficiency projects. Journal of Asynchronous Learning Networks, Vol. 2 No. 2, September, 1998, http://www.aln.org/alnweb/journal/vol2_issue2/arvan2.htm.

ACKNOWLEDGEMENTS

We are grateful to Giuli Dussias, Deanna Raineri, Larry DeBrock, Jerry Uhl, Steven Zumdahl, Christopher Jones, and Gary Gladding for their willing participation in the interviews that were the basis for this paper. We are especially indebted to Olin Campbell, whose detailed comments on an earlier draft greatly improved this version of the paper.

ABOUT THE AUTHORS

Lanny Arvan is the Director of the Sloan Center for Asynchronous Learning Environments (SCALE), Director of the new campus Center for Educational Technologies (CET), and an Associate Professor in the Department of Economics at the University of Illinois at Urbana-Champaign. In 1997-98, under Arvan's leadership, SCALE pursued a variety of experiments with ALN instruction (the Efficiencies Projects) aimed at showing that ALN can lower the cost of instruction in high-enrollment courses. (See "The SCALE Efficiencies Projects," with John C. Ory, Cheryl D. Bullock, Kristine K. Burnaska, and Matthew Hanson, Journal of Asynchronous Learning Networks, Vol. 2, No. 1, September 1998, www.aln.org/alnweb/journal/jaln_vol2issue2.htm#arvan

Arvan has also led SCALE to the next phase of ALN implementation. Most of the initial SCALE success was with highly motivated faculty who required little support from SCALE other than a bit of initial training and access to servers. SCALE, and now the CET, have turned to getting mainstream faculty engaged in ALN teaching. This change in audience is necessitating a change in the nature of support, where training is but a component, and helping faculty think hard about their course re-design is more of a focus.

Contact: Department of Economics, University of Illinois at Urbana-Champaign, 182 Armory Building, 505, East Armory Street, Champaign, Illinois 61820; Telephone: 217-333-1078; Fax: 217-333-5123;
E-mail: l-arvan@uiuc.edu.
 
Diane Musumeci is Associate Professor and Associate Head in the Department of Spanish, Italian and Portuguese at the University of Illinois at Urbana-Champaign. She teaches undergraduate and graduate courses in Italian linguistics, second language acquisition theory and research, and foreign language teacher education. Her research focuses on second language acquisition in the classroom context. She is the author of Breaking Tradition: An exploration of the historical relationship between theory and practice in second language teaching (McGraw-Hill 1997). Since 1998 she has also directed The Spanish Project, a campus initiative to deliver basic language instruction in Spanish more efficiently through instructional technology.

Contact: Department of Spanish, Italian and Portuguese, University of Illinois at Urbana-Champaign, 4080 Foreign Language Building, 707 South Matthews, Urbana, Illinois 61801; Telephone: 217-333-3390; E-mail: musumeci@uiuc.edu.

ENDNOTES

(1) In the Chemistry course, which is offered only in the fall semester, the instructor allowed the students to start the course in the summer. (Indeed, in subsequent offerings she increasingly anticipated the starting date so that the students could do the necessary catch up work at their leisure.) In the Differential Equations course, students who had dropped out of traditional sections of the course and who preferred to attempt the ALN version, rather than postpone taking the course until the subsequent year, comprised the asynchronous C&M version, offered only during the spring. Again, as in the Chemistry course, the ALN version of the Differential Equations course allowed the students to take the course in a self-paced mode. As a result, most of the students completed the course, but not until several weeks into the summer term.