Home

ALN in a Small On-campus Engineering Class

by Sloan-C

ABSTRACT
A custom-designed ALN (Asynchronous Learning Network) was introduced as an enhancement to an electrical circuits course at the University of Louisville. The goal was to combine ALN techniques with traditional classroom methodologies. The experiment was a mixed success. While students reported that they found the ALN elements to be effective in their learning, there were no significant differences in test and homework scores with the addition of ALN. This paper documents the study, and provides a discussion of the design and selection of ALN features that may be helpful to others integrating ALN into small classes.

I. INTRODUCTION

In the transition from traditional teaching methods to asynchronous learning networks (ALN), many instructors will gradually introduce ALN into their traditional classes. This is in contrast to the exclusive use of ALN with no classroom component. There is ample evidence that ALN works well for off-campus educational delivery [1], [2], but how well does it work for serving on-campus students as an adjunct to a traditional course? It is also clear that ALN can enhance efficiency and effectiveness in large classes [3], [4], [5], [6], [7], but how well does it work in smaller classes (up to 40 students)? What special role does ALN play in the teaching of engineering [8]? This paper describes the author's experience with these issues and makes recommendations as to what elements of ALN seem to work best with on-campus students in small classes in general, and in engineering classes in particular.

The University of Louisville is a large urban university offering a wide range of degrees and programs. Speed Scientific School, the engineering school of the University, serves approximately 1800 students, including MS and Ph.D. students. Most of the engineering students are commuters. Although there are a few large calculus sections, classes are generally small by modern standards, most having fewer than 40 students.

Many colleges and universities are similar to the University of Louisville, in that most of their students commute or reside at the institution. Further, these institutions frequently offer small classes. For such institutions, it may not be expedient to abruptly switch the teaching of a small class from a traditional lecture-based mode to an ALN-only mode. As a practical matter, elements of ALN are more likely to be gradually introduced.

II. BACKGROUND

It was decided to attempt to introduce ALN into an introductory electrical engineering class, EE 220, Network Analysis I. This course has a typical enrollment of 25 students, and is a required first course for electrical engineering students. These students have little or no previous exposure to ALN.

Perhaps naively, the author chose to develop new ALN delivery tools rather than to purchase commercial software. This decision, as seems to be common in academia, was driven in part by fiscal considerations--no funding was available to purchase software.

The author was aware of the components that make up an ALN, although he had limited experience with them. He had read the literature on Mallard [9], CyberProf [10] and other ALNs [11]. The components of an ALN may include class notes, syllabi, conferencing, registration, gradebook, interactive tutorials, quizzing/testing, and instant homework grading, with all these components being available online at any time.

With the aid of a graduate student, the author began developing the ALN environment containing the above components. It is called RAISE, for Remote Asynchronous Instruction in Science and Engineering. It provides online lessons, tutorials, links, conferencing, and registration. An electronic gradebook is implemented that allows students to check their own grades online. RAISE does not yet provide online testing or rapid-feedback homework grading, but a module is being developed to include these features. RAISE is implemented on a Windows NT system using HTML, JavaScript, and Active Server Pages.

The online lessons are a substitute for photocopied class notes, but they also allow the instructor to make changes easily and to provide hyperlinks to other web-based material. The online lessons emphasize text and lecture material and provide homework problems. Including the homework in the online lessons made it necessary for students to access the lessons.

Twelve interactive tutorials were prepared for EE 220. Each tutorial is a set of linked HTML pages, with one question on each page. Typically, a tutorial guides the student through the solution of a problem, giving progressively more specific hints with each wrong answer. JavaScript is used for answer entry and answer checking. This module is not tracked in any way; therefore it cannot be used for grading. In its present form, it is not even possible to determine if any specific student has attempted a tutorial. More will be said about this later. Interested readers are invited to download a template of the tutorial from http://RAISE.spd.louisville.edu/SysTom/Educator/freetut.htm and use it for their own classes.

The conference module provides threaded discussions that can be reviewed by everyone in the class. It does not permit file attachments, in-line graphics, or styled text. It is intended for student-student and student-instructor dialogue.

In the Spring of 1998, RAISE was first used as an adjunct to the teaching of EE 220, Network Analysis I. During this term, RAISE was in a rudimentary form. The electronic gradebook had not been implemented, and only about 70% of the lessons and tutorials had been written.

In the fall of 1998, the EE 220 course was again taught, but using the full RAISE implementation with fully developed course materials. The analysis that follows documents experiences with these two classes.

III. METHODS AND OUTCOMES

Three approaches were taken in order to determine the strengths and weaknesses of our use of ALN as an adjunct to a traditional course:

• Comparison of the scores of students in the ALN-enhanced classes with those in classes without ALN enhancement,
• Analysis of student reaction to ALN, as evidenced by an anonymous questionnaire, and
• Subjective and anecdotal analysis based on the teacher's experience.

A. Analysis of Student Scores
The author has taught EE 220, Network Analysis I, six times in recent years. Throughout this time, the course has covered substantially the same material, and has been taught using substantially the same methods, except that in the last two semesters, ALN was introduced. EE 220 is a lecture-based course, consisting of three fifty-minute lectures in each of the fourteen to fifteen weeks of the semester. Twelve to thirteen homeworks are assigned, collected, and graded. Thirteen to fourteen twenty-five-minute quizzes are given. A two and one-half hour final exam concludes the course.

Table 1 presents the scores achieved by EE 220 students.

Table 1. Student Scores in EE 220.

The first four columns correspond to pre-ALN classes, and the last two columns correspond to classes taught with ALN as an add-on. No consistent pattern can be found in these data. Variations in scores may be due to many causes: Since different grading assistants were used in each term, differences in grading standards can be expected. The quality and preparation of the students can be expected to change from term to term. Some term-to-term difference in average difficulty of testing instruments can also be expected. The effect of introduction of ALN in the last two terms may therefore be buried in the noise of the other variables.

Another factor that may play a role in failing to find a significant improvement in student performance is that the pre-ALN version of the course was quite successful. Students reported great satisfaction with the elements of the pre-ALN course. Regardless of the quality of the new ALN elements, it is not surprising that many students chose to use the traditional elements.

B. Analysis of Student Questionnaires
In the fall semester of 1998, a questionnaire was administered to EE 220 students. The questionnaire attempted to determine students' attitudes about the various activities in the course, and particularly about the ALN activities. Seven activities were considered:

• Lectures - The presentations by the instructor
• Textbook - Reading and studying from the text, including assigned readings
• Homework - Weekly homework assignments
• Practice Quizzes - Before each quiz, a take-home practice quiz was distributed.
• Online Lessons - Web-based class notes
• Online Tutorials - Interactive tutorials accompanying each lesson
• Other - Anything else the student did, like studying for quizzes.

Each activity was scored in four categories:

• Effectiveness - How important the activity was in the student's learning, given a rank from 1 (most effective) to 7 (least effective)
• Efficiency - How quickly learning occurred for the activity, given a rank from 1 (most efficient) to 7 (least efficient)
• Time - Time spent per week on the activity
• Percent Accomplished - How much of the assigned activity the student actually did

The results of this questionnaire are presented in Table 2.

Table 2. Results of Student Questionnaire.

These results clearly show that students believe that lectures are the most effective modality for learning the course material. This is not surprising as students spend the most time in this activity (147 minutes per week). Students also report that lectures are their most efficient means of learning. Nevertheless, as can be seen from the standard deviation of this entry and from the efficiencies reported for other activities, many do not perceive lectures as being particularly efficient.

It is also abundantly clear from these data that students do not like reading the textbook. On the average, students completed about a quarter of the assigned reading. Furthermore, the students ranked this activity as the least efficient in learning the material. This outcome cannot be blamed on a poor choice of textbook [12]. It is one of the most popular texts in the field. One possible explanation is that the students felt that the textbook material was redundant with material presented in other activities (lectures and online lessons).

Homework was ranked second in effectiveness, which correlates with the relative amount of time spent on homework--122 minutes per week, second only to the time spent in class attendance. Homework also ranked high in efficiency. This suggests that instructors should take great care in selecting homework problems.

In the class period before a quiz was to be administered, a "practice quiz" was handed out. A practice quiz is a copy of a quiz that was administered in a previous term. Many students found this to be an effective means of studying for the upcoming quiz.

Online lessons are class notes posted on the Web. They summarize the material for the weekly unit, provide hints, and contain the homework assignment. Students reported these lessons as being of moderate effectiveness and efficiency.

Each online tutorial is an interactive multi-part problem based on the material of the current unit. HTML and JavaScript are used. Students reported that these tutorials were of moderate effectiveness and efficiency.

The "other" category was a catch-all for everything else the student did in the class. Surprisingly, students spent a mean of one hour per week in these activities.

Five of the seven activities, Lectures, Homework, Practice Quizzes, Online Lessons, and Online Tutorials, had roughly equal efficiency ranks. This seems to reflect students' different learning styles. One student may learn best from a lecture, while another learns best from working homework problems, while yet another learns best from online materials. This well-known result suggests that instructors should provide a wide variety of activities for students to meet the needs of their various learning styles.

1. Student Access to the ALN
In addition to student attitudes toward the course activities, it was also important to determine student access to the ALN. The following entry appeared on the questionnaire:

Access to a computer to do online lessons has been a problem.

Students answered this on a scale from 1 (definitely yes) to 7 (definitely no). The result was a mean of 6.09 and a standard deviation of 1.79. As expected, engineering students have ready access to the technology necessary for an ALN-enhanced class.

2. Quality and Desirability of ALN Materials
Using the same scale as above, students also responded to the following:

The online materials were easy to use.

This got a mean score of 1.65 with a standard deviation of 1.47. This indicates students had no trouble using the ALN.

Another entry was the following:

The overall quality of the online materials was good.

This received a score of 1.71 with a standard deviation of 1.09, indicating student satisfaction with the quality of the ALN.

Another item read:

More online lessons should be developed to replace lectures.

This got a score of 5.09 with a standard deviation of 2.31. This suggests that most students were opposed to migration to a full ALN course. They still prefer the lecture paradigm. This attitude may be due, in part, to the students' unfamiliarity with ALN-only courses.

3. Use of the ALN Conference Module
As mentioned earlier, the RAISE ALN includes a conference module that allows for threaded forum discussions. This module was available in both the Spring and Fall terms of 1998. It was underutilized in both terms, generating only 10 messages in the Spring, and 16 in the Fall. To determine why this was so, the following item appeared on the questionnaire:

The online forum is rarely used. It is intended as a place for students and the teacher to chat about issues related to the course. Please comment on this below.

Typical comments were:

• I have never used the Forum.
• I do not have time.
• It's usually easier talking at school.
• I'm not sure what should be discussed in the Forum.
• I prefer to talk out my problem in person.

As has been mentioned elsewhere [13], [14], the mere presence of conference capability does not guarantee its use. Since some [15] have achieved success using conferencing, and others [16] have had mixed results, it would seem that the students need a clear statement of the purposes of the Forum, and then need incentives to use it. More will be said about this later.

C. Subjective and Anecdotal Analysis
Each of the interactive tutorials contained a feedback form where students were asked to anonymously comment on the tutorial. Approximately 70% of the responses expressed approval, such as, "Very helpful." Approximately 20% offered suggestions for improvement, such as, "Don't make the answers case sensitive." Approximately 10% requested more or longer problems.

These responses suggest that students think they learn well from interactive tutorials. Although it is also clear that our tutorials can be improved.

In general, the ALN appears to be a valuable enhancement to the course. Very few negative comments and many positive comments have been received. Students voluntarily use the ALN activities that they believe will best facilitate their learning.

IV. DISCUSSION

In many ways, the deck is stacked against the application of ALN to enhance existing, small, on-campus courses. The economics of scale available to large classes are not available to small classes. The effort necessary to create the web-based materials is about the same, regardless of class size. For the small class, it is not economically feasible to have large numbers of student facilitators to monitor the online conference and answer questions.

ALN-only classes have the advantage that students must use the ALN environment. In ALN-enhanced classes, students have other options for learning, such as attending lectures, viewing printed materials, interacting with other students one-on-one, and visiting the instructor rather than using e-mail or an online conference. Therefore, instructors should not be surprised if adding ALN elements does not dramatically improve their small classes. Nevertheless, improvements, if not dramatic, are still possible with good planning and a dedication to the purpose.

The author's plan for adding ALN to his courses was often desultory and unfocused. The attempt was to design and implement a full-featured ALN, without giving a lot of thought to how those features would be used, what improvements in learning would be made, or what outcomes were expected. The successes, partial successes, and failures of this attempt are discussed next.

The basic structure of the ALN appears to be highly successful. The modules for grading, announcements, calendar, syllabus, registration, and lessons (class notes) all work well and are easy for students and faculty to use. The interface is clean, uncluttered, and intuitive.

A basic shortcoming of the entire RAISE ALN is that there is (at present) no student accountability. Other than downloading the homework assignments, there is no requirement for students to use the ALN. There is no tracking of student use. It's difficult to require use where there is no tracking. Students may perceive the ALN as an add-on that does not directly affect their grades. It is therefore remarkable that students reported that the online materials were effective in their learning.

The interactive tutorial module is a qualified success. Students report that it is easy to use, and aids them in their learning. Nevertheless, it currently has the serious shortcoming that tutorials cannot be tracked. In its present configuration, the instructor cannot know who is using them, nor can the instructor use it for grading purposes. As a result, students do the tutorials only on a voluntary basis. This might imply that only motivated students will do them. It was therefore gratifying to note that students reported completing an average of 64.7% of the tutorials.

At the present, there is no online quiz module and no online instant-feedback homework module, although these are in development. This is a serious shortcoming in an ALN. Although in-class quizzes and paper-and-pencil homework are valuable teaching tools outside of ALN, research has shown that rapid feedback can have a profound effect on student performance [8]. Once the RAISE quiz/homework module is completed, it may be used to reinforce and eventually replace the interactive tutorial module.

The conference module was a dismal failure. Few students knew what it was for, and those few didn't find it useful. Student indifference to this module is probably due to three factors: its use was not required, students could obtain information more easily through other means, and the teacher (the author) had no clear plan for the use of this module. The small size of the class made active threaded conferences among students less likely, and the absence of student assistants made rapid answers to posted questions impossible. A significant improvement to the conference module would be automatic e-mail to the instructor whenever a new post is made (this is in development). On the other hand, the teacher had a policy of welcoming students in his office at anytime, and many students had their questions answered that way. It seems inappropriate to require conference use merely for the purpose of proving that it is a worthwhile feature of ALN. The author therefore concludes that online conferencing is of limited usefulness for ALN enhancement of small on-campus classes. Nevertheless, there can be many advantages to conferencing in other environments and with larger classes. Karen Schwalm [17] has articulated many of them.

V. CONCLUSIONS

The studies conducted here relate to how well ALN integrates into an existing small on-campus engineering class. Some of the conclusions may generalize to larger, off-campus, or non-engineering courses, but such generalizations should be made with caution.

The data do not justify the conclusion that the introduction of ALN improved students' performance, although the author believes that such improvements have actually taken place. Other researchers have found significant improvement with larger classes in other environments [4].

Engineering students have easy access to the equipment they need for ALN communication. They are accepting of the ALN technology and find it easy to use.

ALN conferencing will not be successful unless students clearly see its advantages, or are given incentives to use it. It seems to be of limited usefulness in small on-campus classes.

Given the choice of many learning activities, different students make different choices. Some select traditional activities (lectures, homework), others select ALN (online lessons and tutorials), and still others select some mixture of these activities. To maximize learning, instructors can make a wide variety of learning activities available to the student.

VI. REFERENCES

1. Sener, John, and Stover, Merrily, An AS Engineering Degree Program Via ALN, FIE '97 Frontiers in Education Conference, Nov. 1997.
2. Hislop, Gregory W., Evaluating an Asynchronous Graduate Degree Program, FIE '97 Frontiers in Education Conference, Nov. 1997.
3. Arvan, Lanny, Ory, John C., Bullock, Cheryl D., Burnaska, Kristine K., Hanson, Matthew, The SCALE Efficiency Projects, Journal of Asynchronous Learning Networks , Vol. 2, Issue 2, September 1998.
4. Kashy, E., Thoennessen, Michael Tsai, Y., Davis, Nancy E., and Wolfe, S. L., Using Networked Tools to Enhance Student Success Rates in Large Classes, Proceedings of the Frontiers in Education, 1997.
5. Bourne, John R., Net-Learning: Strategies for On-Campus and Off-Campus Network-enabled Learning, Journal of Asynchronous Learning Networks, Vol 2, Issue 2 - September 1998.
6. Bourne, John R, McMaster, Eric, Rieger, Jennifer, Campbell, J. Olin, Paradigms for On-Line Learning: A Case Study in the Design and Implementation of an Asynchronous Learning Networks (ALN) Course, Journal of Asynchronous Learning Networks, Vol. 1, Issue 2, August 1997.
7. Campbell, J. Olin, Evaluating Costs and Benefits of Distributed Learning, FIE '97 Frontiers in Education Conference, Nov. 1997.
8. Oakley, Burks, A Virtual Classroom Approach to Learning Circuit Analysis, IEEE Transactions on Education, Vol. 39, No. 3 - August 1996, pp. 286-296.
9. Swafford, Michael, Graham, Charles R., Brown, Donna J., and Trick, Timothy N., Mallard: Asynchronous Learning in Two Engineering Courses, FIE '96 Frontiers in Education Conference, Nov. 1996.
10. Raineri, Deanna M., Mehrtens, Bradley G., and Hubler, Alfred W., CyberProf - An Intelligent Human-Computer Interface for Interactive Instruction on the World Wide Web, Journal of Asynchronous Learning Networks, Vol. 1, Issue 2 - August, 1997.
11. Barker, D. Steven, CHARLIE: A Computer-Managed Homework, Assignment and Response, Learning and Instruction Environment, FIE '97 Frontiers in Education Conference, Nov. 1997.
12. Nilsson, James W. and Riedel, Susan A., Electric Circuits, Fifth Edition, Reading, Massachusetts: Addison-Wesley, 1996.
13. Hiltz, Starr Roxanne, Impacts of College-level Courses via Asynchronous Learning Networks: Some Preliminary Results, Journal of Asynchronous Learning Networks, Vol. 1, Issue 2 - August 1997.
14. Burrows, V. A., Electronic Conferencing in Undergraduate Engineering Classes, FIE '97 Frontiers in Education Conference, Nov. 1997.
15. Ory, John C., Student Use of and Attitudes about On-Campus ALN, FIE '97 Frontiers in Education Conference, Nov. 1997.
16. Wheeler, Bruce, Magin, Richard, Osborne, Margery, and Bruce, Bertram, Asynchronous Learning in the Small Engineering Classroom, 1996 ASEE Annual Conference Proceedings, Session 2532.
17. Schwalm, Karen, Using Computer Conferencing to Enhance Classroom Instruction, 1995. http://staff.gc.maricopa.edu/~kschwalm/ccguide/

VII. ABOUT THE AUTHOR

Dr. Thomas G. Cleaver is a Professor of Electrical Engineering at the University of Louisville. He teaches classes in electric circuits and computer design. His areas of research include microcomputer design and instructional technology.