Home

The WebTester and the Linear Algebra WebNotes

by Sloan-C

ABSTRACT
This article describes the on-going project to create comprehensive web-based mathematics courses. These will include HTML books, interface to Computer Algebra Software (Maple), and software for administering non-multiple choice web-based tutorials and tests. Featured is the Linear Algebra Web course, which serves as a prototype for future courses including Basic Algebra, Calculus, and Differential Equations.

I. GOALS AND OBJECTIVES: THE PROBLEM

In 1996 while teaching from Anton's linear algebra book, I decided to put my own notes for the course on the Web. Only the first version of HTML was available then, so in the first version of WebNotes there were no subscripts or superscripts. The first version of the Linear Algebra WebNotes was completed by May 1996. Students liked the notes, so work on them continued. As the Notes grew, several pieces of software were added. Rex Dieter created an interface to Maple, Maple Front End. Now students could use Maple while working on their homework assignments and take-home tests. Travis Fisher, an undergraduate student, made the first testing software, which simply sent a previously written test to a student and sent me a note saying that a certain student started taking a test at certain time. When a test was completed, the student sent me the solution by e-mail. This way take-home tests with time limits could be administered. Travis Fisher also created the first version of the Grades database, which allowed students to look at their grades (only) on the Web and allowed an instructor to change and add grades. Later the WebNotes were converted to HTML-2, some pictures were added, and I designed and Rex Dieter implemented the WebTester, an engine for administering and automatically grading web-based non-multiple choice tests. This article presents the main features of the Linear Algebra WebNotes http://www.math.vanderbilt.edu/~msapir/cgi-bin/v.cgi, the WebTester http://www.math.vanderbilt.edu:8888/, and ideas about how to use the WebTester to administer calculus and pre-calculus placement exams.

A. Why Linear Algebra?
Linear algebra is one of the most important college math courses. It introduces students to the concepts of matrices, systems of linear equations, vector spaces, and linear transformations. These concepts are crucial in engineering, business administration (especially in dealing with investments), physics, mathematics, and other disciplines. Therefore, linear algebra is a fundamental course for students specializing in these disciplines.

Every year, thousands of students in the U.S.A. take linear algebra. However, linear algebra classes are usually smaller than calculus classes, and more suitable for experimenting with technology.

B. What Makes Linear Algebra Difficult?
To master linear algebra, students must solve not only computational problems, but also many proof-oriented problems. Unlike calculus, which is based on concepts (e.g., equation, function, graph of a function, trigonometric functions) already familiar from high school, the concepts of linear algebra are completely new for students, and these concepts are much more abstract. To familiarize themselves with these concepts, students need the hands-on experience of solving many computational problems. Literature in cognitive science shows that hands-on experience is extremely important preparation for understanding concepts that are introduced in subsequent discussions or lectures (e.g., [1] and [2]). The time that an instructor or a teaching assistant can spend grading these problems is limited, so students generally do not spend enough time solving computational problems. This is a point where technology can help: A computer can automatically (randomly) generate a problem, check the solution, and give a grade. Leaving computational problems to a computer, an instructor can concentrate more on concept-oriented "proof problems." For most students, linear algebra is the first truly deductive mathematics course, i.e., the first encounter with abstract proofs. (The proofs in high school geometry are much more concrete. Typically, a student describes the diagram, and thus translates from pictures to words and/or symbols. This translation process is not applicable to the proofs in linear algebra, a less pictorial subject.) The concept of a proof is very complicated in itself, and it becomes more difficult when the subject is unfamiliar. "Proof problems" are also more difficult to grade than computational problems: each solution is a little essay. Computational problems in linear algebra usually require a great deal of time; for instance, a typical system of three linear equations with three unknowns takes at least 10 or 15 minutes to solve. Consequently, an instructor usually has little time during lectures to talk about applications of linear algebra to other disciplines. However, most students in the class major in engineering or business management, and they use linear algebra in other courses and in labs. Generally, these other courses do not explain why this or that formula is used. It would be helpful to have time to explain at least some of these applications in the linear algebra course.

C. Pre-calculus and Placement Exams
Another major difficulty experienced by most math instructors in the U.S.A. is that students are badly prepared for the courses they take. In particular, many students of linear algebra do not remember essential material from the third semester of calculus, and students taking calculus do not remember pre-calculus (algebra and trigonometry). Many math departments have placement exams that allow them to check what level of calculus a student should take.

The Vanderbilt University Department of Mathematics spends the first two weeks of the calculus sequence on reviewing pre-calculus material. At the end of these two weeks, the students take a placement exam. Our experience shows a good correlation between a student's score on this pre-calculus exam and the student's performance in the calculus course. A web-based pre-calculus tutorial could save valuable time spent in many math departments on teaching such minicourses and on grading the placement exams. Another obvious advantage of web-based tutorials is that students can spend as much time solving different problems (and getting immediate and detailed responses) as they feel necessary.

II. PROJECT DESCRIPTION

A course based on the Web incorporates features that can be used as a display by an instructor in a classroom, or as a textbook by students with or without guidance from an instructor. It also provides several channels of communication between students and instructors and among students, in the same class, other classes, or other campuses. A web-based course can be used in a classroom or a distance learning environment. There is a large literature devoted to web-based courses. For example, see articles in the Journal of Asynchronous Learning Networks http://www.aln.org/alnweb/journal/jaln.htm [3], [4], [5], [6], [7], and [8].

The first version of the web-based Linear Algebra course was written and used during the spring semester of 1996 at the University of Nebraska-Lincoln (UNL). It has subsequently been used several times both at UNL and at Vanderbilt. Unlike an ordinary book, a WebBook can "grow" and change. The Linear Algebra Web course has grown into a large but convenient tool. This courseware can be used both in a proof-based linear algebra course (the beginning linear algebra for math majors) and in a more computationally oriented linear algebra course.

A. The Main Features of the Linear Algebra Web Course
The Linear Algebra Web course consists of a complete set of lecture notes with Maple examples (interactive worksheets), Maple Front End, a set of homework problems and typical exams with solutions, the WebTester, and discussion pages where students and instructors can post their comments, questions, and answers. (See http://www.maplesoft.com)

In the Linear Algebra WebNotes, complete proofs are given for all statements. However, the proofs do not appear in the main text; they are linked to it (like leaves to branches). Consequently, a student can first read the text without reading the proofs. The text actually feels small, although it contains more than 300 pages if printed. It can actually be used as a short introduction to linear algebra, as a text in a computation-oriented linear algebra course, and as a text in a proof-oriented linear algebra course.

The organization of material has a tree-like structure (see Figure 1). The "trunk" contains all concepts, statements, and examples used in the course (without proofs). The branches include proofs, the index of concepts, homework assignments, solutions, discussion pages, the Maple Front End, the grades database, and the WebTester that contains midterm exams and web tutorials. The analogy with a tree is not completely accurate because the branches and leaves are connected by numerous links.

Figure 1. The Organization of Linear Algebra WebNotes.

B. Teaching Using Computer Algebra Systems
Technology should assist traditional teaching, not substitute for it. One feature of the Linear Algebra Web course is the extensive use of a computer algebra system, Maple. Computer algebra systems (Mathematica, Maple and Mathlab, for example) are currently very popular in mathematics courses. There is a large literature devoted to this topic (e.g., [4], [9], [10]).

Students have access to Maple either by purchasing a copy of the student edition or by viewing the Maple Front End Web page. One version of the Maple Front End was created by Rex Dieter at UNL. Another version was created for our project by Kirill Kopotun. There are two benefits of using a computer algebra system in math classes (especially linear algebra) which seem to be largely overlooked by other researchers.

1. Concentrating on Major Problem Solving Steps
Using a computer algebra system such as Maple during the lectures (with a computer and a projector) allows the instructor to concentrate on major problem-solving steps instead of doing each elementary computation by hand. For example, after the concept of a row operation has been explained and understood, and the instructor begins to explain the Gauss-Jordan method of reducing matrices to reduced row echelon form. There is no need to do all row operations by hand. Students should consider row operations as elementary steps. They need to concentrate on understanding the Gauss-Jordan procedure, and not waste time on numerous calculations. Maple's commands allow users to do row operations in one step. Consequently, using very little time in class, an instructor can demonstrate the Gauss-Jordan procedure on large matrices. If students use Maple to do homework assignments, they can do more "drill" problems on the Gauss-Jordan procedure, and obtain a deeper understanding of this important concept. After the Gauss-Jordan method of solving systems of equations has been studied, the instructor can use one Maple command (gaussjord) to solve any system of equations. Most linear algebra problems require the solution of systems of linear equations; Maple allows an instructor to consider this as an elementary step.

2. Symbolic Computations and New Theorems Using Discussion Pages
Another important advantage of using Maple is that students can do symbolic computations and discover new theorems in the process. For example, several students in my linear algebra class discovered the Cayley-Hamilton theorem (that every square matrix is a root of its characteristic polynomial. Using a discussion page, I organized a group work by correspondence and students solved this problem working together (at a distance).

One of the best features of online courses is the set of discussion pages where students and instructors can exchange ideas, solutions, questions, and answers. When a student posts a comment on a discussion page, everybody else in the class can immediately see it and respond. This feature serves to organize group work in linear algebra classes.

If students use Maple (or any other computer algebra system) at home, they can do more computational problems. Of course, the more computational problems students do at home, the more time the instructor needs to spend grading. Our goal is to make the WebTester handle the administration and grading of all computational homework. For example, the WebTester can randomly choose a matrix and ask a student to list a sequence of row transformations (encoded in a simple manner) which reduce the matrix to its reduced row echelon form. The WebTester automatically grades students' answers; if the answers are wrong, it allows students to solve a similar problem again. The WebTester can even show what the correct solution was and how to get this solution. A student can take an exam arbitrarily many times. No human instructor can grade so many problems.

The environment provided by the Linear Algebra Web book is very flexible. An instructor using this environment can choose many different ways to teach a class. In my classes, for example, all midterm tests have two parts: the computational part is administered via the WebTester, and the "proof" part is administered in class. In addition, I use 10-minute in-class quizzes to check students' understanding of the material.

C. The Main Features of the WebTester
The WebTester is the main component of the Linear Algebra course. It is designed in such a way that it can be used in any mathematics course, and in other courses as well. For example, The Learning Company offers non-multiple-choice K-3 tests in math and reading http://12.6.240.11:8080/. More than 6,000 people have taken at least 10,000 tests there. More than 85% of parents who responded to a survey consider the tests extremely useful for their children.

The tests administered via the WebTester are essentially take-home tests. Most problems have millions of different combinations, and since the problems are generated randomly, students can take tests as many times as they want. Only the highest score counts (although the instructor can see the average and the number of times a student took the test). The computer grades tests automatically. Students have an incentive to work more because this yields a better grade. Students get the hands-on computational experience that they need to master new and abstract concepts of linear algebra.

The answer to a question in a test administered by the WebTester can be anything that is algorithmically verifiable, e.g., a number, a formula or a sequence of formulas. In particular, there can be infinitely many possible answers and even infinitely many forms of correct answers (see II. C. 3. Non-multiple Choice Tests). In fact, every question in a test is a pair of programs: the first program generates the question; the second program checks the answer. In principal, both programs can be written in any programming language. It has turned out that Maple provides a better environment for programming of this sort than other languages, so most of the questions are currently written in Maple. The questions can contain pictures and any other special web effects, including Java scripts, movies and sound.

It is important to stress that the non-multiple-choice nature of the WebTester problems makes students think more about the concepts instead of trying to guess the answer. In contrast, multiple-choice environments require only recognition memory and cued recall.

1. How to Log In?
The WebTester web site http://www.math.vanderbilt.edu:8888/ is user-friendly. The student needs to choose a user name and a password, and the WebTester lets the student enter; then the student needs to fill in the Personal Info Form (see Table 1).

Table 1. Personal Information Form.

The field "advisor" contains the login name of the student's advisor. A student can leave it blank, in which case the advisor will not be able to see student's results. An advisor can see the grades of only those students who list him/her as their advisor.

The field "group" contains a code name of a group. An advisor can make an exam available only for students from a certain group. In addition, the name of a group helps the advisor to sort grades of his/her students. Advisors need to enter "advisor" in the group field.

2. Non-multiple-choice Tests
After the form is submitted, the student is given a list of exams. The first three tests in this list actually form a tutorial on third semester calculus.

Other tests are the midterm tests. Each of these has two parts. The computational parts consist of non-multiple-choice questions. Table 2 shows some of the main features of questions from a typical Linear Algebra (LA) 2.1 test.

Table 2. A Typical Linear Algebra Test Using Web Tester.

Notice that these questions are essentially non-multiple-choice. In an ordinary (non-computerized) linear algebra exam, one may see simpler versions of these questions.

The answer to the first problem is a triple of formulas. A student can use any letter as a parameter. In this problem and in others, it is impossible to list all possible answers because there are infinitely many different forms one can use to write the answer. The checker program in this case is relatively sophisticated and uses the advantages of Maple.

In the second problem, the answer is a vector with real coordinates. There are infinitely many possible answers, and the program checks whether the student's answer is within the acceptable range from the true answer.

The answer in the third problem is a sequence of row operations encoded in a simple way, explained before the problem. Again, the checker program is non-trivial; it employs a deep result from linear algebra. This is a typical problem, showing a chief benefit of computer algebra systems such as Maple: A student can think in terms of big steps instead of elementary computations. Also a chief benefit of the WebTester is that no instructor has time to grade five or six such problems per student.

Grading the answers to these problems seems to require flexibility that one might expect to find only in a human being. Yet, the answers are graded automatically by the WebTester.

3. Grading
A student can save the answers, check syntax, and ask the computer to grade the answers. There are special buttons on the exam page for each of these functions. As usual when a human being works with a computer, the question of syntax is very important. The questions are designed in such a way that a student needs to obey only a few simple syntax requirements. Nevertheless, students make syntax mistakes; the syntax checker help students avoid such mistakes.

When a computer evaluates a student's answers, it gives the total grade and also gives correct answers, so that the student can compare his or her answers with the correct ones.

4. Tutorials
An instructor can include a help part in the checker program for each question. The help part can show a way to get a correct answer, refer to a certain part of the Web book, and show the probable cause of a student's mistake. This feature turns a web test into a tutorial tool. Table 3 shows a typical response to an incorrect answer on the test LA1.2.

Table 3. The Web Test as a Tutorial.

5. What Does an Instructor See?
An instructor can see either the overall scores of the students or the question-by-question performance. Table 4 shows how the overall score pages can look.

Table 4. Overall Score Page Example.

Table 4 shows that virtually all students in the class eventually got A's for this exam, but some students needed to take the exam many times.

Using the question by question summary of results of students, an instructor can see which questions are the most difficult for the students. Thus, the teacher can determine how well the students mastered a particular topic in linear algebra, and can decide whether it is worthwhile to spend additional time on that topic in class.

An important feature of the WebTester is that while students are at home working on the tests, an instructor (also at home) can watch students working as if they all were in the same room. Of course, if a student does not want to be watched, he or she can remove the instructor's name from the Personal Info Form.

D. Related Web Sites
1. WebNotes
There are many books on the Web devoted to different math subjects (see Table 5).

Table 5. Online Math Books.

Note that http://dir.yahoo.com/Science/Mathematics/Education/College_and_University /Courses/ lists only two complete courses, Orr's analysis course (see Table 5) and my Linear Algebra course. Orr's and Wachsmuth's sites contain virtually complete courses in real analysis. Orr's site also has most of the features of my Linear Algebra WebNotes, except for the testing capabilities and the Grades database. Notice also that the real analysis sites have many more interactive tools than the linear algebra sites. The linear algebra sites listed above, and other sites that I have seen contain lectures on some of the topics, but they do not contain complete courses. They have no computer algebra examples. Discussion pages are missing in all these sites except the first one, and none of these sites has anything comparable to the WebTester.

It is also important to mention http://www.umassd.edu/SpecialPrograms/Atlast, the NSF sponsored project ATLAST, which promotes use of software in teaching linear algebra. Since ATLAST currently does not have a WebNotes component and an automatic testing component, our project can be considered complimentary to ATLAST. Several ideas generated by ATLAST (for example, animated illustrations of the main linear algebra concepts) will be crucial for our project.

2. Online Tests
Aside from http://www.math.vanderbilt.edu:8888 and http://12.6.240.11:8080, web sites powered by the WebTester, only three web sites contain non-multiple-choice tests. These are University of Rochester's "WebWorK," http://webwork.math.rochester.edu:8088/, Temple University's "COW," http://chaos.math.temple.edu/cgi-bin/tourguide, and University of Cincinnati's "Online Exercises System" (OES), http://math.uc.edu/onex/demo.html.

These sites have only calculus and pre-calculus exams; none of these sites has linear algebra exams. Computational questions in linear algebra are usually much more complicated than in the low-level calculus courses (see II. C. 2. Non-multiple-choice tests).

Several features of the educational environment provided by WebWorK, COW and OES are very useful--for instance, before the student actually begins any tests, COW permits the student to familiarize himself or herself with the input syntax, by practicing typing. OES has a very nice test authoring system, and is the most impressive among these three sites. The tests include interactive exercises (using Java) which make tests feel more like lab experiments. The WebTester also permits the use of Java and Java script applets.

The WebWorK and COW sites administer tests, but only a few forms of answers are possible. The programmed response to an incorrect answer is either an invitation to try again, or a statement (without explanation) of the right answer. Thus, these programs can only be used for rote drilling, not for making tutorials. In contrast, OES and WebTester can accept answers in a variety of forms, and can distinguish between the many different correct answers to a problem and the many different incorrect answers. However, only WebTester can respond to different kinds of mistakes with different kinds of tutorial explanations.

III. BENEFITS AND FUTURE ACTIVITIES

A. Benefits of the New Technology
1. Benefits of Web Courses
The benefits of web support for traditional courses may be summarized as:

1. Use of the Web environment changes how one uses class time. Using Maple or any other computer algebra system, an instructor saves time by concentrating on big steps in solving problems instead of doing all the elementary steps by hand (see II. B. 1. Concentrating on Major Problem Solving Steps).
2. The discussion pages help students and instructors work on homework problems together. Each student can post an idea on the discussion board, and every other participant can immediately see the idea, respond to it, and participate in a discussion (see II. B. 2. Symbolic Computations and New Theorems Using Discussion Pages).
3. Students can use the WebNotes as supplementary reading for their courses. In fact, many students prefer WebNotes over the standard books. The WebNotes can be much more flexible than an ordinary book. The tree-like structure, with its many cross-references, can make WebNotes useful both for studying new material and for a review.
4. The WebNotes are very good for distance learning courses. Even without testing software, a web-based course can turn distance learning into virtual "in-class" learning: the instructor and all the students are in different places, but they communicate with each other via discussion boards or e-mail. The testing software can make distance learning much more effective.

We have received many positive comments from instructors and students of the Linear Algebra WebNotes.

2. Benefits of the WebTester
Several important features of the WebTester make it an excellent educational tool:

1. Students can take tests as many times as they want. Every time they answer a question incorrectly, the Webtester shows them help/tutorial pages.
2. Students' scores improve, not because they are getting used to solving problems of a specific type after repeating a test a number of times, but because their understanding of the material improves.
3. Rote memorizing is circumvented by randomness in the problems. Not only the numeric parameters of a question can be randomized, but also the wording and even the type of questions. An online test is thus equivalent to a large number of in-class tests.

Students have commented on how they appreciate being able to take tests at their convenience, without anxiety. Both students and instructors recognize the drill and tutorial features as great learning opportunities.

B. Future Activities
1. Improving the Linear Algebra WebNotes
The current web technology allows one to include pictures, sound, movies, Java scripts, and Java applets in an HTML document. It is interesting to investigate the effectiveness of using this technology in web courses. One of the difficulties of linear algebra compared with, say, calculus, is a lack of still images that can demonstrate the most abstract concepts of linear algebra. On the other hand, a movie or even a multiple GIF image can make the concepts much more visual. Thus, it is important to develop interactive demonstrations of linear transformations of the plane and of R³ (possibly using the VMRL language), eigenvectors and eigenvalues, determinants (as volumes of parallelepipeds), and other concepts. Note that there already exist Matlab routines to visualize several important concepts of linear algebra (see [10]). Some ideas used in designing those routines are very useful.

It is also necessary to improve the online grades database. This database will contain all grades for all the students in a class. It will be easy for an instructor to add new grades and correct existing grades. The database will be connected to the WebTester, which will send grades to the database automatically. The database will be useful in small classes and in large classes with many recitation sections. A student will be able to access his/her grades in the database on line, but will not be able to see grades of other students. A TA will be able to access and change grades in his/her section, and the professor will be able to access and change all grades. The database will automatically compute the average, max, min, median, standard deviation, etc. A less sophisticated version of the grades database has been designed by me and created by Travis Fisher, and has been successfully used in my linear algebra classes.

Our investigation of the effectiveness of using technology in web-based courses in linear algebra and pre-calculus will have an important impact on the creation of web-based courses in other topics. Most of the software components of the Linear Algebra WebBook (the Maple Front End, the grades database, the discussion pages, and of course, the WebTester) can and will be used in web-based courses on other subjects.

2. Linear Algebra Tutorials
The effectiveness of web-based tests can be increased in at least three areas:

1. Include more questions with significant visual components, e.g., graphs of functions or movies demonstrating linear transformations.
2. Improve the response part of the test, make it more intelligent. Even now, the WebTester can provide highly specialized help in the case of a wrong answer. It can say not only how to solve a generic problem, but also how to solve the particular problem given to the student. In solving a particular type of problem, students usually make only a few types of mistakes. The WebTester has a potential to "guess" where a student has made a mistake. We propose to improve the linear algebra tests by adding this capability. We will also use it in pre-calculus tutorials that we will create.
3. Design test banks that are instructor friendly. Essentially, a large collection of questions needs to be created (each question is a couple of Maple programs, as described in II. D. 1. ). An instructor will be able to combine questions in a test.
4. Create a better syntax checker. According to the students, one of the major problems with any computer-aided instruction is the problem of syntax. Even a small syntax error can ruin results of a hard work. Together with Alex Likhterman, we designed a very simple but universal syntax checker, which will be written in Java script. Every question in a web test will be accompanied by a description of syntax of a possible answer written in a special language. The WebTester will not allow a student to write answers, which do not fit the syntax description. We are going to describe the syntax checker in the next article.

3. Other Subjects
The technology based on the WebTester can be applied to virtually any other subject, especially subjects involving a lot of computation. One of the obvious candidates for such an "expansion" is pre-calculus (algebra and trigonometry).

IV. REFERENCES

1. Bransford, John, and Schwartz, Daniel, Rethinking Transfer, in press.
2. Schwartz, Daniel, and Bransford, John, A Time for Telling: Cognition and Instruction, in press.
3. Bourne, John R., Net-Learning: Strategies for On-Campus and Off-Campus Network-enabled Learning, JALN, Volume 2, Issue 2, 1998.
4. Crooke, Philip, Froeb, Luke, and Tschantz, Steven, Pedagogy Using Mathematica Through the Web, ALN Magazine, Volume 2, Issue 2, 1998.
5. Mason, Robin, Models of Online Courses, ALN Magazine, Volume 2, Issue 2, 1998.
6. Turgeon, A. J., Web-Based Technology for Engaging Students Across Vast Distances, ALN Magazine, Volume 2, Issue 2, 1998.
7. Goldberg, M. W., Using a Web-based Course Authoring Tool to Develop Sophisticated Web-based Courses, in Web-Based Instruction, B.H. Khan (ed.), Educational Technologies Publications, Englewood Cliffs, N.J., 1997.
8. Brown, D. J., Swafford, M. L., and Mallard, T. M., Asynchronous Learning on the Web, Washington, D.C., ASEE Annual Conference: Capitalizing on Engineering Education, 1996.
9. Abel, Martha L., and Braselton, James P., Maple V By Example, AP Professional, 1994.
10. Leon, Steven, Herman, Eugene, and Faulkenberry, Richard (editors), ATLAST Computer Exercises for Linear Algebra, Prentice-Hall, 1997.