Home

Learners' Performance and Evaluation of Attitudes Towards Web CourseTools in the Delivery of an Applied Sports Science Module

by Sloan-C

I. INTRODUCTION

In 1997 the National Committee of Inquiry into Higher Education (NCIHE) emphasised the importance of the role of communication and information technology (C&IT) in further and higher education (HE). The NCIHE suggested C&IT would play a key role in meeting the demands faced by HE such as greater student numbers, lower budgets [1], [2], [3] and the increasing demand for transferable skills.

Applied Sports Science is a compulsory, preliminary level module in the Sports Science and Sports Therapy courses at the University of North London. The module consists of approximately 24 hours of formal lectures with an additional 30 hours of practical and tutorial time. Practical sessions consist of practical application of taught training methodologies in the gymnasium and take up the majority of staff contact time outside lectures.

The aim of the present study was firstly to determine the effects of providing learning materials and support to learners through the use of the Web CourseTools (CT) environment on their performance in a preliminary level sports science module. Secondly the experiences of those learners using the package were also evaluated.

The learning objectives of the Applied Sports Science Module are identified as follows:
Students should be able to:

• Demonstrate a factual knowledge and practical demonstration of training principles.
• Relate the individual components of exercise training to specific sports, identify which aspects of fitness are most important in given sports.
• Demonstrate a knowledge of how to improve specific components of fitness through systematic training.
• Design an annual plan and a micro cycle for training in a sport of their choice.
• Justify the implementation of this plan through a demonstration of the physiological principles of training on which it is based.
  

In the last five years student numbers in the life sciences area at the University of North London have increased by 300%. In an effort to maintain pedagogic efficiency several staff have utilised C&IT based paradigms. In particular the use of Web CT is receiving considerable institutional support at this time.
Web CT is a tool facilitating the creation of WWW-based educational environments in three ways:

• Providing an interface allowing design and presentation of courses
• Providing tools to facilitate learning, communication and collaboration
• Providing administrative tools to allow course monitoring and improvement.

For the purpose of the Applied Sports Science module not all available student tools were utilised. Those used include:

• Electronic mail.
• A course conferencing system or bulletin board allowing communication between students. Status can be monitored showing the instructor articles read and posted and presents by default unread materiel to the viewer.
• Student self evaluation, a non assessed test which can be linked to any set of notes. Answers to questions in this tool are marked instantly by Web CT and feedback on the answers can also be given.
• External references and links, which can come either directly from titled icons on the homepage or from hypertext links within documents.
• A calendar tool, which can be used as a personal planner with both private and public entries possible by instructor and students.
  

Tools available to the instructor included Progress tracking which allows monitoring of student use of the site including first and last access, articles posted, read and time spent on each. Students can be sorted by any of the above categories for ease of identification. For example, those who have stopped making access to the course materiel can be identified. The tools available to the instructor make Web CT an ideal tool for carrying out pedagogical research, by giving quantitative data on learners' use of the package.

Student access to the course is controlled by the instructor and mediated by initial passwords which the student can then change. Web CT runs entirely through the WWW. and all users can access the course using a Web Browser such as Netscape 2.0 or above. The only requirements are a networked computer for both instructor and students to access the Web CT server. The Web CT package allows the creation of online materiel for individual modules or courses to be created quickly, with minimal knowledge of Web design or HTML.

Although the use of the Internet in general and computer aided learning (CAL) in particular has often been portrayed as a cure all for HE and society [4], it is also agreed that an essential requirement is an understanding of student needs to assess what the Internet can do for them. In addition to its confidence in the proposed increased use of CAL in HE, the NCIHE report also stresses the need for, and importance of pedagogical research into the application of computer technologies in learning.

Through investigative case studies of CAL, a number of benefits over traditional teaching methods have been proposed. The CTI handbook [5] lists six benefits of CAL which are of interest in the present study:

• Allowing students to learn at a time and place that suits them
• Increasing the potential for smaller group learning by reducing staff teaching time
• Increasing interest and motivation in learning using multimedia technologies
• Allowing students to learn at their own pace and in their own style
• Providing a huge information resource for assignments through accessing the WWW
• Extending the students' transferable skills by using C&IT in learning

All of the above benefits of CAL have been both supported and challenged through various pedagogical research [6]. However, it is argued that the validity of such research in this setting may be questionable [7]. Traditional quantitative research analysis shows weaknesses in analysing the complexities of the learning processes alone, without attempting to analyse the effects of the intervention of computers [8]. In the light of this statement it is suggested that action research which aims to improve the practice of teaching and learning in a systematic way may be a suitable method for evaluating CAL. Advocates of action research suggest any learning tool must be appraised in the context of its use [9]. In addition the use of research to evaluate the effects of CAL can bring an integrative approach to development and evaluation of C&IT [10].

There are numerous ways in which CAL can be used in education [8]. The CTI handbook [5] lists fourteen possible constituents of CAL which can be presented either individually to supplement the learning experience or combined to create a full online course. In the present study five of the listed roles of CAL were identified as being utilised in the Web CT package used to supplement the course. These were:

• Lecture notes and other material were posted on Web CT to help students prepare for lectures and tutorials.
• Students were guided to information points such as online journals and documents to supplement class notes allowing them to learn at their own pace.
• Public and private e-mail and a noticeboard were provided to encourage discussion.
• A mock examination providing feedback on questions was provided close to the examination date.
• Use of the Web CT environment, its tools and navigation throughout the WWW encouraged the development of C&IT skills.
  

Numerous models have been developed to categorise the various roles CAL can play in education. Based on the identification of the roles which Web CT filled in this module the package can be categorised as: "the enriched classroom model" in which open learning techniques with the aid of the WWW are used to complement traditional classroom-based teaching. This can be further classified as: "Supplementary and Complementary" according to Kings' levels of involvement for academic staff [11].

The aim of the present study is to combine quantitative and qualitative research methods in a single piece of action research into the use of CAL, completed in situ. The results of which will be used to inform further CAL development for both existing and future courses. This study will aim firstly, to evaluate the impact the addition of Web CT has had on students' performance in and attitude towards the module using quantitative research methods. Secondly it aims to qualitatively evaluate the attitudes to CAL in course delivery in those students who had access to Web CT.

II. METHODS

A control group took the module in semester A while the experimental group took the module in Semester B supplemented with Web CT. Attitudes towards the module were assessed in both using a standard end of module questionnaire. Attitudes of the control group compared to the Web CT experimental group were assessed using an online questionnaire constructed by the author.

Age, gender distribution and A-Level points were not significantly different between the groups. Although the test group scored higher on both coursework and examination assessment in the module the differences were not statistically significant. Correlational analysis showed a significant (p < 0.05) correlation between the use of Web CT and coursework assessment score. A similar relationship with examination performance was also shown which narrowly missed statistical significance (p = 0.051).

A. Participants
Eighty, first-year Sports Science / Sports Therapy students (Median age 19.5 I.Q. Range 19 - 23) took part in the study. The information regarding age, gender and A-Level points can be found in Table 1. The students were split into two groups (control and test) depending on their discipline. Sports Scientists took the Module in semester A (control) and Sports Therapists in Semester B (test). All subjects gave informed consent for their course work and examination results to be used in the study.

The aim of the present investigation was to determine the impact of supplementary CAL on student performance in a first year model and to evaluate the perceived impact of CAL on the module delivery. Experimental pedagogic research is often problematic as it is unethical to deprive one group of learners of a possible improvement in their learning experience while allowing a test group access. In the present study an opportunity to compare two similar groups arose. When group A took the module in the first semester of their first year, Web CT was still under development and not in place to support the course. Group B took the module in semester B following the completion of the module Web CT site.

B. Procedure
Information concerning participant A - level and other access grades was gained through a short questionnaire in order to assess previous academic achievement. Both sets of students were assessed through one piece of course work and a short answer format examination in the 14 week of both semesters. Student attitudes towards the module delivery, assessment and practical were assessed using a standard faculty end of module questionnaire marked on a 1-5 Likert scale with a maximum positive score of 55. The test group had their attitude toward Web CT assessed using an online questionnaire constructed by the author using the Perception online questionnaire package. The questionnaire was marked on 1 - 5 Likert type scale with 1 being a positive and 5 being a negative response to question. Additionally qualitative information was gained through the use of open and multiple answer questions.

Quantitative data on the use of the Web CT environment was gained from the programmes' built in Student Tracking system. This allows the Instructor to monitor number and timing of hits on the site. It also allows the instructor to monitor which articles are being accessed most frequently.

C. Statistical Analysis
Gender and age distributions within the groups were tested using chi squared and Mann Whitney U-test analyses respectively. Independent t-test comparing A-Level points were carried out to determine if any differences in previous academic achievement existed between the groups. Differences between the two groups in attitude towards the module in either semester were analysed using an independent t-test. Correlations between the frequency of use of the Web CT site and performance in both course work and examinations were carried out in the test group. Descriptive statistical analysis was performed on the results of the questionnaire to assess the attitudes of the test group to the use of Web CT.

III. RESULTS

A. Quantitative Analysis
Mann-Whitney U-Test and Chi squared analysis revealed no statistically significant (p >0.05) differences between the 2 groups in either age or gender mix respectively.

The results of the independent t-test carried out revealed no significant (p >0.05) difference in A-Level points between the two groups. On the basis of this result it was not necessary to control for this difference in the two groups and further analysis could be carried out using independent t-tests.

Table 1. Descriptive Statistics for Control and Test Groups.

An independent t-test revealed no significant difference between either the course work or examination marks for each group. (See Table 2.)

Table 2. Mean Course Work and Examination Scores
for Control and Test Groups.

Analysis of the responses to the standard end of course questionnaire showed no difference in evaluation of the course between the two groups on the selected items on the questionnaire referring to course delivery (See Table 3).

Table 3. Course Evaluation From Faculty Questionnaire.

Pearsons' product moment correlation coefficients were calculated between access frequency to the Website and performance in the examination and course work in group 2 (See Table 4.) A significant positive correlation was found between Website access and course work marks. A similar relationship existed with examination results. This however narrowly missed reaching statistical significance.

Table 4. Pearson Product Moment Correlation Coefficients
Between Web CT Hits and Performance in
Module Assessment for Test Group.

B. Qualitative Analysis
Fifty percent of the test group completed the online questionnaire. The following is a summary of their comments:

• Prior to using Web CT in course delivery most students indicated they used computers and the Internet only once a week, 78% of respondents stated that the use of Web CT in the course had improved their understanding of C&IT.
• Thirty-three percent strongly agreed and 61% agreed that the use of Web CT had improved the delivery of the course.
• Ninety-four percent of students either strongly agreed or agreed that Web CT made it easier to access course materiels and similarly 78% allowed Web CT helped them to take control of their learning.
• Sixty-seven percent stated that the use of Web CT had increased their motivation to learn.
• Only 28% of respondents stated that Web CT altered course delivery as much as they had expected with 66% having no opinion on this matter.
• The element of Web CT utilised most was were the online course notes, followed by the external links provided within them. The use of most of the other facilities were reported less in student feedback.
• On the subject of proposed increased interactivity in Web CT through the use of online peer assessment, the majority (78%) of students indicated that they would be happy for members of staff to review work submitted online but not for their peers to see it. Of this 78%, 50% were in favour of this activity being made part of their assessment and 50% were against.

IV. DISCUSSION

Attitudes toward the inclusion of Web CT were mostly positive, students stating that CAL had allowed them to take control of their learning and learn at their own pace, in their own style. Some students reported an increased motivation to learn. The majority of students felt however that the inclusion of CAL did not alter the delivery of the course as much as was expected.

A. Assessment Performance
Supplementing traditional teaching methods with Web CT did not significantly affect student performance on either the course work or examination assessments in the given module. It has been previously stated that CAL increases students' motivation to learn [6]. Vroom [12] in turn has demonstrated that increased motivation can be translated directly into increased performance. It has been suggested that the use of CAL may increase motivation to learn [13]. However, Davies and Crowther [6] warn against overplaying the motivational capability of CAL, especially when courseware is merely an extension of course notes and textbooks. In such cases they warn the motivational factors of CAL may stem from the novelty of the new medium.

A second proposed mechanism by which it is hypothesised that CAL may enhance performance through facilitation of active and experiential learning [14]. Although the present model of CAL used can be said to have promoted active learning through encouraging students to use wider resources available to them (the WWW) in completing tasks, this is not strictly a different procedure to consulting text books or course notes.

In addition to students' learning being facilitated through being actively engaged in learning activities, it is also suggested that multimedia may aid experiential learning [6]. Kolb's [14] model of experiential learning consists of four stages. Experiencing is one of these stages, and the use of CAL can be an example of this. The lack of interactive activities involved in the present model could explain why performance did not improve as may have been expected through the implication of experiential learning.

The courseware was not of a particularly interactive nature, one criticism of this type of CAL is that it does not allow the learner to apply and use knowledge gained. In fact although some interaction was encouraged through the use of self tests this type of activity is more geared toward the recognition and recall of facts than the application of knowledge. This use of such a surface learning approach is perhaps unfortunate, but is suitable for the way in which many preliminary level modules are assessed.

Conversely it cannot be said that there is a lack of interaction when looking into the acquisition of transferable skills. Although no difference in academic assessment performance was noted, transferable skills such as Web browsing, searching and general computer skills were used experientially by all students who used the Web CT package. In evaluation, the majority of students indicated that the use of Web CT had improved their understanding of computers and the Internet. A significant positive correlation was found between Web CT use and coursework assessment performance. A similar relationship between Web CT use and examination performance was found which narrowly missed reaching statistical significance (p = 0.051). Although no causal relationship can be deduced from these relationships it can be seen that those who achieved highest were also those who used the CAL package the most. Such a relationship warrants further investigation.

B. Evaluation of Web CT
Anecdotally from the opinions of teaching staff it was felt that the delivery of the course was enhanced simply by students not being forced to make notes throughout lectures, and more importantly being able to prepare for lectures, allowing more interaction, dialogue and group work. This was supported by comments from students in their course evaluation [15]. In response to the question: "In what way did the use of Web CT enhance the quality of your learning experience?" we received such responses as:

• "Course notes were the same as the lecture, therefore I could listen and understand without rushing down lecture notes."
• "Being able to have course notes in advance enabled me to read up on areas in advance of the lecture, also I did not need to spend the whole lecture taking notes and therefore able to listen to the tutor."
• "It increased my quality of learning a huge amount as it made it more interesting and less formal than class lectures."

We infer from these responses that the classroom learning experience had been enhanced. Following the work of Bligh [16] and later Jenkins [17], it has been recommended in teaching large groups that short lead lectures be given followed by group work. The absence of note taking facilitates this approach. Conversely the statement that "It has made the process of learning more interesting and is much better than looking through books all the time. If something was not understood during the lecture, then it could be looked up and explained by Web CT," shows that the learning experience outside the classroom had also been enhanced.

Evidence for Web CT allowing learning to take place at the learner's pace was also given: "It saved having to take down notes in lecture and also I could access the lecture notes from home."

Problems noted with the use of Web CT included access to terminals, problems with passwords, the temptation not to attend lectures and the habit of filing course notes provided without annotation or reading.

Among the mainly positive findings it is interesting to note that the majority of participants felt that the inclusion of Web CT had not changed the delivery of the course as much as they expected. These findings are similar to those of previous studies [9] in which the impact of CAL has been evaluated, and found not to be as great as learners had expected using the enriched classroom model.

In considering the validity of such responses to the evaluation on CAL in education it must be noted that not all potential participants successfully completed the online questionnaire. This was despite being given tutorial time in which to complete it, and it being linked directly from the self-test in Web CT. On the basis of this, the process of self selection may have created a biased sample. Those most likely to complete the questionnaire are those who are more at ease with the use of C&IT and also those who attended an online tutorial which replaced a formal lecture. Reasons given for non-completion of the questionnaire included difficulty in locating it and problems with passwords and access to the server. Technical knowledge and IT competence may therefore have biased the sample. Previous case studies into the use of the enriched classroom model of CAL have shown that students expect the On Line element of the course to have a greater effect on course delivery than that which they experience [9]. The data on course evaluation obtained in the present study supported this.

In conclusion, it can be seen that the inclusion of CAL into a first year module did not significantly alter student performance on the assessment for that module in either course work or examination. It should however be noted that the assessment used was not specifically testing deep or experiential learning, which have been identified as possible benefits of using CAL. Additionally the module assessment was not testing the acquisition of transferable skills in C&IT which have previously been shown to be developed through the use of CAL in unrelated courses. Finally student responses to Web CT were on the whole positive and demonstrated that supplementing traditional teaching methods with ourWeb CT module successfully produced most of the benefits identified in the CTI handbook which may be derived from CAL. However, caution must be taken in the analysis of questionnaire data, in that the sample of respondents may be biased towards those participants who are more at ease with the use of C&IT.

Further work in this area could include analysis of data collected in the present study dividing the group who received CAL by those who completed and those who did not complete the questionnaire to identify use of and competence in C&IT. This is however beyond the scope of this paper. It is further proposed that analysis of the use of Web CT in this population will continue next year when all participants in the present study attend a second level module again supported by Web CT.

V. ABOUT THE AUTHORS

Mr G. Sandercock is a lecturer in Sports Science at the University of North London. He has just finished his first year of lecturing following moving from a research position in the Ministry of Defence. Although he maintains an interest in research within the field of sports science it is his feeling that in the current HE environment pedagogic research should be of great importance. Having been both a student and a lecturer within the past 3 years it is evident that from the learner's viewpoint good teaching practice in HE should be as important if not more so than a good research profile for lecturers.
It is Mr. Sandercock's feeling that action research within HE should be an integral part of any lecturers practice, especially when experimenting with new technologies in course delivery such as Web CT.

Both Dr. Shaw and Mr. Sandercock have courses supplemented by Web based materiels. Mr. Sandercock currently has 2 courses running in the Web CT environment. Both authors are involved in the creation of On-Line modules for delivery in the University's new technology tower, due to open in semester B of the year 2000.

Biographical information about Dr. G.Shaw can be found at his homepage site.

VI. REFERENCES

1. Margolis, M., Brave New Universities, 1998. http://www. firstmonday.dk/issues/issue3_5/margolis/index.html
2. Noble, D.F., Digital Diploma Mills: The Automation of Higher Education, First Monday, 3, 1, 1998. http://www.firstmonday.dk/issues/issue3_1/noble/index.html
3. Turoff, M., Alternative Futures for Distance Learning: The Force and the Darkside, Keynote Presentation at the UNESCO / OpenUniversity International Colloquium, April, 1997. http://eies.njit.edu/Papers/darkaln.html
4. Carswell, L., The "Virtual University" Towards an Internet Paradigm? Changing the Delivery of Computer Science Education, ACM SIGCSE/SIGCUE Proceedings, Dublin, Ireland, ItiCSE, 1998.
5. The CTI Handbook, 1999. http://www.cti.ac.uk/
6. Davies, M. L. and Crowther, D.E.A., The Benefits of Using Multimedia in Higher Education: Myths and Realities. Active Learning, 3, December 1995.
7. Lousberg, M. And Solwe, J., Action Research and the Evaluation of IT Projects, Active Learning, 8, July 1998.
8. Bowers, C.A., The Cultural Dimensions of Educational Computing: Understanding the Non-neutrality of Technology, New York, Teachers College Press, 1988.
9. Retalis, S., Madrakis, V., Papaspyrou, N. and Skordalakis, M., A Case Study of an Enriched Classroom Model Based on the World Wide Web, Active Learning, 8, July 1998.
10. Introduction to Web CT, 1998. http://www.webct.com/webct/intro/hardware.html
11. King, B., Establishing UniSAnet: the On?line Environment of the University of South Australia, Paper presented to the Information Technology Advisory Committee, University of South Australia, 1998.
12. Vroom V.H., Work and Motivation, Wiley, London, 1964.
13. Filipczak B., Different Strokes: Learning Styles in the Classroom, Training 32, 3, 43?48, 1995.
14. Kolb D.A., Experiential Learning--Experience as the Source of Learning and Development, Englewood Cliffs, 1984.
15. Perception Online Web CT Evaluation Questionnaire, Full Results, 1999. http://horus.unl.ac.uk/apps/perception/gavin_webct.cfm
16. Bligh, D., What's the Use in Lectures ? Penguin, Harmondsworth, UK, 1972.
17. Jenkins, A., Active Learning in Structured Lectures, In Gibbs, G., and Jenkins, A. (Editors), Teaching Large Classes in Higher Education, Kogan Page, London, 1992.