USING RUBRICS AND CONTENT ANALYSIS FOR EVALUATING ONLINE DISCUSSION: A CASE STUDY FROM AN ENVIRONMENTAL COURSE

Maha Bali
Center for Learning and Teaching, American University in Cairo, Egypt

Adham R. Ramadan
Department of Chemistry, American University in Cairo, Egypt

ABSTRACT

This paper presents a case study of using course-specific rubrics combined with content analysis, together with instructor and student feedback, to assess learning via online discussion. Student feedback was gathered via Small Group Instructional Diagnosis, and instructor feedback was collected through formal interviews. Content analysis used emergent coding with different assessment criteria for each phase of the online discussion. Student participation was high, with a number of students feeling they learned beyond what was discussed in class. Some students however were overloaded by the large number of postings and repetitiveness during some of the phases of the discussion. The instructor was pleased to find students who were quiet in class being active in the online discussion. However, he found that student contributions demonstrated insufficient reflection and critical thinking. Content analysis showed that students met, on average, 59-82% of the essential assessment criteria in their postings, and that their contributions significantly improved as the online discussion progressed. However, a limited number of postings reflected critical thinking. In using online discussion, the use of assessment criteria is therefore commendable, as it was found that content analysis gave an insight beyond student and instructor perceptions. The insights gleaned from the methodology indicate its usefulness in assessing online discussion activities more objectively, and with respect to specific learning objectives.

KEYWORDS

Action Research; Asynchronous Discussion; Computer Conferencing; Content Analysis; Environment; Online Discussion; Rubrics

I. INTRODUCTION

The American University in Cairo (AUC) is an American liberal arts university based in Cairo, Egypt. Its language of instruction is English. It follows a semester system, encompassing 15 working weeks. Classes usually meet two to three times per week, and consist of up to 40 students. No degree courses are offered via distance learning, but some instructors add an online component to their courses without reducing face-to-face contact time, using the learning management system WebCT.

The course ‘Man and the Environment’ is an introductory course to environmental science that students of non-science/engineering majors can choose to take in order to fulfil the general science requirement of their liberal arts degree. The course aims at presenting the principles of environmental science, together with the primary concerns of natural resource management and the major challenges for environmental protection on a global scale. At AUC, this is carried out through four modules. Module I presents the fundamentals of ecosystems and how they operate particularly with respect to energy and nutrient cycles. Module II focuses on the major natural resources and their management. Module III is concerned with environmental degradation and global initiatives to combat it. Module IV relates environmental management and protection to diverse issues such as economic development, policy, law, and sustainable development. The four modules are of different sizes with module III being the largest. This primarily stems from the fact that environmental degradation is reaching serious dimensions globally, and particularly so in Egypt. Students’ understanding of details and dynamics of such degradation, as well as ways and initiatives to combat it, is of crucial importance to their commitment to environmental protection. In this respect, it was considered important that students be able to identify issues of primary environmental concern and critically reflect on why these issues persist, while realizing the complexity of their interdependence. Egypt was used as an example of a developing economy faced with serious environmental challenges. Proposing possible solutions to issues of concern would help students appreciate the complementary roles of different entities such as individuals, civil society, regulatory bodies, private businesses, the political establishment, and others.

Class time was not enough for extensive discussion and exchange of experiences and opinions, especially concerning a topic of such complexity. The instructor therefore needed to find an ‘innovative’ way to carry this out outside of class time.

II. Why Online Discussion?

In this respect, and for the purposes of this course, online discussion was particularly suitable as a significant number of students seemed hesitant to speak up in class. In addition, recent research [6] found that 71% of surveyed AUC students (n = 99) who had online discussion as part of a course they took agreed or strongly agreed that it enhanced their learning experience. However, the investigators were aware of the most commonly cited limitations of online discussion [1, 7] , namely, the effect of its text-based dryness on learning particularly for visual learners; too many unread postings causing information overload, potentially de-motivating students and causing them to stop participating; the reluctance of some students to participate altogether; and the fact that some students read but do not post (also known as ‘lurking’) which is often seen as unfair to the other students.

A majority of students taking the course were found to be already familiar with WebCT, the learning management system used at AUC (74% were moderately or very experienced with it). They were also generally experienced with web technologies such as email (100%) and internet (89%), so no technical access issues were anticipated which might have reduced their participation in the online discussion.

Online discussion had been used within the context of this course for three semesters before this research was carried out. Students were awarded bonus marks for submitting ‘good quality’ postings. However, this was subjective, and it was considered important to measure the impact of this pedagogical innovation on student learning and motivation. In this respect, it was decided that an ‘action research’ project be carried out in the Spring 2005 semester, as a collaboration between the course instructor and the Center for Learning and Teaching (CLT) at AUC, to collect feedback from students and conduct a content analysis of the online discussion transcripts. This would result in data gauging student perceptions and measuring actual student learning that had occurred.

A significant number of studies have been conducted on online discussion. However, they have either focused on totally online courses (e.g. [2, 8, 9]) or on blended courses where in-class engagement was difficult because of large numbers of students (a number of the examples in [5, 10]. In addition, there is insufficient research addressing explicit learning gains achieved [5]. Some researchers conducting a study on blended courses with small classes, focused on learner perceptions only [11, 12, 13], and did not go further to conduct content analysis on student learning. Others [14, 15] did recommend the use of rubrics to assess student learning objectively, but did not go further to use rubrics in conducting content analysis. Although some researchers conducted content analysis of online discussion for small numbers of graduate students in a blended learning context, they based their content analysis on a predetermined framework rather than a course-specific rubric set [16, 17]. Others developed rubrics and analyzed them to assess the effectiveness of interaction in distance courses, but not learning [18]. Meyer developed a rubric set to assess the quality of online courses, but again, this was not directly applied for assessing learning [19].

The investigation presented here is unique: course-specific rubrics combined with content analysis, as well as student and instructor perceptions, are used in order to measure learning via online discussion. This is carried out in a course with a small number of non-native English-speaking students meeting three times per week.

III. Purpose of this Study

This study aims to investigate how online discussion can be used to achieve learning goals and specific learning objectives. In this respect, one of the overarching learning goals for using online discussion was to encourage students to apply what was presented in class to specific examples from their daily experiences in order to deepen their awareness of the issues at hand. A second goal was to assist them in developing their reasoning skills, and encourage critical thinking and analysis instead of memorization. These overarching goals were also translated into specific learning objectives as specified in section IV-B.

This study is unique in that content analysis of the online discussion is carried out based on ‘rubrics’ specific to the activity’s learning objectives, rather than applying a general framework. It is hoped that the results of this study demonstrate the benefit of combining rubrics with content analysis in assessing learning. Moreover, this study explores the usefulness of using online discussion in small classes where students meet regularly, as opposed to large classes and distance courses.

IV. Research Methodology

A. Structure of the Online Discussion Activity

B. Content Analysis

In this respect, a number of assessment criteria, ‘rubrics,’ were developed by emergent coding to be used in the content analysis. Emergent coding is a methodology that builds coding categories from existing transcripts instead of applying a pre-existing external framework to the transcripts. In the case of this investigation, the transcripts are the online discussion transcripts, and this approach was deemed more appropriate for the investigation since none of the existing frameworks could directly meet the learning objectives set by the instructor.

The authors read through the online discussion transcripts together, keeping in mind the primary learning objectives, as specified in Table 1, and then devised a set of rubrics to measure these learning objectives. The rubrics for the different cycles were therefore different, depending on the set learning objectives for that cycle. However, they also encompassed some generic criteria applicable to all cycles, such as correctness of information supplied, and originality. The posting was used as the unit of analysis, and all the cycle rubrics were applied to it. Scoring was carried out in a binary fashion with a rubric receiving a score of either ‘1’ (yes) or ‘0’ (no) depending on the content of the posting. Some of the rubrics were essential in assessing the learning objectives, whereas others were a ‘bonus’ (i.e. the students were not expected to meet them, but those who did were considered more advanced). For example, one rubric "Original" was defined as "posting contains at least one original idea,” as opposed to one which is repetitive of another posting, even if reworded, and this “originality” criterion was expected of all students. An example of one of the "bonus" rubrics is one that was named "Building" and is determined on whether a "posting builds upon what is in another posting, developing it and/or adding new information," with the purpose of demonstrating if students have built on what their peers have contributed as opposed to either ignoring it or repeating it. The details of these rubrics and the points they investigate are presented in Table 2.

Table 1. Cycles I to IV: Questions, Primary Learning Objectives, and Rubrics Used for Posting Content Analysis

Table 2. Details of Rubrics Used in Posting Content Analysis

The rubrics were developed taking into account the learning objectives, and then applied to sample sets of postings for each cycle by both investigators collaboratively. These postings were chosen so as to form as much of a representative sample with regards to length and content as possible. This collaborative coding approach was carried out to hone the rubric definitions, increase objectivity, and achieve a level of inter-rater agreement on how to interpret each rubric and assess postings. Rubrics developed for each of the four cycles are presented in Table 1. All postings, including the sample set, were then coded using the finalized rubrics. To ensure intra-rater reliability, the resulting coding of the sample sets was then compared to their initial coding. Coding was not carried out by both authors on purpose, since in an authentic situation, assessment of online discussions is likely to be carried out by the course instructor only, with the emergent coding approach ensuring consistency of the application of each rubric. The finalized coding was then used to analyze patterns in all postings. A tailor-made Access-Excel package was developed for coding and for analyzing patterns in all postings.

C. Instructor and Student Perceptions

The instructor’s perceptions were collected via two formal interviews and several informal discussions throughout the semester. On the other hand, student feedback, giving an insight into the educational situation [24] was collected via Small Group Instructional Diagnosis (SGID). This is a structured group interview process [25] that takes place in the absence of the instructor. Groups of students are asked about what they believe helps them learn in a course and how improvements could be made. SGIDs produce both written and oral feedback. For this research, a number of questions directly pertaining to the students’ perceptions of the online discussion activity were added:

1. Do you feel that the online discussions have helped you connect the course with real life (Egyptian environmental problems) more than if you hadn't had them?
2. Did you know beforehand that the instructor uses online discussions and that it could improve your grade?
3. Do you feel the online discussions have motivated you?
4. Is your opinion online affected by what the instructor seems to think is a good answer?
5. What could have been done to improve online discussions? [this question was created on the spot to address the issues that students said came up with online discussions]

In addition, a demographic survey was conducted at the beginning of the semester. Laurillard [26] advocates conducting such surveys to understand more about students’ backgrounds and what they bring to the course. The survey included questions on students’ familiarity with technology, to rule out its lack as a possible hindrance to effective student participation. The majority of students were very familiar with web technology, and therefore it can safely be concluded that it does not present a hindrance to any of them for contributing to the online discussion.

V. Results and Discussion

A. Instructor Perceptions

According to the course instructor, student engagement in online discussion can demonstrate significant variations from semester to semester. For the Spring 2005 semester, during which the current research was conducted, a significant number (25%) of students were quiet in class but very active online. This was primarily due to limited spoken English skills, which made these students reluctant to speak up in class. The course instructor was satisfied with the overall student performance in online discussion, although his perception was that their critical thinking skills were not developed well enough, and that some students just posted without giving their writing much thought. This was based on the absence of clear indications in the postings that students were capable of recognizing relations existing between environmental challenges which they had successfully identified; that they were aware of multiple causal relations, as for example between one cause and several effects, and/or several causes to one effect; that they were aware of the limitations and/or difficulties of some of the solutions they were putting forward for addressing identified environmental challenges; and that they were aware that some challenges could be addressed through different possible solutions. Another issue of concern was the recurring convergence of a significant number of students on one issue, failing to look beyond it into others, or to develop it further. This was of particular concern when the issue was an obvious one, not necessitating much debate, or discussion (e.g. the lack of awareness being a cause of environmental degradation), thus leading to considerable repetitiveness in the postings.

B. Student Perceptions

When students were asked for feedback about online discussion in an SGID conducted during the fourth cycle of the discussion, ten students wrote that online discussion was an important factor in helping their learning in the course; six said online discussion helped them connect the course with environmental issues in Egypt, and six said the activity motivated them. These numbers need to be considered in light of the number of students who contributed to online discussion (32 students) rather than the total number of students in class (40 students). Furthermore, some students complained of ‘information overload’, stating that they stopped contributing to online discussion when postings became too many and too repetitive, with postings concurring to stated opinions without any further contributions. This is indeed valid
feedback considering that the SGID conducted during the fourth discussion cycle, came right after the third cycle, which had the highest percentage of repetitive postings (Figure 1).

Figure 1. Number of ‘Repetitive’ Postings for the Total of Student Postings

Figure 2. Student and Instructor Postings for Cycles I to IV of the Online Discussion

C. Content Analysis

Table 3. Quantitative Details for Each of the Four Cycles of the Online Discussion

Table 3 presents quantitative details for each cycle. The number of student postings and the number of students participating are indicative of the degree of engagement of the class as a whole, whereas the average number of postings per student and the average word count per student posting are indicative of the level of engagement for each of those students participating in the discussion. Figure 1 presents the number of ‘original’ versus ‘repetitive’ postings, and Figure 2 presents the number of student postings and those of the instructor for each cycle. The overall trends in these variables throughout the four cycles are summarized in Table 4.

Table 4. General Trends for Contributions to Online Discussion

1. Student Engagement

The results show that as online discussion progressed, an increase in student engagement occurred. This is reflected in the increase of the total number of postings, as well as the average number of postings per student. The decrease in the average number of student postings per day should not be considered a contradiction to the general increase in student engagement. It is believed that this is due to external factors such as the approach of the end of the semester with all the associated deadlines of term papers and projects. Students getting busier with other required course work (as opposed to the ‘bonus’ online discussion), would post fewer messages per day. Nevertheless, the total number of postings per cycle increased and the average number of postings per student increased steadily, demonstrating a continued interest and engagement in the online discussion. Fewer student postings per day resulted in prolonging the last two cycles: it took more days to discuss the topics at hand thoroughly. Instructor engagement in the first three cycles was similar, and decreased significantly in the fourth cycle (Figure 2). This ruled out the possibility of the instructor’s engagement in the discussion as being the cause of the increased student participation.

2. Repetitiveness

Regarding the repetitiveness issue which some students raised during the SGID, it was found that it started by being exceptionally low (13%) in the first cycle, increasing to 46% in the second cycle, and reaching a noticeable maximum of 64% for the third cycle, then decreasing once more to 40% in the fourth cycle. These figures need to be considered in light of the instructor encouragement (in his postings) of students reacting to one another’s postings. Though a number of students succeeded to do so, using the content of their colleagues’ postings to further develop ideas and express opinions, a significant number agreed to others’ posting contents, repeating (though through rewording) the same ideas of the posting reacted to with no additions. The correlation between the exceptionally high occurrence of repetitions and the topic of Cycle III is noteworthy and is further discussed below.

3. Meeting Rubrics

Tables 5 and 6 present details of posting content analysis for the four cycles of the online discussion. It is clear from Table 5 that as the online discussion progressed, the quality of the postings generally improved. The percentage of rubrics met, on average, increased. In Cycle I, 63% essential rubrics were met on average, whereas for Cycle IV 81% essential rubrics were met on average. The exception to this upward trend is Cycle III where 60% of essential rubrics were met.

Table 5. Summary of Posting Content Analysis

The mode of rubrics met, as well as the frequencies at these modes, also improved as cycles progressed. In Cycle I, six messages (out of 23, i.e. 26%) met five of the seven essential rubrics. For Cycle IV, 28 messages (out of 55, i.e. 51%) met all six essential rubrics. Again, Cycle III was the exception to this upward trend, where 15 messages (out of 51, i.e. 29%) met only three of the six essential rubrics.

Cycle III data could be a reflection of a limited understanding of students of the topic under discussion “How do you think the causes for the persistence of environmental challenges in Egypt today could be addressed?”(i.e. why are these environmental challenges persisting?). Indeed, the percentage of postings meeting the rubric [OnTopic] is the lowest (82%) for all four cycles, as shown in Table 6. This limited understanding might explain the high level of repetitions seen in Cycle III. Some students not being very clear about the topic of discussion for the cycle, would tend to react to their colleagues’ postings (particularly if they have been encouraged to do so by the instructor), by mostly agreeing to what is in them without further elaboration, criticism or development. These reactions would present the basis for further reactions, etc.

Table 6. Percentages of Essential and Bonus Rubrics Met in Student Postings

4. Critical Reasoning Skills

A number of rubrics reflecting critical reasoning skills need to be more closely considered. For Cycle I, the essential rubrics [Totality] and [Effects] were met by 39% and 30% of the postings respectively, demonstrating that a significant number of students, though successfully identifying valid environmental challenges, were not clear about how the identified challenges related to one another and/or contributed to the detrimental environmental conditions. For Cycle III, the essential rubrics [Implementable], [Original], and [RelatesDiff] were met by 41%, 35%, and 8% of the postings respectively. The [Original] rubric reflects the high level of repetitions as discussed above. The [RelatesDiff] and [Implementable] rubrics scores demonstrate that multiple cause-effect relations were very seldom considered by students and solutions were suggested without giving much thought to whether they were implementable. Indeed this is also reflected by the very low scores of the bonus rubrics [Critical], met by 12% and 9% of postings in Cycles III and IV respectively, and [MultipleSolutions], met by 10% of postings in Cycle III.
 

5. Connecting the Course to Egypt’s Environmental Problems

The overall objective for using online discussion was to help students make connections between the course contents and Egypt’s environmental problems, as an example for a developing country with significant environmental challenges. The rubric [Relevant] was one directly addressing this point. It was met by 78% and 90% of postings in Cycles I and II respectively, and then met by 100% of postings in Cycles III and IV, demonstrating that this overall objective was successfully met by those students taking part in the online discussion, and that student increased engagement through the four cycles assisted in this.

VI. Conclusions

This investigation combines the use of rubrics and content analysis to both assess individual student learning and to gauge the entire class learning outcomes from online discussion. This is applied to a small class meeting regularly.

Using rubrics allowed objective assessment of learning against the instructor’s cycle-specific learning objectives. Rubrics proved more beneficial than the instructor’s own subjective assessment because they showed more learning than expected: the instructor could see how well students were doing with respect to the over-arching learning goals, but the detailed rubrics pointed out specific areas of strength and weakness in each student's learning.

The use of emergent coding also proved more beneficial in practice than the use of pre-determined frameworks as it allowed an assessment specific to the activity's learning objectives. In addition, content analysis captured the wider picture of the entire class learning from the online discussion. It allowed the instructor to identify the learning objectives which were most difficult to achieve and the ones which were widely met. In this respect, the combination of rubrics and content analysis demonstrate that student learning was greater than student and instructor perceptions, and pinpoints a number of possible improvements to the activity which the instructor can apply in the future. These encompass informing students of the assessment criteria (rubrics) in advance in order to guide them towards the learning objectives, which is common practice for rubrics [27]; focusing questions and feedback on addressing specific deficiencies; dividing students into a number of smaller discussion groups to limit repetitiveness and information overload, as suggested by McConnell [1]; and referring to ideas expressed in the online discussion during class meeting times, thus encouraging students who made contributions [16].

In addition to the above, results also demonstrate that online discussion can be beneficial even when classes are small and meet regularly as it allows more in-depth appreciation of content implications, and a venue for quiet students to interact. Although some research [28] warns that learners for whom English is not a first language may suffer in online discussion, this investigation shows that, on the contrary, online discussion seems to be a better forum for such learners to express themselves, as they have more time to reflect and compose their answers than they would have for an oral discussion in the classroom.

V. REFERENCES

1. McConnell, D. Implementing Computer Supported Cooperative Learning, 2nd Ed., London: Kogan Page, 2002.
2. Hiltz, S. R. Impacts of college-level courses via Asynchronous Learning Networks: Some Preliminary Results. Journal of Asynchronous Learning Networks1(2): 1–19, August 1997. http://www.sloan-c.org/publications/jaln/v1n2/pdf/v1n2_hiltz.pdf.
3. Green, L. Online Conferencing: Lessons Learne, 1998. http://www.emoderators.com/moderators/lessonse.pdf.
4. Meyer, K. The Ebb and Flow of Online Discussions: What Bloom Can Tell Us About Our Students’ Conversations. Journal of Asynchronous Learning Networks 9(1): 53–63, March 2005. http://www.sloan-c.org/publications/jaln/v9n1/pdf/v9n1_meyer.pdf.
5. Hammond, M. A Review of Recent Papers on Online Discusson in Teaching and Learning in Higher Education. Journal of Asynchronous Learning Networks 9(3): 9–23, 2005. Online http://www.sloan-c.org/publications/jaln/v9n3/pdf/v9n3_hammond.pdf.
6. Bali, M. and A. Ellozy. Does WebCT Enhance Learning? Case Studies at AUC. 3rd International E-learning Conference, Egypt, January 2005.
7. Preece, J., B. Nonnecke and D. Andrews. The top 5 reasons for lurking: Improving community experiences for everyone. Computers in Human Behavior 20(1): 201–223, 2004.
   
8. Offir, B., I. Barth, I. Lev and A. Shteinbok. Teacher–student Interactions and Learning Outcomes in a Distance Learning Environment. The Internet and Higher Education 9(2): 65–75, 2003.
   
9. Nisbet, D. Measuring the Quantity and Quality of Online Discussion Group Interaction. Journal of eLiteracy 1(2): 122–139, 2004.
   
10. COHERE. Briefing on Blended Learning,2004. Online: http://www.cohere.ca/briefing.html.
11. Wu, D. and S. R. Hiltz. Predicting Learning from Asynchronous Online Discussions. Journal of Asynchronous Learning Networks8(2): 139–152, April 2004. Online http://www.sloan-c.org/publications/jaln/v8n2/pdf/v8n2_meyer.pdf.
    
12. Fredericksen, E., A. Pickett, P. Shea, W. Pelz and K. Swan. Student Satisfaction and Perceived Learning with Online Courses: Principles and Examples from the SUNY Learning Network. Journal of Asynchronous Learning Networks 4(2): 7–41, September 2000. Online http://www.sloan-c.org/publications/jaln/v4n2/pdf/v4n2_fredericksen.pdf.
    
13. Young. A. and C. Norgard. Assessing the Quality of Online Courses from the Students’ Perspective. The Internet and Higher Education 9(2): 107–115, 2006.
    
14. Monroe, B. Fostering Critical Engagement in Online Discussions: The Washington State University Study, 2003.
    
15. Hein, T. L. and E. S. Irvine. Assessment of student understanding using on-line discussion groups. Proceedings of Frontiers in Education Conference, Tempe, Arizona, USA, 130–135. (Nov. 4-7, 1998). Online: http://ieeexplore.ieee.org/iel4/5943/15885/00736819.pdf?arnumber=736819.
    
16. Hara, N., C. J. Bonk and C. Angeli. Content Analysis of Online Discussion in an Applied Educational Psychology. Instructional Science 28(2): 115–152, 2000.
    
17. Meyer, K. Evaluating Online Discussion: Four Different Frames of Analysis. Journal of Asynchronous Learning Networks 8(2): 101–114, April 2004. Online: http://www.sloan-c.org/publications/jaln/v8n2/pdf/v8n2_meyer.pdf.
    
18. Roblyer, M. D. and W. R. Winecke. Exploring The Interaction Equation: Validating a Rubric to Assess and Encourage Interaction in Distance Courses. Journal of Asynchronous Learning Networks 8(4): 24–37, December 2004.
    Online: http://www.sloan-c.org/publications/jaln/v8n4/pdf/v8n4_roblyer.pdf.
    
19. CSU. Rubric for Online Instruction, CSU, Chico. (2003) Online: http://www.csuchico.edu/celt/roi/.
    
20. Henri, F. Computer conferencing and content analysis. In A Kaye (Ed.) Collaborative Learning through Computer Conferencing: The Najaden Papers, 117–136. London: Springer-Verlag, 1992.
    
21. Lally V. Analysing teaching and learning in networked collaborative learning environments: Issues and work in progress. In V. Lally and D. McConnell (eds.), Networked Collaborative Learning and ICTs in Higher Education: The Edinburgh Papers, 5–26. Sheffield: School of Education University of Sheffield, 2002a.
    
22. Fahy, P. J., G. Crawford and M. Ally. Patterns of Interaction in a Computer Conference Transcript. International Review of Research in Open and Distance Learning 2(1): 2001.
    
23. Gunawardena, C. N., C. A. Lowe, and T. Anderson. Analysis of a global debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research 17(4): 397–431, 1997.
    
24. Parlett, M. and D. Hamilton. Chapter 1.1: Evaluation as illumination: a new approach to the study of innovatory programmes. In D. Hamilton, D. Jenkins, C. King and B. MacDonald (Eds.), Beyond the Numbers Game, 6–22. London: Macmillan Education Ltd., 1977.
    
25. White, K. Mid-Course Adjustments: Using Small Group Instructional Diagnoses To Improve Teaching and Learning. In Washington Center's Evaluation Committee (Ed.) Assessment in and of Collaborative Learning, 1995.Online: http://www.evergreen.edu/washcenter/resources/acl/c4.html.
    
26. Laurillard, D. What students bring to learning. In Rethinking University Teaching: A Framework for the Effective Use of Educational Technology, 30–47. London: Routledge, 1993.
    
27. Pearson Education. The Advantages of Rubrics: Part one in a five-part series, 2005. Online: http://www.teachervision.fen.com/page/4522.html.
    
28. Morse, K. Does One Size Fit All? Exploring asynchronous learning in a multicultural environment. Journal of Asynchronous Learning Networks 7(1): 37–56, February 2003. Online: http://www.sloan-c.org/publications/jaln/v7n1/pdf/v7n1_morse.pdf.

VIII. Acknowledgements

The authors would like to express their thanks to Dr. Aziza Ellozy, Director of the Center for Learning and Teaching of the American University in Cairo for invaluable discussions and comments, as well as continuous support and encouragement throughout the course of this investigation.

The authors would also like to express their gratitude to Mr. Nicholas Bowskill, Mrs. Emma Pincott, and Mr. Gerard Clarke for their invaluable input throughout this research process through their roles in the M.Ed. in the eLearning Program at the University of Sheffield, UK.

IX. About the Authors

Maha Bali is a Senior Instructional Technologist at the Center for Learning and Teaching of the American University in Cairo.

Adham Ramadan is an Associate Professor at the Department of Chemistry of the American University in Cairo.