QUESTIONING THE HYBRID MODEL: STUDENT OUTCOMES IN DIFFERENT COURSE FORMATS

Saxon G. Reasons, EdD
Programming Manager, Instructional Technology Services
University of Southern Indiana

Kevin Valadares, PhD
Assistant Professor of Health Services/Administration
University of Southern Indiana

Michael Slavkin, PhD
Assistant Professor of Education
University of Southern Indiana

ABSTRACT

Current literature suggests that a hybrid model of distance education supports student learning more effectively than any other format. It also seems likely that the hybrid format can be used in most courses. The purpose of our study was to examine the strength of these assertions. This study examined the outcomes of two introductory courses in teacher education and health services employing similar pedagogical methods within three delivery formats (face-to-face, internet-based, and hybrid) in an effort to compare each of these modes of instruction. Results demonstrate that significant differences exist among the various formats and that the internet-based format could possibly lead to better student outcomes compared to face-to-face and hybrid formats.

KEYWORDS

hybrid, student outcomes, course format, distance education, internet

I. Introduction

Questions abound throughout most fields in higher education as to the efficacy of online learning. Whereas traditional courses may be limited in the use and integration of technology, other courses use internet-based technologies to directly support pedagogical practices; still other courses use a combination of traditional pedagogies and internet-based services and are often referred to as hybrid courses. This study reviews the results of a recent examination of the differences that exist among student outcomes for courses taught using traditional (face-to-face), internet-based, and hybrid formats. Researchers compared two courses in different academic disciplines that use similar pedagogical methods within three delivery formats. Current literature suggests that the hybrid model supports student learning more effectively than any other format and that a hybrid format can be achieved for most courses. The purpose of our study was to examine the strength of these assertions.

Little doubt exists that distance education has grown exponentially during the last decade. According to the recent National Center for Educational Statistics report Distance Education at Degree-Granting Postsecondary Institutions: 2000–2001 [1], 56 percent (2,320) of all two-year and four-year Title IV-eligible, degree-granting institutions offered distance education courses during the twelve-month 2000–2001 academic year. Eighty-nine percent of public four-year institutions offered distance education courses during this time period, and an estimated 945,000 enrollments in distance education courses at public four-year institutions made up 31 percent of total enrollments at two-year and four-year institutions (3,077,000) [1].

The University of Southern Indiana is an example of a public four-year institution that offers distance education opportunities to its student body (numbering approximately 10,000). Using primarily internet-based delivery, USI Distance Education currently offers seven certificate programs within the School of Nursing and Health Professions, three undergraduate degree programs, and three graduate degree programs. Enrollments in distance education courses have grown steadily from 338 in 1996–97 to 5,445 in 2003–4.

As enrollments in distance education have increased nationwide, so has interest in researching various aspects of student participation in these types of courses. For example, the NCES report A Profile of Participation in Distance Education: 1999–2000 [2] examined student participation in distance education in terms of numerous indicators, including student demographics and academic characteristics. As one would expect, characteristics associated with family and work responsibilities (such as being independent, older, married, or having dependents) were associated with higher rates of participation in distance education for undergraduates. The report also examined the rate of student participation as related to delivery method, and both graduate and undergraduate students were more likely to participate via the internet than either live or prerecorded TV or audio.

A. Traditional, Hybrid, and Internet-Based Courses

Traditional face-to-face courses are the mainstay of colleges and universities. The NCES report Distance Education at Degree-Granting Post-Secondary Institutions: 2000–2001 [1] indicated that 31 percent of institutions did not offer distance education nor did they plan to offer distance education at any time over the next three years. Of the institutions that did offer distance education courses, internet delivery was clearly a preferred format. Ninety percent of these institutions reported offering internet-based courses using asynchronous computer-based instruction, and increased use of this mode of instruction is anticipated in the future [1]. The National Education Association conducted a poll of 400 distance education instructors and found that “faculty believe Web-based courses do a better job than other course formats of giving students access to information, helping students master the subject matter, and in addressing a variety of learning styles. Further, faculty continue to believe that traditional courses do a better job than other course formats of strengthening group problem-solving skills, verbal skills, and oral presentations” [3].

B. Prior Research on Course Formats

Research studies comparing student outcomes among traditional, internet-based, and hybrid sections of the same course are not new; however, findings of these studies vary considerably. Most faculty in higher education are aware of the literature that advocates hybrid course format design [5–8]. Although considerable variability exists within styles of hybrid course formats, they are grounded in an anecdotal belief that some component of a traditional classroom presence combined with a differentiated online environment affords a positive learning experience for multiple student learning styles. This belief in effect suggests that students are in a “win-win” situation when given the chance to learn the subject matter simultaneously within both the classroom and the online environment. Although the literature does support divergent practices for differing student learning styles, there is a lack of definitive longitudinal research supporting hybrid course designs. Moreover, it is unclear whether the previously noted literature documents efforts to give students options to learn based on their needs or efforts to allow instructors the chance to use formats that meet their needs [5].

A collection of results from 355 studies compiled by Thomas L. Russell and available at http://www.nosignificantdifference.org/ aptly illustrates that students engaged in online learning can fare differently than students in traditional learning environments depending on the specific course and the outcome measured. For this study, three measurable student outcomes for courses were chosen: level of course participation, final course grade, and frequency of interaction with the course website.

Participation in class discussion can be an indicator of student success in both face-to-face and online settings and can be quantified through the use of grading rubrics and scoring. As a parallel to discussions between students in a classroom, interactive online discussions are critical to successful online learning from both cognitive and social perspectives [9]. Since online discussions allow all students equal opportunity for expression, they may lend themselves more naturally to increased participation, especially among those students who are not likely to engage in face-to-face class discussions [4, 10].

Final course grade is commonly chosen as the best indicator of student success. Research comparing course grades between traditional and online delivery formats has had mixed results, with varying degrees of statistical significance. In some cases, students in face-to-face settings outperformed students in online settings [11]; in others, students in online settings were more successful [12, 13]. More recently, students in hybrid courses have also been part of the comparison [14]. Generally speaking, a primary goal of the research has been to examine how students in online courses fared when compared to their on-campus colleagues, and statistically significant differences in grades seem to be the exception rather than the rule. Though many researchers are understandably eager to accept online instruction as an equivalent alternative to traditional instruction [15], further research comparing traditional, internet-based, and hybrid courses will generate useful information for those interested in online or hybrid course design and development.

Yet another indicator of student success is the frequency of interaction with course websites. One would logically predict that students in solely internet-based sections would access a course website more frequently than students in either traditional or hybrid sections; however, if course websites were available to students in all three section formats, the variable is an important one to examine. Other researchers have examined the relationship between student performance and interaction with a course website. Maki et al. [13] reported that students in online course sections exhibited better grades and that “forcing” students in online course sections to interact with course material may have resulted in better student performance. In his study of a web-enhanced lecture course, Stith [16] examined student use of a course website and the relationship between frequency of use and final grade. The majority of students in the study reported that the course website benefited them; however, the frequency of access did not appear to correlate with the final student grade.

II. HYPOTHESES, METHODS, AND SAMPLE

A. Hypotheses

In this study, we examined the impact of three course formats (face-to-face, internet-based, and hybrid) on student outcomes as measured by course participation, final course grade, and frequency of interaction with course website. Some literature suggests that hybrid formats provide stronger opportunities for students. After considering prior research findings and our firsthand observations, we proposed the following three hypotheses:

1. Course participation will differ significantly based on the format of the course, with the internet-based sections promoting stronger course participation levels across all students than either traditional or hybrid sections. Course participation is the grade for discussion board postings in both internet-based and hybrid course sections, and by a participation grade in traditional course sections. A standard rubric that measured both the quality and quantity of the discussion board postings is used to support the reliability of course participation grades.
2. Final course grades will differ significantly based on the format of the course, with hybrid sections promoting stronger final course grades than either traditional or internet-based sections.
3. Interaction with the course website will differ significantly based on the format of the course, with internet-based sections promoting greater frequency of interaction than either traditional or hybrid sections. The interaction with the course website is measured as the number of times the site was accessed (“hits”).

B. Methods

Current data were collected over the course of three academic years at the University of Southern Indiana, a public four-year institution of average size (approximately 10,000 students). Two introductory courses, Introduction to Educational Psychology (EDUC 201) and Introduction to the Health Care Delivery System (HP 211), were offered in three delivery formats over the course of six academic terms, with the faculty researchers serving as the instructors of the courses. Blackboard course management software was used to support internet-based instruction. The questions under investigation were reviewed in conversations between the two instructors. Because the study stemmed from teaching and not from a research question, situational constraints required the present format. As much as possible, the researchers attempted to review the data based on current research requirements for teacher action research and phenomenological research investigations.

1. Traditional Sections

Courses were offered in a face-to-face format with lectures, discussions, and hands-on individual and group assignments. Tests composed of multiple-choice questions selected by the instructor from the publisher’s test bank were administered in class. Course materials were distributed in class and also published on the web for student convenience.

2. Internet-Based Sections

Courses were offered using the internet to distribute course materials, such as video-streamed lectures and lecture notes, to facilitate interaction through discussion boards, and to support testing. Coursework was completed primarily via discussion forum reviews of material, distributed group assignments, and individual projects e-mailed between instructors and students. No face-to-face meetings occurred.

3. Hybrid Sections

Courses were offered using a combination of face-to-face and internet-based delivery formats. Over the course of a sixteen-week semester, classes met on seven or eight occasions for lectures, presentations, and discussions. Students performed some group work online and some group work in class. Course materials, discussions, group projects, and tests were administered online. Exams and assignments in these course sections were the same as those used for the internet-based sections. In both courses, students were expected to be involved approximately equally online and in face-to-face meetings.

Prior to teaching online, both faculty researchers participated in USI’s Institute for Online Teaching and Learning, a two-week intensive professional development opportunity for faculty preparing to teach online education courses. Topics covered during the institute included basic instructional design, creation of online learning communities, student learning styles, organization and distribution of online learning materials, use of online discussion boards, e-mail management, copyright issues, integration of an effective assessment plan for online learning, and effective use of course management software.

C. Sample

The sample was composed of students, primarily freshman and sophomores, enrolled in five traditional sections, four internet-based sections, and four hybrid sections of two introductory courses, Introduction to Educational Psychology (EDUC 201) and Introduction to the Health Care Delivery System (HP 211). The numbers of students enrolled in each course section are reflected in Table 1 below. Sections were offered over the course of six academic terms. Introduction to the Health Care Delivery System is a required course for the Bachelor of Science in Health Services. Introduction to Educational Psychology is a required course for education majors.

Table 1. Number of Students in Course Sections

D. Dependent Measures

For this study, three dependent measures were reviewed: (1) course participation, (2) final course grades, and (3) interaction with the course website.

Course participation for both courses was graded by a holistic scoring rubric system. Measurements were based on a Likert scale in which 5 indicated extensive participation with all assignments completed, 4 indicated participation in discussion forums and assignments adequate for week, 3 indicated participation generally appropriate but some problems with completion of week’s assignments, 2 indicated generally weak performance with limited follow-up of week’s activities, and 1 indicated poor performance overall with little to no involvement. Although quality of participation was reviewed through course participation, quantity of participation was evaluated using interaction with the course website (see below).

Final course grades were taken from university records of the students’ performance in the course based on the university grading scale. All sections of the courses used the following standard grading system: A, 90 to 100 percent; B, 80 to 89 percent; C, 70 to 79 percent; D, 60 to 69 percent; and F, 59 percent and below.

Interaction with the course website was based on statistics collected by the Blackboard course management system throughout course facilitation. While students were signed into the system, Blackboard collected the number of times students moved from section to section in the site or participated within various areas of the course site.

III. RESULTS

Using firsthand anecdotal evidence and preliminary analyses, we predicted that the student outcomes measured would differ significantly based on the format in which the course was taught. The results of our study are detailed below.

A. Course Participation

Course participation was predicted to differ significantly based on the format of the course, with internet-based sections providing stronger course participation levels across all students than either hybrid or traditional sections. Results indicated that significant differences did not exist for the participation of students during a course based on the format of the course (F (2, 400) = 0.94, p = .91).

B. Final Course Grades

Final course grades were predicted to differ significantly based on the format of the course, with hybrid sections providing stronger final course grades than either traditional or internet-based sections. Results indicated that significant differences did exist on the final grades that students earned in a course based on the format of the course (F (2, 400) = 8.48, p = .0001), though the differences were not as significant as expected. Using the Tukey HSD post hoc analysis, it was found that students in internet-based sections outperformed those in both traditional and hybrid sections (see Table 2), with no difference found between traditional and hybrid sections.

Table 2. Homogeneous Subsets Table – Final Course Grades

C. Interaction with the Course Website

Finally, interaction with the course website was predicted to differ significantly based on the format of the course, with internet-based sections showing greater frequency of interaction than either traditional or hybrid sections. Results confirmed that significant differences existed in the interaction that students have with a course website based on the format of the course (F (2, 400) = 5.41, p = .005). Using the Tukey HSD post hoc analysis, it was found that students in internet-based sections interacted with the site to a greater extent than those in either traditional or hybrid sections (see Table 3), with no difference found between traditional and hybrid course sections.

Table 3. Homogeneous Subsets Table – Interaction with the Course Website

IV. Conclusions

A. Course Participation

Course participation was predicted to be significantly different based on the format of the course, with internet-based sections providing stronger course participation levels across all students than either hybrid or traditional sections. Although participation differed, no significant differences existed in the participation of students during a course based on the format of the course.

Differences in participation that occurred may have been due to a variety of factors. The frequency of the discussion board assignments varied based on the format of the course section, with traditional courses having fewer online assignments. Traditional courses used a combination of in-class participation, attendance, and online discussion board exercises in determining overall course participation. The lack of significant differences could stem from the lack of participation of some students in either the internet-based or hybrid sections. The course instructors provided the infrastructure (grading rubric), detailed instructions on how to participate, and reminders that presupposed meaningful discussion activities, but they were disappointed in the overall lack of student participation.

B. Final Course Grades

Final course grades were predicted to be significantly different based on the format of the course, with hybrid sections providing stronger final course grades than either traditional or internet-based sections. Results indicated that significant differences did exist on final course grades based on the format of the course. Students in internet-based sections outperformed those in both traditional and hybrid sections, with no difference found between traditional and hybrid sections.

Course grades have traditionally been effective indicators of student learning (performance). And although current opinion may hold that students in hybrid sections would likely do as well or better than traditional students, the preceding results suggest that a consistent format of instruction combined with the internet-based format helped students succeed. Internet-based students know that the only involvement they have with the class is through the course website. Hybrid students may have to adapt to learning in multiple formats and may experience confusion in that students generally report feeling unclear about what is expected of them from week to week, and indicating that the differing formats held different expectations of their performance.

C. Interaction with the Course Website

Interaction with the course website was predicted to differ significantly based on the format of the course, with internet-based sections showing greater interaction than either traditional or hybrid sections. Results confirmed that significant differences existed regarding the frequency of interaction that students had with the course website based on the format of the course. It was found that students in internet-based sections accessed the course website more frequently than those in both traditional and hybrid sections, with no difference found between traditional and hybrid course sections.

This result may be a foregone conclusion since by its design the internet-based model requires no face-to-face component. Therefore, since interaction with a course website is defined by the total number of hits, internet-based students should be engaged in the course on a more frequent basis. Although some caveats to this notion exist (e.g., the eager student who checks his or her grade on repeated occasions, thus generating a hit each time), on the whole, internet-based students tended to be involved in more aspects of the online course (content, group work, discussion board, testing) despite the fact these sections were equally available and encouraged for traditional and hybrid students.

Moreover, differences between the three format groups investigated were less extensive than those differences within the groups; some students in the hybrid group accessed the course website more often than internet-based students, and some in the traditional group accessed the site more than hybrid students. The presence of this dynamic is in itself an interesting finding.

D. Limitations of the Study

The limitations of this study vary in scope. From a procedural perspective, some of the traditional course sections in both EDUC201 and HP211 were taught prior to the faculty’s attending USI’s Institute for Online Teaching and Learning.

From an integrative perspective, one could argue that the level of student participation is in fact wrapped up with the frequency of the course website interaction, thus making its analysis redundant. Although this is true when strictly examining the total number of course website hits, each outcome was evaluated separately in order to assess as precisely as possible the varying behaviors of students.

The results offer both opportunities and challenges; however, this study was limited by the fact that research questions grew out of conversations between faculty as they were developing and teaching their courses in a natural laboratory setting (i.e., the real world of teaching). Changes in format were made from semester to semester, although often these changes were similar across the courses and sections. Although this retrospective examination does not provide the objectivity of a constructed experiment (and thus impacts the results of the study if one reviews them from a positivist perspective), the opportunity to analyze the course formats from an experiential framework provides faculty with the chance to examine their pedagogy.

V. RECOMMENDATIONS

A series of commonalities existed between the two courses. EDUC201 and HP211 are required introductory-level courses in the teacher education and health services programs respectively. Both courses were taught in the three formats (face-to-face, internet-based, and hybrid) by the same instructor over six academic terms. Preliminary analyses revealed that differences in student demographic makeup of the two courses were limited. Finally, from an equitable grading perspective, the instructors used similar rubrics for evaluating discussion board and writing assignments, albeit geared to different content.

Hybrid courses are often touted as offering students the best of both worlds, and, intuitively, this claim seems to make sense. However, this study suggests that this is not necessarily the case. Two of the three outcomes examined demonstrated a significant difference, based on course format. Perhaps as McDonald [15] suggests, comparative studies measuring the efficacy of distance courses alongside traditional courses may not be relevant for educational purposes. However, much can be gained from analyzing the delivery methods of courses and asking relevant questions, such as what is valued in the face-to-face environment and how might these characteristics be brought to distance learners—and vice versa. Shared experiences presented among various methods of delivery may provide the opportunity for individualized student learning and standardized faculty practice, which benefits all aspects of higher education [17]. We believe that comparing formats is useful, especially considering the dynamics of hybrid courses.

VI. REFERENCES

1. Waits, T., and L. Lewis. NCES Report, Distance Education at Degree-Granting Post-Secondary Institutions; 2000–2001 (2003). Available online : http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003017.
2. Sikora, A. NCES Report, A Profile of Participation in Distance Education: 1999–2000 (2002). Available online: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003154.
3. National Education Association.Confronting the future of distance learning—placing quality in reach, 2000. Available online: http://www.nea.org/nr/nr000614.html.
4. Young, J. ‘Hybrid’ Teaching Seeks to End the Divide Between Traditional and Online Instruction. Chronicle for Higher Education 48(28): A33, 2002.
5. Burton, L., and D. Goldsmith. Experience in Online Courses: A Study Using Asynchronous Online Learning Focus Groups, 2003. Available online: http://www.ctdlc.org/Evaluation/StudentExperience.pdf.
6. DeVries, J.and G. Lim. Significance of Online Teaching vs. Face-to-face: Similarities and Differences, 2003. Available online: http://ole.tp.edu.sg/courseware/teaching_guide/resources/article/TP%20Staff/F2FandOnline.pdf.
7. Frydenberg, J. Quality Standards in E-learning: A Matrix of Analysis. International Review in Open and Distance Learning, October 2002. Available online: http://www.irrodl.org/content/v3.2/frydenberg.html.
8. Johnson, S.D. et al. Comparative Analysis of Learner Satisfaction and Learning Outcomes in Online and Face-to-ace Learning Environments. Journal of Interactive Learning Research 11(1): 29–49, 2000.
9. Stacey, E. and M. Rice. Evaluating an Online Learning Environment. Australian Journal ofEducational Technology 18(3): 323–340, 2002. Available online: http://www.ascilite.org.au/ajet/ajet18/stacey.html.
10. Carnevale, D. Online Students Don’t Fare as Well as Classroom Counterparts, Study Finds. Chronicle for Higher Education 48(27): A38, 2002.
11. Brown, B. and C. Liedholm. Can Web Courses Replace the Classroom in Principles of Microeconomics? American Economic Review 92(2): 444–448, May 2002.
12. Koory, M. Differences in Learning Outcomes for the Online and F2F Versions of “An Introduction to Shakespeare.” Journal of Asynchronous Learning Networks 7(2): July 2003. Available online: http://www.sloan-c.org/publications/jaln/v7n2/v7n2_koory.asp.
13. Maki, R., W. Maki, M. Patterson, and P. Whittaker. Evaluation of a Web-based Introductory Psychology Course: I. Learning and Satisfaction in On-line versus Lecture Courses. Behavior Research Methods, Instruments, & Computers 32: 230–39, 2000.
14. Rivera, J., M. McAlister, and M. Rice. A Comparison of Student Outcomes and Satisfaction between Traditional and Web Based Course Offerings. Online Journal of Distance Learning Administration 5(3): 2002. Available online: http://www.westga.edu/%7Edistance/ojdla/fall53/rivera53.html.
15. McDonald, J. Is “As Good as Face-to-Face” As Good as It Gets? Journal of Asynchronous Learning Networks 6(2): August 2002. Available online: http://www.sloan-c.org/publications/jaln/v6n2/v6n2_macdonald.asp.
16. Stith, B. Web-Enhanced Lecture Course Scores Big with Students and Faculty. THE Journal 27(8): March 2000.
17. Twigg, C.Innovations in Online Learning: Moving beyond No Significant Difference. Center for Academic Transformation, Rensselaer Polytechnic Institute, 2001. Available online: http://www.center.rpi.edu/PewSym/Mono4.html.

VII. Acknowledgements

The authors wish to thank Charley Harrington for his assistance with statistical analysis and editing suggestions for this paper.

VIII. ABOUT THE AUTHORS

Saxon G. Reasons is Instructional Technology Services Programming Manager at the University of Southern Indiana. She holds an EdD in curriculum and instruction from the University of South Carolina, an MS in ecology from Vanderbilt University, and a BS in biology from Furman University. Dr. Reasons assists colleagues in distance learning course development and leads technology-related training for faculty, staff, and the community.

Kevin Valadares is an Assistant Professor of Health Services/Administration in the School of Nursing and Health Professions. His background is in health care ethics and health care administration. He teaches both undergraduate and graduate-level courses, all of which have various components of distributed technology. He is currently interested in the use of online discussion forums and online group work as a tool to improve student learning and writing.

Michael Slavkin is an Assistant Professor of Education in the School of Education and Human Services. His background is in developmental psychology and assessment. He teaches both undergraduate and graduate-level courses that employ varying levels of online technologies. He is currently interested in identifying pedagogical strategies that can assist in improving student outcomes in distributed coursework.