AN INTERPRETATIVE MODEL OF KEY HEURISTICS THAT PROMOTE COLLABORATIVE DIALOGUE AMONG ONLINE LEARNERS [1]

Sarah Haavind
Lesley University

ABSTRACT

One of the more challenging aspects of teaching online is promoting content-focused, collaborative dialogue among students. How do we move discussants beyond initial brainstorming toward more focused, deepened dialogue that clearly supports a course’s instructional goals? Garrison and Anderson’s framework for communities of inquiry illuminates the critical interplay among social presence, cognitive presence and teaching presence for learning in asynchronous, online courses [2]. This paper describes aspects of teaching presence, specifically explicit teaching of how to engage in collaborative dialogue, collaborative activity designs and evaluation rubrics or feedback that, supported by attention to maintaining social presence, helped to promote substantive, collaborative dialogue, or cognitive presence, in Virtual High School™ classes [3].

KEYWORDS

Online Learning, Community of Inquiry, Discourse, Teaching Presence, Cognitive Presence, Discourse Analysis

I. INTRODUCTION

Learning to teach online is not a simple transition for instructors. We can no longer assume that what works in a classroom will work online. This challenge is evident in secondary online courses as well as in higher education. A visual scan of the threaded course discussion areas in the archives of the 112 spring 2003 classes taught in the Virtual High School (VHS) revealed just 22 classes (19%) that indicated some collaborative dialogue or extended peer review activity may have occurred, based on the presence of occasional thread depths of four or higher.

I sought incidents with thread depths of over three for two reasons. Thread depths of one or two, the predominant trend found in 51 (46%) of the class discussion areas that semester, appeared too minimal to be called “dialogue.” Thread depths of three predominately represented a single exchange with the author of the initial post returning to add a “thank you” or “I agree.” That also did not amount to “extended” dialogue. Once the thread depth passed three, at least a minimum of extended dialogue appeared to have occurred. I, therefore, set the standard for collaborative dialogue at a minimum thread depth of four postings, indicating that students were replying to one another.

The results of my survey suggested that VHS teachers might want to adopt new strategies that support usage of text-based dialogue to foster a classroom-like “community of inquiry in which students listen to one another with respect, build on one another’s ideas, challenge one another to supply reasons for otherwise unsupported opinions, assist each other in drawing inferences from what has been said and seek to identify one another’s assumptions” [4]. In their evaluation of VHS, Zucker and his associates at SRI [5] confirmed that fostering content-focused dialogue was one of the most significant challenges to secondary level online instructors. To pursue Lipman’s vision, using collaborative dialogue online, exchanges among students must have sufficient depth and levels of engagement that enable discussants to build on one another’s thinking and deepen their mastery of new material.

In just three of the 22 classes that contained some extended discussion threads, I found a regular pattern of extended threading involving thread depths of four or more, along with a low volume of instructor postings in content threads. These thread maps suggested dialogue occurred regularly across the full fifteen-week semester. Indications in each case included the presence of more than 25 occasions where thread depth was four or higher, spanning the 15 weeks. In addition, instructor postings in each of the three classes totaled less than 30 in content for across the semester, or two or fewer each week. The lower level of active instructor involvement in content-based dialogues suggested other types of scaffolding may have supported learner-to-learner collaboration in those discussions, such as clear rubrics, explicit teaching and/or assessment related to co-constructing understanding through dialogue. More instructor involvement would have suggested extended dialogue depended on an instructor actively pushing the discussion forward. A high level of teacher talk within content-related dialogues defeats or at least interrupts the process of engaging students in dialogue with one another—the outcome that I wanted to study. From a practitioner’s viewpoint, intensive and ongoing teacher participation in student dialogue is simply too much work for the teacher, making teaching online more time consuming and challenging than necessary. A regular pattern of lower instructor activity in content-related dialogues was therefore a useful filter, along with the appearance of regular extended dialogue to identify the most interesting classes for further examination.

The analysis of the three classes that appeared on the threaded discussion surface to be highly collaborative could potentially reveal elements of teaching presence and social presence that lead to cognitive presence among the learners. In order for these courses to be instructive, the discourse analysis of collaborative events would need to show that learners were focusing their comments on new content and building on one another’s responses, insights and questions.

This paper describes a preliminary study designed to begin addressing the challenge of fostering cognitive presence among secondary learners. It concludes with an emerging interpretative model that synthesizes key elements that support sustained collaborative dialogue found in all three classes. The resulting model combines three critical elements: the explicit teaching of how to engage in collaborative dialogue, collaborative activity designs, and evaluation rubrics or feedback directly linked to contributions to the forward movement of class discussions. Each of these elements is a facet of teaching presence according to Garrison [6]. These elements of teaching presence were supported by attention to maintaining social presence in the classes examined. They were found to be the most likely elements to have fostered such highly collaborative dialogues and cognitive presence in the context of VHS.

The paper is organized in the following way. First I describe the VHS context. Then I describe the theoretical framework that I used for understanding online collaborative dialogue. Third, I describe the methods employed for transcript analysis and the qualitative approach for identifying and analyzing other elements of teaching presence. Fourth, I present the findings leading to the emerging model of elements most likely needed for fostering collaborative dialogue. Finally, I discuss implications of the results and recommendations for practitioners.

II. THE VIRTUAL HIGH SCHOOL

The Virtual High School is a collaborative effort among public high schools in which teachers at participating schools develop and offer online classes that are open to secondary students from all participating districts. The VHS was first launched ten years ago with a 5-year, $7.4 million dollar Technology Challenge Grant from the U.S. Department of Education to the Hudson Massachusetts Public Schools in partnership with The Concord Consortium. The concept was that accredited high schools would enter into a collaborative relationship in which teachers from those schools would lead virtual classes in exchange for seats for local students in the VHS, twenty seats for each semester class taught.

In 2001, the success of the project enabled VHS to become a non-profit, independent organization. In 2006, over 7,500 students are enrolled in more than 200 VHS classes. Nearly 400 member schools include schools in 30 states. Twenty-five member schools are international schools from around the world. The student body is currently 57% female and 43% male. Additional demographics are not readily available. Course offerings include such titles as History of Pop Music to 101 Ways to Write a Short Story, Fractals, Introduction to Programming in Visual Basic, Advanced Placement Calculus, Biotechnology, the Holocaust and the Vietnam War [2]. VHS is considered an exemplary model for pre-college online learning [5].

I wanted to know what design and facilitation strategies VHS teachers who were highly successful at fostering and maintaining asynchronous online dialogue about course content used to promote collaborative dialogue focused on course learning objectives. In particular, I assessed the presence of content exploration evident in the collaborative exchanges in order to then be able to weigh the effectiveness of some strategies over others for promoting focused discussion on and deepened exploration of stated knowledge goals.

III. FRAMEWORK

Nearly all of the researchers studying online learning have focused on higher education. However, secondary level online course offerings have been increasing rapidly. Distance education opportunities existed in about 36% of US public school districts during the 2002-2003 year. The numbers are growing annually [7]. The community of inquiry model, and the delineation of the critical roles of social presence, cognitive presence and teaching presence in higher education courses, is useful for understanding collaborative constructivist approaches to teaching secondary-level virtual learners. At the secondary level, the content may be less complex. Yet, the inquiry-oriented processes of teaching and learning are equally appropriate, on- and off-line [8, 9, 10, 11]. Social presence, or the establishing and continued nurturing of a sense of community among learners must be fostered for people of any age to feel comfortable taking the risks needed to learn deeply [12]. Cognitive presence grows among learners as they co-construct their understanding of new content. Bransford and the National Research Council, in How People Learn [13], point out the value of such social interaction for cognitive engagement in the K-12 arena:

Teachers must attend to designing classroom activities and helping students organize their work in ways that promote the kind of intellectual camaraderie and the attitudes toward learning that build a sense of community. In such a community, students might help one another solve problems by building on each other's knowledge, asking questions to clarify explanations, and suggesting avenues that would move the group toward its goal [14, p. 25].

The use of dialogue as a learning activity is also not new [15]. Harasim [16] identifies the emerging role for computer-mediated conversation described by Brown [3] as “the shift from seeing technology as a cognitive delivery system to using it as a means to support collaborative conversations about a topic and the ensuing construction of understanding.” Online, asynchronous collaborative dialogue is a potent, new form of collaborative work. Bruffee [17] highlights the potential of conversation for deepened thinking. Aviv describes asynchronous learning networks as “cooperative learning enhanced by extended think time” [18] since the asynchronicity provides learners the opportunity to reflect and think through a response before responding. Bender suggests, “we can think of teaching and learning as being comprised and communicated by the words that flow between teacher and student, as well as student and student.”[19]. Specifically, invitations to learners to post comments to discussions of class readings, science investigations, or math problems ; to peer-review one another’s assignments [20]; or to share questions and insights about a learning experience can prompt participants to collaborate, or “co-labor.” That co-laboring online becomes collaborative dialogue. The researchers cited here focused on higher education. However, the same reflective advantages of the online text-based medium can likely be nurtured in younger learners, when appropriate teaching methods are used.

When describing the community of inquiry framework, Garrison points to design features, discussion facilitation, and pedagogic leadership that must blend social and cognitive issues and expectations in order to achieve an effective teaching presence in an asynchronous, text-based environment. He notes these considerations go “well beyond deciding what content will be covered” [6, p. 26] . The interplay among collaborative, constructive processes that effectively shape inquiry experiences can be revealed through case studies. Therefore, the goal for this study was to analyze case studies of potentially exemplary VHS classes.

IV. METHODOLOGY

A. Transcript Analysis

Before I could examine pedagogy, I needed to confirm that the thread depths of four or more appearing on the surface of thread maps in the three classes I had selected actually contained collaborative dialogue. My approach to transcript analysis was to conduct three passes through the transcript data of all the collaborative events, or threaded dialogue amounting to a thread depth of four or more, in each of the selected classes with the goal of establishing the overall quality of collaborative dialogue in each.

In the first pass I employed Harasim’s [16] categories of collaborative postings: content-based idea generating, idea linking and convergence. According to Harasim, idea generating messages typically introduce ideas or understandings, offer an opinion, or provide an example to the discussion. In VHS discussions, initial postings to discussion assignments were generally idea generating in that they offered student thinking on an assignment question or task.

According to Harasim, postings characterized as idea linking referred directly to previous messages, perhaps referencing the name of the author. They included agreement, disagreement, a statement of enhanced understanding, elaboration, or a brief quote from a previous post with a comment. In the VHS postings, the reference was often implicit, as in “I agree that…” or “What do you mean by…”.

Evidence of intellectual convergence, according to Harasim, is reflected in shared understandings, where ideas of a group of students are combined. She notes that convergence is most likely to appear in the context of participants “engaged in co-production, whether it be producing a report, a presentation, a point of view, a work of art or a scientific theory ” (p. 195). None of the classes I selected for this study included these types of team assignments within discussion fora. In addition, I added two additional categories, social (e.g., “Thanks!”) and evaluative (e.g., “I agree!”), to better fit the VHS data.

For the purposes of my study, I wanted to know whether the content of sustained, threaded conversation was collaborative dialogue. To answer that question, I needed to quantify the amount of dialogue that was on-topic and collaboratively linking or building on the ideas, questions or insights of others. I would then have comparable profiles of dialogue quality. Were participants engaging socially or were they constructing understanding together? Posts, or parts of posts, that offered simple positive evaluations or agreement without extending any ideas were discounted. It served my purpose to conduct a simple assessment of whether initial posts presented an initial response to the discussion topic or not, and whether linked posts extended that initial thought in some way as a linked idea, question, or insight. In other words, would discourse analysis bear out the surface perception that threaded posts represented collaborative, co-construction of understanding new content? Put another way, how much of these collaborative events signified cognitive presence among the learners in each class?

To find out, I first I color-coded all collaborative events with thread depths of four or more, according to Harasim’s framework and my additional categories. In a second pass, I made visual assessments and assigned percentages to each portion of every post by category in order to quantify the content. I made a separate, new pass through the data to match stated learning goals from the three classes with each printed line of the contents of each posting. I chose the unit of a printed line in order to avoid subjective interpretations of “units of meaning” such as sentences, while at the same time gaining a more precise measure than paragraph or message units would provide. Tangential comments were separated out. I assigned a new set of percentages based on direct alignment of discourse with learning goals. In general, the VHS posts were concise and targeted making coding less difficult.

For example, the following post met the stated literature course objective, “Reflect on your life experience, applying the author’s philosophies, in writing and discussion” by 100% (every line of text was directed toward stated learning goals) in one of the three classes. The discussants were asked to compare the meaning of love romantically and realistically, in the context of an analogy made to salt crystals on a branch conceived by the French philosopher Stendahl. This learner built on a previous discussant’s comment stating,

This theory was really interesting to me because it sounds so familiar! It really reminds me of the part in Gone with the Wind (if any of you have read that) where Scarlett discusses what she felt for Ashley as being “in love with a pretty suit of clothes.”

I think we’ve all been there before. It’s so easy to fall “in like” with someone when all you know about them is what you yourself made up It’s not so fun when the “crystals shake off” and you realized you just wasted so much time on someone who wasn’t really worth it!

In an example from a class in nuclear physics, the topic was depleted uranium (called “DU” in the comment). The discussion explored the question, “What responsibility do we have to workers who are exposed to radiation or have been exposed to radiation in the past?” Again, the following post was made by a discussant building on the comment of previous participants. This post was also 100% on topic. Note the explicit linkage to a previous comment in the opening:

[Name] mentioned that DU is a toxic element, which is the main reason why DU is dangerous. Think about the substances people consume everyday that are toxic…tobacco and alcohol are two prime examples of toxic substances (consumed daily by many) Can we blame the government for getting diseases from using these toxic elements? Sure, these are by choice – but DU has not shown strong evidence of illnesses directly from the substance.

Although 100% content-related postings predominated generally in VHS content forums I examined, more socially oriented exchanges did occasionally appear. For example, in the following set of three linked posts, the instructor first re-visited a discussant’s comment, the learner responded, then the instructor followed-up. The instructor’s first intervention was 100% on topic. The learner’s response was about 50% social (apologetic) and the instructor’s follow-up was 100% social in nature, essentially intended to be reassuring to the learner. At the same time, the dialogue has moved off-topic (albeit, not far) and no longer directly targeted the stated objective that “students will be able to solve half-life problems.”

Rate of decay (Instructor): Are you sure there is a direct relation? I’ll agree the activity is higher for an isotope that decays faster, but if something decays faster, does it mean it has a long half-life or a short half-life?  (100% on topic/2.5 lines )
My Mistake (Learner): I feel pretty stupid contradicting myself.
Obviously, as half-life gets smaller, activity increases: inversely proportional. Sorry. (50% social/1 line social; 1 line content-related)
Don’t (Instructor): Don’t feel stupid. Your mistake was quite a common one. Many of my f2f students confuse inverse and direct relations (and I hate it when they use the term indirect instead of inverse). With these two quantities, it is even easier to get them mixed. (100% social/3 lines social)

The purpose of this additional analysis was to be sure the selected classes showed clear evidence that learning about the target goals was consistently and clearly supported by learner dialogue.

I trained a second individual to code a random sample of 10% of the full data set, in order to ascertain inter-coder reliability using the Cohen’s kappa measurement. Following training, and an initial coding session, I revised my training to clarify what counted as substantive linking phrases, and what was simply social or evaluative. For example, in the training, I was thinking of “evaluative” as evaluating the previous participant’s comment. In the first round of coding by my second coder, she coded all statements that evaluated the subject matter of the class (e.g., “I liked this story because…” or “I disagreed with this author since…”) as “evaluative.”  Another correction I had to make was to clarify how a linking comment had to be substantive, in that it was directly related to course content. For example, a student had posted, “Okay, I’m not confused anymore. I figured out what was confusing me. Good story!” My partner coded that as a linking comment, however, according to my schema, it does not add anything to the co-construction of understanding new material; instead, it is a social response to a helpful clarification. With a follow-up post of “Great!” we would have a thread depth of four with only the first two posts being substantive. In a similar case, my partner labeled a comment as linking that stated, “Thanks for the heads up on the movie. I’m definitely going to rent it.” The movie mentioned was a movie that had a plot similar to that of the story under discussion, used in order to make a connection. That connecting post contained a substantive, linking comment. However, it was not a substantive build to respond with plans on renting the video. That again was social, according to my schema. Postings were coded again. The final reliability measure was .8865 or a percentage agreement of 92.68% according to Cohen’s kappa.

I also analyzed instructor postings to understand how discussion facilitation was conducted in each of the three classes and to determine the impact of instructor interventions into dialogue on collaborative discourse. Garrison and Anderson [2] highlight the importance of  “cognitive presence” for online learning, defining it as:

…the intellectual environment that supports sustained critical discourse and higher-order knowledge acquisition and application. More specifically… facilitating the analysis, construction, and confirmation of meaning and understanding within a community of learners through sustained discourse and reflection largely supported by text-based communication. (p. 55)

The framework developed by Collison et al. in Facilitating Online Learning: Effective Strategies for Moderators [21] adds specificity to the general guidelines indicated by Garrison and Anderson. Taxonomies of voices and critical thinking strategies described by the authors are intended to both engage and probe student thinking to further collective exploration of content. Therefore, they were useful in analyzing the specific pedagogical moves at work in the VHS dialogues. Collison et al. identified six “voices” or roles and a set of critical thinking strategies that effective online instructors can use in combination to further collective thinking and learning. These instructor voices are presented in Table 1, below, followed by a brief description of the set of critical thinking strategies.

Table 1: Instructor Voices

The critical thinking strategies identified by Collison et al. included three that are useful for sharpening the focus of a dialogue. Identifying a Direction for dialogue points toward but does not reveal essential elements for improved understanding of a problem or a process. Focusing on Key Points posts draw from participant input related to specific learning goals to highlight essential comments and connections made to date and thus clearing the way forward toward additional learning goals. Sorting Ideas for Relevance posts identify candidates for further exploration that are suggested in initial postings.

Three additional strategies serve to deepen a developing dialogue. Full-Spectrum Questioning offers five general categories of questioning: “so what?”; clarification of meaning; exploration of assumptions; identification of causes and effects; and questions that consider appropriate action. By modeling these richer modes of probing discussants, the facilitator points toward new approaches for viewing or questioning current thinking. The Making Connections strategy challenges discussants to move away from contributing “more of the same” to explore inferences or tensions that will help participants shift to more systemic layers of content exploration and recognition. The Honoring Multiple Perspectives strategy suspends belief, holding up tensions in discussants’ conceptual frameworks for further consideration.

Instructors foster collaborative content-related dialogue using one of the voices in conjunction with a critical thinking strategy. For example, in a VHS language arts assignment, students read a short story by Edgar Allen Poe, “paying close attention to how Poe introduces and carries out the conflict” and “support [ing their] interpretations with specifics from the text.” The resulting discussion was off-target and the instructor intervened to re-focus the group more specifically on topic. In Facilitating Online Learning it states, “If your diagnosis is that plenty has been posted but little of value has been revealed…you can list current directions or contributions, and seek alternative ordering principles, extensions, or interpretations, with the goal of targeting likely conceptual blocks or assumptions” [21, pp. 106-7] . The instructor used a Generative Guide voice to Identify a Direction for more content-related dialogue:

It’s fine to exchange stories about revenge strategies in the lounge, but here I want you to focus on what happened in the story. How does Poe use innuendo? How does he reveal the conflict? What in the character’s personality can we identify?

In this case, the instructor’s initial directions were focused but open-ended (“paying close attention to how Poe introduces and carries out the conflict” and “support(ing) your interpretations with specifics from the text”). The intervention was intended to sharpen a dialogue that was active but not focused on targeted learning goals. She acknowledged their contributions, suggesting that their exchange to this point was acceptable in a social forum, but not for the assignment. She clarified the original assignment. The goal was to generate more focused discussion by identifying the more appropriate direction for students to explore.

In another example, students were discussing Shakespeare’s Anthony and Cleopatra. They produced an active brainstorm and were ready to dig more deeply into their analysis. In Facilitating Online Learning it states,

Most facilitators of online discussions read, sort and assign value to discussion entries and their components based on the context of the topic or the course goals. The sorting ideas for relevance critical-thinking strategy addresses a very different process: Crafting an entry that explicitly, but informally, makes public the sorting mechanism, leaving options open for collective input [21, p. 133] .

In the following example, the instructor prodded the students to consider how the bits of evidence they had gathered as they described the relationship between the two main characters fit together into a consistent picture. He offered them a Reflective Guide post to help them Sort for Relevance:

Hi all...
I'm reading the analyses below, and we've come up with an impressive array of poetry readings...A caution about temptation? (Names Student 1)
An author with an axe to grind against males?  (As Student 2 notes) Male Goblins as a necessary evil for childbearing? (Did I get that right, Student 3?)
(Student 4) suggests there's an inherent weakness in rationality and moral behavior—exposed in the poem. A tale of sibling love? (Names Student 5)
Have we made progress?  We've got a *wide* array of responses on the table... which should we take deeper? And those of you whom we haven't heard from... do get in soon so the rest of us can post our follow-ups!

This instructor guided by mirroring or reflecting back to learners an “array” of insights gleaned from the dialogue, naming contributors and citing their words. He avoided telling them where to focus and instead asked them to sort through their collective gems for relevant pathways worthy of more careful consideration by all. In this case, the instructor helped to sharpen the focus of discussants confronted with a barrage of postings and ideas by narrowing their consideration to the key ideas already on the table. This type of intervention can leverage the group to a more reasoned level of deepened discourse on the important ideas already on the table without narrowing their focus down a single pathway or toward “the one right” answer.

For the VHS data, I first categorized all the public postings of the instructors in each class. I then looked at any moves instructors made in the content-related fora toward focusing or deepening dialogue and assessed alignment between such postings and the voices and strategies described above. I also found other types of instructor comments in public discussions, such as social comments, probes for clarification, public praise, or suggestions of resources. I categorized these to uncover other patterns that might explain the presence of collaborative dialogue. I conducted an open-coding approach to label each public posting made by the instructor. Table 2, below, organizes my final set of categories by the general instructional processes identified by Garrison and Anderson.

Table 2: Indicators of Teaching Processes in Content-based fora.

Once again, to assess inter-rater reliability for this coding procedure I trained a second coder to code a random sample of 10% of instructor postings. Cohen’s kappa agreement was .9415 after two rounds of training. Confusion was reduced for coding these postings with the use of Table 2, above. After a review of the categories, we achieved strong agreement.

After transcript analysis was completed, the quality of dialogue in the selected classes was assessed, and instructor postings were analyzed, I turned to additional elements of the course that may have influenced collaborative dialogue. I considered evaluation and feedback, instructional design features, participant influences, and comparable elements in equivalent courses where collaborative dialogue was low. The remainder of this section describes these methods.

B. Evaluation and Feedback

I compared private feedback in the three classes to assess the role of evaluation in fostering collaborative dialogue. The VHS instructors interacted in three ways with their virtual students. First, they posted in the public discussions. Second, they conversed privately with students in individualized discussions. Third, they posted private feedback to each student about their work, including formal grades. In these private discussions, the conversations were predominately of the “Where’s your assignment?” and “Sorry I was late posting this week” genre. Formal feedback about performance in the class appeared in another area of the class designated for grading and evaluative comments. I examined patterns in instructors’ private feedback to learners that directly supported student dialogue about new content. In order to compare the relative emphasis on and support for substantive interactivity, descriptive statistics such as how often an instructor addressed discussion participation or offered suggestions to students about how to collaborate with their peers more effectively were drawn from the archives.

C. Instructional Design Features

All the VHS classes shared similar design features that supported peer interactivity. For example, VHS instructors were required to start with “getting-to-know-you” icebreaking activities to foster a socially-connected community. Research has established social community as a necessary foundation for generating intellectual camaraderie [2, 22, 23, 24]. Most VHS classes, including those in this study, contained at least one social forum such as a student lounge that was active throughout the semester. VHS classes required discussion participation and final marks incorporated student participation. These common features of VHS classes were actively encouraged by VHS administrators and served as a baseline for fostering interaction. Though essential, they were not in themselves enough to assure substantive student dialogue, given the lack of student interactivity in most classes that semester.

To uncover additional supportive design features, I sought design elements that were likely to have contributed to the cognitive presence among discussants. I used the constant comparative method [25, 26, 27] and an open-coding procedure [27, 28]. I considered aspects of overall course design, specific activity designs, and discussion starter postings in order to assess the aspects of teaching presence that potentially played a role in achieving collaborative dialogue in these classes. I coded design features evident in introductory course documents that might have corresponded with increased or decreased numbers of collaborative events. For example, I documented such dimensions as small team discussions versus whole class discussions, detailed grading rubrics and posting requirements, and discussion activities where students were assigned specific roles. I sought out descriptions of grading practices, specifically related to student-to-student interaction. Were postings evaluated, and if so, what percentage of the final grade did they account for? Was the importance of building on one another’s ideas or “active listening” mentioned or emphasized?

D. Cross-Case Comparisons

In order to develop a complete analysis across classes, I mixed case-oriented and variable-oriented approaches [28, 29]. I constructed descriptive cases [30] in order to have a holistic view of each instructor and class essential for contextualizing the complex interplay among variables. I displayed these variables in matrices that compared the relative prevalence of some variables over others. Carefully examining variables such as type of feedback or course design features (team work, emphasis on dialogue in discussion assignments, etc.) helped to isolate which approaches may have had the most impact, when looking across cases or across weeks in a single case.

E. Participant Influences

To check my assumption that collaborative dialogue was a result of course design or instructor facilitation and leadership, I also considered participant influences. This consideration served two roles in the study. First, the positive role that participants might play in furthering learner dialogue (asking their peers questions, engaging active listening, etc.) could be useful to instructors seeking to teach their students how to engage more collaboratively. Second, examining participant roles in extended dialogue made the model development process more robust, helping to avoid the pitfall of inaccurately attributing sustained student dialogue to instructor moves and design features. Did learners initiate or sustain continued peer dialogue? What online, text-based rhetorical behaviors could be encouraged by instructors and learned by discussants to support extended dialogue?

Research has shown that the gender roles may shift when a course moves from a face-to-face environment onto online discussion boards. In the face-to-face environment, where airtime is limited, research has shown men are more at ease with speaking out. However, online, where competition for airtime is removed, women may be more at ease with the collaborative, networking aspects of the environment that emphasize dialogue. A recent review of the literature by Hiltz and Shea [31] suggests that, though findings of new studies on gender and asynchronous learning networks have yet to be replicated, it appears that gender inequalities aren’t great. However, it is intriguing that, as stated in the original proposal for the project, the intention of VHS was to admit an equal number of male and female participants, yet gradually over time the percentage of females over males has increased. Currently, 57% of VHS participants are female and 43% are male. Therefore, I sought out indications of gender imbalance in the collaborative participation of learners.

F. Comparison with Classes Showing Lower Levels of Collaborative Dialogue

In an earlier study [32], I considered two VHS classes from the same original set of spring 2003 archives that were comparable with matching sessions of the same two classes, but were taught by different instructors and hardly exhibited any linked dialogue at all. The purpose of that pilot study was to compare classes with higher levels of collaborative dialogue with equivalent sessions of the same class that appeared to have lower levels of collaborative dialogue. I considered the results of that study as I weighed the likely impacts of the approaches used in these three exemplary classes. If approaches in both types of classes, those with higher levels of collaborative dialogue and those with lower levels of collaborative dialogue were similar, those elements were less likely to be the factors that explained higher levels of collaborative dialogue.

G. Reliability, Validity and Generalizability

Archived online course data offer a unique opportunity for data collection. The researcher has no impact on class activities and can study them in their near totality without intruding. I made numerous passes, or “check-coding” [28] through the same discourse data for open-coding, and re-coding to assess personal reliability and to correct errors. This reduced reliability problems [29]. As described above, a second coder coded 10% of the discourse data in order to assess reliability and the results were strong. I also conducted “member-checks,” sharing my findings with the VHS instructors who taught the three classes included in the study.

The validity of my analysis may seem at risk, as I selected just three classes from VHS to examine, and from which to draw preliminary conclusions. The strengths of the methodology were the completeness of the data set initially surveyed: one full semester archive of VHS classes, and the complete data set of three classes examined that had revealed over 25 collaborative events across the semester with a minimum of direct instructor intervention.

Now that this study is complete, the resulting interpretative model can be tested internally, by VHS instructors. A new study with an experimental design could compare two semesters of the same class taught by the same teacher: the semester in which he or she attempts to shift practice toward the design and teaching practices suggested by the research could be compared with the previous semester. Such a “pre-post” research design would test and strengthen the validity of the model.

The classes examined represented course disciplines, designs, and instructors who were most likely actively seeking a high level of learner dialogue in their courses. They did not represent all academic disciplines, nor did they represent an exhaustive list of possible approaches to increasing collaboration. For VHS instructors who have been frustrated in their attempts to increase student-to-student collaborative dialogue, referenced in Kozma and Zucker’s account of the challenges VHS teachers face, the model developed in this study may be of interest and use.

The study may have internal generalizability within VHS where the format and general culture is constant. However, the constructed model is preliminary and untested. It must be tested for replicability before it can have significant external generalizability. Although not conclusive, this initial research established standards and methodologies that can be used in the future to further develop interpretative models that can improve online learning in a variety of settings.

V. FINDINGS

Sustained, collaborative dialogue among learners, evidenced by thread depths of more than four, and called “collaborative events,” appeared regularly throughout only these three 15-week VHS courses offered in the spring of 2003. In addition, there was a minimum of instructor facilitation, implying that other factors of design, rubrics or assessment fostered sustained peer-to-peer conversation. One course was an honors language arts class open to grades 9-12, examining the theme of love in literature. The others were standard courses open to grades 10-12. One was a social studies course on the United Nations, and one was a course on nuclear physics and society. Examination of the three selected discussion areas revealed that the language arts course and the science course were the two classes with the most robust collaboration levels. Both had 29 collaborative events with thread depth of four or more that spanned the 15-week semester. The average thread depth for both was four, and the average thread length was 5 in the language arts class and 6 in the science class. The social studies class achieved all 26 of its collaborative events within just four weeks during the class when discussions were assigned. This high concentration of collaborative events was unique. The extent of collaborative dialogue, with an average thread depth of five and thread length of 13 within each of these events, was also notably high for VHS that semester.

Although the thread depth in each of the collaborative events varied, many were conversations involving just three linked replies building on an initial post, suggesting that even in these three classes, which may have represented the most robust levels of sustained, collaborative dialogue in VHS classes that semester, there was not much evidence of extended dialogue. These relatively low levels of interaction point to the importance of providing training to secondary instructors. Research in higher education has definitively shown the importance of interaction for learning online [2, 33, 34]. These collaborative learning processes are equally important for younger people [13].

A. Dialogue Quality

I also analyzed the quality of conversations in collaborative events using Harasim’s taxonomy of online discourse [16], seeking posts that were idea generating, idea linking, and possibly reflecting a convergence of ideas. I found that these VHS learners conducted highly focused discussions of content in collaborative events. The middle column in Table 3 below shows the percentage of postings using Harasim’s collaborative categories in all three classes. When I quantified the amount of dialogue within collaborative events (CEs) that was in support of explicitly stated learning goals, the third column shows that I also found the dialogue quality to be high.

Table 3: Collaboration and Learning Scores
*A major digression in the SSt class during March of 2003, a class about world conflict and the United Nations, focused on the controversial start of the war in Iraq. The dialogue was not directly related to course learning objectives. If that “digression” to a related current event is considered a digression from learning goals, only 94% of lines of discussion strictly adhered to stated objectives. If the discussion is considered at least indirectly related to the goals of the class, 99% of the lines of dialogue were directly linked to course goals.

In fact, the three courses selected as a result of the initial visual scan of all the spring 2003 course discussion areas turned out to contain substantive collaborative dialogue about course content, focused on meeting course learning goals. I could, then, focus on my research question and delve into these three classes to determine what aspects of social and teaching presence led to sustained collaborative dialogue, or cognitive presence, in each case.

B. Social Presence

The online medium for learning poses inherent challenges to instructors seeking to foster collaborative dialogue. The social cues that support learner collaboration in a face-to-face classroom are easily taken for granted. These include the facility with which class members exchange eye contact, facial expressions, and body language. Before learners feel comfortable joining in a goal-directed dialogue, alternative methods of fostering a sense of social community and trust must be implemented in a text-based, asynchronous environment [2, 35]. A clear sense of community [22] or social presence [2] has been established as a key factor in measuring the success of learning online [12]. All three classes in this study showed evidence of social presence with fully-attended ice-breaking activities during the first weeks, as well as active social fora that were available and used by the students regularly throughout the semester to discuss non-course related topics. Students enthusiastically discussed new movies, music, college plans, and other matters in all of these classes. However, social bonds only support learning. To locate the elements of teaching presence that built on the established foundation of community bonds, I turned to such instructor moves as discussion facilitation, feedback, evaluation, course activity designs and discussion prompts.

C. Teaching Presence

Teaching presence is described by Garrison [6] as “the crucial process of constructing meaning through collaborative inquiry…the crucial integrating force that structures and leads the educational process in a constructive, collaborative and sustained manner ” (p. 26). He described this leadership role in terms of four elements: design, discourse facilitation, direct instruction and assessment. In each of the three VHS classes I examined, the interplay of these elements was unique. In order to ascertain the elements of teaching presence that most likely contributed to collaborative dialogue among learners I took the following steps: 1) I coded instructor postings using the Collison et al. framework to single out facilitative interventions intended to focus or deepen dialogue; 2) I categorized and compared patterns in private feedback and evaluation rubrics related to dialogic participation by learners; 3) I examined and compared course documents such as those related to expectations for participation, specific discussion activity designs and instructions, and discussion prompts. Findings in each of these areas are described below.

1. Discourse Facilitation

Discourse facilitation can support cognitive engagement by helping discussants focus more sharply on content, or by moving them more deeply toward mastery of new concepts. The role of facilitator is inherently a coaching or supporting role that helps to guide inquiry. It differs from the role of expert, who might provide explanations, clarifications, or other forms of direct instruction within the context of dialogue. The Collison et al. framework of voices and critical thinking strategies identifies specific approaches a facilitator might use to focus or deepen dialogue.

The language arts instructor only interjected three posts into content-related dialogue all semester. They were all facilitative comments according to the Collison framework. For example, “Hi, (Name), What pieces suggest eros later in the play? This is an interesting point…is Shakespeare inconsistent in his characterization of their love?” Using the voice of a reflective guide, he mirrors the point a discussant is making in order to broaden the scope of the discussion, making a connection to the possibility that Shakespeare, the author, might be inconsistent. Two of these facilitative interventions were inserted into Week 3, when content-related discussions were first getting underway. The third appears in Week 6. This early modeling of deepened exploration for the group may have helped to foster collaborative dialogue among his students. It is hard to know, but useful to highlight the minimal active participation needed in this class to achieve collaborative dialogue among learners.

The social studies instructor interjected six comments into content-related dialogue. One served as a generative reminder, “I believe we are getting away from the intended topic of this week’s discussion.” The other five were probes such as, “Why should the same five nations in the (United Nations) Security Council have the only say in what gets a veto? Is this fair?” Again, perhaps these facilitative comments helped nurture collaborative dialogue, but it is likely other elements made more of a difference. It can be said that neither of these two instructors needed to be very actively supporting dialogue.

The science instructor posted more regularly into learner dialogues, with a total of 27 over the semester. Twelve of those delivered direct instruction on a science concept. Nine interjections were evaluations of student comments. Five suggested additional resources. One was public praise. The science instructor did not use focusing interventions as defined by the Collison framework at all. In fact, his approach contrasts in a number of ways with the language arts instructor. Some examples of direct instruction interjections into dialogue were:

A few of you have mentioned accidental discoveries. I’ll agree that they occur, but do they represent the bulk of scientific discoveries? Galileo is frequently given credit as the person who truly started the ‘scientific method.’ They point to his methodical experiments and careful documentation of data and results. When he came to his conclusions, he had the evidence to back it up. That doesn’t sound ‘accidental’ to me.

Rutherford was not using a cathode ray tube to separate the three types of radiation. He simply had the radiation, shooting from a pinhole in a lead container, pass through a magnetic field before striking a fluorescent screen.

The presence of direct instruction within the content dialogues may simply represent a difference between the roles of a science teacher to support accurate understanding of scientific facts, versus the role of a teacher of literature orchestrating dialogue where the meaning of literature is discussed. For literary analysis, more of a focus on interpretation drawn from life experiences or evidence drawn from a text is needed.

When comparing these three instructors, it may seem that 27 interventions into learner dialogues by the science instructor was relatively high, however, over a 15-week semester, his posts amounted to fewer than two per week. Therefore, although the other two instructors entered into learner dialogues very infrequently, all three of these instructors kept their active steering of learner dialogue to a minimum.

In summary, no clear pattern of instructor facilitation appeared to be related to promoting collaborative dialogue. The use of interventions that fit the model described in Collison et al.’s Facilitating Online Learning [21] was minimal in all these classes containing extensive, regular collaborative dialogue, and therefore the analysis of the effect of these interventions remained inconclusive. Other elements of teaching presence appeared to contribute more to overall effectiveness.

2. Evaluation and Feedback

Assessment of participation appeared to contribute to dialogue quality in all three of these cases. Each instructor presented rubrics or expectations for postings differently, yet collaborative dialogue was clearly supported in all three classes by feedback and/or evaluation rubrics.

The language arts instructor offered continuous private feedback that emphasized the importance of dialogue for group learning. It was the single most distinguishing element of teaching presence for that instructor. He offered feedback that was unique in its explicit urging of learners to discuss, not just post, to earn full credit, and he offered it every time he delivered private feedback (bi-weekly throughout the class), with the exception of his final comments in week 14. Here’s an example from the grade book for a student who was not doing well. Dialogue was explicitly encouraged:

I see 5 posts from you in those two weeks. This does not meet the minimum number of required posts for this two-week period. If you intend to stay in the class (and I sincerely hope you do) I need to see a dramatic turn around in the amount of work you are doing. This is going to require a lot of work from you--it is possible to catch up--but you must start sooner instead of later.

I'm not seeing any of the required follow-ups from you that we so urgently need to build momentum and depth in our discussions.

It is important that everyone checks in at least 3 times each week. Cramming assignments in at the end of a week or not doing them on time has detrimental effects on the quality of our discussions, and it affects your participation mark. This class is about interacting, discussing, and engaging ideas with other classmates throughout the week-not merely turning in assignments at the end of a week. Fair enough?  So be sure to spread your participation out more so we can get your grade up. (Italics added.)

In the next example, the student was doing well, and the instructor emphasized what the learner was doing right, specifically pointing to the student’s contributions toward sustained dialogue. The instructor cited specific phrases from the learner’s postings to demonstrate exactly the type of contributions sought.

I see 18 posts from you in two weeks. While you didn't make that minimum number, you have posted a number of quality "builds" in our discussions. This is especially true when you wrote:
Had Goblin Market been written in present times, women would have been portrayed much differently. In those days, the only role women had in sexuality was fulfilling the man’s needs and desires. Things have changed, in a society that both sexes are out to satisfy themselves. Their roles in public are no longer being silent and beautiful but having a voice, opinions and independence are looked highly upon. Christina Rossetti’s poem would have portrayed the females as a little wiser in their actions, a little less dependant on the goblin’s company and their fruit and they wouldn’t have allowed for the mistreatment, swiftly calling a women’s abuse hotline.

Do you see what this post has to offer to the group . . .you have taken the discussion to a new and interesting place and offered direction for others to take up in follow-up posts. This is exactly the type of post we are looking for. Hurrah!!

Collaborative dialogue was most likely related to the frequent private feedback he provided in the case of this class. Therein, he emphasized the importance of dialogue, pointed to what students were doing that was interactive, and described specific ways learners might improve their efforts to discuss content with peers.

The science instructor offered numerical grades for discussion postings. The numerical grade was based on a detailed rubric for discussion participation:

To earn five points, comments were expected to:

• address a topic directly,
• show evidence of higher-order thinking, and
• contain no evidence of misconceptions.

To earn three points, comments must be:

• directed toward the issue,
• could contain minor errors, but
• generally had to demonstrate topical understanding.

Students earned only one point for postings that contained major errors, missed key points, or were too brief to be informative. Information related to enhancing collaborative dialogue was not discussed further.

The social studies instructor, similar to the other two, emphasized the importance of dialogue for maintaining a high grade point average. In this class, 66% of the total private feedback comments focused on class discussion participation including some interactive games, the class discussions, and a final, multiple-week UN simulation where students played roles as nations in the discussion area. Only 33% of private comments focused on written assignments. Dialogue was emphasized and expected in the games, the content-related discussion activities, and the final simulation. The course overview included statements such as the following two. The first clearly described how to interact; the second emphasized what happened to the student’s grade if they did not post more than four times per week:

Participation: For this course to be successful it is necessary for us to interact with each other. This means that you and I must interact as well as you interacting with your fellow students…When discussing, simply responding “I agree” does not constitute meaningful interaction as far as I am concerned. If you are participating in a discussion and you find yourself agreeing with another student it would be better if you stated: I agree with Fred particularly when he stated ________. This shows me that you are indeed paying attention to the discussion not just randomly responding to discussions in order to complete the assignment. [Expectations and Quality of Work in Start Here] (Italics added.)

In another introductory document, the grading policy for the course was described. Students were required to post a public acknowledgement of this document in the discussion area as an assignment. Dialogue was emphasized once again:

Important Note Regarding Discussion Grades: One area of this class that prompts the most student questions to me is that of grading discussion. Students are graded on both the quantity and quality of their participation during discussion weeks. The minimum amount of student participation during a discussion week is four comments posted on different days. Some students in the past have been unsure about what “minimum” participation means. “Minimum” means the lowest possible passing grade will be given for four comments posted on different days during discussion weeks. A minimum passing grade is 60% or “D-” So in other words should a student post four comments on different days during a discussion week they will receive a 60% for that week. I do not anticipate this being a problem for the vast majority of you. However, in the past some students have had trouble understanding this policy and my goal here is to make it as clear as possible. [Grading Policy: Week 2]

Comparatively, the science class weighted participation most heavily, with discussion participation accounting for 40% of the final grade. The language arts class weighted discussion participation as 25% of the final grade, supporting that leverage with extensive private feedback. In contrast, in the social studies class no overall percentage of the course grade for discussion participation was given in the beginning of the class. However, the documents and level of feedback comments directed at participation regularly reinforced the importance of discussion. This analysis shows that one likely answer to the question of what made these classes engage in collaborative dialogue more than most was an explicit link between collaboration and evaluation. This finding is supported by research in higher education as well [24, 36]. The language arts teacher was unique in his use of private feedback to foster collaborative dialogue. It seems additional elements must explain the striking outcomes in the science class and the social studies class.

3. Collaborative Instructional Designs

All three of these higher-collaboration classes were designed with promoting dialogue as a goal. They shared key features for establishing social presence, including social networking activities in the first two weeks and active student lounge-like social forums. These are expected features in all VHS classes. Each of these classes also included discussion prompts that were open-ended and explicitly required students to post responses to the comments of their peers: “Post your initial response and reply to at least one or two of your peers.” Such prompts were typical in many VHS classes I reviewed. However, as the findings in an initial comparative survey [32] indicated, these factors alone did not promote extended collaborative dialogue. The analysis of these higher collaboration classes provided an opportunity to examine which design differences most effectively supported collaborative dialogue among all the classes examined. Here I describe differences in discussion assignment design that may have affected the level of collaborative online dialogue. I begin with the general framework for discussions evident in the social studies class, the class that exhibited the highest concentration of collaborative dialogue with 26 events in just four weeks.

This class was constructed around a United Nations simulation that took place during weeks 11 to 14 of the class. Activities in previous weeks were designed in order to build to the level of collaboration required for the final role-play. For example, during the second and third week of class, students were asked to partner and introduce one another to the group in an activity called “Meet the Delegates.”  The theme of the activity adds a United Nations frame to a common ice breaking task.

During Week 3, a friendly competition called “Who Is That Man” was conducted. A black-and-white image of an unremarkable middle-aged gentleman in a suit at a desk is pictured with the directive,

I would like all students to post a comment as to who you might think this man is. He has something to do with the United Nations. You can post questions and I will answer them. Check the postings of other students and we will see who is the first person to figure out who he is.

The design of this activity fostered interdependence, another facet of getting to know one another, and provided scaffolding for more extended collaborative interactivity later on.

In the fourth week, students were assigned their first collaborative discussion, “What is the United Nations?” Each discussant was required to post four comments to the dialogue on four different days of the week to earn credit. The stated goals of this first discussion were: 1) to demonstrate that students understood the information contained in the introductory reading about the UN; and 2) that students continued to get to know one another by exchanging questions and ideas. The seed post for this forum was:

OK everyone now that you are all “experts” on how the UN is organized let’s see what you think. In this discussion you can post answers to questions or ask questions. I would like to stress that for classroom discussion you should * not * only ask questions. Posting questions and not doing any more than that is easy. I want to know your ideas as well. I’ll get things started with this: What do you think are positive aspects or strengths of the way the Security Council is organized?

Eighteen threads were initiated under this assignment, most of them starting with the question the instructor posed, but some shifting to other aspects of the reading. Seven of these (39%) moved to thread depths of more than four, with students building on the comments of others, asking further questions, or adding their views in the context of previous statements. The facilitative effects of both the grading structure for the week and the inherent interdependency of the activity appear to have been effective, along with the explicit preparation for working in an interdependent way.

In contrast to this open-ended, but interdependent, collaborative discussion design, during weeks eight, nine and ten the discussions were highly structured, yet retained emphasis on collaboration. Each student was assigned the role of discussion leader or expert for a nation-based discussion on such topics as the economy, people, geography, or government of the assigned nation. First, the leader posted a report s/he wrote about the topic. The rest of the class was expected to “critique the reports given on countries as to their usefulness for this course.” The instructor included a standard caveat with these assignments:

I must stress that those students not reporting on a country will be graded on the quality and frequency of their participation in the weekly discussions on the countries assigned last week. You must comment frequently and in a relevant manner on each report…I will be looking for a relevant comment/question from each student on each report listed. You must also check back to see what the reporters have responded to your comment…The person who reported on this country and topic is responsible for facilitating discussion and answering questions students may have.

Each of these assignment and discussion documents was augmented with images of formal and informal gatherings of delegates at meetings in the United Nations, reminding learners of the context about which they were learning and that they would soon be simulating themselves. These highly structured and task-oriented discussions were closely linked to weekly grades, helping to ensure students would jump in and offer their comments “frequently” and “in a relevant manner” as stated above.

Taken together, supporting activities such as “Meet the Delegates” (supporting community building) and “Who Is That Man? (fostering interdependence) provided a foundation for the first open-ended collaborative discussion, “What is the United Nations?” and the similarly collaborative nation report discussions. These nation-based discussions were the final pre-cursor to the United Nations simulation that followed. They included student moderators and roles for participants that generated and sustained particularly high collaborative dialogue.

The overall design of the class reflected a cooperative learning approach [18, 37] successfully enhanced by the affordances of an asynchronous learning network. Previous research on such an approach for online learning has confirmed its effectiveness. In Aviv’s evaluation study, cooperation was also structured so that learners succeeded “if and only if all learners succeed(ed), so that they all must coordinate their efforts” (p. 58). Effective methods that were successful in the case evaluated by Aviv were to assign a deliverable produced by the whole group (in the UN class, such a task was the UN simulation itself); to design task interdependence or a division of labor (as in the nation reports and the more elementary “Who Is This Man” warm-up activity); to build in resource interdependence (for example, the nation reports and follow-up discussions prepared students to participate knowledgably in the final simulation); and to reward interdependence (for example, individual accountability was emphasized from the beginning of the UN class with the email “contract” and grading was explicitly tied to collaborative participation).

In the less successful classes, less structure was evident. One of the 22 higher collaboration classes identified in the initial survey focused on controversial topics related to science. However, it only achieved irregular collaborative dialogue. An example of a prompt in that class was,

Think about the following questions and make a comment to the discussion called “Environmental Problems.” What do you think is Earth’s most serious environmental problem? What caused, or causes, the problem? How do you think this problem should be dealt with?

The limitation evident in this example is that there are no instructions about how (or whether) to build on one another’s thinking , nor is there a generic “respond to the comments of your peers” statement. Collaboration is neither emphasized nor specifically graded (and, as indicated by the wording of the question itself, may not have been the intention in this case). Both explicit teaching about how to post collaboratively and direct linkages to expectations and grading appear to be essential if collaborative dialogue is a goal.

The language arts class and science class examined for this study offered similarly open-ended discussion seed posts with positive results. However, as with the social studies class, the instructors of the other two classes made participation more heavily weighted in the final grade and pro-actively orchestrated dialogue by discussion design (science) or provided individualized feedback on participation (language arts). For example, in the science class’s Week 4 controversy of the week (COW) discussion on “Food Irradiation” prompted the most collaborative events of all the discussions assigned in the class: six in all involving 41 postings. A total of 92 postings, 51 not part of collaborative events, were submitted under the topic. Thus, collaborative dialogue engaged just under half (45%) of the attention given to this activity.

The question for dialogue was, “Should the U.S. allow greater use of irradiation to decrease food contamination?” Seventeen background facts were provided in a list form, such as, “Washing food does not remove all bacteria,” “Irradiated food is given to medical patients with weakened immune systems,” and “At least 40 developed countries use food irradiation although it is not extensively used in the United States today.” The expectations for COW discussions beyond the basic discussion rubric were delineated on every COW assignment document, including the following procedural expectations:

Remember, to complete this assignment you:

• should comment on the topic based upon what you know now.
• may respond to others at any time.
• should do some research on your own.
• check on the “facts” (C an you confirm or dispute information supplied?).
• expand the “facts” (for example: How many people die each year due to food poisoning?
• add to our knowledge (D id you find some relevant information that would help in our discussion?).
• identify websites with related information.
• should return to this thread a couple times before weeks’ end to add comments.

Assessment: Total Points Available=40
10 points for posting your initial comments in the CR (by Thursday)
10 points for adding something to the discussion…knowledge, a website, etc. (by Friday)
10 points for responding to at least two others (by Sunday)
10 points for your final thoughts about the topic (Monday or Tuesday)
Before making your final comment you should read all other comments posted. Your final comments may include something you learned, something that surprised you, a summary, a shared thought, what you consider important about the topic, what you still wonder about, etc. (Italics added.)

This discussion question did not have a right answer or just a few possible avenues of exploration; the net of possible responses was cast widely. Evaluation in the class emphasized participation in such discussions, and collaboration was explicitly valued in the individual activity directive cited above. In particular, the instructions for the “final thoughts” posting involved the expectation postings would reflect that students had considered everything previously posted to the dialogue. By suggesting they highlight new learning, surprises, shared thoughts, or what remains less understood, the instructor re-opened inquiry through the final days of dialogue, holding out the possibility that additional extended discussion could still take place.

In the language arts class, the structure was more open-ended; yet, the private feedback, as was shown earlier, was more specific. For example, the discussion forum that prompted the most collaborative dialogue in that class was titled “Need Love and Gift Love in Anthony and Cleopatra.” It stated,

…This week, we’re trying out a different theory, courtesy of Narnia Chronicles author, C.S. Lewis. What did you think of his theory and description of love and its four types? What impressed you or seems worth discussing further? Are there connections to this theory in the play we’re reading right now (Shakespeare’s Anthony and Cleopatra)?

Post your initial comments to this discussion starter early in the week. Stop back later to read others’ ideas, and post a follow-up that helps us unpack our ideas further.

Assessment: Posting at least two well-developed, thoughtful comments that
1) Show your understanding of C.S. Lewis’ theory in a timely manner and
2) Demonstrate active listening and interaction with your classmates will earn you credit for this required assignment. (Italics added.)

The discussants participated in a whole class discussion for this assignment. There were 38 postings, including one facilitating post (Conceptual Facilitator Identifies Direction according to Collison et al.) from the instructor. Twenty-eight of the posts (74%) were involved in five collaborative events. Nine of the fourteen students (64%) fulfilled the assignment, with eight (57%) students posting more than twice, beyond expectations. In the most sustained dialogue, the thread depth was five and the thread length was nine.

A different version of this assignment appeared in the equivalent class, examined for the earlier comparison study of higher and lower collaboration sessions of the same class taught by different instructors. In that class, the parallel activity was shifted to a structured question-and-answer format. Students were required to post initial thoughts and then respond to two peer submissions. The task stated:

**Note: visit this thread a couple times during week six.
Early in the week, post your initial responses to the web site reading, and then later in the week, stop back to respond to a couple of your group members’ ideas. You need to post at least three well-developed comments for your participation grade.

In your initial post, answer at least three of the questions from group A and answer the question in group B. Please cut and paste these questions in to your comment so we know what you’re responding to.
(Choose three)

• If there are four types of love (Affection, Friendship, Eros, and Charity), how can need and gift love be considered broad categories to sort the four types?
• Is an understanding of God and religion necessary to understand Lewis’ concepts? Why or why not?
• What is the difference between Need-Love and Gift-Love? Is one better than the other? Is this a helpful distinction?
• In Lewis’s Affection an adequate characterization of parent-child love? Is that the only thing it is good for?
• Is Lewis’s definition of Friendship adequate? Does it describe your own?
• How, exactly, is Eros different from sexuality?
• Describe what it would look like for a love to be transformed by charity. (This question is mandatory.)
• Choose a character from the play and explain the type(s) of love that they seem to be engaging in. Choose specific, textual examples to support your ideas. (Italics added.)

This class was divided into two small group discussions for this assignment, even though the class contained only seven active students. The full class participated in their assigned groups. Each posted at least twice with a total of 26 postings, including one probing question for clarification by the instructor. All but one of the students fulfilled the assignment with at least three comments. There was one collaborative event, though it was weak in that it contained just four posts, with the fourth linked post being a probe from the instructor that remained unanswered. Thread depth in all other cases was two: initiating posts with single responses.

It made sense that the conversation might be more stilted based on the version of the assignment provided. Students offered many intriguing ideas and raised a variety of topics. However, the assignment itself did not encourage conversation. Evaluation and feedback did not regularly focus on collaborative dialogue or co-construction of understanding in this class, as it did in its higher collaboration counterpart. The dialogue remained limited to the level of brainstorming thoughts and reactions. That is, in essence, what the instructor sought from the learning activity.

In contrast, the directions in the assignment version in the session with higher collaborative dialogue cast a wide net that encouraged extended discussion by asking, “What impressed you or seems worth discussing further?” Additionally, the instructor asked students to “read others’ ideas, and post a follow-up that helps us unpack our ideas further.” This instructor also clearly linked assessment to collaborative dialogue when he stated, “Demonstrate active listening and interaction with your classmates will earn you credit for this required assignment.” (Italics added.) Feedback then directly supported these interactions with explicit teaching of how to participate in a co-constructive manner.

One of the science classes among the initial 22 classes selected for some evidence of collaboration dealt with ethical issues related to biology. Discussion focused on such controversial topics as organ donation, genetic screening, and cloning. The potential for engaging in collaborative dialogue about such content was high. Student postings could have demonstrated understanding of the facts, used evidence to support opinions, and engaged with the multiple perspectives presented by peers. Emphasis on interaction would have necessitated that they linked their comments by actively listening and thus sustaining conversation rather than just offering individual points of view. Yet the course design in this science class did not directly support that collaborative type of engagement. Typically, discussion prompts read as this one on euthanasia did:

Using the article you found write a short summary of the article and post your summary. To post your summary in the Course Room…
Click the “Comment” button above to enter your question.
Fill in the subject field with a descriptive title that briefly summarizes your posting.
Enter your comments in the large text box below that. Please use as much space as you need—don’t be shy.
When complete, click the “Save & Close’ button, and you’ll be returned to this document.
Use the “Discussions” button to see you newly posted question listed in the view with the other subthreads. Yours should appear just below the subject of this document.
Click back later today or tomorrow for your answer. There will be a twisty beside your original comment indicating that a comment has been made to it.

These directions were repeated regularly in this class. Not only did they not emphasize interaction and dialogue, they focused learners on looking only at their own postings for comments rather than viewing the discussion as a whole for a more collaborative learning experience. In contrast, the set of directives used in the science class selected for its high level of collaborative dialogue, shifted the focus to the collaborative effort of the group. There, directions for “Controversy of the Week” discussions (described earlier in this section) emphasized dialogue and focused on how to co-construct understanding of content by way of peer interactivity. The directions for posting encouraged collaboration.

As in the comparison between the two literature class discussions above, this comparable pair of directives for dialogue about science content contained inherent differences that appeared to affect the amount of collaborative dialogue as one might have expected. The euthanasia topic produced no exchanges with more than two or, at the most, three linked comments. The “Controversy of the Week” discussion activity design from the higher collaboration science class produced 21 of the 29 collaborative events in that classroom, including those contained in the discussion of the question “Should the U.S. allow greater use of irradiation to decrease food contamination?” that was described earlier. Directions for individual dialogues worked in concert with evaluation rubrics and grading to make a difference in the level of collaborative dialogue.

A theme that emerges in all three cases is the importance of explicit teaching about how to engage in collaborative dialogue. Both of these comparisons illustrate how collaboration needed to be clearly linked to evaluation and explicitly taught, either directly within the design of dialogue activities, as we saw here, or delivered individually in private feedback.

Thinking out loud with peers, quick exchanges, brainstorming, and the use of shared visual artifacts are automatic in a face-to-face environment. Online, these collaborative activities take place predominately with text [38]. Therefore, new skills must be developed, taught, and learned to achieve effective collaboration in this literary context [2, 39]. With the support of explicit teaching and direct evaluation of collaborative efforts, collaborative dialogue improved.

D. Comparison of Participant Moves

One final factor was considered to fully understand the elements that influenced collaborative dialogue and to test the validity of the conclusions that emerged from this analysis of the higher collaboration classes. I needed to examine whether patterns existed in the student-to-student interactions themselves that further promoted or discouraged interaction.To find out, I compared patterns across student postings in all three classes. I found participants’ postings did not appear to markedly affect the level of collaborative dialogue in VHS classes.

The participants in these classes were a mix of repeaters and first-time online course takers. Elements other than facility with conversing online with text were therefore considered in order to test the validity of the emerging interpretative model. I sought out patterns of learners engaging one another spontaneously with questions, name-referencing, dominating participants or any other learning community-building or reducing behaviors that might be present. Ascertaining participant involvement in enhancing collaborative dialogue would influence the findings as well as the final interpretative model.

However, no patterns in participant activity appeared to enhance or discourage dialogue in the higher collaboration VHS classes. Students used such strategies as questioning one another and name-referencing relatively infrequently in these classes; and, when they did, there did not seem to be a consistent effect.

VI. DISCUSSION

A. A Balanced Interplay among Key Heuristics

Teaching presence, or instructor leadership, that leaves the responsibility for learning to the learners may be easily recognizable when we experience the challenge of it, yet the approach is somewhat unique in every case, depending on content, activities, and the individual learners in a given class.

In these three cases, a foundation of social community, or a social presence, provided scaffolding for fostering cognitive presence among the learners. Three unique approaches to instructional leadership, or teaching presence emerged. Common elements were involved in all three approaches. In the language arts case, cognitive presence was heavily scaffolded by individualized regular feedback that explicitly showed learners how to collaborate. Early facilitative interventions directly into student dialogue modeled collaborative dialogue. Activity designs were inherently collaborative. Discussion prompts clearly promoted exchange rather than required homework-style individual statements of views and opinions. Evaluation was directly tied to successfully participating in collaborative dialogue.

In the science case, extensive feedback was not used. Grading was numerical, but tied to successfully contributing to collaborative dialogue. Explicit teaching of how to collaborate was woven into the “controversy of the week” type discussion activity designs and rubrics in great detail. Discussion design features thus provided scaffolding upon which students could build dialogue.

In the social studies case, where collaborative dialogue was most densely concentrated during the four weeks in which discussion activities were assigned, the key was again instructional design. Learners explicitly agreed to a grading policy that linked collaboration (not just participation) with grading. Interactive, ice-breaking activities built a collaborative skill base in the early weeks. Structuring specific roles and activities for learners extended the length and depth of collaboration. The structure of the discussions required that students both ask and answer questions of one another. Students moderated their own discussions. A direct connection between that type of engagement and the learners’ grades for the week assured collaborative dialogue would take place.

These three exemplary classes revealed a balanced interplay among the key heuristics for fostering collaborative dialogue. They were 1) social community, or social presence; 2) explicit teaching of how to collaborate; 3) collaborative activity designs; and 4) clear linkages between collaborative dialogue and evaluation. In each of the three cases, a socially-bonded community, design features such as rubrics that explicitly promoted collaborative dialogue, direct linkages in feedback or evaluation to meeting these expectations, and direct teaching of how to collaboratively engage in dialogue around content, were essential to enhance collaborative dialogue.

In comparable classes examined in an earlier study [32], collaborative dialogue was not achieved when instructors did not provide this interplay of stated collaborative expectations, related evaluation, and explicit scaffolding. In those classes, social presence was evident, but it was not enough. This finding echoes findings from higher education [40]. In the cases examined for this study, it was not just any rubric, discussion activity, or participation grade that worked to promote collaborative dialogue. Instead, the quality of these elements made a difference. They needed to be precise, targeted, and explicitly pointed at collaborative interaction to ensure success. To a lesser extent, these elements were also present in the comparison classes, where collaborative dialogue was less. Therefore, it appears that course design elements that emphasize collaboration (“post responses to the initial comments of one or two peers”) must be accompanied by additional supporting elements to be effective. Other research suggests that simply tying grades to number of postings will also backfire [41]. A discussion rubric supported by explicit teaching of how to engage in collaborative dialogue, all clearly linked to evaluation worked in concert to shift interactivity from a regular thread depth of two to more extensively linked postings exhibiting collaborative dialogue focused on learning goals.

B. Next Steps

The examination of the interplay of instructor moves, course design, and participant moves in this study provided a useful research tool for understanding factors that likely promote collaborative dialogue online. However, there are numerous directions for future research that further extend our knowledge about collaborative dialogue and quality online instruction:

Test the Interpretative Model within the VHS or other secondary online classes.
VHS could train interested instructors in course design features, using interventions that promote dialogue, and setting expectations for participation that prompt extended, substantive exchange. Researchers can use an experimental design to perform a comparative analysis in order to ascertain whether collaborative dialogue and learning is enhanced and the findings from this study are replicated.

Test the Interpretative Model beyond secondary classes.
Will the findings of this study be replicable in other educational contexts? As a secondary school program, grades in VHS classes affect learners’ class ranking and college acceptance. Thus, evaluation is a strong extrinsic motivator. Will the other significant factors – collaborative activities that involve interdependence to succeed, clear directives to collaborate, and explicit feedback regarding learners’ effectiveness at collaborating with their peers – make a difference in contexts where evaluation is not a driving force, such as in a pass/fail environment or a program for lifelong learning? How much encouragement to interact is needed in graduate classes involving highly motivated learners who perhaps paid the tuition themselves, though a high GPA isn’t as important? The interpretative model would provide a basis for examining the impact of these factors in other settings.

Test Focusing and Deepening Interventions.
This study found some evidence of instructors using focusing and deepening interventions into learner dialogue as described in Collison et al. [21]. More research is needed where instructors are first trained in the method and then resulting collaborative dialogue is compared with previous sessions of the same class or discussion.

Perform Social Network Analysis.
In this study, the analysis of participant moves was conducted with the goal of eliminating the possibility that higher or lower collaboration was not a clear, direct result of the actions of one or more dominant or particularly engaging participants. A rigorous analysis of the existing social networks in these classes (for example, who addresses whom and how often) was not conducted.

In my analysis, I found it intriguing that the science class examined had an equal number of males and females, and yet neither gender was clearly dominant. Gender is an important aspect of social network analysis. Some research has suggested potential gender issues [22] in online discussion activities. Are gender differences less evident in secondary level online classrooms than they appear to be in undergraduate and graduate contexts? Or did the structure of the discussions in this class overcome commonly observed gender inequalities?  The methodology employed in this study could be productively applied to these research questions.

Extend the Analysis of Learning.
In this study, I examined the quality of dialogue content for the strength of its sustained connection to stated learning goals, without delving deeply into an analysis of levels of higher-order thinking reflected in collaborative events. My judgment was that these data were too preliminary for such an investment of time and effort. Future research is needed to examine evidence of higher-order thinking found in sustained collaborative dialogue.

Extend Research to Other Disciplines.
Evidence of cognitive engagement in online classes is recognizable in learners’ cognitive presence in the class, namely in their comments posted to their instructor and peers. Collaborative dialogue is one type of activity that facilitates cognitive presence. Classes considered in this study include language arts, social studies, and science classes. What about mathematics, music, technology (programming, graphic design, or filmmaking, etc.), or foreign language classes? How might classes in other disciplines differ in their quantity and quality of interactive engagement? Instructor moves, collaborative activity designs, and inquiry-based participation in those classes have yet to be explored. Further research is needed to determine if these factors or additional factors are effective in promoting online collaborative dialogue in other curricular areas and if there are curricular specific strategies.

Test Asynchronous Dialogue Combined with Synchronous Dialogue Activities.
It is a policy of the VHS to avoid synchronous technologies in VHS classes, although administrators acknowledge instant messaging outside of the courseware platform is inevitable and widespread. The reasoning behind denying access to synchronous activities within VHS classes is that student participants span every time zone, making the scheduling of synchronous class meetings next to impossible. Primarily, VHS is concerned about maintaining equity among students who only have access to computers at school and those who can also log on at home.

Another issue unique to the secondary school context is the considerable level of overt control over learners’ daily schedules. For example, a student cannot choose to skip a math class to attend a synchronous chat at the library computer, nor can a young learner from a distant nation easily join his predominately U.S. peers for a mid-day synchronous chat when the time in her country is 3:00 am. While a certain type of equity is gained with the policy, the potential for more collaborative learning opportunities is also lost.

Might there be effective approaches to weaving synchronous conversations with transcripts and asynchronous follow-up dialogue to deepen the cognitive presence and higher-order thinking reflected in class discussions?  That way, all students can join in the conversation. While VHS and other international online programs may have greater difficulty integrating synchronous class meetings, state-based or local online programs may be able to accomplish this more easily. In those contexts, it would be valuable to examine whether particular strategies used during synchronous dialogue supports or detracts from collaborative content-based dialogue.

VII. CONCLUSION

The importance of interaction for learning is well established [11, 14, 42, 43]. Text-based, asynchronous online delivery requires that instructional design and teacher behavior shift to fit the new medium [44]. When learners can’t see one another and may never meet each other or the instructor, moving a group from social communication and direct teacher-student exchanges to a more reflective level of peer interaction requires new approaches. If online learning is to serve as a major educational medium in the future, understanding the pedagogical strategies that foster content-based dialogue online is essential.

Kozma and Zucker [5] reviewed a variety of online secondary programs and found that even the exemplary Virtual High School lacked the level of student-to-student interaction sought by its instructors to support higher-order critical thinking in online dialogue. In their evaluation, Zucker and his associates at SRI found that fostering dialogue was one of the most significant challenges to secondary level online instruction. Parallel challenges have been documented in higher education [40]. They indicated that research on what approaches best support extended student dialogue was needed. The systematic analysis of online learner exchanges and instructor moves conducted in this study addressed that concern and added to recent research that analyzes discourse transcripts from online classes in higher education [16, 45, 46, 47, 48].

The findings from this project begin to reveal elements that support online collaborative dialogue on the secondary level. This study takes a step toward assessing the quality of online learning in achieving content-based collaborative dialogue. It draws insight directly from work of VHS practitioners, with the explicit intent of thereby enhancing the credibility and usefulness of the research for online instructors seeking realistic approaches to enhancing collaborative dialogue in their online classes. The methodology and strategies that were derived in this study can set the stage for the next level of work in improving online instruction and the professional development for secondary online instructors.

This paper mined three VHS classes for teaching elements that most likely influenced the nature of the dialogues among students. In each case, specific approaches to discussion facilitation, activity designs and evaluation rubrics were uncovered that promoted substantive, collaborative dialogue. Yet the interplay among those elements varied widely. Examining all three illuminated a practical model of heuristics useful to practitioners on both the secondary level and possibly also in higher education.

VIII. ABOUT THE AUTHOR

Dr. Haavind is an Assistant Professor at the School of Education, Lesley University. She co-authored one of the pioneering books in the field of online learning: Facilitating Online Learning: Effective Strategies for Moderators by George Collison, Bonnie Elbaum, Sarah Haavind & Robert Tinker [21]. Her doctoral thesis, Tapping Online Dialogue for Learning: A Grounded Theory Approach to Identifying Key Heuristics that Promote Collaborative Dialogue Among Secondary Online Learners[49] was a study of online teacher practice and instructional design in Virtual High School (www.govhs.org) classrooms where asynchronous collaborative dialogue is a core learning activity. Her professional interests include online instructional design and teaching practices that deepen and enrich learning.

IX. REFERENCES

1. Haavind, S. Tapping Online Dialogue for Learning: A Grounded Theory Approach to Identifying Key Heuristics that Promote Collaborative Dialogue Among Secondary Online Learners. Cambridge, MA, Harvard University: Harvard Graduate School of Education,