Keeping Online Asynchronous Discussions on Topic

Download PDF version:

ABSTRACT
The purpose of this research was to identify various techniques recommended and used by online instructors for keeping online learners on topic during asynchronous discussion and to identify what factors affected selection.  A thirty-seven item online questionnaire was developed and completed by 135 online instructors subscribing to an international distance education listserv.  Thirteen techniques for keeping online asynchronous learners on topic were rated using a six-point Likert scale. The results of the study showed that online instructors rated the following as the top four techniques for keeping asynchronous online discussion on topic: 1) Carefully design questions that specifically elicit on-topic discussion, 2) provide guidelines to help online learners prepare on-topic responses, 3) reword the original question when responses are going in the wrong direction and 4) provide discussion summary on a regular basis.  Experience, training and differences between what respondents recommended and used to keep online asynchronous discussion on topic produced statistically significant results at the 0.05 level.

 KEYWORDS
Asynchronous discussion strategies, Managing online discussion, Exploratory study, Self-administered online questionnaire

I. INTRODUCTION

Person-to-person interaction plays an important part in learning online both within the private and public sector [1], [2].  “Research has documented, over and over, when participants make the learning their own, when they get to talk about it their way, without being manipulated and controlled, learning increases.” [3 (p. 174)]. When learners make the learning their own they may have a tendency to lose focus on the original intent of the instructor or the course objectives.  Cantor [4] suggests that the instructor has a responsibility to keep the discussion on topic.  She finds learners expect the instructor to keep discussions on the subject and sees learners maintaining interest in discussion that has direction. 

Finding the fine balance between keeping on topic and allowing learners to talk about learning in their way is a challenge.  This is the case in face-to-face instruction and an even greater challenge when learners are at a distance and the instructor is not physically present.  When students go off-topic in a traditional classroom the instructor can immediately refocus the discussion.  In an asynchronous online classroom where interaction occurs in delayed time the instructor is not immediately available to refocus the discussion. The discussions can lose their original focus and digress for long periods of time.  Romiszowski [5] concluded after reviewing the literature and his own experiences that it is more difficult to bring participants back on track in computer-mediated communication (CMC) seminars than the conventional meeting.

The purpose of this exploratory research was to identify various techniques recommended and used by online instructors for keeping online learners on topic during asynchronous discussion and to identify what factors affected selections.  The research questions for this study were:

1. Which techniques do online instructors recommend to keep online learners on topic during asynchronous communication?

2. Which techniques do online instructors use to keep online learners on topic during asynchronous communication?

3. What are the relationships between online instructor characteristics (gender, experience, attitude, administrative support, training, and expectation) and the extent to which they recommend various techniques for keeping online learners on topic during asynchronous communication?

4. What are the relationships between online instructor characteristics (gender, experience, attitude, administrative support, training, and expectation) and the extent to which they use various techniques for keeping online learners on topic during asynchronous communication?

5. Is there a significant difference between the degree to which online instructors recommend and use various techniques for keeping online learners on topic?

A. Review of the Literature
The online instruction literature was reviewed and no data-based studies were found which investigated techniques for keeping online learners on topic.  However there are conceptual discussion articles that address the topic of improving communication and interaction in asynchronous learning.  Winiecki [6] argues that there are difficulties found with asynchronous learning networks interaction and sees the possibility of losing track of the threaded discussions as a major issue. Several other authors have also raised the issue of the importance of interaction in distance learning [7], [3], [4].

According to Moore and Kearsley [8] there are three types of interaction in distance learning: learner-content interaction, learner-instructor interaction, and learner-learner interaction.  It is obvious that human two-way interaction occurs between the instructor and learners and between learner and learners.  What is less obvious is how interaction occurs between learner and content.  Traditionally text has been the main medium used for learner-content interaction.  In the context of computer-mediated communication the content has been presented using computer-based training (CBT).  There isn't normally any two-way human interaction with text or CBT.  Armstrong [9] reports that web-based CBT can be greatly enhanced by having the option of interacting with a real live person in chat rooms or through asynchronous communication.  It would seem obvious that without some kind of human interaction, after there has been learner-content interaction, learning could be limited.  Even when there is interaction between learner and instructor and learner and learner problems can still exist.

Gilbert and Moore [10] suggest that interactivity online has been a problem in distance education at two levels (social and instructional).  As it relates directly to instruction, they conclude that there needs to be "fine tuning" of the control of interaction within the instructional process.  The control of interaction is more broadly supported from Ragan's [11] research that developed a set of principles and practices with faculty from three universities who were involved in designing distance education programs.  With regard to interaction he concluded that there is a need for frequent and meaningful interactions among the learners, with the instructional materials, and between the learner and the instructor.

The online instructor is key to organizing interaction and Hiltz [7] suggests from her research that having a responsive moderator is key.  The instructor does not necessarily need to be the moderator and Driscoll [12] suggests that participants can be assigned the task.  Driscoll suggests that the instructor weigh the benefits and risks of a moderated listserv.

Hughes and Hewson conclude that “the specifics of online learning interaction is an issue often neglected in the rush to embrace the internet for learning, and therefore, improving upon currently available communication tools is imperative if the online learning experience is to become a serious proposition.” [13 (p. 54)]  Romiszowski [5] concludes that most participants using CMC respond to messages as they read them and while doing so often extend the discussion to other interesting topics leading to digression from the original task.  He recommends careful attention to structure and control in keeping participants on task.  Hence the purpose of this research.

A series of independent variables was identified for the study based on a review of the related literature.  Egan et al. [14] reported that release time for planning was a key indicator of employer support.  Dillon and Walsh [15] suggested that recognition and rewarding of distance teaching as a scholarly activity are also important incentives.  Several authors [16], [17] [18], [19] support the notion that it is essential that organizations provide training related to online instructing.

II. METHOD

An online questionnaire was developed to collect information and opinions from online instructors who have had experience using asynchronous discussion with online learners.

A. Study Participants
A purposive study sample was established from an international distance education listserv.  Tracking the postings to the listserv allowed the researcher to conclude that the membership included experienced online instructors.  A request for study participation was sent to the listserv and willing participants were linked to the online questionnaire. 

B. Data Collection Instrument
A 37-item online questionnaire was developed to measure the level of recommendation and use of specific techniques for keeping online learners on topic. Reaction rating scales were presented with ratings from 1 = very low recommendation to 6 = very high recommendation and 1 = very low use to 6 = very high use. DeVellis [20] suggests that either an odd or even number of choices can be used for the response scale depending upon the phenomenon being investigated and the goals of the investigator.  A six-point Likert scale was chosen for this study to force respondents to take a position as to whether they would recommend and/or use a specific technique.  Content validity was established by the use of a panel of online instructors not part of the study.  The remaining eleven items collected information related to the online instructor's background and work environment characteristics.

C. Data Collection Design and Procedures
An e-mail message was posted to an international distance education listserv requesting the participation of online instructors who used asynchronous discussion.  The e-mail explained the purpose of the study and insured confidentiality and anonymity.  Qualified non-participants were asked to send a pre-coded message, embedded in the original request for participation, indicating they would not participate in the study.  The baseline sample size used to establish the response rate was calculated using the study participants and online instructors who did not wish to participate in the research.  In this case the main assumption for deriving a reasonably accurate sample size is that listserv members usually respond to messages posted. 

III. ANALYSIS AND RESULTS

Descriptive statistics were used to summarize the information collected and multivariate analysis was used to explore relationships between variables. 

A. Descriptive Statistics
The questionnaire was the only means of data collection for the study.  The purposive sample consisted of 135 online instructors who had experience using asynchronous discussion in their online teaching.  A response rate of 93% (135/145) was achieved using the total number of possible respondents to include those who chose not respond and had indicated their reluctance to participate in an e-mail message.  In retrospect the response rate cannot be verified because the listserv had a membership of approximately 2,000 and there was no way of knowing the profile of the listserv membership.  The results of this study are only generalizable to the survey participants and should not be construed as representing opinions and beliefs of all online instructors.

The study population consisted of seventy-two females and sixty-three males with 73% of all respondents employed by nonprofit organizations. Approximately 43% had taught five or more courses using asynchronous communication methods during the past three years and 96% responded that they would choose to teach more online courses using asynchronous communication discussions if given the opportunity.  Sixty-eight percent have participated in formal training for online instructing.  Respondents indicated that their employers supported online instructors by providing training related to online instructing (65%), recognition as a scholarly activity (48%), release time for planning (46%), and additional salary (32%).

Table 1 presents that mean rank order of techniques online instructors recommend for keeping online learners on topic during asynchronous communication. Table 2 shows the mean rank order of techniques online instructors actually use. Comparing both tables it can be seen that the top 4 techniques were ranked the same: 1) carefully designing good questions, 2) providing guidelines for learners to use when preparing their responses, 3) rewording the question when discussions go off topic and (4) providing discussion summaries. It should be noted that the techniques that were ranked 1st and 2nd were activities that an instructor can do before the start of any session

Although not as highly rated as the top 4 techniques items 5 and 6 received ratings to warrant consideration for possible use. Providing an alternative location (café) for off-topic discussions reinforces the all-important concept of providing alternatives to promote learning [4], [21]. 

Table 1. Means and Standard Deviations of What Online Instructors Recommend for Keeping Online Learners on Topic, by Descending Mean Rank Order (N = 135).

Note: A 6-point Likert scale was used to measure recommendation level with 1 = very low recommendation to 6 = very high recommendation. The following techniques were presented for rating:

1. Carefully design questions that specifically elicit on-topic discussion.

2. Provide guidelines to help online learners prepare on-topic responses.

3. Formally present rules of conduct that eliminate off-topic comments.

4. Formally state the expectation that online discussions stay on topic.

5. Provide alternative locations (cafe) for off-topic discussions.

6. Screen all postings and route off-topic posts to alternative locations with an explanation to the submitter.

7. Include a reminder that responses stay on topic with all posted questions.

8. Reword the original question when responses are going in the wrong direction.

9. Provide discussion summary on a regular basis.

10. Privately reprimand and provide corrective suggestions to learners who submit off-topic comments.

11. Provide a reward for keeping on topic.

12. Provide a grade for keeping on topic.

13. Expel offenders from the discussion after "x" number of off-topic submissions.

Table 2. Means and Standard Deviations of What Online Instructors Use for Keeping Online Learners on Topic, by Descending Mean Rank Order (N = 135).

Note: A 6-point Likert scale was used to measure use level with 1 = very low use to 6 = very high use.  See note in Table 1 for questionnaire item wording.

B. Relationship Statistics
Relationships between online instructor characteristics and the extent to which they recommend and use various techniques for keeping online learners on topic during asynchronous communication are reported in the following section.  A series of independent variables were identified and data were collected in an attempt to explain why some techniques were recommended and used over others. 

When the results were controlled for gender (Table 3), a few of the most highly ranked techniques (items 1, 8, and 9) appeared to suggest that there were significant differences between how females and males rated the various techniques. However this table only presents seven out of twenty-six measures where there were differences at the 0.05 level and should not be interpreted as suggesting that there is a significant difference between the degree to which males and females recommend and use the various techniques.

Table 3. Means and Standard Deviations for Techniques for Keeping Online Asynchronous Discussion on Topic, by Gender (Significant Differences Only - Independent Samples t-Tests).

Note: All of the t-tests in the table are significant with a p <. 05 and df = 133. N = 72 females and 63 males. A 6-point Likert scale was used to measure recommendation level with 1 = very low recommendation to 6 = very high recommendation. A second 6-point Likert scale was used to measure use level with 1 = very low use to 6 = very high use. See note in Table 1 for questionnaire item wording.

Experience as represented by the number of courses taught using asynchronous communication discussion attempted to explain why some techniques were selected over others.  Table 4 shows the means and standard deviations of the techniques that were found to be significantly different when the respondent's experience was considered.  Experience ranged from zero to one hundred courses taught with 57% of the study sample having taught four or fewer courses during the past three years.  It is interesting to note that respondents with less experience assigned higher ratings than their more experienced counterparts.  Both groups agreed on the top two techniques for keeping online asynchronous discussion on topic.  The less experienced respondents rated the technique "stating the expectation" more highly than their more experienced counterparts.

Table 4. Means and Standard Deviations for Techniques for Keeping Online Asynchronous Discussion On Topic, 
by Experience (Significant Differences Only - Independent Samples t-Tests).

Note: All of the t-tests in the table are significant with a p <. 05 and df = 130. A 6-point Likert scale was used to measure recommendation level with 1 = very low recommendation to 6 = very high recommendation. A second 6-point Likert scale was used to measure use level with 1 = very low use to 6 = very high use. See note in Table 1 for questionnaire item wording. Online instructors who taught 0-4 courses = 75 and those who taught 5 or more courses = 57.

Whether respondents had formal training for online instructing didn't produce as many significant differences as had experience. When the results were controlled for participation in formal training for online instructors, approximately 27% were significantly different.  The statistically significant findings for formal training for online instructing are shown in Table 5.  Respondents that received formal training tended to assign a higher rating for those variables identified in Table 5 than those who did not receive formal training.  However, only seven of the twenty-six variables had significant differences.

Generally speaking formal training doesn't explain why certain techniques were selected.  However in a couple of instances formal training does seem to impact selection.  Three of the top 4 used techniques identified in Table 2 were statistically significant when it came to whether respondents participated in formal training.  Respondents with formal training rated the top choice technique higher than when all were reported in Table 2.

Table 5. Means and Standard Deviations for Techniques for Keeping Online Asynchronous Discussion On Topic, by Formal Training (Significant Differences Only - Independent Samples t-Tests).

Note: All of the t-tests in the table are significant with a p <. 05 and df = 130. A 6-point Likert scale was used to measure recommendation level with 1 = very low recommendation to 6 = very high recommendation. A second 6-point Likert scale was used to measure use level with 1 = very low use to 6 = very high use. See note in Table 1 for questionnaire item wording.

No statistically significant results were obtained when attitude, administrative support, and expectation were tested to explain why online instructors selected one technique for keeping online learners on topic over another.

The final research question explored whether there is a significant difference between the degree to which online instructors recommend and use various techniques for keeping online learners on topic.  Table 6 presents the results of a paired sample t-test between what was recommended and used by respondents.  It was found that there was a statistically significant difference between how respondents rated each technique as to whether he or she would recommend and use the technique.  Recommending a technique is often rated higher than a report of its use.  What actually happens in practice is different from what is recommended. 

 Table 6. Means and Standard Deviations for Techniques for Keeping Online Asynchronous Discussion On Topic, by Recommend and Use Ratings (Paired Samples t-Tests).

Note: All of the t-tests in the table are significant with a p <. 05 and df = 134. * N = 114 with df = 113 all others N = 135. A 6-point Likert scale was used to measure recommendation level with 1 = very low recommendation to 6 = very high recommendation. A second 6-point Likert scale was used to measure use level with 1 = very low use to 6 = very high use. See note in Table 1 for questionnaire item wording.

IV. DISCUSSION AND IMPLICATIONS FOR PRACTICE

The results of this study suggest that keeping online asynchronous discussion on topic can be best done by carefully designing good questions, providing guidelines for learners to use when preparing their responses, rewording the question when discussions go off topic, and by providing discussion summaries. Although the results are not generalizable beyond the study population there are several implications new and experienced online instructors might want to consider when keeping asynchronous discussion on topic.

The "carefully design questions that specifically elicit on-topic discussion" item (ranked # 1) is a practice that is supported in the literature [1], [23], [24], [25].  The technique of designing good questions is key to good teaching and learning. Taba [23] described questioning as the single most influential teaching act because of the ability of questions to influence the learning process. Therefore online instructors should be encouraged to develop questions that are clear, concise, and directly relate to the purpose of posing the question.

It is not surprising that the study respondents viewed designing good questions as being very important. Good questions promote active participation of the learner by stimulating various levels of thinking and/or by creating cognitive dissonance. To develop questions that stimulate learners to higher levels of cognitive thinking it is suggested that instructors use Bloom’s taxonomy or other schemes [22], [24], [26]. Developing questions that create cognitive dissonance attempts to pierce closed-minded views and provides the learner an opportunity for new insights [6]. Keeping the learner focused through the use of well-designed questions will assist learners in reaching the learning objective.

Providing guidelines to help online learners prepare on-topic responses was ranked 2nd and providing discussion summaries ranked 4th. Both of these techniques could be considered information organizers. Having information organizers fosters learning and retention. These techniques develop the lower level cognitive processes and set the stage for the higher level cognitive processes of analysis, synthesis, and evaluation. The importance placed on providing discussion summaries reminds online instructors that there is a need for continuous monitoring of discussions and more importantly comments by a summerizer.  The practice of summarizing is widely supported in the field of higher education and within business and industry [21], [27].

"Reword the original question when responses are going in the wrong direction” item was ranked 3rd for a both a technique recommended and used by online instructors in the study.  Although it does give good direction for practice when the discussion is going off topic this technique cannot be planned for ahead of time.  The need to reword questions may be reduced by attending to one of the previously highlighted techniques that encourages the careful design of questions in the first place.

When the results were controlled for experience, experienced online instructors rated techniques lower.  This might imply that experienced online instructors should be invited to any professional development activities that focus on the topic of this research.  However, the significant differences may also be due to the fact that inexperienced online instructors over estimate their recommendations and reported use of the various techniques.  This might suggest that further study is needed and a closer look at what experienced and inexperienced online instructors actually do to get the intended results of any asynchronous online program they are delivering.  A further implication is that both experienced and inexperienced online instructors saw benefit in many of the techniques and as a result some of these techniques could be included in training programs for new online instructors.

There is a significant difference between what respondents recommended and used to keep online learners on topic.  This suggests that there was a difference between the degree to which they recommended a technique and the degree to which they rated their use of the same technique.  A possible conclusion could be that online instructors may see that what they are actually doing could be improved upon if they tried an alternative technique.  Implications for practice would be to offer professional development activities that allow for an opportunity to develop skills that would allow them to move toward what they would recommend from what they actually use.

This exploratory study reinforced many of the principles and practices used in face-to-face classrooms to keep discussions on topic and should serve as a reminder that good instructional design is essential whether it is on line or face-to-face.

REFERENCES

1. Green, K. C., 1998 Campus Computing Survey Report, Encino, CA., Claremont Graduate University, Campus Computing Project, 1998.

2. Training Magazine’s Industry Report 1998, Training Magazine (Special Section), Vol. 35, No. 10, pp. 44-76, 1998.

3. Jensen, E., Brain-Based Learning, Del Mar, CA., Turning Point Publishing, 1996.

4. Cantor, J. A., Delivering Instruction to Adult Learners, Toronto, Wall & Emerson, 1992.

5. Romiszowski, A. J. Use of Hypermedia and Telecommunications for Case-Study Discussions in Distance Education. In Lockwood, F. (Ed.), Open and Distance Learning Today, New York, Routledge, pp. 164-172, 1995.

6. Winiecki, D. J., Keeping the Thread: Adapting Conversational Practice to Help Distance Students and Instructors Manage Discussions in an Asynchronous Learning Network, DEOSNEWS (Online serial: www.ed.psu.edu/ACSDE/) Vol. 9, No. 2, 1999.

7. Hiltz, S. R., The Virtual Classroom: Learning Without Limits Via Computer Networks, Norwood, N.J., Ablex Publishing Corporation, 1994.

8. Moore, M. G. and Kearsley, G., Distance Education: A Systems View, Boston, Wadsworth Publishing, 1996.

9. Armstrong, J., Tech Trends: Online Schools for Computer Skills, Training Magazine, Vol. 36, No. 1, pp. 34-37, 1999.

10. Gilbert L. and Moore, D. R., Building Interactivity in Web Courses: Tools for Social and Interaction, Educational Technology, Vol. 38, No. 3, May-June, 1998.

11. Ragan, L. C., Good Teaching is Good Teaching: An Emerging Set of Guiding Principles and Practices for the Design and Development of Distance Education, DEOSNEWS (Online serial: www.ed.psu.edu/ACSDE/), Vol. 8, NO. 12, 1998.

12. Driscoll, M., Web-Based Training: Tactics and Techniques for Designing Adult Learning, San Francisco, Jossey-Bass, 1998.

13. Hughes, C. and Hewson, L., Online Interactions: Developing a Neglected Aspect of the Virtual Classroom, Educational Technology, Vol. 38, No. 4, July-August, 1998.

14. Egan, M. W., Sebastian, J., Welch, M., Page, B., and Nkabinde, Z., Quality Television Instruction: Indepth Views of Administrators and Television Instructors, In Proceedings of the Eighth Annual Conference on Distance Teaching and Learning, Madison, WI, University of Wisconsin-Madison, pp. 40-44, 1992.

15. Dillon, C. L. and Walsh, S. M., Faculty: The Neglected Resource in Distance Education, The American Journal of Distance Education, Vol. 6, No. 3, pp. 5-21, 1992.

16. Lewis, R., Staff Development in Conventional Institutions Moving Toward Open Learning, In Latchem C. and Lockwood, F. (Eds.), Staff Development in Open and Flexible Learning, pp. 23-31, New York, Routledge, 1998.

17. Moore, M. G. and Thompson, M. M., The Effects of Distance Learning, ACSDE Research Monograph, The Pennsylvania State University, Rev. ed., Vol. 15, 1997.

18. Eastman, D. V., Alone But Together: Adult Distance Study Through Computer Conferencing, Cresskill, N.J., Hampton Press, 1995.

19. Houseman, J. G., Infusion, Not diffusion, a Strategy for Incorporating Information Technology Into Higher Education, Journal of Distance Education, Vol. 12, No. 1/2, pp.15-28, 1997.

20. DeVellis, R. F., Scale Development: Theory and Application, Newbury Park, CA, Sage Publications, 1991.

21. Powers, B., Instructor Excellence: Mastering the Delivery of Training, San Francisco, Jossey-Bass, 1992.

22. Arends, R.A., Learning to Teach. (3^rd ed.) New York, McGraw-Hill, 1994.

23. Taba, H., Teaching Strategies and Cognitive Functioning in Elementary School Children, San Francisco, San Francisco State College, 1966.

24. Saunders, R.E., Questioning Techniques In Gailbraith, M.W. (Ed.), Adult Learning Methods. (2^nd ed.). San Francisco, Jossey-Bass, pp. 187-196, 1998.

25. Wassermann, S., Introduction to Case Method Teaching: A Guide to the Galaxy, New York, Teachers College Press, 1994.

26. Rosmarin, A., Profile: The Art of Leading a Discussion, In Morganrouth-Gullette, Margaret (Ed.), On Teaching and Learning, Cambridge, MA, Harvard-Danforth Center for Teaching and Learning, 1985.

27. Reid, S., The Prentice Hall Guide to College Writers, 3^rd ed., 1992.