Identifying Student Attitudes and Learning Styles in Distance Education

Download PDF version:

David Therriault
Doctoral candidate in the Psychology Department at the University of Illinois at Chicago
Department of Psychology
University of Illinois at Chicago
1007 W. Harrison St., Chicago, IL 60607
Phone: (773) 972-8868; Fax: (312) 413-4122

Michael Dieter
Faculty member of UIC's School of Biomedical and Health Information Sciences
School of Biomedical and Health Information Sciences
University of Illinois at Chicago
1919 West Taylor Street, Chicago, IL 60612
Phone: (312) 413-8463; Fax (312) 996-8342

Robert Mrtek
Professor of Medical Education at the UIC College of Medicine in Chicago
Department of Medical Education
University of Illinois at Chicago
808 S. Wood Street, Chicago, IL 60612
Phone: (312) 996-7898; Fax: (312) 413-2048

ABSTRACT
As universities and businesses move toward the use of online education and training, there is need to discover how to make this alternative both more attractive and viable for different populations. Our research efforts examined the cluster of opinions held by students, with respect to technology and its application to education, across two populations: traditional college undergraduate students and adult learners (nontraditional graduate students). None of the students had any experience with online coursework. Q-methodology was used to identify opinions, shared among students, on issues they considered important about the application of technology to course instruction. This research suggests approaches on how an educational program might fine tune its online delivery for maximum suitability and acceptability to the broadest group of learners in post-secondary education.

KEY WORDS
Teaching, Education, Distance; Attitude to Computers

I. INTRODUCTION

The last ten years have seen the widespread development of digital processing and communication coupled to networked computing. This has opened up a broad set of teaching and learning opportunities, allowing a new emphasis on interaction and concept exploration. As is commonly the case in other fields, however, these early extensions have tended to follow the already established distance learning conventions, or those of the classroom. Little work has been done to identify opinion typologies that characterize the student population with respect to lifestyle, workplace, and learning style. Ultimately, understanding the opinion types of a student population permits faculty to optimize effectiveness in the delivery of course content using technology.

Commenting on the psychological satisfactions provided by the classroom setting, Batstone [1] writes that the information contained in Internet offerings usually is not enough. To be most effective, such offerings must provide users with a credible virtual environment, one that gives users a sense of community. He further asserts that in the zeal of universities to build computer and video infrastructures, they run the risk of neglecting the ways in which technology could help them stay connected with students through tailored education approaches. The success of long-distance learning hinges on its capacity to simulate a dynamic campus classroom. Students are not willing to sacrifice that shared experience merely for the convenience of studying at home. Andriole [2] asserts that the uniqueness of technology-based instruction makes it necessary to adopt more rigorous course requirements and design, development, delivery, and evaluation.

The purpose of this study was to identify and categorize the opinions of a sample of students at the University of Illinois at Chicago in order to improve our understanding of their acceptance of or resistance to the application of technology to learning. These results draw attention to opinions of the marketplace. We did not really know--and there was relatively little published research--what the market population of students thinks about computer-mediated instruction and what it would take to meet their needs. Early on, the investigators wondered if there was a relationship between a student's expressed opinion about technology applied to education, and his or her preferred learning style. The application of web-based technology to education introduces a host of administrative, communication medium, one-on-one and face-to-face interaction, as well as technical concerns. The interactions of course presentation with these types of concerns may significantly push individuals away from considering web-based courses. Understanding student opinions, we can better design and provide instruction for web-based courses.

II. LITERATURE REVIEW

Although the body of literature is large and growing, the subset of research literature dealing with student attitudes toward technology and web-based computer-mediated distance learning is small. Others have substantiated this view. Zhang [3] states, "few studies report the actual uses of Internet technologies alone or in combination with other technologies in effective distance learning." Other distance education practitioners [4,5,6] confirm the problem of too little data on the use of technology and its effectiveness. Biner [7] suggests that students' attitudes toward distance education are as important a metric as students' achievements in determining the effectiveness of distance education.

A bibliographic search of the ERIC database turned up 4,059 citations with the major subject heading of "distance education" over the years 1985 - December 2000. Of these citations, only 194, approximately 5%, dealt with student attitudes toward computer-mediated distance learning. Further limiting this set to citations that contained references to the World Wide Web or the Internet resulted in a retrieval of 20 citations, approximately 0.5%. There is an obvious gap in the distance education literature regarding students' opinions of technology, as evidenced by the paucity of relevant citations.

Content analysis of published literature and of websites indicated both positive and negative aspects of the application of technology to distance learning. Among the positive aspects documented were that online courses and distance education provide greater flexibility and student convenience; improved access/interaction with the instructor; better grades; and a more positive overall learning experience. The collaborative learning environment seems to better engage students individually in the learning process. Among the negative aspects documented were reduction in face-to-face interaction; concerns over technology and logistics; an increased student workload; and increased costs to the student. These positive and negative aspects are described in the following pages.

A. Positive Aspects
1. Flexibility and Convenience (time-shifting and associated advantages of time management)
Guernsey [8] found that a large number of distance education students were either already registered in regular classes, or were trying to work full- or part-time while earning degrees. Richards and Ridley [9] found that logistics was the second most common reason for enrolling in online courses. Hiltz [10] reported that 69% of students felt that the courses in the virtual classroom were "more convenient" than traditional courses. Richards and Ridley [9] found that distance education as the only alternative, was the third most common reason for enrolling in online courses.

2. Access/Interaction with Instructor
Students perceive that they receive more individual attention from instructors [8]. Studies [11] have shown that student attitudes toward distance education can be significantly affected by facilitating some degree of interaction among students and teachers. Hiltz [10] found that 71% of students who had just completed an online course felt that asynchronous learning networks provided better access to their professor.

3. Better Performance
Students perceive that they would get better grades than in a face-to-face course [8]. Koch [12] states that distance education students earned higher grades than students in conventional versions of the same classes. Bee [13] found that students who participated in web-based instruction felt that they improved their course performance.

4. Collaborative Learning Environment
Barreau, Eslinger, McGoff, and Tonnesen [14] found that students reported they formed good working relationships, felt equality in their contributions, and felt that groups enabled them to produce higher quality projects. Students prefer engaging in small group discussion or interactive question and answer as opposed to viewing lectures [11]. Hiltz [10] found that only 15% did not "feel more involved in taking an active part" in a virtual class; and that 55% felt more motivated to work hard on their assignments because others would be reading them. Hiltz [10] also found that only 20% agreed with the statement, "I would not take another online class," while 52% disagreed. Asynchronous learning environments allow more time to compose responses to questions [15].

5. Positive Learning Experience
Barreau, Eslinger, McGoff, and Tonnesen [14] state that students found the time spent on class (1 to 27 hours per week) was worthwhile. Barbrow, Jeong, and Parks [16] and Foell and Fritz [17] found that students overall attitudes toward computers in distance education classes were positive. Those who have taken distance courses have generally responded positively to the experience and would recommend it to other students [11]. Richards and Ridley [9] found that 79% of students rated their experience in online courses as "excellent" or "good." Hiltz [10] reported that 58% of students felt that the virtual classroom increased the quality of education (20% felt it did not).

B. Negative Aspects
1. Limitations on Interactivity (text-based communications, asynchronous timelag vs. synchronous)
Guernsey [8] states that younger students had difficulties with online courses and felt that they needed to be with a "live person." Larson [18] cites some students' need for face-to-face interaction. Hiltz [10] reported that the majority (59-64%) of students felt that they made new friendships in courses with a face-to-face element, whereas only 33% of the virtual classroom-only students agreed.

2. Technological Problems
Students new to a particular technology may initially exhibit some concern about the role of technology in the learning experience. If this occurs, these students typically demonstrate a reluctance to actively participate in the distance classroom areas [11]. Mastrian and McGonigle [19] found that the most frequent negative comment related to the overall experience was the early frustration with the use of the computer.

3. Increased Workload
Barbrow, Jeong, and Parks [16] found that students' attitudes were positive with the exception of the amount of time it took to learn new software. Gifford [20] stated that the majority of students felt that more time was spent on the Internet-based class than in the regular classroom. Hiltz [10] reported that only 13% of students in the virtual classroom agreed (67% disagreed) with the statement, "I didn't have to work as hard for the online class." Barreau, Eslinger, McGoff, and Tonnesen [14] reported that students sometimes felt overloaded with information; Guernsey [8] found that students felt online courses required more work.

4. Lack of Logistical Support (administrative and technical)
Larson [18] has found that lack of availability of course resource materials was a negative aspect of distance learning. Hiltz [10] found that 40-50% of students had difficulty accessing course materials due to busy signals at the dial-in. Hiltz [10] also reported that 52% of virtual students felt that it was easier to fall behind in virtual classes due to the ease of postponing or procrastinating.

5. Costs (equipment, online phone charges, etc.)
Bee [13] found that students who chose not to take advantage of auxiliary materials placed on the Web felt that the university should provide financial assistance to offset the associated costs of going online. Hiltz [10] found that 13% of students indicated that access to a PC was a serious problem.

III. METHODS

A. Measuring Subjectivity
Q-methodology, applying a hybrid of qualitative and quantitative statistical techniques, is used to uncover commonly shared opinions regarding a specific topic. The qualitative methods of Q allow participants to express their subjective opinions and the quantitative methods of Q use factor analytic data-reduction and induction to provide insights into opinion formation as well as to generate testable hypotheses. Studies employing this method typically use small sample sizes as the method emphasizes the subjective opinion of a population, not how many in the population share the opinion. The methodology involves three stages: developing a set of statements to be sorted; having participants sort those statements along a continuum of preference (agree to disagree); and analyzing and interpreting the data [21]. An extensive discussion of the definition and application of Q-methodology can be found in Valenta and Wigger, and Barbosa, Willoughby, Rosenberg, and Mrtek [22,23].

The subjects in a Q-methodology research study are asked to rank order a group of subjective statements on a continuum from strongly agree to strongly disagree. Unlike the Likert survey technique, the Q-method permits examination of the statements relative to one another. The set of instructions for sorting the statements, given to the research participant, is called the Condition of Instruction.

In this study, the researchers developed, using nominal group technique, a large collection (concourse) of items relating to the application of technology to education. From the original concourse, the researchers selected, through content analysis, 23 statements that represented aspects of lifestyle, workplace, and learning preference. These 23 statements represented the final Q-set (appendix A). The final Q-set was distributed to the respondents, along with the Condition of Instruction (see appendix C): Which issues are important and which issues are not important to you when thinking about the application of web-based technology to education? Respondents were to arrange the statements within the response grid (appendix B) with those on the left side being items considered most unimportant and those on the right side being items considered most important. Statements were not ranked within the columns. The result of this process, the Q-sort, was analyzed using PQMethod, a statistical program that allows data entry in a way that reflects the response grid, computes intercorrelations among participants' responses, and results in a definition of factors [24].

B. Participants
Seventy-four students from the University of Illinois at Chicago participated voluntarily (30 males and 44 females, 40.5% and 59.5%, respectively). None of the participants had previously taken a web-based course. Two populations were sampled: graduate and undergraduate students. Participants ranged in age from 17 to 63. Fifty-four of the 74 participants were graduate students enrolled in coursework in the graduate health informatics specialization (20 males and 34 females). These participants were categorized as adult learners. The average age was 36 (standard deviation = 8.72 years). The remaining 20 participants were undergraduate students enrolled in an introductory psychology class (10 males and 10 females). The average age was 19 (standard deviation = 2.88 years). The average age for the entire sample was 31 (standard deviation = 10.5 years).

C. Procedures
All students received the same Q-set, which was administered within the first week of the course, before students were fully exposed to any particular web-based technology. The instructions for the Q-instrument were given to the participants to read and follow. Total time for administering the instrument did not exceed 45 minutes.

IV. RESULTS

Q-methodology results in the identification of participant opinion profiles based on the similarities and differences by which they sort the statements in the Q-sample [21,25,26]. By-person factor analysis and varimax rotation identified three opinion types among our participants that represented three different views regarding the use of web-based instruction. 35 of the 74 participant sorts (47%) were accounted for in the three opinion types (also called factors). The remaining 39 sorts did not show any significant correlation with these three factors. Table 1 (on next page) summarizes the rankings among statements for each factor or opinion type, as generated by the statistical software. In Q, an understanding of participant viewpoints results from the examination of that factor's statements, after ranking ordering the statements from +3 to -3. The three factors (opinion types regarding the application of technology to education) were titled: (1) Time and Structure in Learning; (2) Social Interaction in Learning; and (3) Convenience in Learning.

Factor 1: Time and Structure in Learning
Most important to the Time and Structure group was that web-based education provides flexible time management. It is important to these students that they can work at home when they want to and at their own pace. They are very much aware that it requires self-discipline and active learning and initiative. Unimportant to this student group were issues such as having access to the Internet only through work, paying home phone bills, or attaining quiet computer time at home. This group was neutral on issues regarding social interaction in the classroom.

Factor 1 participants recognize some benefits in the application of technology to deliver educational content, in terms of time management and flexibility. They do not appear to be concerned over the loss of face-to-face classroom interactions; however, they recognize the need to exercise discipline to be successful.

Nineteen participant sorts loaded significantly on this factor. Of these sorts, there were: 16 (84%) graduate students and 3 (16%) undergraduates; 13 (68%) female and 6 (32%) male. Figure 1 illustrates this factor.

Factor 2: Social Interaction in Learning
Most important to the Social Interaction group was the potential for less participant discussion and that with web-based education there would be fewer subtleties in teaching, i.e., instructor observation, speech inflection and immediate feedback. Also ranked important were less enrichment from other perspectives and potential interference with work. At the same time, this group reacted positively to the opportunity to be able to work at home. Least important to this group were issues such as being able to work in their bathrobe and learning to use the Internet. They were not concerned about having trouble accessing the Internet from home, paying home phone bills, or their need to be self-disciplined in learning. This group reacted in a neutral way to statements such as being able to learn at one's own pace and having less of a sense of self-assessment in comparison to others.

Factor 2 participants recognize few benefits in the application of technology to deliver educational content; the only benefit appears to be the ability to work at home. This group is quite concerned over the loss of face-to-face classroom interactions.

Ten participant sorts loaded significantly on this factor. Of these sorts, there were: 9 (90%) graduate students and 1 (10%) undergraduate; 6 (60%) female and 4 (40%) male. Figure 2 illustrates this factor.

Factor 3: Convenience in Learning
Most important to the Convenience group was that web-based education lets them work at home and save travel time. It provides flexible time management and saves commuting cost. They recognize potential interference when logging in at work. Least important to this group were basic computer troubleshooting and Internet skills. Also unimportant were issues such as paying home phone bills and having Internet access only through work. Being less able to assess oneself vis-à-vis another was unimportant. Neutral reactions centered on issues of social interaction, the need for active learning, and limited computer time at home.

Factor 3 participants see much benefit and few drawbacks to the application of technology to deliver educational content. They are concerned neither about loss of face-to-face classroom interaction nor of the need to exercise self-discipline.

Six participant sorts loaded significantly on this factor. Of these sorts, there were: 5 (83%) graduate students and 1 (17%) undergraduate; 2 (33%) female and 4 (66%) male. Figure 3 illustrates this factor.

Consensus Statements
Important to all three groups was the ability to work at home. Unimportant to the three groups was paying home phone line costs.

V. DISCUSSION

Three opinion types were identified in this study: Students who identified with issues of Time and Structure in Learning, Social Interaction in Learning, and Convenience in Learning. These opinions can be used to aid educators in reaching their students and increasing the effectiveness of their online courses. At UIC, this insight had direct application to the evolution of course materials. Early application of technology merely supplied a web site on which were posted syllabus, readings and assignments. No opportunity existed for conferencing; thus, there existed no opportunity for social learning. In a subsequent semester, conferencing software was made available to the class, in addition to the website. Thus, the opportunity was added for social learning. The faculty learned, however, that every time a new technology was added, it experienced an increase in the level of effort necessary to support the student. Ultimately, the University made available a course management system, which significantly streamlined the effort on the part of faculty to make course materials available to the student. The system provides through a single URL the student's access to course materials, discussion forums, virtual groups and chat, testing, grades, and electronic communication.

This study is qualitative and confined to University of Illinois at Chicago graduate and undergraduate students. The three opinion types identified through this study, however, correlate closely with results reported in the literature. All three groups of students, representing the three opinion types, shared a belief in the importance of being able to work at home. The studies of Richards and Ridley [9] and Hiltz [10] described flexibility and convenience as both reasons students enrolled in online courses and as the perception of students once enrolled. On the other hand, all three groups of students thought unimportant the need to pay home phone bills incurred in online education, whereas Bee [13] found that students felt the university should provide financial assistance to offset the associated costs of going online. There is evidence in the literature (viz., studies by Guernsey [8] and Larson [18]) that support the opinion identified in this study of the need by some students for face-to-face interaction. Since none of the students taking the Q-sort had ever taken an online course, they were unaware of the opportunities provided by technology [8,10] to potentially increase individual attention from instructors above that normal in face-to-face course offerings. Since no post-enrollment Q-sorts were administered, there was no way to tell whether students continued to hold that opinion, or whether that opinion has changed. It is anticipated that even if the Q-set were administered to a larger number of students, similar viewpoints would still emerge.

The authors wondered whether there was an association between the opinion set held by the student and his or her learning style. Preliminary data using the Canfield Learning Styles Inventory [27] show that the factor one group--Time and Structure in Learning--exhibited a much higher than expected proportion of independent learners. (74% of the students who had high factor loadings on factor one were also classified as independent learners. This difference was significant Z = 3.00, p < .025.) One might be tempted to hypothesize a relationship between being an independent learner and having the time and structure opinion of technology and education. Similarly, one might also expect that individuals who had high factor loadings for factor two (Social Factors in Learning) would be more likely classified as social learners. Further research is necessary to understand how learning styles contribute to the experience of online education.

There is a movement in both education and business to harness the power of the World Wide Web to disseminate information. Educators and researchers, aware of this technological paradigm shift, must become invested in understanding the interactions of students and computing. The field of human-computer interface design, as applied to interaction of students in online courses, is ripe for research in the area of building better virtual learning communities (thus addressing the needs of the social learner) without overwhelming the ability of the independent learner to excel on his or her own.

VI. ACKNOWLEDGEMENTS

This research was underwritten by a grant provided by the University of Illinois Council for Excellence in Teaching and Learning.

VII. REFERENCES

1. Batstone D., Link - and - learn. Chicago Tribune. March 16, 1997;sect 2:1,7.
   
2. Andriole S.J., Requirements-driven ALN course design, development, delivery & evaluation. Journal of Asynchronous Learning Networks [serial online]. August 1997;1(2). Available at: http://www.sloan-c.org Accessed April 21, 2000.
   
3. Zhang P., A case study on technology use in distance learning. Journal of Research on Computing in Education [serial online]. 1998;30(4):398-419. Available from: WilsonSelect via OCLC FirstSearch. Accessed June 14, 1999.
   
4. Bergen C., Kingston P., A framework for analyzing the contribution of educational technology to learning. British Journal of Educational Technology. 1994;25(1):58-60.
   
5. Neal L., Ramsay J., Preece J., Distance learning: A CHI 97 special interest group. SIGCHI Bulletin. 1997;29(4):76-78.
   
6. Stubbs S. T., Burnham B. R., An instrument for evaluating the potential effectiveness of electronic distance education systems. The American Journal of Distance Education. 1990;4(3):25-37.
   
7. Biner P. M., Re-assessing the role of student attitudes in the evaluation of distance education effectiveness. Distance Education Review [serial online]. 1997. Available at: http://www.distance-ed-review.com/biner.htm Accessed June 14, 1999.
   
8. Guernsey L., Distance education for the not-so-distant. Chronicle of Higher Education [serial online]. 1998;45(3):A29-30. Available from: Periodical Abstracts via OCLC FirstSearch. Accessed April 7, 1999.
   
9. Richards C. N., Ridley D. R., Factors affecting college students' persistence in on-line computer-managed instruction. College Student Journal [serial online]. December 1997:490-5. Available from: WilsonSelect through OCLC FirstSearch. Accessed April 9, 1999.
   
10. Hiltz S.R., Impacts of college-level courses via asynchronous learning networks: some preliminary results. Journal of Asynchronous Learning Networks [serial online]. August 1998;1(2). Available at http://www.sloan-c.org Accessed September 27, 1999.
    
11. Lucent Technologies Center for Excellence in Distance Learning (CEDL). Student attitudes toward distance learning. Distance Learning Abstracts [serial online]. Available at: http://www.lucent.com/cedl/stdtatt.html Accessed June 16, 1999.
    
12. Koch J. V., How women actually perform in distance education. Chronicle of Higher Education [serial online]. 1998;45(3):A60. Available from: Periodical Abstracts via OCLC FirstSearch. Accessed April 4, 1999.
    
13. Bee R. H., Differing attitudes of economics students about web-based instruction. College Student Journal. 1998;32(2):258-269.
    
14. Barreau D., Eslinger D., McGoff K., and Tonnesen C., Group collaboration in the virtual classroom: An evaluation of collaborative learning in the virtual classroom of CMSC 828S and the technology that supports it. Journal of Virtual Experience [serial online]. 1993. Available at: http://www.hitl.washington.edu/research/knowledge_base/virtual-worlds/JOVE/index.html Accessed December 6, 1999.
    
15. Jaffee D., Asynchronous learning: Technology and pedagogical strategy in a distance learning course. Teaching Sociology. 1997;25(4): p262-77.
    
16. Barbrow E., Jeong M., and Parks S., Computer experiences and attitudes of students and preceptors in distance education. Journal of the American Dietetic Association [serial online]. 1996;96(12):1280. Available from Expanded Academic ASAP via Infotrac. Accessed April 7, 1999.
    
17. Foell N.A., Fritz R.L., Association of cognitive style and satisfaction with distance learning. Journal of Industrial Teacher Education [serial online]. 1995;33(1):46-59. Available at: http://scholar.lib.vt.edu/ejournals/JITE/v33n1/foell.html Accessed April 21, 2000.
    
18. Larson D. K., Distance Education and the WEB. Available at http://www.uis.edu:1967/~dlarson/dlpaper.htm Accessed December 6, 1999.
    
19. Mastrian K.G., McGonigle D., Older student perceptions of technology based learning assignments. On-Line Journal of Nursing Informatics [serial online]. 1997;1(2):no pagination. Available at http://cac.psu.edu/~dxm12/percep2.html Accessed April 21, 2000.
    
20. Gifford L.J., Graduate students' perceptions of time spent in taking a course by Internet versus taking a course in a regular classroom. Paper presented at: Annual Mid-South Educational Research Association Conference; November 4-6, 1998; New Orleans, LA.
    
21. Brown S. R., A primer on Q-methdology. Operant Subjectivity. 1993;16:91-138.
    
22. Valenta A. L., Wigger U., Q-methodology: Definition and application in health care informatics. Journal of the American Medical Informatics Association. 1997;4:501-510.
    
23. Barbosa J. C., Willoughby P., Rosenberg C. A., Mrtek R., Statistical methodology: VII. Q-methodology, a structural analytic approach to medical subjectivity. Academic Emergency Medicine. 1998;5:1032-1040.
    
24. PQMethod, freeware for data entry and analysis developed specifically for use in Q-methodology, available at http://www.rz.unibw-muenchen.de//~p41bsmk/qmethod/, Accessed June 23, 2000.
    
25. McKeown B. F., Thomas B. D., Q-methodology. Newbury Park, CA: Sage Publications, 1988.
    
26. Dennis K. E., Q-methodology: Relevance and application to nursing research. Advances in Nursing Science. 1986; 8:6-17.
27. Canfield A.A., Learning Styles Inventory Manual. Los Angeles, CA: Wester Psychological Services; 1988.
    

VIII. ABOUT THE AUTHORS

Annette L. Valenta is an Associate Professor and Assistant Director for Academic Programs for the School of Biomedical and Health Information Sciences at the University of Illinois at Chicago. Dr. Valenta is program coordinator for UIC's graduate programs in Health Informatics, having developed, implemented, and taught in both the Masters in Health Informatics and the Health Information Management Specialization within the Master of Business Administration. In 1997, Dr. Valenta was awarded a U.I. OnLine/Sloan Foundation grant to translate the face-to-face core informatics coursework to a distance model. The coursework offers online education in the application of information systems and information systems management to the health care industry. Dr. Valenta earned her Certificate in On-Line Teaching and Learning from California State University, Hayward.

Contact: School of Biomedical and Health Information Sciences, University of Illinois at Chicago, 1919 West Taylor Street, Chicago, IL 60612; Telephone: (312) 996-1452; Fax (312) 996-8342; E-mail:valenta@uic.edu

David J. Therriault is a doctoral candidate in the Psychology Department at the University of Illinois at Chicago and an independent consultant. His interests include exploring the psychology of reading, and evaluating web-based training programs and educational courses. He holds a Master's Degree in Psychology from the University of Illinois at Chicago.

Contact: Department of Psychology, University of Illinois at Chicago, 1007 W. Harrison St., Chicago, IL 60607; Telephone: (773) 972-8868; Fax: (312) 413-4122; E-mail: davidt@uic.edu

Mike Dieter is a faculty member of UIC's School of Biomedical and Health Information Sciences, teaching in the graduate program in Health Informatics. The majority of the program's courses are offered in distance education format, and provide perspectives on topics and issues arising from the convergence of healthcare organizations, healthcare information systems, and medical informatics. Mr. Dieter's interests include knowledge management in healthcare organizations, and the interrelationship between information literacy and distance education. He holds a Master's Degree in Library and Information Science from Dominican University in River Forest, Illinois, as well as a Master's Degree in Business Administration from the University of Illinois at Chicago.

Contact: School of Biomedical and Health Information Sciences, University of Illinois at Chicago, 1919 West Taylor Street, Chicago, IL 60612; Telephone: (312) 413-8463; Fax (312) 996-8342; E-mail :miked@uic.edu

Robert Mrtek is Professor of Medical Education at the UIC College of Medicine in Chicago, the largest US College of Medicine. He teaches Evidence Based Medicine (EBM) in the undergraduate medical curriculum as well as running EBM conferences and Journal Clubs in General Internal Medicine for Residents and Fellows. He also works closely with volunteer clinician faculty learning about EBM at hospitals used for core clerkships. For Graduate College level programs in health professions education, Dr. Mrtek offers elective courses in research design for both the quantitative research paradigm as well as a separate course in qualitative experimental design and data analysis emphasizing the use of Q Methodology as a research strategy for the study of human subjectivity. Dr. Mrtek's design experience and skills in teaching methods put him in high demand with graduate students and Research Fellows, as is evidenced by joint appointments he holds in the Department of Internal Medicine, and on the faculties of the School of Biomedical and Health Information Sciences in the College of Health and Human Development Sciences as well as in the College of Pharmacy. Dr. Mrtek is Editor of Operant Subjectivity (ISSN 0193-2713), the peer-reviewed scholarly journal devoted to Q Methodologic studies and research. The Journal is sponsored by the International Society for the Scientific Study of Subjectivity.

Contact: Department of Medical Education, University of Illinois at Chicago, 808 S. Wood Street, Chicago, IL 60612; Telephone: (312) 996-7898; Fax: (312) 413-2048; E-mail: mrtek@uic.edu

APPENDIX A
PARTICIPANTS' Q-SAMPLE STATEMENTS

1. Less sense of self-assessment in comparisons to others.
2. Fewer subtleties in teaching - instructor observation, speech, inflection, and immediate feedback.
3. Fewer opportunities to meet new people - social interaction.
4. Less enrichment from other perspectives
5. Less informal learning - side comments by teacher and students.
6. Less discussion with participants
7. Sometimes hard to find quiet time at home or work.
8. Sometimes computer time hard to get at home.
9. Provides flexible time management.
10. Potential interference with work obligations.
11. Saves travel time.
12. Can work at home when I want.
13. Trouble getting access to Internet at home.
14. Requires basic skills in computer troubleshooting.
15. Must pay home phone line costs.
16. Access to Internet only through work.
17. No set class time.
18. Requires self-discipline.
19. Requires active learning and initiative.
20. You'll sure learn to use the Internet.
21. Can learn at my own pace.
22. Saves commuting cost.
23. Can work in your bathrobe.

APPENDIX B
SORTING ANSWER SHEET
(Please fill out all of the following information)

APPENDIX C
SORTING INSTRUCTIONS

Which issues are important or not so important to you when thinking about the application of web-based technology to course instruction? The goal of this study is to help us understand and incorporate your needs and concerns into the planning and implementation of new ways of delivering education to you.

The objective here is to sort the statements along the continuum from the ones that are Most Important to the ones that are Least Important to you.

1. Look at all the opinion statements to familiarize yourself with the range of issues.

2. Sort the issues into 2 piles. One should contain the statements that you find Important in one way or another--for any reason. The other pile contains those statements that you find Not Important for any reason. The piles do not have to contain equal number of statements.

3. From the pile of statements you find Important, select the two items (only 2) that you find Most Important. Place them in a two-item column at the extreme right hand of your workspace.

4. From the remaining Important pile, select three (3) more issues that are now more important to you than the others in the pile. Place these 3 statements in another column just to the left of the two already selected in step 3 above.

5. Next, select from the remaining Important pile the four (4) statements that you now feel are Most Important. Place these 4 statements in another column just to the left of the three already selected in step 4 above.

6. Next, select from the remainder of the Important pile the five (5) statements that you now feel are Most Important. Place these 5 statements in another column just to the left of the four already selected in step 5 above.

If you have run out of statements in the Important pile and cannot finish step 6, proceed immediately to the next step.

If you have extra unsorted statements at the end of this step, combine the extras with the Not Important pile and go on to the next step.

7. Now, work with the pile of statements you feel are Not Important. Begin by selecting the two (2) statements you find Least Important. Place them in a two-item column on the far-left side of your work area.

8. From the remaining Not Important pile, select three (3) more issues that are now less important to you than any others in the pile. Place these 3 statements in another column just to the right of the two already selected in step 7 above.

9. Next, select from remaining Not Important pile the four (4) statements that you feel are Least Important. Place these 4 statements in another column just to the right of the three already selected in step 8 above.

10. Place the remaining 5 issues in the middle of your grid.

11. Now, look at your arrangement. Feel free to move issues around to make sure that your opinion is reflected correctly.

12. When everything is sorted as you want it to be, write the statement numbers in the blank boxes in the grid on your answer sheet and answer the remaining questions on the form.