An informal survey was taken of student reactions to technological enhancements in an upper-level course on Psychopathology. Responses were generally favorable, with many positive comments made on use of NetForum. Server unreliability was a commonly-expressed concern. Requiring that daily, one-point Good Faith Effort assignments be done on NetForum appeared to facilitate student use of it.

In the Fall 1999 semester, I instituted the following technology-related changes in PSY 350, Psychopathology: (a) a course web site which included the syllabus, announcements, assignments, handouts/study guides, grade posts, and some lecture slides and (b) NetForum assignments. I had been using PowerPoint to present lecture material in class for several semesters. I also incorporated the idea of daily Good Faith Effort (GFE) assignments worth one point each. Often, but not always, these were done on NetForum.

As part of my assessment of the technology-related changes I made in this course, I administered an informal, voluntary survey of student reactions specifically to instructional technology on the last day of class. In addition to open-ended items, 11 rating scales 1 to 5 were used (1 = "strongly disagree," 5 = "strongly agree," and no other anchors were provided). Some of these items were adapted from a draft version of an assessment instrument circulated by Kathleen McKinney in conjunction with the State Farm program. I added negatively-worded items to assess more directly possible negative reactions.

I structure this course with an optional cumulative final; students drop their lowest exam score. To ensure that students are able to make an informed decision on taking the optional final, I schedule the last Unit exam for the second-to-last class meeting. At the final class meeting, I distribute grade summaries for the entire semester, allowing students to determine (a) their grade if they forego the optional final; (b) the score they would need on the optional final to change their grade; and (c) whether their grades for semester have been accurately recorded. After this information was distributed, surveys were distributed with an emphasis on the following: participation was voluntary, responses were anonymous, the survey was not part of official course evaluations, and data would not be examined until after grades were submitted. Twenty five individuals (56% of the total class of 45) provided responses.

Overview of Open-ended comments

How did technology enhance your learning of the course material? A variety of positive responses were provided, and most respondents provided at least one. The most common response to this question was on the use of NetForum. Eleven individuals (44% of respondents) indicated that they liked or benefited from the web-based interaction with me and their classmates. Some of these indicated a preference for computer-mediated interaction over in-class interaction. Other fairly common responses included: clarity of material, facilitation of note taking, increased accessibility of course information, increased organization of course material or work (e.g., keeping up with reading that would be assigned for GFE NetForum discussions).

What other benefits did you get from the use of technology in this course? There were fewer responses to this item; most individuals who responded identified learning about the using web or other technology (e.g., e-mail) as other benefits.

What problems were associated with the use of technology in this course? Twenty individuals (80% of respondents) identified server problems in response to this question (many of these responses included recognition that there was little I as instructor could do about this problem except be flexible). Three individuals indicated that computer access (either at home or in labs) was sometimes problematic. Clearly, as more courses incorporate more computer technology, these hardware issues must be addressed, as they will likely become even more pressing.

Any other comments? Seven individuals provided comments here, although two of these were complaints about an exam. The five relevant to instructional technology indicated generally positive reactions, with some suggestions. It was noted that while the NetForum/GFE assignments could be difficult or time-consuming, they were interesting and worthwhile. One suggestion was to reduce the frequency of using NetForum, in part due to server and access problems. Another was to discontinue posting assignments on the course web site so that individuals would be rewarded for class attendance (or punished for skipping class).

Responses to rating scales

The responses to the 11 rating scales are summarized in the following Table.

Response Frequencies

      
 Item

      
 M

      
 SD

      
 Mdn

      
 1

      
2

      
3

      
4

      
5
The
      technology used by the instructor aided my learning in
      this course.

      
 3.96

      
 0.84

      
 4

      
 0

      
 1

      
6

      
11

      
7

      
The technology I used aided my
learning in this course. 

      
 3.76

      
 0.88

      
4

      
 0

      
2

      
7

      
11

      
 5
The benefits of using technology
      in this course outweighed the hassles or difficulties.

      
 3.72

      
 0.89

      
4

      
0

      
2

      
8

      
10

      
5
Technology got in the way of my
      learning.

      
 1.88

      
 0.78

      
 2

      
8

      
13

      
3

      
1

      
0

      
I would like to take technology intensive
courses in the future.

      
 3.24

      
1.23

      
3

      
3

      
2

      
11

      
4

      
5

      
I think this course would have been less effective
without technology. 

      
 3.24

      
 1.13

      
 3

      
 2

      
4

      
8

      
8

      
3

      
Additional assignments and instruction using
technology should be added to this course.

      
 2.88

      
1.13

      
 3

      
1

      
10

      
8

      
2

      
0

      
Technology was overemphasized at the expense
of course content.

      
 2.28

      
 1.89

      
2

      
 5

      
10

      
8

      
2

      
0

      
The technology used in this course seemed
appropriate for the objectives or assignments.

      
 4.16

      
 0.75 

      
 4

      
 0

      
1

      
 2

      
 14

      
 8

      
The technology related expectations for students
(hardware, software, and expertise) were
inappropriate for this course.

      
 1.72

      
  1.02

      
1

      
 14

      
 7

      
 1

      
 3

      
 0

      
I received sufficient assistance to be able to use
the technology required in this course.
 3.80

      
  0.96

      
4

      
 0
  3

      
5
 11
 6
Note. N = 25. Rating scales were anchored by "Strongly Disagree" for 1 and "Strongly Agree" for 5.

Reactions tended to be favorable; on every clearly positively-worded item, the median was 4, indicating agreement; on every clearly negatively worded item, the median was 2 or 1, indicating disagreement. In general, respondents also seemed to think that the use of instructional technology was at an appropriate level for the course and their preparation. Interestingly, the majority (16 of 25, or 64%) seemed undecided or negative about taking technology-intensive courses in the future.

Students who responded to the survey provided a positive reaction to the use of computer technology. It appears that the features they liked most were the opportunity to learn while interacting with their classmates in an asynchronous environment and the organization and availability of information. They expressed some frustration with server problems, but typically acknowledged that this was not a course-specific problem. Potentially useful suggestions included removing assignment postings on the web to make class attendance more rewarding and to reduce the frequency of use of NetForum assignments in light of server unreliability and other access problems.

The positive reaction to NetForum bears some comment, in light of reports from colleagues that students often do not use it. In this course, NetForum use was specifically assigned and counted toward students' grades as part of a Good Faith Effort system. A typical GFE on NetForum would entail providing three posts: (a) a brief answer to question I posted,(b) a comment on another student's answer, and (c) a reply to a comment on one's original answer. It was instructive that on days when additional comments or posts were not required, hardly any students ever made them. In other words, it appears that students will gladly discuss the course material if they believe that it directly affects their grades, even when it is only worth one point (out of 491!). However, they are less likely to use NetForum when the payoff seems remote or indirect. It might also be useful to make clear that exam questions will be based on material discussed on NetForum. These considerations also highlight the importance of using technology in the context of the entire course and its pedagogical goals.