City University of New York
The National Project on Computers and College Writing
This evaluation and dissemination project aimed to assess the effectiveness of computer-assisted composition programs and to showcase outstanding college programs that employ this instructional strategy. A network of 15 competitively selected institutions of all sizes and types, chosen from among 90 applicants, participated by joining in a centrally designed assessment program and by sharing their computer-use models and instructional strategies with each other and with wider audiences.
Each of the 15 institutions agreed to carry out an assessment activity according to a common plan. Each site identified at least six sections of English composition, three taught using computer-assisted instruction and three by conventional methods. Each section enrolled similar kinds of students within a single institution, but this was not necessarily the case across institutions. A pre-course and post-course test common to all 15 sites was administered to each student, who in addition completed the Descriptive Test of Language Skills Sentence Structure Subtest and questionnaires regarding attitudes toward writing and writing anxiety. Each participating institution gathered site-specific qualitative data. The final sample consisted of 1,700 students, equally distributed between computer-instructed and regular sections.
The essays were evaluated centrally, all 1,700 holistically and 10% using analytic assessment methods. While overall the holistic scores showed no difference between experimental (computer instruction) and control (traditional instruction) groups, the scores did reveal significant advantages of computer instruction for those in developmental classes, adult learners and community college students. The analytic scoring did show significantly better performance by students in the experimental sections. Students in computer-based classes also showed a significant reduction in writing anxiety as compared with those in the control groups.
The dissemination activities of the project were concentrated in a conference, "Computers and College Writing: Curriculum and Assessment for the 1990's," and a monograph, Computers and College Writing: Selected College Profiles. The conference enrolled 600 participants from North America and Europe in June, 1990 and was followed by additional conferences in fall, 1990 and spring, 1992. The monograph contains accounts of computer-based composition technology and instructional strategies at 49 institutions in the United States.
The project is unique in its efforts to assess the effects of computer instruction on student learning and attitudes across a large number of institutions of varied size and type. No base of comparable size exists. The project both addressed questions about the value of computer-based composition instruction and created opportunities for the exchange of information about the way computers are being used in many institutions. Thus it provides an experimental base to justify the use of computers in teaching introductory composition courses as well as furnishing practical resources for teachers and institutions.
The assessment plan and its implementation were not themselves the object of independent external evaluation. Project personnel have, however, been careful to point out the problems inherent in carrying out such a study and employing the kinds of assessment described. Specifically, they point to problems inherent in holistic assessment. While the methodology, when used with trained evaluators, works well in handling a large volume of essays, when used in a pre-test/post-test situation it tends to produce a regression (and advancement) toward the mean pattern. To counteract this effect the project used a number of measures of student growth in addition to the essays.
Holistic scoring, in focusing on overall impressions, fails to identify particular advances in student writing, such as organization, copiousness, or stylistic sophistication. For this reason, 10% of the essays were subjected to analytic assessments, which revealed more significant differences between control and experimental groups. This result led project staff to assess the entire sample analytically, an effort still in process.
Project personnel were well aware of the problems inherent in trying to measure gains over a single semester of instruction, but the logistical problems of maintaining contact with students and administer a common assessment some semesters later were impossible to manage.
The number of participants in the conferences sponsored through the project and subsequent to its completion reflects its far-reaching effects. Project activities and results have also become known through numerous articles written and produced by project personnel.
The project has produced a firmly established network of institutions engaged in computer-based instruction, and a data base of both institutional strategies and assessment results. This information is a valuable resource for colleges and universities to justify purchase of computers for composition instruction and for those seeking information on the technology, models and strategies of such instruction.
Apart from the difficulties in finding fully satisfactory and manageable assessment mechanisms, which were acknowledged in advance, the project worked much as planned. Integrating the large volume of information gathered, an effort that continues to the present, required considerably more time and energy than expected.
The project has produced convincing evidence of the value of computer-based composition instruction, particularly for less well prepared groups of students.
In the spring of 1992, the Project sponsored "Computers Across the Curriculum: A Conference on Technology in the Freshman Year," which addressed computer-based instruction in a range of introductory college courses.
An effort is now underway to use the project's findings as the basis for a program of technology-driven writing instruction in the New York City Public Schools.
The project has generated a number of articles, as well as the major monograph, Computer and College Writing: Selected College Profiles. General information about the project and copies of the publications may be secured by writing to:
Max Kirsh, Director
National Project on Computers and College Writing
Office of Academic Computing
City University of New York
555 W. 57th Street, 14th Floor
New York, NY 10019
Each of the project sites has developed curricular materials, from videotapes to course syllabi. These are available from the individual institutions or, in some cases, from the project director, who will gladly supply the list of participants.