University of Rhode Island
Chem-Tutor: An Expert System For Teaching Freshman Chemistry
After a freshman chemistry course, most students are able to use the information they have acquired to solve simple, carefully-structured problems. When confronted with an exercise that requires them to use more than one principle, or that does not explicitly state necessary relationships, however, students are likely to falter and give up without even attempting to think their way through the problem.
Research indicates that success in solving complex problems is highly correlated with hours of practice--thousands of hours of practice--and with appropriate feedback. Yet often introductory science classes number in the hundreds of students, and teaching assistants typically teach three laboratory sections per semester. Thus, neither faculty nor teaching assistants are able to provide students the amount of practice and feedback they require to learn to solve complex problems. This project addressed that situation through the design, writing and testing of software for an intelligent computer-assisted instructional system, Chem-Tutor, that enables first-year chemistry students to improve their ability to solve complex problems.
hem-Tutor integrates an intelligent tutoring system into a traditional academic setting. It improves the complex problem-solving ability of the student by articulating its own reasoning skills and explanations, coaching the student's problem-solving behavior directly, and diagnosing his or her errors.
Chem-Tutor consists of a domain expert, which contains a knowledge base that serves as a model for problem-solving concepts and procedures, and a problem manager, which controls a library of over 1,200 problem templates. It also includes a tutor expert, which comprises the rules and procedures for interaction with the student, and the user environment--on-line tools such as a full-screen Periodic Table of the Elements. The program runs on a DEC VAX computer, contains over 50,000 lines of Common Lisp code, and is designed to be used at least one hour a week over an entire semester.
In the course of the project, a separate, complementary program, Chem-Tutor Instructor Expert, was also developed. This tool allows the instructor to reorganize, add, or delete areas of Chem-Tutor, to generate reports on the status of individual students or of the entire class, and to intervene when Chem-Tutor indicates that a student is in difficulty.
Approximately 600 students in URI's beginning chemistry classes rated their satisfaction with Chem-Tutor 6.8 on a scale from 0 (poor) to 10 (excellent). There was a large amount of variation from the average, and this variation correlates with the quality of the student's relationship with the teaching assistant.
After using Chem-Tutor, student performance on the final exam in the fall 1988 semester was compared to the final exam grade distribution for the fall of 1986, before Chem-Tutor was introduced. A check of the SAT scores for these two classes separated by three years shows no significant difference and the sections were taught by the same instructor at the same time. One can therefore assume that the 1986 and 1988 students were equivalent in background, ability, and SAT scores. The mean exam score for 1986 was 52.6 percent, and the mean exam score for 1988 was 56.6 percent. This is almost a 10 percent change in the average grade. Using a t-test, the comparison revealed a significant difference between students that had used Chem-Tutor and those students that had not (t(683)=2.79, p<0.01).
Surprisingly, despite the availability of Chem-Tutor the percentage of completed homework assignments per student was as low as 63.5 percent. Nevertheless, final grades did correlate positively with the percentage of homework completed--that is, students who completed more homework received higher final grades.
Chem-Tutor has been used by over 4,000 URI students to solve some half million chemistry problems. The required use of Chem-Tutor has increased the time that students spend solving problems, and the time spent tutoring on the part of instructors has decreased proportionately. The project has been institutionalized at the University of Rhode Island since 1989.
Several other institutions, including the University of Wisconsin, SUNY Albany, the University of New Hampshire, and Milan Polytechnic University have purchased Chem-Tutor and are adapting it to their own needs.
In the course of the project, efforts to develop a natural language user interface had to be abandoned due to the complexity of natural language processing. The designers of Chem-Tutor are attempting to overcome the system's limited ability to use artificial intelligence.
Because Chem-Tutor is text-based and keyboard-driven, its usefulness in certain areas requiring use of graphics is insufficient. Future systems will operate in a Graphical User Interface environment.
Major Insights And Lessons Learned
As a result of experience gained in the development and use of Chem-Tutor, it has become apparent that any successful tutoring system must take into account the four elements that make up an integrated learning environment--the discipline, the available technological resources, the instructor, and the learner--and their interactions with each other.
An analogous program, Math-Tutor, was created after Chem-Tutor, with support from the Rhode Island Board of Governors for Higher Education Fund for Excellence. A new FIPSE grant will fund the development of ChemScholar and MathScholar, two PC-based programs with a more powerful and sophisticated graphical user interface. These programs will actively engage the student in the direct manipulation of scientific objects, concepts, and problem-solving processes. FIPSE is also supporting a project entitled Students Learning and Teachers Working Using Networked Computer Technology.
The Chem-Tutor software system for VAX/VMS, including the Instructor Expert program, is available for $595. User guides may be purchased separately for a nominal fee. Address requests to:
James L. Fasching
Department of Chemistry
University of Rhode Island
Kingston, RI 02881