The assessment process was generally discussed by teachers and administrators as a set of assessments which exists on two levels, standardized exams at one level and in-class curriculum-based assessments, such as tests and quizzes, at another level. While these tests typically serve different purposes, most teachers, principals, and even parents recognized that these two very different forms of assessment were not totally unrelated to one another.
No formal evaluation of progress towards Goals 2000 currently exists. However, an ongoing monitoring of scholastic achievement in schools in the United States occurs through the National Assessment of Educational Progress (NAEP). Since it began in 1969, NAEP has offered the only nationally representative and continuous assessment of student performance in various subjects (Mullis, Dossey, Foertsch, Jones, & Gentile 1991). The NAEP data, reported in the form of statistics aggregated by states, are used by the states to compare the performance of students in each state to that of the rest of the country.
Students in the NAEP sample, who are 9, 13, and 17 years old, take tests of knowledge and skills in reading, mathematics, science, writing, and history/geography. In addition, students provide information about themselves, their families, and their schools. Proficiency in each of the subject areas is broken down so that the scores of geographic regions (Northeast, Southeast, Central, and West), as well as the scores of each state, can be differentiated.
Of all the standardized exams which schools use or are required to participate in, the NAEP test appears to have the least impact on local school policy. It was seldom mentioned by school teachers or administrators when we asked them about the role of standardized exams.
Most standardized exams in use in the United States have been created by a state department of education or a professional testing organization. They are used widely in U.S. schools to assess achievement levels. In fact, exams such as the California Achievement Test, the Iowa Test of Basic Skills, and minimum competency tests developed by various states have become a regular feature of the academic year for most schools.
We found that the California Achievement Test, the Iowa Test of Basic Skills, and the Stanford Achievement Test were administered by many of the Case Study schools, particularly at the elementary and middle school levels. Teachers noted that test results from these exams were often used to assess the achievement of individual students and to make recommendations concerning placement in courses at the middle and high school levels. For example, sixth-grade students at Vanderbilt Middle School took the Iowa Algebra Aptitude Test. Those who received a certain score then took another standardized achievement test. Students who achieved the prescribed math and combined (math and verbal) scores were allowed to participate in an advanced-track math class at the neighborhood high school. Although this was an unusual program in the schools we visited, we did find that standardized test scores as well as teacher's recommendations were often used to divide middle school students into math, science, and language courses of different levels of difficulty. Schools also sometimes used the test results to develop new classes for target populations, such as a prealgebra two-level class.
Statewide minimum competency tests are the primary method by which states currently assess their standards and their schools. These tests assess basic skills in reading, mathematics, and English (although Spanish-language versions are available in some states for nonnative speakers of English). In the lower grades they are used to monitor learning, while at the high school level they may set minimum achievement standards that are required for high school graduation.
As of 1992, 40 states had adopted mandatory minimum competency testing at several grade levels (U.S. Department of Education 1995). In this study, we found that two of the three states in which the field research sites were located required students to pass a minimum competency test for high school graduation. In addition, the minimum competency tests were used at all grade levels to assess individual and group (class) achievement levels and to monitor overall school achievement both through successive years and in comparison to other local schools.
Teachers from around the state in which Metro City was located were involved in the development of the State Goal Assessment Program. This program seeks: to measure what students know and are able to do with respect to the seven goals for learning; to use student test results to describe how students, schools, and districts are performing in mathematics in comparison to the state and nation; to monitor mathematics progress of schools, districts, and the state over time; and to generate information on mathematics learning outcomes that will be used for accountability, policymaking, and school improvement.
A brief description of the assessment test used to evaluate achievement in math serves as an example of the testing format and procedure of this particular standardized state test. Tests for grades 6, 8, and 10 contained 70 items, which were divided into two 40-minute test sessions of 35 items each. The tests contained an equal number of multiple choice items (10) for each goal being assessed.
State requirements for East City and West City schools specified that an end-of-year exam be administered to students in each grade or class. In West City, these tests were offered in Spanish as well as in English because of the high percentage of nonnative English-speaking students enrolled in schools in this state. The tests were used primarily by the district to assess the overall achievement level of the schools. The end-of-year tests administered in the schools in East City were, in contrast, used to assess the achievement level of individual students. As one principal stated, "we are being much more specific about skills we want kids to master." He also stated that a students' performance was rated along four levels of mastery of the subject: excelled in mastery (level four); just mastered (level three); not mastered (level two); and has significant problems (level one).
The tests and rating criteria were new and the principal indicated that he did not yet know what they would do with students who received the lowest rating, because they were a year-round school and did not have the typical 3-month summer break that lent itself to remedial work. State end-of-year tests here were described as comprising a combination of fill-in-the blank, multiple-choice, and open-ended questions. In order to receive a level three rating, students must achieve the equivalent of 70 percent correct. We were also told that the end-of-year (or course) test at the middle school would generally count for 20 percent of a student's overall class grade.
Views on standardized exams. The use of standardized tests was among the most passionately discussed topics of the Case Study research among parents, teachers, and administrators. Opinions varied widely, according to each person's view of education, their role within the education establishment, and experience with standardized tests. In spite of the differences over content and form and the uses and abuses of test results, most agreed that they are important as a means of evaluating the extent to which certain standards have been met and of establishing a degree of comparability between schools throughout the state.
We found that principals, teachers, and parents were all well aware of the multiple uses of results of standardized tests. While some of the standardized tests were believed to be useful for instructional guidance and student placement in courses of varying degrees of difficulty, the end-of-year exams mandated by the state were generally believed to provide a less-than-accurate assessment of their schools and their teachers. This view was shared by teachers and administrators at all the schools we visited, regardless of the school's history of achievement. Teachers at most schools viewed the state-mandated tests as a "necessary evil," something to be tolerated. However, teachers from schools with many low-achieving students expressed a deep sense of resentment and unfairness at the public comparison of schools based on these test results. These feelings came across in many different interviews. A principal at South Central said the tests are "unfair to me and my staff." A teacher at Parks Elementary school stated, "comparisons between the worst and the best schools are ridiculous." Teachers at these schools often pointed out that their students faced many social problems that interfered with learning and that it was impossible to judge the effectiveness of the teachers or the school by comparing them with schools whose students did not face the same obstacles.
Despite the general dislike of state-mandated testing programs, most of the teachers and principals in Metro City said that they felt some pressure to maintain or improve the students' performance each year. Test results for each school were published in the local newspaper, and parents and community members were said to hold the schools accountable when the results did not meet expectations. Teachers and principals at Rockefeller Elementary and Vanderbilt Middle Schools in particular indicated that parents compared their school's test results to other neighborhood schools and complained if their score results were slightly lower. The chairman of the math department at Hamilton said that each year he must go before the school board, answer questions, and defend what the department is doing. "The roughest part is answering the question: how come you didn't improve as much this year as last year?"
Test preparation was not an issue in East City schools, since the state end-of-year test was more closely linked by the state to the curriculum being taught in the schools. According to the principal at East Middle School, "the state is moving away from norm-referenced testing toward criterion-referenced testing based on the state curriculum which is expected to be taught."
Although there were differences between the schools, teachers and principals at most of the schools in Metro City said that students were given some in-class preparation for the state test. In several schools, old copies of test questions were used to prepare students for the type of questions they were likely to encounter. Although it was difficult to get a firm idea of the amount of time teachers spent doing this, it was clear that at some schools the test preparation was intensive for several weeks before the test was administered. School administrators at Vanderbilt Middle School downplayed the school's activities in preparing students for the test, but we found that both the assistant principal and several teachers had materials for providing such preparation. In addition, one of the students we interviewed said that she and her classmates had received test preparation "from day one." Another student at this school reported that his teacher had been preparing the class for the state test for 3 months before the exam. Reports from other schools indicated that teachers may spend 1 to 2 days a week going over practice questions in the weeks proceeding the exam. Thus significant amounts of time appeared to be spent in preparing students for these tests.
Although many teachers commented on the lack of fit between the state assessments and the curriculum, the principal at Vanderbilt Middle School pointed out that since the State Goals Assessment Program was not specifically designed to measure coursework, a "smart" local district would try to figure out what it was testing and try to incorporate the material into the local curriculum.
In contrast to standardized tests, which were administered in schools once or twice a year, in-class tests and quizzes were found to be administered at the elementary and middle school levels every 2 to 3 weeks or at the end of every unit covered in the textbook. The frequency of quizzes was more variable. However, among math teachers in particular, quizzes were often given on a weekly basis to keep students focused and up-to-date on the material in the week's lesson. In-class tests at the high schools were said to be administered less frequently than at the middle and elementary schools and sometimes consisted of only a midterm and a final test at the end of the semester. In general, teachers at all school levels had considerable discretion over both the frequency and the content of in-class tests.
A teacher at Parks Elementary, like many other teachers, said that she used the unit tests in math that come with the textbook series. She also gave a lot of quizzes which she developed herself. Midtown Elementary School used the same textbook series as Parks did for math, and the teachers there also said that they used the exams that came with the text. They added that at their school "70 percent must pass each unit test or the unit must be retaught." In contrast to these elementary schools, heavy reliance on prepackaged curriculum and tests, the teachers at Rockefeller Elementary School rarely relied on textbooks for instruction. The Rockefeller teachers also constructed performance-based tests and used student constructed portfolios to assess learning rather than traditional paper and pencil tests.
Many states and school districts are turning to performance-based assessments to complement traditional testing programs. Performance-based tests require students to demonstrate what they know by responding to problems they have not previously met by calling on the skills and information they are supposed to have acquired in class. The tests are based on the premise that testing should be closely related to things students are learning and may include portfolios of student work, exhibitions, science experiments, oral interviews, and performances.
The state in which our primary research site was located recently informed schools that they must develop, based on the state's goals for learning, two forms of assessment for the measurement of desired outcomes in six curriculum areas. One of these two forms of assessment must be performance-based. According to one principal, "the schools are just getting this off the ground, and one problem is that the staff must undergo training in alternative assessment."
Among the teachers with whom we spoke, there seemed particular eagerness among science teachers for performance-based assessments and dissatisfaction with more traditional types of tests. A common complaint from science teachers was that the memorization and testing of vocabulary was boring and not relevant to what students were learning in their labs.
Standards in the classroom. Course grades were typically determined by the results of both quizzes and exams. However, teachers often had other criteria, which they included in the final grade decision. For example, a middle school teacher in Metro City said:
There are three important factors, which I consider in addition to test and quiz scores. They are: whether the student has asked questions when they are lost, whether they have participated in the classroom, and whether they have completed homework assignments. Students who do well in these areas can be bumped up to a higher letter grade if they are borderline.
At Rockefeller, students created their own portfolios with guidance from the teacher. Students were also given an opportunity to set goals and reflect on their work. To inform parents of their student's progress, teachers wrote narrative reports twice a year about each child in each subject area. The narrative report coincided with a parent conference at the middle of the year, and at the end of the year students presented their portfolio work to their parents, and the parents received a second narrative report from the teacher.
Although teachers indicated that it was possible to retain a student in a grade as a consequence of poor performance, it was clear that it was a more feasible option in some schools than in others. In East City Middle School, the principal said that a student's course grade must average 70 percent or they may be required to repeat the class. Remediation was often described as occurring over summer vacations or during intersession breaks (for schools on a year-round calendar). However, in some schools students were passed from grade to grade despite their low levels of achievement. Teachers at the schools with the poor test results expressed a strong concern about the effects of retention on students' self-esteem. At Parks Elementary, a teacher said he would not take the responsibility of holding students back in fifth grade when their classmates were leaving for the middle school. "If I were a kid who had been held back, I would stop coming to school. That would be too big a blow." A science teacher at Metropolitan School stated firmly, "I will not fail students who put forth effort."
The principal of South Central Vocational High School indicated that a large percentage of the student population at South Central read at the elementary school level when they entered high school (ninth grade). These students had been passed through the grades regardless of their level of achievement on in-class or standardized tests.
Although some students were passed on from grade to grade regardless of their level of achievement, most the teachers and principals we spoke with in Metro City had clear criteria by which they judged student performance, were identifying students who needed extra assistance, and were contacting parents about students' academic problems to enlist their support.
Statistics on funding sources for public elementary and secondary schools for the 1992-93 school year indicate that nationally 46 percent of school funding came from the state level, 47 percent from the local level, and 7 percent from the federal level (USDE 1995). However, according to the same source, the distribution of funding for schools in the states in which our research sites were located was quite different. Schools in East State and West State received approximately 62 percent of their funding from the state. However, schools in the state in which our primary site, Metro City, was located, received approximately 29 percent of their funding from the state, 7 percent from the federal level, and the remaining 62 percent from local tax revenue in the 1992-93 school year. This heavy reliance on local tax base revenues meant that school budgets varied across school districts, with the high-income, high property value school districts able to support higher spending levels for their schools.
State money, local money, and federal money were all funneled to individual schools through the district office. Schools typically received state and local funds from the school district based on the number of students enrolled. Federal funds were awarded to individual schools, although the application generally went through the school district, based on the number of students they enrolled who had special needs. Schools enrolling special education students, students from low-income families, and students whose native language was not English received federal funding to support programs for these students. Although the funds were distributed by the districts, it was the responsibility of the principals and vice principals of the individual school's to manage the budgets for each of these allocations. Principals in Metro City indicated that this recent shift to school-based management had its benefits, but had also greatly increased the administrative workload in the schools. Budget expenditures were approved by local school councils (boards) and, in some cases, also by the district's school board.
The fact that schools within particular districts in Metro City shared the same tax base and received over half of their funding from local tax revenues meant that schools within the same school district generally worked with fairly equitable budgets. However, there were often large differences in funding between districts in Metro City. These differences in funding were often reflected in differences in schools' physical facilities, educational resources, and teachers' salaries.
In the 1993-94 school year, per pupil spending was $3,835 in the school district to which Parks Elementary School belonged. In contrast, per pupil spending, for the same time period, was $8,035 in the district to which Rockefeller Elementary School belonged. A brief description of two classrooms from these schools in Metro City illustrates the markedly different environments in which children attend school.
Parks Elementary School is located in an inner-city neighborhood surrounded by apartment buildings, built on average over 70 years ago. In contrast, Rockefeller Elementary School serves a very affluent suburban neighborhood, surrounded by large single-family homes with extensive lawns and well-tended yards. At Rockefeller, the windows of the classrooms open to a view of a nearby woods which surround the school. The classrooms are carpeted, include a round worktable for projects, a television and VCR. In contrast, the classrooms at Parks were crowded with aging chairs and desks, the wooden floors provided no sound absorption, and a window made of glazed-over plastic blocked the view to the outside.
At both the school district and the school, decisions are made in regard to areas such as curriculum content, the selection and use of textbooks, homework requirements, and the use of technology in the classroom. These decisions, which establish school policy, inevitably affect some of the basic operations related to learning and standards.
The elementary schools we visited differed in the amount of structure that they imposed on teachers. These differences were often reflected in the degree to which they relied on textbooks for the delivery of the curriculum and the degree to which teachers were required to demonstrate to school administrators that they had successfully presented the curriculum. The teachers in the highest-achieving elementary school had the most freedom from structure. They were allowed, and even encouraged, to teach without relying on a specific textbook. Instead, they used materials from a variety of sources. Teachers at this school also described their own standards for the curriculum they taught as being above those required by the state's curriculum guidelines. As an example of these higher standards, they spoke of a project, which they were working on that involved the creation of a framework for a new math curriculum based explicitly on NCTM standards.
On the other hand, teachers at the middle- and low-achieving schools said they used textbooks that have been chosen specifically by the school (or a committee of teachers at the school) to teach the curriculum. Some of the these schools also required teachers to demonstrate through in-class tests that at least 70 percent of their students had learned the information presented in each of the units; otherwise they were required to reteach the unit. Most of these teachers said that while they were free to add to the curriculum using materials from other sources, coverage of the required curriculum was first priority.
Teachers in Metro City and West City elementary schools indicated that the textbooks were often specifically chosen because they incorporated materials required by the state's curriculum guidelines. In fact, in West City, the district gave the schools a list of approved textbooks, which incorporated materials specified by the state's curriculum guidelines.
Most of the elementary schools we visited had explicit homework policies. Often these were in the form of general homework guidelines for each grade. Very little homework was generally assigned in the first and second grade, but by third grade, students were usually required to do approximately 30 to 45 minutes per day and by the fifth grade, students could expect up to 1 hour of homework per day. In addition, most schools recommended that students read for pleasure at least 30 minutes a day at home.
The one elementary school that differed significantly from the above pattern was the school with the lowest scores on the state assessment, Parks Elementary. Homework assignments were a debated issue at Parks. Some members of the parents' council argued that it was unfair to assign homework when some of the children did not have parents who could help them. As a result, many teachers did not assign homework. In response to one of our questions, the principal replied:
All I can say is that it's easy to teach kids who are well prepared, who have the parents helping them, who have the advantages of the average middle-class child. It's hard to teach kids who don't have advantages, who can't go home and say `help me.'
Technology in the form of computers and calculators was used by all of the elementary schools we visited. Although all had computer facilities for students, the highest-achieving school, located in the wealthiest school district, had the best computer facilities of any of the elementary schools. Computer learning games were commonly used at most of the elementary schools, although the frequency with which students had a chance to use computers varied from school to school. In addition to the use of computers for in-class instruction, teachers at Midtown Elementary also used computers to help students develop library skills.
We also found that calculators were incorporated into the math instruction in most of the schools we visited. However, they did not play a dominant role. For example, teachers at Rockefeller said that calculators were used for "drill and skill" work and to familiarize students with the different mathematical operations.
Curriculum policies at the middle school level often pertained to textbook selection and to the division of students into course levels by their level of ability. We found that, as in the case of the elementary schools, the high-achieving middle school (Vanderbilt) gave teachers the greatest autonomy in the development of courses and the use of textbooks. Vanderbilt also was the most active school in coordinating curriculum content with the high school to which their students would go following the eighth grade.
We found that most of the middle schools provided at least some courses at more than one level of difficulty. The criteria used to divide students into different course levels most often were the student's previous grades, their test scores on standardized exams, and their teacher's recommendations. Students were most commonly divided into ability levels for mathematics, science, and language arts classes, although there was variation among the schools. Vanderbilt tracked students into five levels of math courses in the eighth grade, and students with the highest test scores were offered an opportunity to begin taking math classes at the high school through a special program.
At King Junior High, students were tracked in math and in languages but not in other subjects. Math was offered in three levels beginning in the seventh grade. Similar tracks in math were found in East City Middle School. However, in the lowest-achieving middle school that we visited, math was offered at only two levels. The students enrolled in accelerated math were part of a special program at this school to separate and challenge "those students who are working at or slightly above grade level, and raise the expectations and the work requirements." The parents of students in this school were also required to commit to support their student in writing when their student joined the accelerated track.
Most of the middle schools we visited had explicit homework policies that were spelled out in their Parent/Student Handbook. The eighth-grade students we spoke to in Metro City and in East City said that they could expect 1 to 2 hours of homework per night. Only one school did not have a written homework policy. In this school, the length of homework assignments was left up to the discretion of the teachers.
Teachers we spoke to also indicated that they adjusted homework requirements to the students in the course. For instance, one teacher at King Junior High, a middle-achieving school, said:
I let prealgebra 1 students do homework in class, but not the prealgebra 2 or scholars math kids. If I didn't let them do it in class, 80-90 percent of the prealgebra 1 kids wouldn't do it, because they can't keep track of their stuff.
Completion of homework was also a controversial issue at the lowest-achieving middle school, Metropolitan, where completion of homework assignments was a chronic problem among students who lacked parental support.
All of the middle schools in Metro City had computers available for in-class instructional purposes. However, the actual use of computers varied a great deal from school to school, and some schools had much more extensive computer facilities and offered more opportunities for students to interact with computers. One example of in-class computer use was demonstrated by a science teacher at Metropolitan, who used a computer program that simulated the dissection of a frog in her biology course. Another very different usage was observed at Vanderbilt, where students learned how to use an electronic spreadsheet in the computer lab as part of their math lesson. Vanderbilt also offered a computer course as an elective course for students; however, we did not find that this was a common offering at other middle schools.
Calculators were used in the middle schools we visited, and most of the middle school math teachers we spoke with indicated specifically that they allowed students to use calculators as a time-saving device, particularly when working with fractions.
We found the greatest differences between schools at the high school level. The highest-achieving schools, such as Springdale and Hamilton, offered the greatest variety of courses and more levels of courses in subjects such as math and science than those offered at South Central or Uptown.
According to its principal, Springdale has traditionally upheld high standards and resisted attempts to erode those standards. In addition, Springdale offered over 200 hundred courses, ranging from advanced-placement courses in calculus and astronomy to vocational courses in food service management and interior design.
At the opposite end of the spectrum were the schools with the lowest-achieving students, such as Uptown High and South Central Vocational High School. Uptown had a wider range of students than South Central in terms of levels of ability and had adapted its course offerings for this range. Among the courses were many math and science courses at two levels, regular and honors. Calculus, chemistry, and physics were also offered at the advanced-placement level. Both Uptown and South Central enrolled students in several 2-year vocational programs, such as metal working and auto mechanics. South Central also offered regular academic courses; however, the principal at South Central said that many were remedial courses, since the majority of their students required courses at this level. A consistent response in the interviews with teachers at South Central was that they were "doing what we can."
Homework expectations were quite high at the high- and middle-achieving schools we visited, although not all schools had a written policy on homework. At Springdale, the school catalog stated that students can expect "an average of 40 minutes of out-of-class homework per subject each day, and that students in basic, regular, and accelerated classes are expected to spend about the same amount of time on homework." At Hamilton, we were told the amount of homework assigned depended upon the level of the course. The prevailing view at Hamilton was that "it was better to assign quality problems rather than just a large number of problems." Again, in contrast to these two schools, were the remarks of teachers at South Central, who indicated that they did not often assign homework because the students seldom completed the assignment. They went on to explain that most students left their books in their lockers, because they did not want to be seen on the street with a textbook under their arm. There was a stigma attached to being a good student. Teachers said that such students were ostracized and "put in their place" by their classmates.
As at the elementary and middle school levels, high schools generally encouraged the use of calculators and computers. Students in accelerated math courses at most schools were using graphing calculators, and computer usage was often incorporated into the curriculum and instructional methods of particular courses. At Hamilton, computer courses were offered through the mathematics, fine arts, business education, cooperative education, and industrial technology departments. At the lowest-achieving school, South Central, computers were used for testing students' academic abilities and often to offer remedial instruction.