|Agreeing on the Results to Measure|
The goal of the fourth step of the Compact process is to evaluate your Compact. Evaluation offers your school community an opportunity to determine the strength of your Compact in improving student learning, communication, and capacity building.
The first step in evaluation is to develop a list of questions about the impact of your Compact on student learning, communication, and capacity building. You may want to know, for example:
Continually evaluating your Compact allows you to check on whether the Compact is working as you intended it to work. This evaluation process enables you to catch problems early, so that you can correct your course for stronger and more effective outcomes. Also, as students, parents, and teachers see the progress that is being made, they have a greater incentive to keep working, with tools that help them work better. Moreover, evaluation sends a signal that the activity you are examining is important.
After basic questions are developed, you will want to develop performance indicators that can give you information on factors that may be associated with the success of your Compact. For example, an indicator of "upward movement in students' skills" would be increases in student test scores in reading (see the section following on "Individual Student Performance"). An indicator of "strengthened lines of communication" would be increases in the numbers of families who report that they are informed about the school's reading standards. An indicator of "a growing capacity of families to help their children in reading" would be increases in the percent of families that help their children complete the School-Home Links activities. See Activity Sheet 4A entitled "Evaluating the Quality of the Compact Process" for more performance indicators that can be used to collect such information.
A word of caution: Gathering and analyzing data can be expensive and time-consuming. Therefore, your school should measure only what is important and make certain that what is important is measured. While some data can be collected and analyzed quite easily by members of the Core Team, other information can be more difficult to get and may require the help of your district or state evaluation staff or university partners who specialize in research and evaluation.
|Assessing How Well Your Students Are Reading|
The ultimate goal of your Compact for Reading is for more students to be reading at higher levels of proficiency. Basic readers should be developing into proficient readers, and proficient readers should be acquiring the skills of advanced readers.
You need to look at your students' reading progress on two levels:
Individual Student Performance
Many different kinds of information can be obtained on student performance in reading, ranging from very informal measures of progress, like skills checklists, to more formal measures of performance, such as standardized reading tests administered by the school, typically for the district or state. Each of these measures has particular strengths as well as weaknesses. There is not simply one good test of reading ability.
To measure performance in your school for your students, a range of informal and formal assessments is most useful. Some quick-to-administer, informal assessments enable teachers to easily gauge student progress in early literacy skills and get immediate feedback to guide further instruction. More formal group assessments provide an overall picture of where groups of children stand in reading ability and achievement.
Informal Checklists of Skills
Informal checklists of skills that most children have by grade, developmental, or proficiency level can indicate very quickly which skills the child has or needs further work on and can show at what level a child is reading.
Informal Reading Inventories
These quick-to-administer reading tests, such as the Running Record and the Informal Reading Inventory, permit teachers to determine students' reading strengths, weaknesses, readability levels, and fluency. The Running Record, for example, requires that each child read 100 words of text out loud to the teacher. The teacher notes the time it takes the child to read the passage and the accuracy with which the child reads the passage. Teachers can administer Running Records multiple times during the year to note progress in fluency as well as accuracy in reading.
Student Home Reports
Several times a year, teachers develop grades or scores that reflect students' performance on classroom reading, written work, and classroom tests. Grades are very broad indicators of performance and are more likely to be subjective than other assessments.
Observations of individual students' reading behaviors and attitudes can be helpful in determining reading progress. Observations that obtain information on students' enthusiasm for reading, confidence in reading, and recognition that reading requires hard work and practice can be strong indicators of reading success.
Formal Standardized Reading Assessments
Many states and most large districts administer formal assessments in which performance in reading and other language arts is assessed in one section of the overall assessment. Some examples are:
Your school's performance in reading is determined by adding up the progress made by individual students in reading. A good way to understand your school's reading progress is to chart your students' scores at the school level and across a number of subpopulations in the school (e.g., by grade, ethnicity/race, special populations such as Title I, limited-English-proficient, and disabled students).
When reviewing student scores before and after the implementation of the Compact for Reading, your Core Team should ask about:
|Supporting Learning, Communication, and Capacity Building|
Your Core Team should also look at the extent to which the Compact has led to improved support of learning activities, to improved communication among the partners who signed the Compact and are responsible for improvement in reading in your school, and improved capacity to advance the effective teaching and learning of reading.
At least once a year, the Core Team should conduct an evaluation of how well your school-family partnership is working to fulfill the commitments made in your Compact for Reading.
Activity Sheet 4A provides indicators your school may use to summarize the quality of your compact's implementation. Activity Sheet 4B provides a worksheet for detailing what the Compact partners are doing well and what areas need further improvement.
To determine how well your Compact is working, (1) ask participants how it is working (i.e., conduct surveys); (2) review routinely collected information (i.e., examine administrative records); (3) observe students, staff, and parents (i.e., keep anecdotal notes); and (4) conduct in-depth discussions (i.e., hold focus-group sessions).
Surveys ask school staff, parents, participating community members, and students their opinions about and experiences with reading and learning. Survey responses tell about partner expectations, provide judgments about the quality of learning experiences and materials, and address whether the partners are communicating well and respectfully with each other. Make sure that your surveys are truly representative of your school's population by language, race, and parents' educational level. Remember that everyone's views count. In general, companies strive to have at least half of their questionnaires returned. High-quality surveys receive at least 7 out of 10 returned.
You may use the records your school routinely collects. Routine records may cover students' attendance rates and completion of home learning activities; amount of training for teachers, parents, and volunteers; and levels of participation in the Compact. Such data should be reviewed periodically for accuracy and completeness.
Direct observation provides rich information that cannot be obtained through surveys or written records. Observations reveal student attitudes and behaviors and are often part of the informal reading assessments previously described.
When observing students, observers must have set criteria for what to look for and what criteria indicate success or quality. Furthermore, it is worthwhile to review the materials themselves, to assess their quality and value.
Focus groups are small-group discussion sessions to test and review materials, such as the School-Home Links activities. A group of parents and teachers sitting around a table assume the role of market testers who assess the clarity, appropriateness, and usefulness of materials. Focus groups are also useful for discovering and addressing barriers to communication. Focus groups should be representative of your school's population or, upon occasion, targeted to groups of families who are harder to reach.