Accountability systems that provide rewards, sanctions, or assistance to schools based on student achievement outcomes are currently being developed and implemented in nearly every state. The purpose of this study was to identify and describe these systems, as well as to determine the degree to which publicly available documents clearly articulate whether students with disabilities are included in accountability calculations. A search was conducted on each state educational agency Web site for information pertaining to school-level accountability. State accountability and Title I directors were also contacted to provide additional information. One of the greatest challenges in studying accountability systems is their transitory nature.

Statewide accountability systems designed to promote student achievement are beginning to play a very significant role in the way resources are allocated to schools. The rationale for implementing accountability systems stems from a belief that education can be improved when clear standards for student achievement are communicated to both students and educators, achievement toward those standards is measured, and appropriate consequences are linked to levels of student achievement. The intent of these accountability systems is to promote behavior change among schools in ways that will increase positive outcomes for students.

One of the challenges of researching accountability and assessment systems is that these systems are constantly being revised due to legislative and administrative action. Historically, states and local school boards have determined education and educational accountability policies. However, in the past decade the federal government has played an increasingly prominent role in this area.

The Web sites of major organizations such as Education Commission of the States (http://www.ecs.org), the National Conference of State Legislatures (http://www.ncsl.org) and the Council of Chief State School Officers (http://www.ccsso.org) provide interesting perspectives on the increased attention to assessment and accountability. Many states are attaching high stakes consequences to students for their performance on assessments, such as withholding diplomas (Center on Education Policy, 2002; Guy, Shin, Lee, & Thurlow, 1999; Johnson & Thurlow, 2002) and grade retention (Quenemoen, Lehr, Thurlow, Thompson, & Bolt, 2000), as well as attaching significant consequences to schools and school districts for the overall performance of students (Krentz, Thurlow, & Callender, 2000). The latter approach, known as system accountability, occurs when educators, administrators, schools, or districts are held responsible and consequences assigned for various student performance indicators. For example, school accreditation may be based on test scores; teachers may receive cash rewards for student performance; administrators may be either rewarded or sanctioned based on school improvement, which, in turn, is measured to a large degree by student performance.

As states have proceeded in constructing their student and school or district accountability systems, federal requirements have made it clear that students with disabilities must be included in accountability. Initial efforts to include students with disabilities in assessment systems have realized significant progress in assessment participation and the development of alternate assessments for those students unable to participate in regular assessments (Thurlow, 2000). In order to promote the positive intended consequences of accountability systems for all students, Linn (2000) points to the importance of including all students in accountability calculations. The next step, therefore, is to identify ways in which to include the scores of students with disabilities in accountability systems.

The concept of including the assessment scores of students with disabilities in accountability is simple on the face of the matter, but there are several complications associated with including the scores obtained by students with disabilities that challenge accountability systems. These complications include what have been called "non-standard" administration of assessments (i.e., assessments in which students have used accommodations that are considered to change what the test is intended to measure), and alternate assessments, which generally are designed for students with significant cognitive disabilities, and which therefore seem to be "different" from regular assessments.

Despite these complications, it is important to include students with disabilities in accountability systems, particularly those with school or district consequences. Therefore, it is also important to examine states’ progress toward this end. The purpose of the study reported here was to do just that. We focused on statewide system accountability with rewards, sanctions, or assistance provided for schools or districts. We examined Web sites of state educational agencies to determine whether accountability systems were in place, were being phased in, or were in the development phase. We looked at various indicators used by states to determine rewards or sanctions, how these indictors were determined, whether a growth factor was included, and also noted the various labels applied to schools or districts. We also examined what states are using to determine whether schools receiving Title I services are making adequate yearly progress (AYP).

We then investigated the status of the inclusion of students with disabilities in these systems. With the reauthorization of the Individuals with Disabilities Education Act in 1997 (IDEA 97), states and districts must include students with disabilities in their assessments. They are also required to report on the performance of students with disabilities, both aggregated with other students, and disaggregated. Although it is clear that students with disabilities are required to participate in statewide assessment systems, IDEA 97 did not specifically require that their scores be incorporated into system accountability. Because schools are likely to target resources toward those student groups that are included in the determination of rewards and consequences, a potential consequence of failing to include students with disabilities in accountability systems is that their instructional needs will not be met. Our goal, therefore, was to determine the extent to which students with disabilities are clearly included in accountability systems.

In 2000, we did a similar analysis (Krentz et al., 2000) and found that it was very difficult to locate evidence on state Web sites that clearly stated that students with disabilities were included in accountability systems. Although students with disabilities may have been included in assessments, their scores may not have been included in the accountability systems; most states did not specifically state whether students with disabilities were included when defining the indicators used for accountability purposes.

In 2001, the passage of the Elementary and Secondary Education Act (ESEA), also known as "No Child Left Behind," federally mandated accountability as one of its reform pillars. Annual testing is its cornerstone. While this legislation is in its early stages of being interpreted and implemented, it is apparent that nearly every state will be required to revise its accountability system for all students, including students with disabilities and low-income students receiving services through Title I. According to Wenning, Herdman, and Smith (2002), the required disaggregation of assessment data by various student groups as articulated in No Child Left Behind (NCLB) represents a significant improvement over past practices in which the use of schoolwide averages for accountability purposes frequently masked the performance of various student groups.

As a result of NCLB, many states that have developed accountability systems for schools receiving funds through Title I will now have to broaden their systems to include all students. States may simply include the additional students within their existing accountability systems, or may decide to undergo significant revamping of their current systems in order to comply with ESEA. In several states, the legislature was no longer in session by the time the details of ESEA implementation were shared with state departments of education. Many states have not yet begun to make decisions about how their current policies and procedures will need to change. State policymakers in many states will likely be faced with enacting new legislation to align their current accountability systems with the federal legislation.

In the current study, we additionally examined states’ Title I accountability systems and their measures of AYP that are required by federal legislation. When we started to collect information from states, we were interested in seeing how their state accountability systems were related to the Title I accountability system. As we collected data, not only were states in flux with respect to their accountability systems, but federal clarification through regulations was in process as well. In many states accountability criteria were undergoing revision, and in several others AYP was being altered in order to include all students, not just those of low socioeconomic status.

With changes occurring across the country, the data included in our report admittedly represent a snapshot in time of what states were currently doing or planning to do. Nevertheless, it is an important snapshot to take, as states proceed in their implementation of standards-based educational systems.

Method