Nearing the Target in Disaggregated Subgroup Reporting to the Public on 2004-2005 Assessment Results

Technical Report 46

Gretchen R. VanGetson • Martha L. Thurlow

April 2007

All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

VanGetson, G. R., & Thurlow, M. L. (2007). Nearing the target in disaggregated subgroup reporting to the public on 2004-2005 assessment results (Technical Report 46). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Tech46/


Table of Contents

Executive Summary
Overview
Method
Results
Assessment Participation in 2004-2005
Assessment Performance in 2004-2005
Assessment Performance: Trends
Other Information Collected for 2004-2005
Discussion
References
Appendices


Executive Summary

This report marks the ninth analysis conducted by the National Center on Educational Outcomes (NCEO) of the public reporting of state assessment results for students with disabilities. This is the fourth analysis that NCEO has conducted since the passage of the federal No Child Left Behind Act of 2001 (NCLB). Overall, all 50 states and 5 unique states reported some disaggregated assessment results for students with disabilities for the 2004-2005 school year. Thirty-six states reported participation and performance data for all their general assessments, 12 reported participation and performance data for some of their general assessments, and 2 reported only performance data. When considering only assessments that were part of states’ NCLB accountability systems, more states reported all assessment data publicly: 44 states reported participation and performance data for all these assessments, 4 reported participation and performance data for some of these assessments, and 2 reported only performance data for all of these assessments. For alternate assessments, 47 states reported some disaggregated assessment results for students with disabilities. Forty-one states reported both participation and performance data for their alternate assessments, 2 states reported these data for some of their alternate assessments, 3 states reported only participation data, and 1 state reported only performance data for their alternate assessments. Three states did not report participation or performance data about their alternate assessment.

When reporting general assessment participation data for students with disabilities, 43 states reported the number of students tested, the most common way of reporting participation information, while 35 states reported either the percent of students tested or not tested for at least one of their assessments. For states’ alternate assessments, the most common way of reporting participation information was to give the number of students tested; this was the approach of 41 states. Thirty-eight states gave a percentage (either a percent tested or percent not tested).

In terms of reporting general assessment performance data, all 50 states reported on the percent of students with disabilities who were proficient, and 40 states reported the number of students with disabilities in each achievement level. For states’ alternate assessments, 42 states reported on the percent of students with disabilities who were proficient, while 27 states reported the number of students with disabilities in each achievement level. When we examined student performance on general assessments, we found that sizeable and variable gaps existed between students with disabilities and general education students. Gaps in the percentage of participating students reported as proficient tended to be larger at higher grades; however, meaningful comparison of these gaps across grade levels is limited by differences across grade levels in both the composition of the compared subgroups (Bielinski & Ysseldyke, 2000) and by differences in the assessments themselves at different grades. This difference by grade level in the gap in rate of proficiency was not present in the alternate assessment performance data. Percentages of students with disabilities achieving proficiency on the alternate assessment were typically higher than on the general assessment. In examining seven years of data from states that had publicly reported information using the same assessment (11 states), we found that, in general, there was a trend in both reading and math toward higher rates of proficiency among participating students with disabilities.

Overall, this report reinforces what was found in 2003-2004. States continue to improve their public reporting practices, especially for alternate assessments, but it is gradual, and there are still improvements to be made. This report discusses the results of the study and provides recommendations for how states can continue to improve their public reporting practices.


Overview

The end of the twentieth century saw education shift toward holding all students to high standards, and holding all schools and school districts accountable for students’ progress toward those standards (Goertz & Duffy, 2003). For example, Title I of the Improving America’s Schools Act (IASA) of 1994 required states to create assessment systems aligned with high standards to measure student progress at least once in elementary, middle, and high school in both reading and mathematics (Goertz & Duffy, 2003). Also in 1994, the Elementary and Secondary Education Act (ESEA) required that students with disabilities be included in standards-based assessment systems, answering a call from researchers that stressed the importance of including all students (McGrew, Thurlow, Shriner, & Spiegel, 1992; Zlatos, 1994).

It was not until 2001, when ESEA was reauthorized as the No Child Left Behind Act (NCLB), that standards-based assessment and accountability solidified its presence in the educational landscape. Under NCLB, by the 2005-2006 school year, any school accepting federal dollars must demonstrate adequate yearly progress in the number of students meeting proficiency standards in the areas of reading and mathematics in grades 3 through 8 and once between grades 10 and 12 (Peterson & Young, 2004). The ultimate goal is to have all students achieve proficiency by 2013-2014 (Goertz & Duffy, 2003). Further, these results must be disaggregated by student group (i.e., gender, disability, limited English proficient, ethnicity, economic status, migrant status); included in the state’s accountability system; and publicly reported in a clear, timely, and useful manner (Peterson & Young, 2004). “Public reporting is the most basic form of accountability” (Goertz & Duffy, 2003, p.6) in that it allows for public knowledge of and school responsibility for student achievement. Public reporting of state assessment participation and performance information for students with disabilities has been tracked by the National Center on Educational Outcomes (NCEO) since 1997.

Not surprisingly, since the passage of NCLB, there has been an increase in the number of states that publicly reported participation and performance data for all of their assessments; there was a jump following implementation from 28 in 2000-2001 to 35 in 2001-2002. This reporting hit a plateau in 2002-2003 and 2003-2004 (36 and 35 states reporting, respectively) (Klein, Wiley, & Thurlow, 2006; Thurlow & Wiley, 2004; Thurlow, Wiley, & Bielinski, 2003; Wiley, Thurlow, & Klein, 2005). The number of states that reported these data for all of their alternate assessments has also shown considerable improvement. For example, while 22 states reported this information in 2001-2002, 33 states reported it in 2003-2004 (Klein et al., 2006; Thurlow & Wiley, 2004).

The 2004-2005 school year was the third year that states were required to report on the performance of students with disabilities on standards-based assessments, and the last year before states were required to test in all grades 3 through 8, and once between grades 10 and 12. This report marks the ninth in a line of NCEO reports that document state public reporting practices, and pays particular attention to present and upcoming federal requirements in public reporting. Additional analyses were conducted as well, including an examination of accommodations reporting, a look at disaggregated student group reporting, and a Web site analysis to determine the accessibility of states’ public reporting of participation and performance data.


Method

We began our search for information by reviewing every state’s Department of Education Web site as well as the Web sites for the 11 unique states (i.e., American Samoa, Bureau of Indian Affairs, Commonwealth of Northern Marianna Islands, Department of Defense Education Activity, District of Columbia, Federated States of Micronesia, Guam, Palau, Puerto Rico, Republic of the Marshall Islands, Virgin Islands). We began collecting data in October 2005 and collected information for the 2004-2005 school year. We recorded the names and other information about the assessments that were administered, documented whether participation and performance information were reported for students with disabilities, and noted any other subgroups that were included in disaggregated reporting. We also examined the way in which participation and performance were reported, whether participation and performance information were reported for students who took the test with accommodations, and how many “clicks” it took to arrive at disaggregated assessment results from the Department of Education Web site homepage. In our initial review, a little over half of the states had already posted their 2004-2005 assessment data online in a way that made the data easy to locate and understand.

On February 10, 2006 we mailed a letter to each state director of assessment (see Appendix A) outlining our findings from the state’s Web site. We asked directors to review our findings, correct any misinformation, and provide the public document or Web page in which the correct information was available. We asked that they send us these changes by March 3, 2006. We received replies via fax, e-mail, letters, or phone calls. Many states directed us to a Web page that we had not found in our search. While a few sent paper copies of information, we were able to find this information on state Web sites as well. A few states gave us dates by which they expected their disaggregated assessment results to be posted. Overall, we received responses from 22 directors of assessment.

To ensure that our findings were as accurate as possible, we followed up these efforts with a letter to each state’s director of special education (see Appendix B). These letters were mailed on March 24, 2006. The letters asked the directors to review our findings and make any changes by April 14, 2006. For the 17 states from which we had already received a response from the director of assessment, we noted that in the letter by stating that “these tables have been verified by your state’s Assessment Director; please notify me if you have anything to add.” For states from which we did not hear from the director of assessment, we sent the same letter to the director of special education as we had sent to the director of assessment. Of the 50 states and 11 unique states to which we sent letters, 32 responded with either corrections or to verify that the information was correct.

Finally, there were still 10 states and 7 unique states for which we had heard back from neither the director of assessment nor the director of special education. For 9 of these states we found information on students with disabilities for all of their general and alternate assessments. For the remaining state, we reported all of the information we could find on its Web site. The unique states were not included in the full analysis. Also, a few states directed us to their Part B State Performance Plan (SPP) for information. The 2004 reauthorization of the Individuals with Disabilities Education Act (IDEA; PL 108-446) required states to create a long-term special education performance plan and:

Make the State’s performance plan available through public means, including by posting on the website of the State educational agency, distribution to the media, and distribution through public agencies. [Section 616©(ii)(I)]

Indicator number three of Part B includes the participation and performance of children with disabilities on statewide assessments, providing a form of reporting of these data. In this report, data from SPPs are provided only for states that specifically asked us to review their SPP.


Results

Characteristics of State Assessment Systems

Appendix C lists all the 2004-2005 state mandated general assessments that we identified for the 50 states and the 11 unique states. This list includes the state, the name of the test, the grades and content areas tested, whether the state had publicly available disaggregated participation and performance data for students with disabilities, and whether the assessment results were used for state accountability purposes. For the 50 states, we identified 97 different statewide assessments and 107 assessments total. (Ten are repeat assessments – three Terra Novas and seven ITBSs.) Thirty-three states had more than one general assessment. For the 11 unique states, six different assessments and eight total assessment systems were found; only one of these unique states gave more than one general assessment.

Because few unique states publicly reported complete assessment data, the following results include only information from the 50 regular states. Figure 1 breaks down the 107 testing systems by type: norm-referenced tests (NRT), criterion-referenced tests (CRT), exit tests used as a gate for graduation or earning a particular type of diploma (EXIT), and hybrid tests that combined standardized NRTs with additional state-developed test items (NRT/CRT). Exit exams were included in a separate category only if the state had a distinct test designed specifically for high school completion requirements. However, it should be noted that some states’ CRTs or NRTs also include an EXIT component.

Figure 1. Types of General Assessments

Chart showing types of general assessments

Criterion-referenced tests (CRTs) comprised 63% of all the assessments that states administered in 2004-2005. In fact, only seven states (Arizona, Delaware, Florida, Iowa, Missouri, North Dakota, and South Dakota) did not administer a separate CRT, though six of those states administered a NRT/CRT hybrid test (only Iowa administered solely an NRT). Norm-referenced tests comprised 17% of the assessments, exit exams comprised 13%, and 7% were a NRT/CRT hybrid. These numbers are similar to the 2003-2004 assessment pattern in which 61% of the assessments were CRTs, 16% were NRTs, 16% were exit exams, and 7% were hybrids (Klein et al., 2006).

States That Reported Disaggregated Regular Assessment Data for Students with Disabilities

Figure 2 summarizes the different ways in which regular assessment data were reported in all 50 states. Overall, 72% percent of states reported disaggregated participation and performance data for students with disabilities for all their assessments, 4% percent reported performance data for all assessments (but not participation data), and 24% percent reported participation and performance data for some, but not all, of their assessments. Figure 3 indicates how each of the 50 states reported their disaggregated participation and performance data for students with disabilities.

Figure 2. States that Disaggregated Assessment Results for Students with Disabilities

Chart showing states that disaggregated assessment results for students with disabilities

Figure 3. States that Reported 2004-2005 Disaggregated Regular Assessment Results for Students with Disabilities

Map showing states that reported 2004-2005 disaggregated regular assessment results for students with disabilities

Figure 4 shows the number of states that reported participation and performance data for the assessments that were included in their statewide accountability systems. In many states, only a subset of assessments was part of their NCLB accountability system. When we examined just the NCLB assessments, we found that 44 states reported participation and performance data for students with disabilities on all of these assessments. The states that reported disaggregated data for their regular accountability assessments did so regardless of whether they had just one assessment or multiple assessments (i.e., 21 of the 44 had more than one assessment included in their accountability system), and regardless of whether they tested in just a few grades or in as many as 10 grades. Of the four states that reported participation and performance information for some of their accountability assessments, Mississippi, Montana, and Utah were missing participation data for at least one test.


Figure 4. States that Reported 2004-2005 Disaggregated Results for Students with Disabilities in their NCLB Accountability Systems

Map showing states that reported 2004-2005 disaggregated results for students with disabilities in their NCLB accountability systems

Unique States That Reported Disaggregated Assessment Data for Students with Disabilities

This report is only the second to include unique states in the analysis of publicly reported data for students with disabilities. Because many of the unique states did not have information that was found to be publicly reported, we will only briefly mention them here. Table 1 contains a summary of the unique states and whether they reported participation or performance data for students with disabilities. More detailed information is included in Appendix C for general assessments and Appendix D for alternate assessments. Of the 11 unique states, 5 states publicly reported disaggregated data on the participation and performance of students with disabilities on statewide assessments: American Samoa, Bureau of Indian Affairs, Commonwealth of the Northern Mariana Islands, District of Columbia, and Guam. For all of these unique states, with the exception of Guam, these data were gleaned from the SPPs.

Almost all of the unique states administered an NRT, such as the Stanford Achievement Test, 9th Edition (District of Columbia) or 10th Edition (American Samoa, the Commonwealth of the Northern Mariana Islands, Guam). The only unique state to administer a CRT was the Commonwealth of Northern Mariana Islands. In looking at the reporting practices of the five unique states that reported participation and performance data for students with disabilities, all reported the number and percent of students with disabilities who participated in statewide assessments. American Samoa, the Commonwealth of Northern Mariana Islands, and Guam all reported both the number and percentage of students with disabilities who were proficient, while the Bureau of Indian Affairs and the District of Columbia reported only a percent. Further, American Samoa, the Commonwealth of Northern Mariana Islands, the District of Columbia, and Guam all administered an alternate assessment for students with disabilities covering the same content areas and grade levels as the regular assessment.


Table 1. Unique States that Reported Disaggregated Participation and Performance Data for Students with Disabilities

State

Regular Assessment

Alternate Assessment

Participation

Performance

Participation

Performance

American Samoa

Yes

Yes

Yes

Yes

Bureau of Indian Affairs

Yes

Yes

   

Commonwealth of the Northern Mariana Islands

Yes

Yes

Yes

Yes

Department of Defense Education Activity

No

No

 

 

District of Columbia

Yes

Yes

No

No

Federated States of Micronesia

No

No

 

 

Guam

Yes

Yes

Yes

Yes

Palau

No

No

 

 

Puerto Rico

No

No

 

 

Republic of the Marshall Islands

No

No

 

 

Virgin Islands

No

No

 

 


 

States That Reported Disaggregated Alternate Assessment Data for Students with Disabilities

Appendix D lists all the 2004-2005 state mandated alternate assessments that we identified. A total of 47 states reported some data for students with disabilities’ participation or performance on an alternate assessment. There were a total of 59 different alternate assessments administered in 2004-2005; six states administered more than one alternate assessment. As shown in Figure 5, results from our Web searches and mailings revealed that 41 regular states publicly reported both participation and performance data at the state-level for their alternate assessment. An additional three states reported participation only, one state reported only performance data, and two states reported some participation or performance data. Three states (i.e., 6% of all states) did not report any type of information about their alternate assessment. However, 80% of states did report both participation and performance for their alternate assessment, which is an increase over 66% in the 2003-2004 school year. Figure 6 illustrates how each state reported alternate assessment participation and performance data.

Figure 5. States That Disaggregated Alternate Assessment Results for Students with Disabilities

Chart showing states that disaggregated alternate assessment results for students with disabilities

Figure 6. States That Reported 2004-2005 Disaggregated Alternate Assessment Results for Students with Disabilities

Map showing states that reported 2004-2005 disaggregated alternate assessment results for students with disabilities

 


Assessment Participation in 2004-2005

General Assessment Disaggregated Participation Results for Students with Disabilities

Among the states identified as providing participation data for students with disabilities, the way in which this information was reported varied (see Appendix E). Figure 7 illustrates the number of assessments with disaggregated participation data and how those participation data were reported. Information is presented in terms of the number of assessments for which participation data were available, not in terms of the number of states. For example, in Alabama there are four assessments and each is counted separately. We used this approach because not all states report participation in the same way across assessments. For instance, one state might report only a count of students tested for one assessment, but for another assessment it might report a count tested, a percent tested, and a percent not tested.

Figure 7. Participation Reporting Approaches for General Assessments

Chart showing participation reporting approaches for general assessments

Thirty-seven states (59 assessments total) reported either the percent of students tested or the percent not tested for at least one of their assessments, which is a slight increase from the thirty-four that reported rates in 2003-2004. For 50 of those assessments, the percent of students tested was given, and for 30 assessments the percent of students not tested was given. For 21 assessments, both percent tested and percent not tested were provided. Forty-three states (73 assessments) provided the number of students tested, making this the most frequent way of reporting participation data. Sixteen states (21 assessments) provided the number of students not tested. The number or percent of students who were exempt or excluded from assessments was given in seven states (9 assessments) and the number or percent of students absent was given in 12 states (18 assessments).

Figure 8 illustrates the participation rates reported in those states for which this information was clearly reported. It is important that states report the percentage of students tested, in addition to just a count, because this presents a more accurate picture of how many students are participating. To summarize participation rate information, we selected one grade, 8th grade, and one subject, math, to portray in Figure 8. This grade and subject area were chosen to remain consistent with past years’ reports. In one state (Kansas) the math assessment was administered in 7th grade; that participation rate was used here. However, not all states provided data broken down in this way. Some states provided math assessment participation data but multiple grade levels were aggregated; or a rate was provided but it was a percent of all students tested who were students with disabilities, rather than a percent of all students with disabilities who were tested. Some states that otherwise reported clear participation rates for students with disabilities did not administer a middle school level math test. States which for any of these reasons did not report clear participation rates for the 8th grade math test are not included in Figure 8. During the 2004-2005 academic year, participation rates ranged from 83% to 100%; 10 of the 20 states had participation rates of 95% or higher.

Figure 8. Percentages of Students with Disabilities Participating in Middle School Regular Math Assessments in Those States with Clear Reporting of Participation Rates

Graph showing percentages of students with disabilities participating in middle school regular math assessments in those states with clear reporting of participation rates


Alternate Assessment Disaggregated Participation Results for Students with Disabilities

Figure 9 illustrates how states reported participation for their alternate assessments. Many more states provided participation information this year compared to the previous testing year, 2003-2004, in which only 35 states provided participation data. In 2004-2005, 46 states provided participation information for some or all of their alternate assessments, for a total of 53 assessments. Appendix F outlines in more detail the methods used in reporting this information.

Figure 9. Participation Reporting Approaches for Alternate Assessments

Graph showing participation reporting approaches for alternate assessments

Similar to reporting for the regular assessment, the most common way of reporting participation information for the alternate assessment was to give the number of students tested. This was done by 41 states on their 46 alternate assessments and the number of students not tested was reported by 15 states on their 16 alternate assessments. Thirty-eight states gave a percentage, which was either the percent of students tested (reported for 39 alternate assessments), not tested (reported for 19 alternate assessments), or both. Five states provided the number or percent of students who were exempt or excluded on their five alternate assessments, and seven states provided either the number or percent of students who were absent for their seven alternate assessments.

Thirty-eight states provided a percentage of either students tested or not tested. However, not all of these states reported the data clearly for each grade and content area, either aggregating across grade or content, or presenting the percent of all students tested who were students taking the alternate assessment. Figure 10 displays the participation rates for students with disabilities for the 8th grade math test in those states that provided clear participation rates The nine states shown provided information on the percent of students with disabilities who participated in the alternate assessment out of all students with disabilities. (Note: Washington’s data were from a 7th grade assessment).


Figure 10. Percentages of Students with Disabilities Assessed with the Alternate Assessment out of the Total Number of Students with Disabilities

Graph showing percentages of students with disabilities assessed with the alternate assessment out of the total number of students with disabilities
 


Assessment Performance in 2004-2005

Regular Assessment Performance Results

As with participation data, the way in which states provided performance data for students with disabilities varied (see Appendix G). Figure 11 illustrates the number of assessments with disaggregated performance data and how those performance data were reported. Information is again presented in terms of the number of assessments for which performance data were available, not in terms of the number of states.

Figure 11. Performance Reporting Approaches for Regular Assessments

Graph showing performance reporting approaches for regular assessments

All 50 states (88 assessments total) reported a rate of proficiency, such as percent of students with disabilities who are proficient, which was also the most common performance reporting method. Reporting the percent of students with disabilities in each achievement level was the next most common method, used by 40 states (70 assessments). The number proficient, used by 20 states (31 assessments), and number not proficient, used by 18 states (25 assessments),were much less widely used as a reporting method. The NRT reporting methods, representing percentile ranks rather than proficiency status, were the least frequently used, probably because only 7% of the state assessments were NRTs.

For all states that clearly disaggregated performance data for students with disabilities, we examined both the performance of general education students and students with disabilities. When examining performance across states, it is important to remember that state assessments are different, in terms of both content and proficiency levels. The assessments may emphasize different standards and are likely to differ in difficulty. In addition, there is variability across states in the percentage of students with disabilities whose scores are reported. Thus, it is not appropriate to compare performance across states. It is possible, however, to examine the differences in percent of students with and without disabilities achieving proficiency within each state, although it should be noted that this gap is also affected by variability between the participation rates of the two groups. Clearly, comparisons of the gaps across states are also inappropriate.

Performance results are reported for both reading and math assessments because these content domains are the ones assessed by most states and are the first content areas required by NCLB to be assessed, reported, and included in accountability. We included English language arts assessments as reading if the state did not have a specific reading assessment. All of the assessments were CRTs except in Iowa, which used an NRT. For 2004-2005, we do not report performance on exit exams because the distinct exit exams do not include all of the exams used to hold students accountable for graduation requirements. Many of the other graduation requirement exams are also used for NCLB accountability requirements, and we have reported results along with the regular statewide exams used for accountability.

We separated grade levels into three categories: elementary (3-5), middle (6-8), and high school (9-12). For our summary, we chose to present only one grade for each level. When available, 4th grade was used to represent the elementary level, 8th grade to represent the middle school level, and 10th grade to represent the high school level. These grades were chosen because they are the grades at which the most states test students. If data from those grades were not available, the next grades used were 5, 7, and 11. The 3rd grade assessment was used for Missouri’s elementary level reading performance data. Additionally, some of the high school assessments did not specify grade level; these results were included as 10th grade. (See Appendix K for the actual grade used.)

Reading Performance. Figures 12-14 present the reading performance of students. The performance of students with disabilities in reading is generally much lower than the performance of general education students. At the elementary level, New York had the largest difference in percent proficient (49 percentage points) between general education students and students with disabilities. At the middle school level, Alabama had the largest difference in rates of proficiency (58 percentage points). At the high school level, the largest difference in rate (45 percentage points) was shown by Alabama. The states with the smallest difference in percent of students with and without disabilities achieving proficiency were Texas at the elementary level (11 percentage points), Nebraska and Texas at the middle school level (23 percentage points), and Pennsylvania at the high school level (16 percentage points). In general, the reported gaps in student proficiency rates are variable with few evident patterns. One evident pattern is that the gaps increase with grade level. Such a trend must be interpreted with caution, given differences in composition of the two compared groups across increasing grade level (Bielinski & Ysseldyke, 2000). Additionally, there are observably lower proficiency rates in reading at the middle school level than at elementary; however, the assessments at different grade levels may differ in terms of both content and proficiency levels, making comparisons of proficiency rate across grades inappropriate.

Figure 12. Elementary School Reading Performance on the Regular Assessment

Graph showing elementary school reading performance on the regular assessment

 

Figure 13. Middle School Reading Performance on the Regular Assessment

Graph showing middle school reading performance on the regular assessment

Figure 14. High School Reading Performance on the Regular Assessment

Graph showing high school reading performance on the regular assessment

Mathematics Performance. Performance of general education students and students with disabilities on states’ 2004-2005 mathematics assessments is shown in Figures 15-17. In general, the gaps in proficiency rates between students with disabilities and general education students on math assessments were quite similar to the gaps found for reading assessments; they vary considerably from state to state. In elementary grades, the largest difference in the reported percent of students achieving proficiency was 45 percentage points in Alabama. In middle school, the largest difference was 51 percentage points, shared by Alabama and Wisconsin; in high school it was 60 percentage points in Alabama. The states with the smallest differences in percentage of students with and without disabilities reported as proficient were Texas at the elementary level (10 percentage points), Missouri at the middle school level (12 percentage points), and Missouri at the high school level (17 percentage points). As on the reading assessments, the percent of students both with and without disabilities who achieve proficiency is observably lower at middle school than an elementary; interpretation of this trend is again limited by differences in the assessment used at different grade levels.

Figure 15. Elementary School Mathematics Performance on the Regular Assessment

Graph showing elementary school mathematics performance on the regular assessment

Figure 16. Middle School Mathematics Performance on the Regular Assessment

Graph showing middle school mathematics performance on the regular assessment

Figure 17: High School Mathematics Performance on the Regular Assessment

Graph showing high school mathematics performance on the regular assessment

Alternate Assessment Performance Results

Figure 18 illustrates the number of alternate assessments with disaggregated performance data and how those performance data were reported. Information is again presented in terms of the number of assessments for which performance data were available, not in terms of the number of states. Appendix H provides details about the assessments and reporting for each of the alternate assessments.

Figure 18. Performance Reporting Approaches for Alternate Assessments (Number of Tests = 59)

Graph showing performance reporting approaches for alternate assessments

Forty-two states (48 assessments) reported a rate of proficiency—such as percent of students with disabilities who are proficient—which was also the most common performance reporting method. Reporting the percent of students with disabilities in each achievement level was the next most common method for 27 states (33 assessments). The number proficient (20 assessments) and number not proficient (17 assessments) were much less widely used as a reporting method. There were no NRT alternate assessments.

We examined alternate assessment performance data for all states that clearly disaggregated (i.e., by grade level and content area) these data for students with disabilities. Figures representing reading and math performance on the alternate assessment are included in Appendix I. On average, students with disabilities achieved higher rates of proficiency on the alternate assessment than on the general assessment. On the alternate assessment, there were no observable general differences in rates of proficiency across grade levels.


Assessment Performance: Trends

In an earlier analysis (Thurlow et al., 2003), the performance of students with disabilities for states that had at least three years of publicly reported data was examined. In 2000-2001, 13 states had publicly reported information on their statewide assessments for the past three years (California, Delaware, Indiana, Kentucky, Louisiana, Maryland, Minnesota, Missouri, New Jersey, New York, Texas, Utah, and Washington) for both math and reading. Colorado reported three years of information only for the reading assessment and Kansas reported information only for the math assessment. For the current period ending with 2004-2005, 11 states have seven years of publicly reported information on their statewide assessments (California, Colorado, Delaware, Indiana, Kansas, Kentucky, Louisiana, Missouri, New Jersey, New York, and Washington). For Colorado, full data for this period is available for the reading assessment only, and for Kansas for the math assessment only. States included in the trend analysis for 2000-2001 but excluded from the current analysis were the following: Texas and Maryland, which have since changed their assessments and cannot be included in the analysis; Minnesota, which did not report performance information in 2002-2003 (Wiley et al., 2005); and Utah, which aggregated its performance information across all grades in 2003-2004 (Klein et al., 2006).

Reading Tests. Figures 19-21 show the percentage of participating students with disabilities who achieved proficiency on statewide reading assessments at the elementary, middle, and high school levels over the past seven years. In general, the rate of proficiency for participating elementary school students with disabilities continues to increase in reading; 80% of the states included in this analysis have shown such an increase compared to the 2003-2004 school year and most of the states show an overall trend toward greater rates of proficiency for participating students. A few states have spikes or drops in the data across time (California, Louisiana), which may reflect altering assessments for NCLB. At the middle school level, all of the states reported a higher percentage of participating students achieving proficiency than in 2003-2004. While there are evident spikes in 2001-2002 for most states, and other years in California and Louisiana, several states show a trend across these years of increasing rates of proficiency among participating middle school students with disabilities. At the high school level, each of the three states with complete data demonstrate a trend across years toward higher rates of proficiency among those participating.

 

Figure 19. Seven-Year Trends of the Percentage of Elementary Students with Disabilities who Achieved Proficiency on Statewide Reading Exams

Graph showing seven-year trends of the percentage of elementary students with disabilities who achieved proficiency on statewide reading exams

Figure 20. Seven-Year Trends of the Percentage of Middle School Students with Disabilities who Achieved Proficiency on Statewide Reading Exams

Graph showing seven-year trends of the percentage of middle school students with disabilities who achieved proficiency on statewide reading exams

Figure 21. Seven-Year Trends of the Percentage of High School Students with Disabilities who Achieved Proficiency on Statewide Reading Exams

Graph showing seven-year trends of the percentage of high school students with disabilities who achieved proficiency on statewide reading exams


Math Tests. Figures 22-24 show the percentage of participating students with disabilities who achieved proficiency on statewide assessments at the elementary, middle, and high school levels for math over the past seven years. The math trends are similar to the trends noticed in reading performance. In general, it appears that rates of proficiency for participating elementary school students with disabilities continue to increase in reading: 90% of the states included in this analysis report higher rates of proficiency than in 2003-2004, and most of the states show a trend across these years of higher rates of proficiency for participating students. Again, data from some states show spikes or drops (California, Delaware, and Louisiana) that do not support a trend. At the middle school level, 70% of the states had a higher percentage of participating students achieving proficiency than in 2003-2004, and most states show an overall trend across these years of increasing rates of proficiency among participating students with disabilities. At the high school level, all four states with complete data show an overall trend across years of increasing rates of proficiency among participants.


Figure 22. Seven-Year Trends of the Percentage of Elementary Students with Disabilities who Achieved Proficiency on Statewide Mathematics Exams

Graph showing seven-year trends of the percentage of elementary students with disabilities who achieved proficiency on statewide mathematics exams

Figure 23. Seven-Year Trends of the Percentage of Middle School Students with Disabilities who Achieved Proficiency on Statewide Mathematics Exams

Graph showing seven-year trends of the percentage of middle school students with disabilities who achieved proficiency on statewide mathematics exams

Figure 24. Seven-Year Trends of the Percentage of High School Students with Disabilities who Achieved Proficiency on Statewide Mathematics Exams

Graph showing seven-year trends of the percentage of high school students with disabilities who achieved proficiency on statewide mathematics exams


Other Information Collected for 2004-2005

In our analysis of state reporting for 2004-2005, we looked at additional characteristics of states’ publicly reported information. Specifically, we looked at information available on accommodations used and how states publicly reported disaggregated student group assessment data. We also conducted a “click” analysis to determine the ease of access to Web-based reporting.

Accommodations

Sixteen states provided state-level information about students who took the general assessment with an accommodation. The same number of states provided accommodations information in 2003-2004. In some cases, states reported on standard accommodations (those considered appropriate and not ones that change the constructs measured by the assessment). In other cases they reported on nonstandard accommodations (which generally were considered to change the constructs measured—and might be referred to as “non-allowed” accommodations or “modifications”—although IEP teams could select them). Last, some states reported on both or did not specify which.

Table 2 summarizes the information the 16 states provided. Appendix J contains additional information about the data provided by these states, with details about the participation and performance of students. Five states reported student participation and performance by specific accommodation (e.g., directions read orally, braille, extended time), and three states indicated that this information was available in their SPP.

Table 2. States that Reported State-Level Information about Accommodations

 State

 Terminology Used

By Content/
Grade?

 Participation

 Performance

 Comments

Colorado

Specific Accommodations1

Yes/Yes

Yes

Yes

 

Non-Approved Accomm/Mod

Yes/Yes

Yes

No

 

Idaho

Accommodation

Yes/Yes

Yes

Yes

Reading only; Provides information for Fall 2004 and Winter 2005SWD

Adaptation

Yes/Yes

Yes

 

Indiana

Accommodations

Yes/Yes

Yes

Yes

Includes only “special ed”

Kansas2

Accommodations

Yes/No

No

Yes

Includes on IEP students

Kentucky

Accommodations

Yes/Yes

Yes

Yes

End of Primary and Gr 6 only

Louisiana

”Calculator Used”

Yes/Yes

Yes

Yes

Only on ITBS

Michigan

Standard Accommodations

Yes/Yes

Yes

Yes

Data broken down by All students, Non sp ed students, and spec ed students & ALL

Non-Standard Accomm

Yes/Yes

Yes

Yes

 

Mississippi

Specific Accommodations1

No/No

Yes

No

 

Missouri

Accommodations

Yes/Yes

Yes

No

 

Nebraska

Accommodations

Yes/Yes

Yes

No

ALL

Nevada

Not-Tested: Modified

Yes/Yes

Yes

No