|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Use of Accommodations in State Assessments: What Databases Tell Us About Differential Levels of Use and How to Document the Use of AccommodationsNCEO Technical Report 30Published by the National Center on Educational OutcomesPrepared by: Martha Thurlow August 2001 Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as: Thurlow, M. (2001). Use of accommodations in state assessments: What databases tell us about differential levels of use and how to document the use of accommodations (TechnicalReport 30). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Technical30.htm This report is based on a paper presented at the annual meeting of the National Council of Measurement in Education, Seattle, Washington, April 11, 2001. Executive SummarySince the enactment of IDEA 97, students with disabilities are being included in state and district assessments more than ever before, often with accommodations designed to give them access to the test. In the past, most states did not collect data on the use of accommodations. This is changing. As part of its most recent survey of states, the National Center on Educational Outcomes found that data were available from 12 states on the number of students using accommodations during state assessments. The extremely variable rates in accommodations use (8-82%) indicated that in some states more accommodations are being used than in others; still there was some relationship of accommodations use to level of schooling (elementary, middle, high school). Publicly reported data on accommodations use are unique to each state, making other comparisons difficult. Further study of how data on the use of accommodations are collected in states points to concerns about the accuracy of information and possibly the need for better methods for recording accommodations use. OverviewFor some time now, the participation of students with
disabilities and students with limited English proficiency in national, state,
and district assessments has been a topic of considerable discussion and
research effort, to the point that the National Research Council formed
committees and conducted studies to address the issues (e.g., August & Hakuta,
1999; McDonnell, McLaughlin, & Morison, 1997). These and numerous other efforts
were aligned with federal initiatives that require the participation of students
with disabilities (both IDEA 97 and Title I of IASA 94) and students with
limited English proficiency (Title I of IASA 94) in state and district
assessments. Accommodations frequently are cited as one of the key
avenues for increasing the participation of students with disabilities in
national and state assessments (Anderson, Jenkins, & Miller, 1996; Elliott, &
Thurlow, 2000; Mazzeo, Carlson, Voekl, & Lutkus, 2000; Stancavage, McLaughlin,
Vergun, Godlewski, & Allen, 1996; Thurlow, Elliott, & Ysseldyke, 2000).
Generally defined, accommodations are changes in testing materials or procedures
that allow students to show their knowledge and skills rather than the effects
of disability or limited English proficiency. While there are a variety of other
terms in use (e.g., adaptation, modification), some of which may mean the same
thing and others of which may define the comparability or acceptability of a
given change (Elliott, Thurlow, Ysseldyke, & Erickson, 1997), the essence of the
concept is that either the materials or the procedures of testing have changed.
In this paper, I use “accommodations” as a generic term to cover all types of
testing changes. Although the importance of accommodations for both
students with disabilities and English language learners is now recognized, the
requirement that accommodations be provided has a much longer history for
students with disabilities, a history entwined with the law (Section 504 of the
1993 Rehabilitation Act, Americans with Disabilities Act, IDEA). For the past
decade, the National Center on Educational Outcomes (NCEO) has examined the
issues surrounding the participation of students with disabilities in
large-scale assessments. All states now have written policies guiding the
provision of assessment accommodations for students with disabilities (Thurlow,
House, Boys, Scott, & Ysseldyke, 2000). Indeed, assessment directors are much
more cognizant of their policies on assessment accommodations (Olson, Bond, &
Andrews, 1999), and many states are conducting or participating in research on
the effects of accommodations. Although our knowledge about what accommodations
are allowed in states and which are most controversial has increased
dramatically (see Thurlow & Bolt, in press-a), as has our recognition of the
research that has been conducted on a variety of test changes (Thurlow & Bolt,
in press-b; Tindal & Fuchs, 2000), this information tells us nothing about the
extent to which accommodations are used, nor about the specific accommodations
that are used most frequently. The purpose of the present paper is to examine the use
of accommodations in state assessments by students with disabilities, and to
explore several related factors. Five primary topics are addressed: 1. To what extent do states have data on the use of assessment
accommodations by students with disabilities during state assessments? 2. How variable is the use of assessment accommodations during
state assessments by students with disabilities across states? 3. What information
exists on the use of specific accommodations? 4. What data are
reported publicly on accommodations used during state assessments? 5. How do states
collect data on the use of assessment accommodations by students with
disabilities? State Data on the Number of Students with Disabilities Using Assessment AccommodationsBackgroundData collected in the past from states’ written policies
on assessment accommodations (Thurlow, Ysseldyke, & Silverstein, 1993; Thurlow,
Scott, & Ysseldyke, 1995; Thurlow, Seyfarth, Scott, & Ysseldyke, 1997; Thurlow,
House, Boys, Scott, & Ysseldyke, 2000), as well as information collected from
surveying state assessment directors (Olson, Bond, & Andrews, 1999; Roeber,
Bond, & Braskamp, 1997; Roeber, Bond, & Connealy, 1998) consistently indicate
that the most frequently allowed accommodations across states are (1) large
print, (2) Braille, and (3) reading directions. However, a look at the
distribution of disability categories among the school-age population (U.S.
Department of Education, 2000) suggests that these probably are not the most
frequently used accommodations. Most students have learning disabilities, speech
and language impairments, emotional or behavioral disabilities, and mental
retardation, not
visual disabilities, as the most frequently allowed accommodations might
suggest. To better understand the use of assessment
accommodations, it is important to look at the extent to which states have data
on the use of accommodations by students with disabilities during state
assessments. Further, it is important to look at the extent to which students
with disabilities are using accommodations during state assessments. Data SourceData on the use of accommodations were collected through
a survey conducted by NCEO during 1999 (Thompson & Thurlow, 1999). State
directors of special education, or their designees, in the 50 states were asked
to indicate whether their state assessment systems collected data on the use of
accommodations during assessments, and if so, to indicate the percentage of
students using accommodations for each state test. ResultsTwelve states indicated that they collected data on the number of students with disabilities using accommodations in their state assessments. Further, they each supplied the actual percentage of students with disabilities using accommodations in their state assessments, for the grade levels in which they were administered. For summary purposes here, the percentages are averaged and grouped in Table 1 according to the level of school (elementary, middle, and high school). Table 1. State-Reported Levels of Use of Accommodations
As is evident in Table 1, all 12 states had data
available at the elementary level, while only 11 did at the middle school level,
and only 9 did at the high school level. The cells without data in Table I
correspond to the school levels for which accommodations data were not available
in Maryland, New York, and West Virginia. In general, there is a downward trend in the percentage
of students using accommodations across grades. For the 16 tests with data on
percentages of students using accommodations for at least three grades, 11 show
a downward trend in percentages across school levels; 2 additional show a
downward trend from elementary to middle school, but minimal difference between
middle and high school. Thus, a downward trend either across two or three of the
three school levels is evident in over 95% of the possible comparisons. In the
other three cases where three levels of data were available on the same test,
the trends in percentages of students using accommodations show upward trends
from elementary to middle to high school; two of the three cases are from one
state (Indiana) while the other is from Rhode Island. An examination of the relative levels of the percentages of students using accommodations also is of interest. Table 2 summarizes the state data in terms of the approximate percentage of students using accommodations at each school level. There are several states that hover consistently at either the lower, middle, or higher ranges of percentages. For example, Kansas is consistently within the lowest three percentage levels, regardless of school level. Only two states stay consistently above the 50% level, regardless of school level (Kentucky and Pennsylvania). Four states hover around the 40-60% range for at least two of the school levels (Nevada, Pennsylvania, Rhode Island, South Dakota).
Table 2. Approximate Percentages of Students with Disabilities Using Accommodations at Each School Level
Note: West Virginia is not included in this
table because its data were combined across all grades (3-11).
In addition, Maryland and New York are not included in the high school column
because they did not have data at that school level. Use of Specific AccommodationsBackgroundWith the apparent increase in use of accommodations
during state and district assessments, there is interest not only in how many
students are using accommodations, but also in exactly which accommodations are
being used. This information will help us understand not only which specific
accommodations are used most, but also the extent to which combinations of
accommodations are used. Data SourcesThree states that have collected data on specific
accommodations that are used by students with disabilities have either analyzed
or supplied data on accommodations to NCEO to analyze – Kentucky (Trimble,
1998), Rhode Island (Elliott, Bielinski, Thurlow, DeVito, & Hedlund, 1999), and
Missouri (Bielinski, Ysseldyke, Bolt, Friedebach, & Friedebach, in press). The
findings of these studies are presented here, and the results examined in
relationship to each other and to contextual characteristics of the states. Each
of these states has slightly different accommodations policies. Kentucky
is a state with relatively broad accommodations policies. Districts are
instructed that assessment accommodations should be aligned with instructional
accommodations – those accommodations used during instruction may be used during
assessment. The exception is that reading the test aloud to students is not
appropriate when the test is a measure of reading decoding skills. Rhode Island is one of a handful of states that allows
accommodations for all students. Students or their teachers must be able to
demonstrate need for a specific accommodation before it can be incorporated into
the testing procedures for the student. As a result of this uniquely inclusive
approach to accommodations, Rhode Island has important data about the numbers of
students with and without disabilities using accommodations. Missouri
initiated its new Communication Arts assessment in 1998. Missouri coded the
specific accommodations used by students on bubble sheets at the time of
testing; also coded for these students was the student’s category of disability
and the disability area (math, reading, behavior). As a result of the richness
of its information, Missouri has important data about the numbers of students
with specific disabilities who used accommodations during state testing. ResultsThe data on specific accommodations used during
statewide assessment are collected in slightly different ways in each state,
which in turn affects the kind of information available on use of accommodations
during testing. Therefore, each state is presented separately here, with some
indication of how data on accommodations used were collected. Kentucky. Although Kentucky’s accommodation policies have been in
place since 1992, it did not begin coding accommodations used during assessments
until 1994-95. For coding purposes, Kentucky groups its accommodations into
seven categories: •
Reader/Oral. The assessment is read to the student in a way comparable to
the manner in which normal instruction is delivered. •
Scribe/Dictation. The responses to the assessment are dictated by the
student and written down by a scribe in a way comparable to the manner in which
normal instruction is delivered. •
Cueing. The student with disabilities uses mnemonics, problem-solving
organizers, semantic organizers, webs, or templates in responding to the
assessment in a way consistent with daily instruction and the student’s IEP or
504 Plan. •
Paraphrasing. The content of the assessment is paraphrased for the student
with disabilities in a way comparable to the manner in which normal instruction
is delivered. •
Interpreter. The content of the assessment is signed for the student with
disabilities in a way comparable to the manner in which normal instruction is
delivered. •
Technological. Technology typically used by the student with disabilities in
daily instruction is made available during the administration of the assessment. •
Other. Other accommodations normally made available in the delivery of
instruction are made available in the administration of the assessment.
(Trimble, 1998, pp. 23-24) According to the 1999 NCEO survey data (see Table 1),
Kentucky is one of the states with the highest percentage of students using
accommodations during state testing (50–82%). The analysis of specific
accommodations used during two earlier testing years (Trimble, 1998) indicated
that across all students using accommodations, the most frequently used
accommodations were (not necessarily in order): Reader/Oral, Scribe/Dictation,
Paraphrasing, Oral & Scribe/Dictation, Paraphrasing & Reader/Oral &
Scribe/Dictation, Paraphrasing & Reader/Oral & Other, Paraphrasing & Reader/Oral
& Technology & Scribe/Dictation. Clearly, many students used combinations of
accommodations rather than a single accommodation. The percentages of students
using each of the most frequently used accommodations and combinations of
accommodations during 1995-96 are presented in Table 3. The data in Table 3 support NCEO’s finding that the percentage of students using accommodations decreases across grades (Thompson & Thurlow, 1999). In addition, the data in Table 4 suggest that combinations of accommodations are used by larger numbers of students than are single accommodations: in grade 4, the largest percentage of students used the Paraphrasing & Reader/Oral & Scribe/Dictation, while in both grade 8 and grades 11/12, the largest percentages of students used Paraphrasing & Reader/Oral. Table 3.
Estimated Percentages of Students in Kentucky Using Specific Accommodations
Note: The percentages in this table are
estimated from the 1995-96 numbers presented by Trimble (1998) in Table 7
(accommodations use) and in Table 3 (number of students participating in
assessment). Cells with less than 100 students were not estimated. The numbers
in this table are considered to be estimates because it is possible that there
were discrepancies in coding the number of students actually using specific
accommodations or combinations, as well as some variation in the actual number
of students participating in the assessment.
Rhode Island. The policy of allowing any student for whom the need
for an accommodation could be demonstrated to use the accommodation during
testing began during the 1995-96 school year, at the same time that the state
first administered new performance assessments in Grade 4 mathematics, writing,
and health education. At that time, Rhode Island had a list of several dozen
accommodations that could be used by students. According to the 1999 NCEO survey data (see Table 1),
Rhode Island is a state with moderate percentages of students using
accommodations during state testing (39-61%). The analysis of Rhode Island’s
data revealed that while students without
disabilities most often used just one accommodation, students with disabilities most often used between
one and four accommodations (Elliott, Bielinski, Thurlow, DeVito, & Hedlund,
1999). Table 4 presents the percentages of students with and without disabilities using each of the most frequently used accommodations and combinations of accommodations. These data show that relatively small percentages of students without disabilities used accommodations; the largest percentage of these students used the timing accommodation. For students with disabilities, the percentage of students using timing only was comparable to the percentage of students without disabilities using timing. However, there were other accommodations used by percentages of students with disabilities that were just as high (Oral & Repeated Directions), and a couple of combinations of accommodations that had much higher percentages of students with disabilities using them (Oral & Repeated Directions & Setting; Oral & Repeated Directions & Setting & Response). Table 4.
Estimated Percentages of Students in Rhode Island Using Specific Accommodations
Note: The percentages in this table are
estimated from 1995-96 numbers presented in Table 6 by Elliott et al. (1999).
They are considered estimates because there may have been discrepancies in
coding the number of students actually using specific accommodations or
combinations, as well as some variation in the actual number of students
participating in the assessment.
Missouri. In Missouri a list of about two dozen accommodations was
in place during the 1998 testing. IEP teams decided which accommodations
students would use, and then marked these on a Student Information Form, which
also contained information on the student’s disability category and area of
disability. Missouri did not provide data to NCEO’s 1999 survey on the percentage of students using accommodations during state testing. However, Bielinski, Ysseldyke, Bolt, Friedebach, and Friedebach provided data on the percentages of students with disabilities who had valid test scores using accommodations in grade 3 (53%), grade 7 (53%), and grade 11 (33%). These data indicate that greater percentages of students use accommodations in elementary and middle school than in high school. Table 5 shows the percentages of students with various disabilities using the four most frequently used accommodations and combinations of combinations. These data were not broken down by grade by Bielinski et al. (in press).
Table 5.
Estimated Percentages of Students in Missouri Using Specific Accommodations
Note:
The percentages in this table are directly from Table 1 in Bielinski, Ysseldyke,
Bolt, Friedebach, and Friedebach (in press), and are based on only those
students who received valid scores on the Missouri Comunications Arts test. Publicly Reported Data on AccommodationsBackgroundWhen the Individuals with Disabilities Education Act
(IDEA) was reauthorized in 1997, it included a requirement that states report on
the number of students with disabilities included in state and district testing,
and the performance of students on those tests in the same way and with the same
frequency as they did for students without disabilities. NCEO started tracking
public education reports in 1997 (Thurlow, Langenfeld, Nelson, Shin, & Coleman,
1998; Thurlow, Nelson, Teelucksingh, & Ysseldyke, 2000; Ysseldyke, Thurlow,
Langenfeld, Nelson, Teelucksingh, & Seyfarth, 1998). These reports revealed that
few states reported any information about students with disabilities other than
enrollment data. Those that did report on the performance of students with
disabilities often did not make clear whether they had included only those who
took the assessments in the standard way, or had also included students who used
some kind of accommodation. Of the 13 states that provided some type of test data in
1998, Ysseldyke et al. (1998) found that only Delaware included any information
on the use of accommodations. Delaware reported that “of the 3,395 special
education students who took the test, 1,033 had some type of accommodation and
2,362 tested with no accommodation” (Delaware State Department of Education,
1997, p. 25). This indicates that about 30% of students with disabilities in
Delaware used accommodations. Oregon provided combined data on the numbers and
percentages of students who took the 1993-94 reading assessment with
modifications or who were exempt (Oregon Department of Education, 1997, p. 52).
It is not possible to separate modifications from exemptions. Further, the use
of the term “modifications” suggests that the report is focusing on the number
of students whose results were not being reported. How many students used
accommodations that were considered acceptable and thus were included in
aggregate scores cannot be determined. When another review of state reports was conducted in
1999 (Thurlow, Nelson, Teelucksingh, & Ysseldyke, 2000), 13 states (but not
necessarily the same ones as in 1998) provided test-based data on students with
disabilities; 12 provided participation data. Only South Dakota provided
information about accommodations use (South Dakota Department of Education and
Cultural Affairs, 1999). South Dakota disaggregated the scores for students who
took the test under standard conditions, standard accommodations, and with
nonstandard accommodations. In a report that was released after NCEO’s data
collection period, North Carolina provided data on the numbers of students using
modifications for its Tests of Computer Skills for the classes of 2001 and 2002
(State Board of Education, 1999). In North Carolina, “modifications” is the term
used for both acceptable and not acceptable changes in testing conditions: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||