 |
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
 |
 |
 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
Special Topic Area:
|
|
>Introduction |
Frequently Asked Questions
1. How many students are tested out of
level?
Pinpointing the exact number of
students who are tested out of level in statewide assessments for any given
school year is difficult. Only a few states make out-of-level test
participation and performance data available to the public. Finding these
test results even when available is very difficult. Sometimes the results
are included with other test results without identifying how many students
were tested below grade level. Only a few states identify the numbers of
students who take below-grade level tests and indicate how they perform.
Initial research results (from the 2000-2001 school year) demonstrated very different rates of students tested out of level, from approximately 30% in one state to approximately 60% in another state. Rates varied according to the grade level and the content area. This research showed that the percentage of students tested out of level was influenced by the states' out-of-level testing policies. It also showed that in some states many of the students with disabilities who took out-of-level tests performed very well, suggesting that these students could have been tested on grade level.
Findings from research during the 2002-2003 school year showed that more states were able to produce out-of-level test results. Typically this is accomplished by special data analyses rather than through states’ regular public reporting. Preliminary research results suggest that the number of students tested out of level is decreasing. Changes in states' policies that either limit or disallow the use of out-of-level testing for large-scale assessment and accountability purposes have played a big role in this decrease. Nevertheless, rates of students tested out of level continued to vary widely across states that were still testing students out of level in standards-based assessments.
2. How are the results of out-of-level
tests reported in states' data reports?
In the past, out-of-level test
results were not reported – at least not in a clearly identified way. This
statement is based on both searches of state data reports and interviews
with state education personnel. In fact, some states viewed the reporting of
out-of-level test scores as not feasible due to specific statistical
concerns about the resulting test scores. This issue arose primarily within
those states that used criterion-referenced tests for statewide assessment
programs.
If out-of-level testing is occurring in state-level testing, it is important to have information reported publicly about both the number and percentage of students tested out of level (participation reporting) and the performance of those students (performance reporting). Participation reporting is needed to know whether unreasonably large numbers are being tested below grade level. Many states simply fold in students tested out of level with all other students, so it is impossible to know how many were tested below grade. It is also important to have separate reporting on the performance of students tested out of level. Many states claim to report these test results, but do so in formats that mask the actual out-of-level test performance data. For instance, some states equate a below-grade level test score to the grade level at which a student is enrolled in school. These test results are then reported at the student's grade of enrollment with other general assessment test scores from tests administered on-grade level. There is no way to tell how students performed on out-of-level tests when this reporting format is used. Some states do not yet report out-of-level test data at all.
3. How are decisions made in local
schools about out-of-level testing?
There are two types of decisions
that need to be made. First, it must be decided whether to test a student
on-grade or below-grade level. Second, if testing below-grade level, the
grade level of an out-of-level test needs to be selected. Since most
out-of-level tests are presented to students with disabilities, states
typically delegate the decision-making responsibility to the Individualized
Educational Program (IEP) team. Many states provide specific criteria for
IEP teams to use in making decisions about whether to test below grade and a
few states provide guidance in selecting how many grades below.
Despite the policies and guidance provided by states, research suggests that in those states that have allowed out-of-level testing, there has been wide variability in how students with disabilities are selected and assigned to below-grade level testing. Often, the IEP team relied on a special education teacher's recommendation that had been made before the meeting started, or that prevailed during the meeting as the best recommendation for testing after active discussion. Further, the special education teacher’s recommendation tends to be influenced by other teaching factors such as the content area that the teacher worked on with the student or the grade in which the student was enrolled in school - not necessarily the student's achievement level in the classroom.
There are many concerns that can be raised about this kind of decision making. It is critical that administrators, teachers, parents, and students ask questions when out-of-level testing is discussed.
4. How does out-of-level testing
affect the instruction of students with disabilities?
The specific effects of
out-of-level testing are difficult to determine. Yet, we know a few things
from research. Special and general education teachers seem to hold lower
learning expectations for students with disabilities who are tested
below-grade level. Further, students do not receive grade-level instruction.
While it is difficult to separate what came first, it is clear that
instructional decisions and testing decisions are linked. It is unclear
whether the future consequences of taking standards-based measures below
grade-level is understood by parents or IEP teams when decisions are being
made.
5. Do out-of-level tests yield
psychometrically sound test results?
This question is best answered
by distinguishing among three approaches to testing students out of level in
standards-based assessments. The
first approach is an attempt to demonstrate grade-level proficiency based on
performance on a test designed for a different grade. An example would be
having a student take a 2nd grade test to demonstrate proficiency
on 5th grade standards. This approach is difficult to justify.
Psychometricians continue to find the attempt to vertically equate tests
designed for one grade (e.g., 2nd grade) to tests designed for
another grade (e.g., 5th grade) problematic.
The second approach assigns below-grade tests to students who are presumed to be unable to demonstrate grade-level proficiency. No matter what score the student obtains on the below-grade test, the student is reported as not proficient on grade-level standards. Although there are some psychometric concerns about whether a score a fifth grader gets on a 2nd grade test would mean the same thing for that fifth grader as the same score would mean for a second grader, the larger concern is that this approach results in people making judgments about whether students are proficient on grade-level standards without giving the students the chance to show whether they are or are not. In other words, we assume that the student is not proficient on 5th grade standards and assign the student to a 2nd grade test. But we have not given the student the opportunity to surprise us – to show that he or she can perform on a 5th grade test.
The third approach uses assessments that are not designed for a particular grade. The practical and psychometric issues with this approach are more complex. Typically these assessments are thought of as adaptive – matched either to the level of instruction that a student is receiving or to the student’s level of performance. In a computerized format, these assessments may dynamically adjust to the students’ performance during the test. Otherwise a test level is assigned to a student based on teacher’s judgment or on the basis of other test performance. One of the issues with this approach is similar to the problem with the first approach. That is, what psychometrically defensible evidence shows that a particular score on any given level of the test demonstrates proficiency on grade-level standards? Another issue is similar to the problem with the second approach. Might the student’s opportunity to show proficiency on grade level standards be impeded if teacher judgment or the student’s performance on some subset of more primitive tasks prevents the student from ever getting to the more challenging tasks that would be needed to demonstrate grade-level proficiency?
Related NCEO Publications
- Out-of-Level Testing: Pros and Cons (Policy Directions 9 / 1999)
- Past and Present Understandings of Out-of-Level Testing: A Research Synthesis (Out-of-Level Testing Report 1 / 2000)
- How Out-of-Level Testing Affects the Psychometric Quality of Test Scores (Out-of-Level Testing Report 2 / 2000)
- A Review of Out-of-Level Testing State Policies (Out-of-Level Testing Report 4 / 2001)
- Testing Students Out of Level in Large-Scale Assessments: What States Perceive and Believe (Out-of-Level Testing Report 5 / 2001)
- Testing Students with Disabilities Out of Level: State Prevalence and Performance Results (Out-of-Level Testing Report #9) / PDF Version
- Reporting Out-of-Level Test Scores: Are These Students Included in Accountability Programs? (Out-of-Level Testing Report #10) / PDF Version
|
About NCEO
|
Publications |
Special
Topics |
Projects
|
Presentations |
State
Policies |
Accommodations
Bibliography |
Teleconferences |
Tools |
Related Sites
| Indicator 3
|
|
|
||
