The advent of standards-based reform during the past 10 years has ushered in a variety of challenges for policymakers and practitioners alike. Such concerns were accentuated by the legislative mandates of the No Child Left Behind (NCLB) Act which required year-to-year academic performance to be measured by states’ standards-based large-scale assessments for all subgroups of students in U.S. schools. While schools and their staff generally support the theory of action that underlies initiatives like NCLB, such as that explicated in Testing, Teaching, and Learning (Elmore & Rothman, 1999), it is still a challenge to provide instruction based on challenging, grade-level content standards. In addition, there is often a concern that some students, especially those with disabilities or limited English proficiency, may not be capable of achieving the academic content deemed appropriate for the grades in which they are enrolled in school. It is also suggested that these students are not able to fully participate in large-scale assessments that were designed for their peers. While concerns have been raised for students with disabilities as a subgroup and English language learners as a subgroup for some time, it is only recently that including students with both disabilities and English learning challenges in states’ large-scale assessment and accountability programs have been considered.

With the requirements of the Individuals with Disabilities Education Act (IDEA) and NCLB, both of which clearly required that students with disabilities be included in state assessments, states are making progress toward including all students in their standards-based testing. Rates of participation for students with disabilities and English language learners have been improving over time (Thompson & Thurlow, 2003). However, there are few data that can demonstrate improved academic results for English language learners with disabilities. In fact, there are few sources of public data that report results for these students (Albus & Thurlow, 2005). Only recently, English language learners with disabilities have begun to receive marginal attention in the literature (Minnema, Thurlow, Anderson, & Stone, 2005). The National Center on Educational Outcomes (NCEO) has, for the past four years, conducted research on large-scale assessment and instructional issues for English language learners with disabilities, but neither NCEO’s research nor any other research study has yet described large-scale assessment experiences at the local school level for English language learners with disabilities.

This study was designed, in part, to clarify some of the issues that surround including English language learners in states’ large-scale assessment programs. More specifically, we gathered practical information at the local school level to understand these students’ large-scale assessment experiences from a variety of perspectives, to describe the characteristics of English language learners with disabilities as well as the characteristics of their schools, and to make known the level of awareness that students and their families have about large-scale assessments.

This study was conducted in a large western state with a total estimated population of 35,484,453 in 2003. The breakdown by ethnicity in 2000 was 59.5% White persons, 6.7% Black or African American persons, 1.0% American Indian and Alaska Native persons, 10.9% Asian persons, 0.3% Native Hawaiian and Other Pacific Islander, 16.8% persons reporting some other race, 4.7% persons reporting two or more races, 46.7% White persons, not of Hispanic/Latino origin, and 32.4% persons of Hispanic or Latino origin (U.S. Census Bureau, 2005). Across these subgroups, 26.2% were foreign born for an estimated total of 8,864,188. The region in which these individuals were born was Europe (7.9%, a total of 696,578), Asia (32.9%, a total of 2,918,642), Africa (1.3%, a total of 113,255), Oceania (0.8%, a total of 67,131), Latin America (55.6%, a total of 4,926,803), and North America (1.6%, a total of 141,779). Of those 5 years of age or older, 39.5% (a total of 31,416,629) of this state’s population spoke a language other than English at home, 60.5% spoke only English at home, and 39.5% (a total of 12,401,756) spoke a language other than English at home. Again, as of 2000, there were 5,923,361 individuals in this state who were 5 years old or older.

The school district in which we collected data is an urban unified school district located in the southern region of the state. The district serves approximately 97,000 students in 95 public schools from four different cities. As the third largest school district in the state, it serves the most diverse large city in the United States (U.S. Census Bureau, 2005). The students in the district speak 46 different languages and are required to wear school uniforms through grade 8. Any student at the end of grade 3 who is reading below grade level attends mandatory summer school, because the district no longer practices social promotion. This district was the first in the nation to introduce these district-wide reforms. Because of outstanding educational practices and student outcomes, this district recently received a national award for being one of the top urban school districts in the United States.

The Elementary School

The elementary school in which data were collected educates 1,204 students in kindergarten through grade 5. The school is located in a residential neighborhood on the west side of the city. Students represent a wide range of ethnic groups, including 73.9% Hispanic/Latino, 10.4% African-American, 9.8% Filipino-American, 2.4% Pacific Islander, 2.1% White (Not Hispanic), 1.3% Asian-American, and 0.1% American Indian or Alaska Native. A recent push for class size reduction has resulted in a teacher/student ratio of 1:20 in kindergarten through grade 3. The teacher/student ratio for grades 4 and 5 is 1:33. All 56 classrooms in the school are described on the school district Web site as staffed by dedicated teachers offering direct, explicit instruction in basic skills and higher level thinking skills. Students are instructed on the state content standards with the goal of every student achieving grade level expectations as defined by the state’s performance standards. Standard proficiency is measured by the state’s large-scale assessment program, district benchmark tests of basic mathematics facts, performance tasks in mathematics and writing, and student portfolios.

The Middle School

The middle school that participated in the research project serves 1,760 students in grades 6 through 8. It is a year-round neighborhood school in the northern area of the city that is noted for its cultural diversity. Students represent a wide range of ethnic groups that include 58.8% Hispanic/Latino, 26.1% African-American, 5.2% Asian-American, 4.8% Pacific Islander, 3.9% White (Not Hispanic), 0.8% Filipino-American, 0.4% American Indian or Alaska Native, and 0.1% Other. The average class size at the middle school varies by subject area, from 31 in English classes to 40 in science classes. Daily goals for the students and staff are driven by the school’s mission, "to educate all students to enable them to participate in their education and contribute to their school and society." Their education is guided by the district’s content standards, and students recently met their targeted growth areas in reading, language, and mathematics according to the state academic performance index. Departmental and grade level teacher-developed tests are administered in addition to the state’s large-scale program.

The High School

Data were collected in a high school that is a state distinguished school that educates 4,376 students in grades 9 though 12. The school operates under the vision that, the "high school will foster a positive and open atmosphere that guarantees academic success, enhances self-esteem, and promotes respect for others within a culturally diverse society." The diverse student body is comprised of 30.5% Asian-American, 29.2% African-American, 17.5% Hispanic/Latino, 13.1% White (Not Hispanic), 6.6% Filipino-American, 2.8% Pacific Islander, and 0.3% American Indian or Alaska Native. In grade 9, each student is assigned to a four-year academy according to a particular field of interest. The academies include: math and science, business world preparation, media, visual performing and applied arts, special education, English language development, and a partnership academy. These academies prepare each student for a future at a four-year university. Vocational training is also available at this high school. Ultimately, the school curriculum is determined by state and district content standards. Students are assessed using the state large-scale assessment program and must pass the state high school exam to receive a high school diploma.

Students in grades 11 or 12 who wish to accelerate their studies in a focus on passing the high school exit exam may attend the upper division academy. This 18 classroom program is located one block off of the high school campus. Approximately 550 students attend this academy to take advantage of the accelerated instructional pace.

Students may complete a year long course in a semester, as each 10 unit class meets daily for 90 minutes. Instead of the typical 60 credits a year, these students may complete 80 plus credits each school year. Students at the upper division academy are able to participate in all activities offered to students at the high school, as well as additional activities special to the academy such as the student commission. The goal of this program is to graduate each student with regular high school diplomas.

Large-Scale Assessment Program

We collected data in a state where the large-scale assessment program for the school year 2003–2004 consisted of four core assessments: (1) a standards-based measure administered to students in grades 2 through 11 in English-language arts and mathematics, a written composition portion for students in grades 4 and 7, a history-social science portion for students in grades 8, 10, and 11 and a science portion administered to students in grade 5, 9, 10, and 11; (2) an alternate assessment that is designed for students with severe cognitive disabilities in grades 2 through 11 that assesses English-language arts and mathematic skills; (3) an "off-the-shelf" test that serves as the norm-referenced segment of the standards-based measure and that assesses skills in reading/language, spelling, and mathematics in grades 2 through 8, and reading/language, mathematics, and science in grades 9 through 11; and (4) a norm-referenced test used to assess Spanish-speaking students for primary language assessment component of the state’s large-scale assessment program. This test measures skills in reading, spelling, language, and mathematics for Spanish-speaking students in grades 2 through 11. These students must be identified as English language learners (ELLs) who have been in the states’ schools for less than 12 months.

Extending beyond the large-scale assessment program is an English language development test, that is a standards-based assessment of listening and speaking in grades kindergarten through 1, and listening, speaking, reading, and writing in grades 2 through 12. This assessment helps determine a student’s level of English language acquisition and comprehension. Lastly, the state requires students in grade 10 to take a state-developed standards-based high-stakes assessment of language arts and mathematics skills that students are required to pass in order to graduate from high school with a regular diploma.

Research Questions

We addressed two broad research questions in our study:

1) What perceptions do educators, parents, and students have about the experiences of English language learners with disabilities who participate in large-scale assessments?

2) What are the characteristics of schools that test English language learners with disabilities in large-scale assessments?

Research Design

Our case study research design, in which one school is defined as a case, used a mixed method approach to collect quantitative and qualitative data from four sources of data. Data were collected on site in three schools and one alternative school program in a large urban school district located in a large western state.

Sample

Using a purposive sample, we included students with disabilities (n = 24), their parents (n = 30), special and general education teachers (n = 72), and administrators (n = 5). The schools from which our sample was drawn were recruited by the assistant superintendent of special education and the program specialist of special education/English language learners. Within each school, one staff member served as a contact person for the study. Their primary responsibility was to recruit parents and students for the face-to-face interviews. All participants received a gift card from a local department store as a thank you for their time invested in our research activities.

Instruments

We used a variety of self-developed data collection instruments that included a written survey, interview protocols, and document review data collection sheets (see Appendix A for copies of the survey and interview protocols).

Procedures

The written surveys were distributed in teachers’ school mailboxes with a request to return the surveys to the school’s main office by the end of that school day. As a follow up procedure, we worked with our contact person in each school to encourage those teachers who had not responded on the first day to return their survey by the end of the week. In one school where we were unable to distribute the survey while working on-site, we mailed the surveys to the principal who distributed and collected them during a staff meeting.

Our face-to-face interviews with parents and teachers were conducted at school in either small groups or individually depending on the participants’ preferences. Each interview was tape recorded for subsequent transcription and data analysis. An English speaking researcher conducted all of the school staff interviews. All of the parent interviews were conducted in Spanish except one where a Latina parent indicated that English was her dominant language. A bilingual researcher whose ethnic heritage was the same as the Spanish-speaking parents in our case study interpreted the Spanish and English for the English-speaking researcher who conducted the interviews. All three researchers interviewed the students in English according to their preferences. Students were interviewed at school during noninstructional time. Depending on the size of the group, teacher and parent interviews required from 30 to 45 minutes to complete with individual interviews requiring less time. Student interviews were typically less than 10 minutes in duration.

A school staff member was contracted to collect data for the document review of students’ Individualized Education Programs (IEPs) and cumulative files. Rather than conducting a direct review of students’ files, the assistant director of special education requested to first gather any data available in the districts’ computerized student information database. These variables included language assessment results, special education services, prior school history, attendance, behavior, and grades. If this information was missing in the district database, data were gathered from students’ master special education files or cumulative files directly. We provided data collection sheets for this data collection activity that were completed and returned to us in the mail. Any follow up questions were answered by e-mail or telephone.

Data Analysis

To analyze our narrative data, we first transcribed all English portions of the educator and parent interviews verbatim. For the Spanish portions of our parent interviews, the bilingual researcher transcribed the Spanish data and then translated these data to English. All narrative data were then subjected to a content analysis that yielded themes of results. Throughout the qualitative analysis process, English data were compared back to Spanish translated data to ensure accuracy of our interpretations. We use both the original Spanish and the translated English for any supportive quotations taken from the parent interview data. Since the student interview responses were briefer than the parent and educator narrative data, these interviews were not tape recorded. Instead, student responses were written down during the interviews. To analyze the student interview data, we tabulated categories of responses rather than creating themes of results. We employed descriptive statistics to analyze the document review and survey data.

The findings from the document review, written survey, and face-to-face interviews are each presented in different formats. These varying organizational schemes allowed us to highlight the most interesting and useful findings from each data collection activity.

Document Review

For the document review, results are presented by school level so that we can understand each school at an individual student level. Data for nine students at the elementary level are presented with seven students’ data at the middle school level and eight students’ data at the high school level. Within each school level, student demographic and language assessment results are reported. Student demographic variables included grade level, grade promotion history, school transfer history, disability category, special education services, and school behavioral concerns. Unfortunately, data on the number of years spent in the U.S. and the country of origin, two variables listed on our original data collection protocol, were not available.

Language assessment results were comprised of proficiency levels from a standardized English proficiency measure that was used on a large-scale basis. A second English measure, which consisted of a series of checklists that were organized from levels 0–7, was completed by an educator familiar with a child’s communicative patterns. The second instrument was used when the severity of an English language learner’s disability prevented their participation in the large-scale assessment of English proficiency. These students were typically deaf, nonverbal, or significantly cognitively impaired.

Elementary School. The grade level, years enrolled in the district, disability category, and retention status for each elementary school student are presented in Table 1. These students were enrolled in 1st (n = 1), 2nd (n = 1), 4th (n = 4), and 5th (n = 3) grades. Three students had been retained at some point during elementary school. All but one of the students attended school in only this district since kindergarten. The remaining student enrolled in this school in the 4th grade, after attending elementary school in another city in the state for at least one year prior to moving into this district. Years of enrollment in this school district ranged from one to nine years.

The number of elementary students in each proficiency level based on the scores from the state developed English language proficiency test is presented in Table 2. Each student’s English language development was assessed in the areas of oral language, reading, and writing. They were scored as having beginning, early intermediate, intermediate, early advanced, or advanced emerging English. Generally speaking, most students’ proficiency levels were beginning and early intermediate for both the content areas and overall levels.

Two students were administered the Kendall Conversational Proficiency Levels (KCPL) test in lieu of the state English proficiency test due to the severity of the students’ disabilities. The KCPL measured expressive communicative competence, organized from level 0 to 7, via a series of checklists completed by one or more adults who were knowledgeable about the student’s communication patterns. The district used the KCPL as an alternative to the state English proficiency test for students who are deaf, non-verbal, or significantly cognitively impaired. The KCPL test results are in levels, which are then converted to the states’ English proficiency test descriptor scores. One student scored a KCPL level of 3, converted to an overall early intermediate English proficiency level and the other student scored a KCPL level of 0, converted to an overall beginning English proficiency level. These two students’ test results are only included in the overall category in Table 2.

Middle School. The grade level, number of years enrolled in the district, disability category, and retention status for each middle school student is presented in Table 3. The students at the middle school were enrolled in 6th (n = 2), 7th (n = 3), and 8th (n = 2) grades and had attended school in this district from two to ten years.

Two of the 7 students had attended schools outside of the district in elementary school. One student had no school records prior to 3rd grade where the student presumably attended school in another country, attended 3rd through 5th grades at other schools in the U.S., and enrolled in this district in 6th grade. The other student attended elementary school in another city in the U.S. until being retained in 3rd grade. Upon enrollment in this district, the student repeated 3rd grade.

Most of the English language learners at the middle school had learning disabilities with one additional student who had an orthopedic impairment (OI). Four students had been retained at some point in their school history, and two of those students were retained twice. In terms of special education services, four students, including the student with an orthopedic impairment, attended a segregated special day class in the middle school. One of those students also received special transportation and behavioral intervention services. The other three students received resource specialist program services. Five students were reported as having some behavioral concerns.

The number of middle school students in each proficiency level based on the scores from the state-developed English language proficiency test is presented in Table 4. In terms of English oral skills, the majority of students were at an early intermediate or intermediate level. Most students were functioning from beginning to intermediate levels of proficiency in reading and writing, which was true for the overall level as well.

High School. The grade level, years enrolled in the district, disability category, and retention status for each high school student is presented in Table 5. The students at the high school were enrolled in 9th (n = 5), 10th (n = 2), and 12th (n = 1) grades. Three of the 7 students had attended schools outside of the district prior to 2002–2003 school year. One student attended a school in another state for 7th grade, but school attendance for 6th and 8th grade was not documented. Another student had no educational records prior to 4th grade when