Growing inconsistencies in how disabled and Limited English Proficient (LEP) students are either accommodated or excluded from NAEP testing may soon make it impossible to compare NAEP test data across states.
The possible obliteration of NAEP trend lines from the past 30 years is a matter of growing concern within the National Assessment Governing Board (NAGB), the nonpartisan entity that oversees NAEP.
Minutes of the August 4 meeting of an NAGB subcommittee, obtained by the Lexington Institute, report that federal officials expressed concern not only over the validity of data samples collected for this year’s assessments, but also for the usefulness of future comparisons.
Until 1996, NAEP tested all students under equal conditions. Since then, special accommodations–such as extra time for test-taking–have been made for disabled and LEP students. In 1996, NAEP began splitting its samples between students who took the tests under standard conditions and those who were given special accommodations. In an attempt to maintain valid trend lines, the samples of accommodated pupils were kept as research, while the official reported results included only the scores of students tested without help.
“However,” note the NAGB minutes, “under the federal Individuals with Disabilities Education Act (IDEA), as amended in 1997, all IEPs [Individualized Education Plans] for disabled students must now include a statement on testing accommodations. This appears to have led to a substantial increase in the number of students receiving [accommodations], but there is little consistency in the criteria for making such determinations and the situation varies widely from state to state.” Kentucky, for example, permits reading tests to be read aloud to LEP students, while California bars that practice.
In 1999, NAGB decreed that NAEP national and state results for 2000 would be reported only for those students tested under standard conditions. However, the National Center for Education Statistics (NCES) subsequently recommended the decision be reconsidered because of rising rates at which some states are excluding students from the testing.
At the August 4 meeting, Peggy Carr, acting NCES commissioner, said “preliminary state-by-state data indicate that this increase has occurred and could significantly threaten the trend line of NAEP assessments.” The Educational Testing Service found that since 1996 such exclusions in eighth-grade math had increased in 27 states, fallen in four, and stayed the same in seven.
Such disparities in the 1998 NAEP testing threw a dark cloud over apparent gains in reading–gains that Vice President Al Gore had touted as evidence that federally favored “systemic reforms” were working.
The NAGB committee recommended that the full board release results from NAEP’s 2000 assessments with data from samples both with and without accommodations. Results for students tested without special help would be regarded as the official scores in an attempt to preserve trends.
An NAGB staffer, however, suggested that the percentage of students participating was more consistent state-to-state in samples with accommodations, so those might be regarded as “a new baseline for state comparisons.” Acting NCES commissioner Carr reported future data would be flagged to indicate states with high levels of exclusions and large changes in exclusion rates.
While the future usefulness of NAEP data for state-by-state comparisons is in doubt, the Presidential candidates from both major parties have proposed using NAEP results to judge the worthiness of states for federal education funding.
Robert Holland, a journalist and author who has championed school choice throughout his career, is a Heartland Institute Senior Fellow addressing education policy.
