NCES Comparison Invalid, Critics Say

Published November 1, 2006

A study released in August by the National Center for Education Statistics (NCES) claimed charter school students don’t perform as well academically as public school students. But education reform advocates said the researchers’ choice of comparisons doesn’t accurately reflect charter schools’ performance.

The study, “A Closer Look at Charter Schools Using Hierarchical Linear Model,” is based on a 2003 comparison of academic achievement for fourth-graders enrolled in charter and traditional public schools.

According to the reading comparison, charter schools scored an average of 5.2 points lower than other public schools on the National Assessment of Educational Progress (NAEP). After adjusting for multiple student characteristics, the average difference was 4.2 points. In math, charter schools averaged 5.8 points lower than public schools; after adjusting for student characteristics, the average difference was 4.7 points.

The study sample included 150 charter schools and 6,764 public schools. School participation rates for the NAEP tests were 100 percent for both charter and public non-charter schools; student participation rates were 92 percent for charter schools and 94 for public non-charter schools.

The NCES is a research arm of the U.S. Department of Education.

Wrong Model

Robert Enlow, executive director of the Indianapolis-based Milton and Rose D. Friedman Foundation for Educational Choice, said the study was invalid because it did not actually compare schools. Instead, he said, it compared types of students. A hierarchical linear model examines differences between two different types of subjects being studied, whereas in a longitudinal study the same subjects are observed over multiple time periods.

“Since charter schools seek out the students that public schools can’t handle, the study would have been more substantial if it were a longitudinal study,” Enlow said. “The NCES did not take a bad study–they just didn’t say anything about the quality of the schools.”

Jeanne Allen, president of the Center for Education Reform in Washington, DC, agreed, saying this particular interpretation of government data adds nothing to the question of how well students perform in charters compared to other schools.

“The education establishment–teachers, unions, school board associations–tout these flawed studies in an attempt to discredit new school opportunities for parents,” Allen said. “My message to any parent, especially those with children attending charter schools, is to look at state-level assessments and get a real picture of student achievement. It’s there that charter school students shine like the true stars they are.”

Daschell M. Phillips ([email protected]) is a freelance writer in Chicago.

For more information …

“A Closer Look at Charter Schools Using Hierarchical Linear Model,” National Center for Education Statistics,