A Harvard University study released in August refutes the conclusions of a federal report released earlier this summer that said students in public schools perform as well as or better than their private school counterparts.
The new study, conducted by Paul Peterson and Elena Llaudet of Harvard’s Program on Education Policy and Governance, criticizes the methodology federal researchers used to analyze the data.
“It is unfortunate that a study commissioned by an agency of the United States government has not kept pace with contemporary research standards,” the authors write.
Flawed Methods
The federal study–commissioned by the National Center for Education Statistics (NCES), a research division of the U.S. Department of Education, and released July 14–compared math and reading scores on the 2003 National Assessment of Educational Progress (NAEP) test for fourth- and eighth-grade students in public and private schools.
Private school students boasted higher scores, but when researchers controlled for factors including race and family income, the private school advantage gave way to a draw in most categories. Public schools did better in fourth-grade math, and private schools performed better in eighth-grade reading, according to the NCES.
But when Peterson and Llaudet reexamined the federal researchers’ methodology, they found the NCES study included two major flaws.
First, the NAEP data reflect student performances at only one point in time. In order to examine how effectively schools raise student achievement, the Harvard study says, researchers must look at data collected over time.
Second, to control for differences among students, the NCES study inferred certain background characteristics of students based on their participation in federally funded programs, including Title I programs for disadvantaged students and the free lunch program. The NCES study undercounts the incidence of economically disadvantaged students in private schools and overcounts this population in public schools, the Harvard authors said.
Different Schools
For example, the NCES reported 44 percent of all fourth graders in public schools who took the math test were receiving Title I services for disadvantaged students, compared with 7 percent of students in private schools.
The Harvard researchers say these differences might be more reflective of differences in the schools than of differences in students as the NCES study suggested.
“Often private schools don’t have motivation or interest in these federally funded programs,” Llaudet explained.
Furthermore, “if a public school has a schoolwide Title I program,” Peterson and Llaudet wrote, “something that is permitted if 40 percent of its students are eligible for the free lunch program, then every student at the school regardless of poverty level is said to be a recipient of Title I services.”
Private schools cannot receive Title I funds, the Harvard authors note.
Apples and Oranges
When Peterson and Llaudet assigned other variables to control for students’ backgrounds–including parental education levels and school location–they found private school students performed better than public school students in 11 of 12 categories.
The Harvard researchers do not conclude that private schools are more effective than public schools in raising test-score performance. “NAEP data are too fragile to permit any conclusion about school sector effects, one way or another,” they wrote.
Limited Relevance
After its July release, the NCES study inspired critics of school choice programs to cite it as evidence such programs are futile. However, even the study’s authors included a caveat in its executive summary: “[T]he estimated effects should not be interpreted in terms of causal relationships.”
“It’s a shame anyone thought this study was relevant to the debate about free-market education reform,” said Andrew Coulson, director of the Cato Institute’s Center for Educational Freedom.
School choice advocates endorse a free-market approach to education, which would allow a parent to choose the best school for his or her child from numerous options. The debate is very different from comparing public and private schools, Coulson said.
Matt Warner, director of the American Legislative Exchange Council’s education reform task force, agreed. “Why are we obsessed with knowing what is best for each child nationally?” Warner asked. “We should look at which school is best for each child, on a case-by-case basis.”
Hilary Masell Oswald ([email protected]) is a freelance writer in Evanston, Illinois.
For more information …
The National Center for Education Statistics’ July 14, 2006 study, “Comparing Private Schools and Public Schools Using Hierarchical Linear Modeling,” is available through PolicyBot™, The Heartland Institute’s free online research database. Point your Web browser to http://www.heartland.org, click on the PolicyBot™ button, and search for document #19702.
The August 2, 2006 report, “On the Public-Private School Achievement Debate,” by Paul E. Peterson and Elena Llaudet, also is available through PolicyBot™. Search for document #19703.
