“The results across these studies are so consistent and strong that I’m developing a pretty high level of confidence that offering families choice improves student achievement.”
Without a doubt, the last half-dozen years have been one of the most exciting and productive periods in the history of parental choice in education, both in the creation of new programs and in the reporting of positive research findings on outcomes.
It also has been one of the most contentious periods, with research findings and researchers alike coming under almost constant attack from groups opposed to school choice.
The scientific method demands that research results be robust enough to withstand strict scrutiny over how research is conducted and how results are derived. Nevertheless, researchers in school choice have found their work needs continued defense against ongoing attempts by opponents to discredit its worth.
Jay P. Greene has been a target center in this firestorm ever since he decided to focus on education research after receiving his Ph.D. in political science from Harvard University in 1995, where he was a student of Paul E. Peterson. Greene was part of the team that conducted the first independent evaluation of the Milwaukee Parental Choice Program, and he subsequently coauthored evaluations of school choice programs in other cities, including Cleveland, Ohio, and Charlotte, North Carolina. He has presented the findings from these evaluations in popular outlets like The Wall Street Journal and The Washington Post as well as in books and scholarly journals.
Currently a senior fellow at the Manhattan Institute for Public Policy Research, Greene received his B.A. in history, summa cum laude, from Tufts University in 1988. He has been a professor in the Government Department at the University of Texas and Political Science Department at the University of Houston. He lives with his wife and three children in Weston, Florida.
Findings from Greene’s research on the effects of school choice on racial integration and civic values have appeared in The New Republic, Commentary, and the Brookings Institution book, Learning from School Choice. His most recent publication is The Education Freedom Index, a report from the Manhattan Institute that measures the degree of education freedom in each state and its relationship to educational achievement.
Currently, Greene is conducting evaluations of the new statewide school choice program in Florida as well as long-running programs in Maine and Vermont. He also is the executive editor of Education Matters, a new education journal to be launched this winter. Greene recently spoke with School Reform News Managing Editor George Clowes.
Clowes: How much has the school choice landscape changed over the past six years?
Greene: In the last several years, there’s really been an enormous spurt of high-quality research on school choice. I dare say that there’s more high-quality evidence available on school choice than there is on almost any other issue in education policy. The research designs employed are quite good, and the effort devoted to collecting data has been quite good as well.
In 1995, we had some very interesting and persuasive theoretical arguments available, but the evidence was limited by limitations in research design. We had theories about why private schools might be more successful than public schools. We had theories about why systems of choice ought to improve educational quality in general, for choosers and non-choosers alike. But we didn’t have evidence that persuaded skeptics very well at all.
James Coleman observed differences in student achievement between public and private school students and noted those differences remained even after controlling for observed demographic differences between the kinds of students in the two sectors. The trouble was, skeptics could always complain that unobserved characteristics of families in private schools really explained the better achievement in private schools.
Their main argument was that highly motivated families would be more likely to send their children to private schools, and so the better achievement of private school students would be attributable to the parental motivation, and not to any particular efficiency in the private sector. One can never fully resolve that issue as long as the research design was simply a comparison of students already in the two sectors, controlling only for observable characteristics.
Clowes: When did a better design come along?
Greene: The Milwaukee study, released in 1996, was the first random-assignment experiment in school choice. The design actually came about by good fortune. Policymakers decided that when voucher school seats were over-subscribed, holding lotteries would be a fair way of allocating limited spots. That had the benefit of allowing researchers to compare lottery winners to lottery losers. Doing the best we could with the available data, we observed significant benefits from participating in that school choice program–very strong in math, and weaker in reading.
Cecilia Rouse re-analyzed the data, employing slightly different analytical techniques, and found pretty close to what we found. She confirmed very similar gains in math scores, and found similar effects in reading, except they fell short of statistical significance. So we have two studies showing significant benefits, and the random assignment design gets around the problem of accounting for parental motivation.
Clowes: What if only more motivated parents are applying for vouchers?
Greene: That’s a very good point. There are actually two different issues in drawing conclusions from research. One is called internal validity, and the other is external validity.
Internal validity is the confidence you have that your results are accurate in describing the population being studied. External validity is your ability to generalize from the study to a larger population.
Now, random assignment solves a lot of internal validity problems. That is, if students are randomly assigned to two different groups, then we can have very high confidence that any differences we observe between the outcomes of the groups after a period of time are attributable to the different treatment each student received after being assigned to the group. Some students get one treatment and others get a different treatment based on chance alone.
Clowes: Let’s stay with internal validity for a moment. How do the gains in the Milwaukee voucher program compare to the gains from the Tennessee class-size reduction experiment?
Greene: We’re talking about something that’s bigger. The Tennessee STAR Study, which compared students assigned at random to classes of either 25 or 18 students, produced a benefit of .21 standard deviations after two years. Our findings in Milwaukee were one half of a standard deviation (.5) gain in math and one quarter of a standard deviation (.25) gain in reading after four years.
Using standard deviation units is a convention among researchers that allows you to compare across different kinds of tests where the points might be different. Education researchers think of .2 or .3 standard deviation gains as being moderate, and larger than that as being fairly big. Frankly, it’s hard to find any intervention that produces large gains in education. So to see something big, especially in a short period of time, is rare. It’s very encouraging. It’s worth being excited about.
I think the four random assignment studies that just came out are very exciting. A team from Harvard, Brookings, and the University of Wisconsin released three separate studies of random assignment scholarship programs in New York, Washington DC, and Dayton, Ohio; and I released through the Manhattan Institute the same kind of random assignment program in Charlotte.
The results across these studies are so consistent and strong that I’m developing a pretty high level of confidence that offering families choice–particularly poor, African-American families–improves student achievement.
Clowes: Let’s go back to external validity–the ability to generalize from the study to a larger population. Are voucher students the “cream” of the student body, or are they pretty much the same as the students who don’t apply?
Greene: The evidence from all of these choice experiments is that the students involved are disadvantaged in many ways. The families in Milwaukee were almost 100 percent African-American or Latino. Their average family incomes were below $20,000. About three-quarters were from single-parent families. They’re not what we imagine as elite students.
Now it is true they had slightly better-educated mothers than the general Milwaukee population, but there’s also evidence they had significantly lower initial test scores before they transferred into a private school. A lot of them were in trouble. They were doing badly in the public schools and they had higher rates of behavior problems.
Also, keep in mind that early on in the choice program in Milwaukee the schools involved were secular, urban private schools that did not have fantastic reputations, nor were they well-endowed with resources or institutional history. That was because the rules excluded religious schools, which make up about 90 percent of the private school capacity in Milwaukee. The schools also had to accept very low per-pupil amounts for tuition, and they had to accept students at random. All of this drove away a lot of the higher-quality private schools.
Can we generalize from the results of these random assignment experiments? Since the families are generally highly disadvantaged, we might be able to generalize to other disadvantaged populations. Second, even if the effects are slightly different for the more general population, finding a significant effect for any population is encouraging. It at least says that it’s beneficial for someone.
In each of the cities that were studied in the reports just released, the families that participated in the programs had slightly higher income than the otherwise eligible families that did not participate. But that is true of every anti-poverty program. Of those people who are eligible for food stamps, the people who use them are of higher average family income than people who don’t. That’s simply because the poorest of the poor often have difficulty taking full advantage of programs designed to assist them. So to condemn these programs for failing to serve the poorest of the poor is to condemn anti-poverty programs in general.
Clowes: Is there any evidence to indicate whether public schools change in response to parents choosing other schools?
Greene: None of the existing school choice programs is large enough or has been around long enough to see effects on public schools for good or ill with any confidence. But there are two studies that allow us to address this question.
One is the Education Freedom Index I just released through the Manhattan Institute. The report speaks to the question of whether choice benefits non-choosers as well as choosers. It asks: Is the average student helped or hurt when more choices are available to parents?
When you compare the extent of choice across the 50 states, you find states that offer parents a greater range of options in education have higher student achievement than states that offer fewer choices. This is the case after controlling for household income, racial composition, per-pupil spending, and class size.
The second study is Caroline Hoxby’s work on choice among public school districts. Her research shows there is higher student achievement at lower per-pupil expense in those metropolitan areas that have more school districts for parents to choose from. Again, that result is consistent with the idea that a system of choice improves learning for the average student and not just those who participate as active choosers.
I think it’s worth recognizing that public schools need resources to provide a quality education, and they need an incentive to use those resources effectively. Addressing one without addressing the other is likely to be insufficient. It’s also possible that choice could improve student achievement simply by allowing people to sort themselves into the kinds of schools that are right for them. It’s like restaurants, where different people have different tastes, and part of the benefit of choice in restaurants is that you can find the right kind of restaurant for you.
Clowes: But school choice opponents argue that parents would sort themselves into schools for the wrong reasons, with the result being greater segregation and harm to society.
Greene: Both the evidence and theory suggest otherwise. It’s also hard to imagine how one could have a more segregated system than what our current system encourages.
My own analysis of the National Education Longitudinal Study shows that privately chosen schools are better racially integrated on average than are public schools. Why is that? I think it’s because public schools, for the most part, assign students to schools based on where they live, and housing patterns are highly segregated by race and class. What choice offers is the possibility that people can cross these political lines to attend schools that are better racially integrated.
Two recent studies–one I conducted of the choice program in Cleveland, and another Howard Fuller and George Mitchell conducted in Milwaukee–show those choice programs actually produce better racially integrated schools. So it’s not just something one has to extrapolate from private schools: It’s actually happening with the pilot choice programs.
I’m not saying chosen schools are likely to be wonderfully integrated. In fact, the evidence from existing private schools is they’re moderately well-integrated. They are just not as badly segregated as the public schools.
People are willing to cross political and psychological lines because private schools offer two things that are particularly attractive. One is that they often offer a common purpose that may transcend race, like religion, or a shared pedagogical approach.
The second thing a private school offers that makes racial mixing easier is confidence that the school will be a safe environment. Parents then are willing to “risk” integration, as it were, because they have greater confidence the school will do it well. The difficulty the public schools have, in addition to the fact that they’re constrained by politically drawn lines, is that they’re constrained because they have a weak sense of mission, no over-arching mission, and they also have been losing the confidence of parents in terms of discipline.
Clowes: What about teaching democratic values, something the public schools claim to do well?
Greene: I’ve published a few studies on this issue. One is in the Georgetown Public Policy Review, where I examined a national sample of adult Latinos and looked at measures of tolerance. Adult Latinos were asked, “Name your least-liked group.” Then they were asked if they would let members of that group run for office, hold a rally in their town, and other things like that. The more willing you are to let members of your least-liked group do those things, the more tolerant you’re said to be.
It turned out that the more that Latinos had attended private school–controlling for other demographic differences–the more tolerant they were likely to be. So it appears that private schools actually more effectively convey the democratic norm of tolerance than do public schools–especially to a group with a very high proportion of immigrants, or children of immigrants.
Now, that’s exactly what public schools were set up to do. One of the strong motivations behind establishing universal public schooling was that people wanted to convey democratic values to new generations and particularly to immigrants. It appears as if government control of schools is not necessary, or even beneficial, for conveying those values.
Why is that? My guess is that to convey these democratic norms requires addressing some very controversial subjects. To talk about tolerance of highly disliked groups requires talking about those groups, which raises controversies in schools and in the classroom. Public schools, by virtue of their political governance, may be averse to controversy, and therefore may shy away from teaching these subjects at all for fear of alienating or offending any group as part of the discussion. Private schools may just have greater self-confidence to take on these tough subjects.
George Clowes is former Heartland senior fellow addressing education policy. He served as founding managing editor of School Reform News between November 1996 and January 2005.
