The New York City Charter Schools Evaluation Project

The distinctive feature of this study is that charter schools’ effects on achievement are estimated by the best available, “gold standard” method: lotteries. 94 percent of charter school students in New York City are admitted to a school after having participated in a random lottery for school places. In a lottery-based study like this one, each charter school’s applicants are randomly divided into the “lotteried-in” (who attend charter schools) and the “lotteried-out” (who remain in the regular public schools. These two groups of students are identical not just on dimensions that we can readily observe, such as race, ethnicity, gender, poverty, limited English, and disability. They are also identical on dimensions that we cannot readily observe like motivation and their family’s interest in education. The lotteried-in and lotteried-out students who participated in the same lottery are identical on these subtle dimensions because they all applied to the charter school. They are separated only by a random number.

We follow the progress of lotteried-in and lotteried-out students. We compute the effect that charter schools have on their students’ achievement by comparing the lotteried-in students to their lotteried-out counterparts. This is a true “apples-to-apples” comparison. Lottery-based studies are scientific and reliable. There are no other methods of studying the achievement of charter school students that have similar reliability.

--from the Executive Summary