A study by Education Consumers Clearinghouse (ECC) founder J.E. Stone has concluded nationally certified Tennessee teachers are no more proficient in raising students’ test scores than average teachers in their home school districts.
Stone’s findings caused those with a vested interest in national certification to spring into action, most notably through an Education Commission of the States press release asserting it would empanel an “unbiased” review of the study.
The National Board for Professional Teaching Standards (NBPTS) has been in existence for 15 years and has spent $215.6 million in tax and private funds. Stone’s study, however, was the first to assess the impact of national certification according to objectively measured student achievement.
In January, NBPTS officials had issued an open call for research on how certification might relate to achievement. “We’re not just looking for feel-good research,” said Ann E. Harman, NBPTS’ research director. “We’re ready for whatever the results are.” (Education Week, January 30, 2002)
In Fall 2000, NBPTS President Betty Castor touted a study by researchers at the University of North Carolina at Greensboro, who compared 31 teachers who won certification to 34 who applied unsuccessfully. The study was financed by the U.S. Department of Education and NBPTS. That the nationally certified teachers ranked higher on criteria the NBPTS deems important to teaching, Castor said, merited “the absolute highest confidence” that NBPTS-certified teachers are giving students “a high-quality learning experience.” (Education Week, October 25, 2000.)
In a May 15 advertisement in Education Week, NBPTS boasted National Board Certification is “Comparable to established standards in other professions,” and said the UNC-Greensboro study showed “National Board Certified Teachers outperform their peers in teaching expertise and student achievement.”
However, the touted study counted only student work samples gathered by the teachers themselves and deliberately excluded students’ test scores. Critics have continued to press NBPTS for hard, verifiable data.
In a statement, NBPTS officials slammed Stone’s work as “hardly independent research,” noting he has criticized the NBPTS and advocated market-based reform of teacher preparation and licensing. Stone countered that researchers—including those employed by the NBPTS—rarely study issues about which they lack opinions; the relevant question is whether opinions determined the outcome. The NBPTS also criticized the small number of teachers in Stone’s study—16—although the NBPTS’ own studies over the past decade have included as few as three teachers.
Measuring How Much Students Learn
Stone’s study tapped into the sophisticated statistical analysis pioneered by Dr. William Sanders in the Tennessee Value-Added Assessment System (TVAAS), which provides a measure of how much learning a student gains from a specific teacher. Sanders was not a party to the Stone study and in fact has been commissioned by NBPTS to study the effectiveness of NBPTS-certified teachers in North Carolina using value-added data.
Sixteen of Tennessee’s 40 NBPTS-certified teachers teach in grades 3 through 8 and therefore have value-added reports of teacher effectiveness in the state database.
Stone, an education professor at East Tennessee State University, compared the “teacher-effect” scores of those 16 to the average achievement gains of their local school systems in as many as five subjects and over as long as three years. Tennessee’s performance standards grade an achievement gain of 115 percent or more as an “A” and a gain of 85 percent of less as an “F.” Stone found only 15 percent of scores earned by the NBPTS-certified teachers reached as high as 115 percent, while 11 percent were at 85 percent or below. The remaining three-fourths were within the average range for their school systems.
Noting Chattanooga gives $5,000 performance bonuses to teachers who reach 115 percent gains in math, reading, and language for the preceding three years, Stone pointed out none of the NBPTS-certified teachers would have qualified for that reward.
It is implausible that these 16 teachers are the only mediocre performers among the 16,000 NBPTS-certified teachers nationally, said Stone, but “if a state found that it had 16 certified lifeguards who were only average swimmers, the finding would not be dismissed as statistically insignificant.”
Study “Well-Conceived”
Noted education researcher Eric Hanushek, a senior fellow at the Hoover Institution, told School Reform News that, contrary to the NBPTS’ insinuations, “Stone’s study follows a well-conceived methodology.” He added that “Tennessee is not an aggressive NBPTS state,” and therefore Stone had no choice but to rely on a mere 16 teachers in the database. “This weakness is not Stone’s fault, or his choice. It simply represents the available universe of teachers.”
Hanushek observed, “John Stone’s provocative study underscores one extremely important feature of U.S. education: Widely acclaimed and expensive policies frequently escape any evaluation in terms of their true effectiveness. Stone’s study is far from definitive, but it is the evidence that is available.
“The NBPTS certification process has become an important element of policy in several states, and large financial rewards flow to successful applicants. But all of this has happened without a thorough analysis of its effectiveness.”
States and school districts have awarded pay increases or bonuses in the range of $5,000 to $7,500 per year for teachers winning national certification. After paying the NBPTS a $2,300 application fee—a tab sometimes picked up by their districts—candidates prepare a portfolio of their work, videotape themselves teaching, and take an all-day examination.
Hanushek said now that Stone has asked “the right question,” others—including states with a significant financial stake in the NBPTS process—should look at the evidence also.
“One thinks that, had Stone’s study of 16 teachers supported the certification program, it would have been widely publicized and little criticized,” Hanushek commented.
Review Panel Named
Soon after ECC’s release of the Stone report, the Education Commission of the States (ECS), which bills itself as a nonpartisan education policy organization, issued a press release calling for a review panel. The following panel members were appointed in mid-June: Susan Fuhrman, dean of education at the University of Pennsylvania; Dominic Brewer of the RAND Corporation; Robert Linn of the University of Colorado; and Ana Marie Villegas of Montclair State University.
ECS President Ted Sanders said in a statement his organization had an “obligation to determine the validity of this study and whether ECS constituents can depend on it for altering the course of their work to improve teaching quality.”
Sanders’ own objectivity on the matter could be in question. He was a founding commissioner of the NBPTS’ greatest champion, the National Commission on Teaching and America’s Future, an organization bankrolled by the Carnegie Foundation, as is the NBPTS itself. There is no record of the ECS ever having challenged the validity of NBPTS studies claiming to show the effectiveness of nationally certified teachers.
Robert Holland is a senior fellow at the Lexington Institute, a public policy think tank in Arlington, Virginia. His e-mail address is [email protected].
For more information …
Links to the Stone study and to other material pro and con can be found at: www.education-consumers.com/briefs/may2002.asp.
Robert Holland, a journalist and author who has championed school choice throughout his career, is a Heartland Institute Senior Fellow addressing education policy.
