In one of the hardest-hitting Government Accountability Office reports I’ve ever read, Congress’s auditing organization has, in effect, said the Transportation Security Administration’s Screening of Passengers by Observation Techniques (SPOT) program does not work and should be defunded. Members of Congress asked GAO to answer two questions:
1. To what extent does available evidence support use of behavioral indicators to identify aviation security threats?
2. To what extent does TSA have data necessary to assess the effectiveness of the SPOT program in identifying threats to aviation security?
The answer to the first question is that there is no such evidence, and to the second is that TSA does not have such data. This is laid out in 55 pages of text plus seven appendices. (“TSA Should Limit Future Funding for Behavior Detection Activities,” GAO-14-159, November 2013).
TSA says the purpose of the program is to identify high-risk passengers based on behavioral indicators that indicate “mal-intent.” Accordingly, its Behavior Detection Officers (BDOs) are trained to size up passengers as they await screening using a memorized checklist of behaviors indicative of stress, fear, or deception. Passengers with a sufficiently high point score are taken aside for an interview, pat-down, and search of their belongings.
Assuming nothing bad is found and the person’s behavior does not “escalate,” that’s the end of the process and the passenger gets back in line. But if the behavior reaches a pre-defined threshold, a law enforcement officer (LEO) is summoned to question the passenger further and decide whether an arrest is warranted. The initial (pre-LEO) encounter takes an average of 13 minutes. The program started in 2007 and has grown to about 3,000 BDOs working at 176 airports, at a current annual cost of around $200 million.
TSA’s Studies Lax, Flawed
The GAO team reviewed two TSA studies of the SPOT program and found both to be non-rigorous, with considerable flaws in their methodology. It then carried out a literature review and a meta-analysis of research studies on “whether nonverbal behavioral indicators can be used to reliably identify deception.” They concluded “research from more than 400 separate studies on detecting deceptive behavior based on behavioral cues or indicators found that the ability of human observers to accurately identify behavior based on behavioral cues or indicators is the same as or slightly better than chance.”
GAO provides excerpts from several of these studies, by entities such as RAND Corporation, DOD’s JASON program, and MITRE Corporation. Another section of the report documents the wide variation in referral rates by BDOs at various airports, and presents evidence on the subjective nature of some of the behavioral indicators BDOs are taught to look for.
No Terrorists Identified
But the most damning information of all is who actually gets identified as “high risk” and referred to a LEO. Not a single potential terrorist was identified by the BDOs. Those who ended up arrested were for such matters as possessing fraudulent documents or prohibited or illegal items, having outstanding warrants, being intoxicated in public, being in the country illegally, or engaging in disorderly conduct. While all those things may be law violations, not a single one is, per se, a threat to aviation security. And yet the only measure TSA has for the alleged effectiveness of the program is the referrals to law enforcement.
Nonetheless, TSA recently conducted a “return-on-investment analysis” which it claims justified the SPOT program. Despite zero evidence that the program can detect or deter aviation-oriented terrorists, the analysis assumes that the BDO “layer of security” prevents a catastrophic (9/11-type) attack. GAO dryly notes that “the analysis relied on assumptions regarding the effectiveness of BDOs and other countermeasures that were based on questionable information.”
TSA’s Requests for More
In response to previous GAO and Inspector General criticism, TSA is developing a new set of metrics about SPOT, but says it will require at least three more years and additional resources to report on the program’s performance and security effectiveness. Meanwhile, TSA is asking for a budget increase to add 584 more BDOs so the program can expand to smaller airports. In other words, to paraphrase a familiar line about the recent federal healthcare law, we have to keep running and expanding the program to find out whether it works.
GAO sums up this comprehensive assessment as follows: “10 years after the development of the SPOT program, TSA cannot demonstrate the effectiveness of its behavior detection activities. Until TSA can provide scientifically validated evidence demonstrating that behavioral indicators can be used to identify passengers who may pose a threat to aviation, the agency risks funding activities that have not been determined to be effective.”
Robert Poole ([email protected]) is director of transportation policy and Searle Freedom Trust Transportation Fellow at Reason Foundation. Used with permission of Poole’s Airport Policy and Security News #96.
