NASA plays down its air safety report

Just how safe is it to fly these days? Well, according to Michael Griffin, administrator of the National Aeronautics and Space Administration, it's extraordinarily safe.

"This is one of the safest forms of travel that human beings have ever invented," he says.

That assertion of confidence came this week as Dr. Griffin and his staff released 16,000 pages of raw data from a controversial NASA study that seemed to suggest flying may be far riskier than official statistics indicate.

Called the National Aviation Operations Monitoring Service (NAOMS), it surveyed more than 25,000 pilots from 2001 to 2004. According to Griffin, it found that things such as engine failures, near collisions, and bird strikes occur as much as four times more frequently than thought by the Federal Aviation Administration – which is charged with assuring that US skies are safe.

NASA originally tried to keep the study from becoming public, fearing it would undermine confidence in the airlines. In fact, NASA contends the problem is not with the FAA and its record keeping. In a testy teleconference with about 30 reporters on New Year's Eve, Griffin and Bryan O'Connor, NASA's safety chief, said the problem was the $11 million NAOMS study itself. "No product of the NAOMS project ... should be viewed or considered at this stage as having been validated," says Mr. O'Connor.

And so what is the public to make of this? Not much, according to Griffin – except that NASA did not handle the situation as well as it could.

The study was designed to come up with a new methodology for air safety experts to ferret out potential safety problems. Currently, pilots are promised anonymity, and they self-report problems – things such as planes flying too close to each other. The NAOMS project, on the other hand, regularly surveyed pilots about problems they experienced. It was designed to complement the current FAA reporting mechanisms.

But Griffin dismissed it as "hangar talk" and said the data were full of "inconsistencies." For instance, the survey found four times as many cases of engine failure as the FAA statistics show.

"Engine failures are a very high-profile item," says Griffin. "This is an area where if someone comes in and says we're seeing four times as many as are being otherwise reported, it calls into question the reporting mechanism rather than the underlying rate of engine failures, which we believe we understand."

The NAOMS researchers defend their work, saying their methods were state of the art. They believe the FAA's current reporting mechanism has serious flaws that may be leading to significant underreporting of potential hazards. NAOMS was designed to rectify that by adding regular reporting.

"The NAOMS team made an extraordinary effort to clean and validate the data collected through the survey," Robert Dodd, the former principal investigator for NAOMS, told a congressional hearing last fall. "The resulting data is of good quality and ready for meaningful analysis."

NASA initially turned the study over to the FAA, and it was never made public. But the Associated Press got wind of it and filed a Freedom of Information Act request to see the data. NASA originally denied it, stating release "could materially affect the public confidence in, and the commercial welfare of, the air carriers."

"We inappropriately rejected the FOIA request, with inappropriate language, for the release of the raw data," says Griffin.

After being pressured by Congress, he agreed to release the data by the end of 2007. That's what prompted the unusual New Year's Eve teleconference with reporters. But many were left dissatisfied because NASA released incomplete, raw data in a form that makes it virtually impossible to interpret.

Congress was not completely satisfied with NASA, either. "NASA itself concedes that this is not the most complete data set that they intend to release," said Rep. Bart Gordon (D) of Tennessee, chairman of the House Science and Technology Committee, in a statement.

NASA will now turn the NAOMS data over to the National Academy of Sciences, which will do an independent analysis of the project.

Many aviation experts are glad the study will now undergo a thorough review. "I would look at it from the standpoint to see if it's highlighting problems that maybe we need to be looking at more closely," says Clint Oster, an aviation safety expert at Indiana University at Bloomington.