“Out-of-Control” Controls

How often do you encounter a situation where the attempt to “control” something has itself gotten out of control?  Usually it is the result of an overly zealous attempt to “protect” someone or something from the bad behavior or bad results of someone else’s behavior [whether that behavior was ill-intended or otherwise].

The Sarbannes-Oxley legislation of 2002 immediately comes to my mind.  Enacted in the aftermath of the Enron scandal, and intended to make corporations’ financial reporting processes better controlled and less prone to the risk of unexpected financial reporting errors, the legislation spawned an entire new product line for auditing and consulting firms, who sent their “SOX-trained” consultants to corporations near and far, helping them deal with the mountains of forms they were required to prepare and certify in order to qualify for a satisfactory audit opinion.  I’m sure we’ll never know whether the immense cost of Sarbannes-Oxley compliance was worth it. Looking for the Enron that didn’t happen because of Sarbannes-Oxley seems to me like listening for the bell that didn’t get rung.

But we never seem to learn.  Overreaction in regulation appears to be the standard, not the exception. OK. OK. I’ll concede that the latest brouhaha in the financial markets, with JP Morgan Chase flushing away $2 Billion as part of an ill-devised “portfolio hedge” strategy, was probably the result of regulators believing the bulls**t of the Big Banks a bit too much when they were pleading the necessity of portfolio hedges to “protect” themselves, but I’ll also say that it’s the exception that proves the rule.

Here’s what got me going on this rant against Over-the-Top-and-Out-of-Control Regulations. [Before I get started, I feel obliged to disclose that the only reason I know about this situation is that I am in investor in Baltia Airlines.  I’ll also warn you in advance that the narrative that follows has a good bit of jargon, but I think it’s worth the effort to plow through it all.]

Baltia Airlines is a fledgling US airline operating out of JFK airport in NY.  Founded by a few former Russian air industry people, their dream is to operate non-stop passenger flights from JFK airport to Pulkovo International Airport in St. Petersburg, with the bigger dream to expand and fly to other Russian airports like Riga, Moscow, Kiev and Minsk.  The business plan looked attractive enough to bring onboard lots of investors, large and small, including me.

Things were moving along for Baltia, not always at a rapid pace, but the progress was steady.  The company acquired an aircraft, a de-commissioned B-747, which was refurbished, mounted with leased jet engines, prepared for operations at one of the world’s largest fitting-out facilities in Singapore, painted with the Baltia logo, furnished with the latest in aircraft seating and passenger amenities, and test flown.  Baltia rented office space at New York’s JFK Airport, secured gates, even went so far as to hang the Baltia sign at Terminal 4 at JFK, anticipating completion of the final Federal Aviation Administration (“FAA”) Certification in about 12 months.

That was three years ago.  Since then Baltia has been stuck in regulatory quicksand, able to do little more than try to avoid sinking deeper, while the FAA seems to be twiddling its regulatory thumbs.  The problem here isn’t what you might expect – budget cuts and staff shortages at the FAA – though FAA staffing issues have certainly been a problem for the Baltia certification. No, the principal culprit is the protocols surrounding FAA implementation of Safety Attribute Inspections [SAI] as an integral part of the FAA certification process.

Now, I’m as much in favor of airline safety as anyone.  I have flown over two million miles in the past 30 years, and it always feels good when the number of successful landings equals the number of successful takeoffs.  By all accounts, and based on the best available research, high-capacity, regular public-transport commercial airline travel is the safest mode of travel in the world; almost 100 times safer than auto travel. (For comparison’s sake, bus and rail travel are about 7 times safer than travel by auto. Other modes of travel are riskier than auto travel: private aircraft travel is 6 times riskier, bicycling is about 10 times riskier, pedestrian travel is 14 times riskier, motorcycling is riskiest of all at 24 times riskier than auto travel.)  But, as witnessed by the situation with Baltia’s certification, the regulatory environment has obviously been strongly influenced by the relative hue and cry that arises on those extremely rare occassions when a commercial aircraft crashes, almost certainly because the number of lives lost in a single event can be relatively large.  Closer examination shows that a strong case can be made that this particular regulatory rigamarole is definitely out of control.

As best that I can ascertain, The SAI approach used by the FAA attempts to “test” the myriad safety and control procedures used by the airline to make sure that flight operations are as safe as possible.  [There are 35 airworthiness and 34 operations SAI elements, comprising more than 3,000 individual questions.]  While presented as a highly objective scientific analysis tool, the SAI is, in fact, an entirely subjective approach which relies upon the judgment of the FAA Subject Matter Experts [SME], who use their experience to make an assessment of the likelihood that any stated procedure in the airlines’ safety and procedures manuals will deliver the desired “safe” result.

The process seems to work in the following manner:

• The airline’s own Subject Matter Expert [SME] formulates specific text, dealing with each of the 3,000+ SAI questions, and that text is included in the airline’s manuals.
• The appropriate portions of the airline’s manuals are transcribed into an SAI submission to the FAA.
• The FAA’s SME determines whether the treatment of the issue in question is “adequate,” with the judgment about adequacy solely at the discretion of the FAA’s SME, that judgment supposedly guided by considerations of the size and scope of proposed operations.
• If any condition is deemed by the FAA’s SME to be less than 90% likely to ensure that the desired control will not fail, the entire review is halted, and the airline has to start over again.

What adds to the problem is the fact that there are no “right answers” to any of the 3,000+ questions in the SAI.  It comes down to a matter of opinion about whose SME is “correct, the airline’s or the FAA’s.  You’d think that in such situations, the best way to resolve the issue would be to have the two SME’s talk it over, and come to some agreement about where to draw the line, and what language should be included in the manuals.  Unfortunately, the FAA’s SAI review protocols provide no mechanism for interaction to discuss, dispute, and otherwise resolve judgments about the arguably ambiguous and subjective judgments.

This works out to be a bad situation for Baltia Airlines.  But I think it’s indicative of a larger problem in our overly regulated world.

We all want to be safe; we all want to be protected; and we all find ourselves dependent upon complex technologies that we don’t begin to understand.  So, we become incredibly risk-averse, often to our collective detriment.  Nobel-prize winning psychologist Daniel Kannemahn, in his recent book, Thinking Fast and Slow, made the point brilliantly in his discussion of hindsight bias:

“Because adherence to standard operating procedures is difficult to second-guess, decision makers who expect to have their decisions scrutinized with hindsight are driven to bureaucratic solutions — and to an extreme reluctance to take risks.  As malpractice litigation became more common, physicians changed their procedures in multiple ways: ordered more tests, referred more cases to specialists, applied conventional treatments even when they were unlikely to help.  These actions protected the physicians more than they benefitted the patients…”

Sadly enough, when these types of situations occur, our preference to be “safe rather than sorry” leaves us not very safe, and more than a little sorry.