Photo: Denver Airport pano, by Doug Waldron, CC license
“Denver Airport” has become synonymous with epic technology failure for those who remember the colossal breakdown of that airport’s ambitious new automated baggage handling system in 1995. Just two numbers explain the magnitude of the failure: 16 months — the delay in opening Denver’s new airport – and $560 million – the extra cost of building the new airport – both a direct result of the baggage system debacle.
Denver Airport has huge lessons for us that can be applied to any software endeavor.
In 1993 a brilliant plan was hatched to fully automate baggage handling at Denver’s new state of the art airport. The key to the automated system was software that would control the movement of thousands of carts moving along miles of track throughout the airport’s three main terminals. The software would direct the movement of the carts to collect or deposit bags at precisely the right time. It was to be a highly orchestrated activity that depended on software that would continuously process complex algorithms. After more than a decade of trying to make the system work, the airport went back to the traditional manual bar code tug and trolley system.
Key Points
Hubris can lead to very bad decisions. The airport’s new system was to be state of the art, the most automated passenger baggage system in the world. Through 26 miles of underground track, bags would move from plane to carousel or gate to plane without human handling. Tours were given to the public to show how advanced the new system would be.
BAE Systems, the company hired to design and build the baggage system, had built several traditional systems at airports throughout the world, but none were as advanced as the Denver project’s design. A sign of early trouble came when the airport bid the design out for construction, and several companies either declined to make a bid or responded that construction of the complicated system within the airport’s stated timeline was impossible. The Denver team was proud of its futuristic design and even these clear signals of danger ahead did not dissuade them from their plan.
Complexity greatly increases risk of failure. All along the miles of track of the new system, thousands of small carts would deposit or pick up baggage at precise points in the network, ostensibly at just the right time. This required complex algorithms that had to account for travel distance, expected flight arrivals and departures, sorting rules and routings, and canceled flights. Scanning devices positioned at just the right locations would read bar code labels applied to bags and route them to the appropriate conveyor. This type of technology, while standard today, still isn’t perfect; you can imagine its relative immaturity in the 1990s.
A ‘Big Bang’ launch of a new system adds to likelihood of failure. Denver’s airport authority had a great opportunity to start small and prove out its advanced system design. The team could have sliced the project into digestible parts in two ways: An end-to-end prototype system on a small portion of the airport’s baggage traffic, or a fully functional piece of the envisioned architecture, such as the bar code label and scanning functionality. Small pieces are easier to focus resources on. Best of all, had the Denver team phased in the system gradually and still faced failures, it wouldn’t have shut down the entire airport. Being able to continue business as normal if the new system fails is an essential but sometimes forgotten aspect of all software projects. Building in pieces or parts allows new learning to be incorporated into the design. Instead, the team gambled on launching the whole system at once.
No backup plan = nasty outcome. Once the Denver team realized that it may take awhile to make the new system work, they rushed to put in place a more traditional trolley system using baggage handlers. But this alone took many months and an extra $70 million before it was completed. In the end, the original advanced-design system was only ever used for outbound baggage at one terminal. Large parts of the airport’s new system simply never worked. But a failure or cost overrun on the original ambitious project is one thing. Because there was no backup, the project was squarely in the critical path of the new airport’s opening. If you’re going to fall out of the boat, don’t drag everyone with you.
OK, you say, your organization isn’t launching anything nearly as ambitious as the Denver Airport baggage system. And of course you would never make the dumb mistakes they made, right? Yes, you would, because every human being makes mistakes; the Denver team just put a lot more at stake. It’s not so much that their software design failed, it’s that they placed a huge bet on their system working – the opening of a new $3 billion airport.
