"You can think of it sort of like a idiots version of Slack"
January 24, 2024 7:55 PM   Subscribe

"Current Boeing employee here – I will save you waiting two years for the NTSB report to come out and give it to you for free: the reason the door blew off is stated in black and white in Boeings own records." In two comments on an aviation blog, someone claiming to be a Boeing employee details the exact steps leading to door plug bolts not being installed on an Alaska Airlines 737 Max 9. The Seattle Times investigated and called the account convincing.
posted by clawsoon (44 comments total) 58 users marked this as a favorite
 
As a business, Boeing is in the "too big to fail" category. But the current management is definitely in "hold my beer!" mode and is willing to give that a test.
posted by Dip Flash at 8:22 PM on January 24 [15 favorites]


Perspective (@Perspective@c.im)
On most aircraft, windshields are installed from the inside to ensure a full, consistent and proper seal that is enhanced by the cabin pressure. They won't blow out.

Why are these door plugs installed from the outside making them ultra reliant on the bolts, pins and pads to hold them in place? 🤔
posted by ChurchHatesTucker at 8:37 PM on January 24 [10 favorites]


As per Dip Flash…737 Max 9: Boeing jets cleared to fly after mid-air incident https://www.bbc.co.uk/news/business-68090175. 🙄 we’re doomed, aren’t we?!
posted by mollymillions at 8:38 PM on January 24 [1 favorite]


Fascinated by the blog comments too. Thanks.
posted by Brian B. at 8:40 PM on January 24 [1 favorite]


so... Jenga but for planes? Cool.

As a (we)a culpa: My partner worked as a lowly graphics assistant at McKinsey on the Boeing account for a few years starting in 1998. We're all reaping the seeds sewn in those years.
posted by turbowombat at 8:43 PM on January 24 [2 favorites]


Why do I have the feeling that in the next 48 hours, Boeing security will spend 10x more effort/cost on trying to track down the whistleblower than anyone else at Boeing will spend on actions to address the systemic issues that led to this incident?

Not that they'll ever be able to reconcile the reality that the pursuit of 'shareholder value' was likely the systemic issue.
posted by jerome powell buys his sweatbands in bulk only at 8:46 PM on January 24 [69 favorites]


The comments make it sound like a chaotic system which will inevitably produce random failures, rather than the incident being a unique one-off failure.

From the industry I'm in, it reminds me a lot of the time pressures and mutual blame games that result in shitty outsourced animation mistakes making it into broadcast episodes.

...except that aircraft shouldn't be built the same way that kid's TV is built! They should have better processes! Aircraft production should never make me think, "Huh, that reminds me of all the shit that went wrong with the Clifford the Big Red Dog reboot"!
posted by clawsoon at 8:46 PM on January 24 [41 favorites]


...finally we get to the damning entry which reads something along the lines of “coordinating with the doors team to determine if the door will have to be removed entirely, or just opened. If it is removed then a Removal will have to be written.” Note: a Removal is a type of record in CMES that requires formal sign off from QA that the airplane been restored to drawing requirements.

If you have been paying attention to this situation closely, you may be able to spot the critical error: regardless of whether the door is simply opened or removed entirely, the 4 retaining bolts that keep it from sliding off of the door stops have to be pulled out.
A removal should be written in either case for QA to verify install, but as it turns out, someone (exactly who will be a fun question for investigators) decides that the door only needs to be opened, and no formal Removal is generated in CMES (the reason for which is unclear, and a major process failure).

Therefore, in the official build records of the airplane, a pressure seal that cannot be accessed without opening the door (and thereby removing retaining bolts) is documented as being replaced, but the door is never officially opened and thus no QA inspection is required.

This entire sequence is documented in the SAT, and the nonconformance records in CMES address the damaged rivets and pressure seal, but at no point is the verification job reopened, or is any record of removed retention bolts created, despite it this being a physical impossibility. Finally with Spirit completing their work to Boeing QAs satisfaction, the two rivet-related records in CMES are stamped complete, and the SAT closed on 19 September 2023. No record or comment regarding the retention bolts is made.

I told you it was stupid.

posted by jerome powell buys his sweatbands in bulk only at 8:58 PM on January 24 [11 favorites]


I looove this “how the sausage gets made” insider info, thanks for posting.
The description of the authoritative workflow process tracking system being used in combination with the frequently-discussed-at-meetings metrics-generating management visibility system (the one you post on when you want people to know that progress is being made) was particularly relatable i mean harrowing.
posted by crime online at 8:59 PM on January 24 [20 favorites]


An official system of record that is painful and another system where people actually get their work done? That is definitely a bad starting point (and all too common).
posted by i_am_joe's_spleen at 9:08 PM on January 24 [20 favorites]


so someone's been poking extra holes in the swiss cheese?
posted by Baethan at 9:15 PM on January 24 [8 favorites]


since we were bought by McD

Took me a few bewildered seconds to realize they weren't talking about McDonalds...

and oh God do Boeing managers love their metrics

If they're like most companies in my experience, "the metrics" serve as more of a red herring and excuse for all sorts of cockamamie ideas to "improve the metrics" rather than contributing to any sort of actual gain in product quality.

Ok, off to read the rest of the post...
posted by Greg_Ace at 9:20 PM on January 24 [11 favorites]


Haven't flown in over a decade. I'll drive anywhere, but flying, just no...
posted by Windopaene at 9:23 PM on January 24


The comments make it sound like a chaotic system which will inevitably produce random failures, rather than the incident being a unique one-off failure.

my first thought is normalization of deviance

Why are these door plugs installed from the outside making them ultra reliant on the bolts, pins and pads to hold them in place

Well they do have to come off the plane sometimes, they're not permanent. You need some mechanism to let you detach them from the plane and take them off. Pushing them inside doesn't seem to help because now they're stuck inside.
posted by BungaDunga at 9:25 PM on January 24 [6 favorites]


Pushing them inside doesn't seem to help because now they're stuck inside.

They're not circular.
posted by ChurchHatesTucker at 9:30 PM on January 24 [15 favorites]


If they're like most companies in my experience, "the metrics" serve as more of a red herring and excuse for all sorts of cockamamie ideas to "improve the metrics" rather than contributing to any sort of actual gain in product quality.

Hey man, MBAs gotta eat too.

Kidding. Having read above about how McKinsey was in there, one can assume that significantly sized domains of function in the company were optimized, if not specifically established, for the sole purpose of giving many an MBA the justification for drawing a Boeing paycheck.
posted by jerome powell buys his sweatbands in bulk only at 9:31 PM on January 24 [5 favorites]


Greg_Ace: If they're like most companies in my experience, "the metrics" serve as more of a red herring and excuse for all sorts of cockamamie ideas to "improve the metrics" rather than contributing to any sort of actual gain in product quality.

In my experience - and I'd be curious to know if yours is the same - an excessive focus on metrics seems to be the result of management having a sense that something was taking too long or costing too much in production, but not caring enough about the actual production process to figure out what the problems were. They wanted a number so that they could feel they were getting useful insights without making the effort to actually get useful insights.
posted by clawsoon at 9:37 PM on January 24 [16 favorites]


Lots of Boeing (or former Boeing) people in the comments on the Seattle Times article; some of them are also interesting reading. One comment about "Stonecipher" and "Boy Scouts" led me to an older Seattle Times article, A behind-the-scenes look at Boeing’s shifting leadership landscape — and its profound effects. It explains why Spirit Aerosystems was split off from Boeing, resulting in this month's mess:
[Boeing's] new leaders were judged by their performance in meeting financial metrics: especially “return on net assets,” or RONA, which measures the ratio of net income to capital employed. In practice, the easiest way to make the number go up was by selling off manufacturing plants.
That seems to answer the question that I've seen in multiple comments on various Boeing stories: Why would Boeing make their processes more complicated by spinning off Spirit Aerosystems? Wouldn't that make everything more expensive? And, indeed, it did:
It might have improved the RONA numbers, but it made everyone’s jobs harder. Now when someone needed a part at the last minute or had an idea for a process improvement, it wasn’t a call with a colleague; it was a negotiation with lawyers, procurement-chain executives, human-resources representatives.
...but it made the right number go up, which brought in more investment dollars. As long as you could appear to be using less money to make more money, the financial markets were happy, no matter how insane the result was on the production floor.
posted by clawsoon at 9:59 PM on January 24 [54 favorites]


That’s a great article Clawsoon.

…The move meant more than a change in time zones. Boeing’s headquarters would be “a new, leaner corporate center focused on shareholder value,” the company’s statement said. The self-conscious emphasis on investors reflected a change within the company’s upper ranks, and especially in its top seat, that took hold in the first decade of the 21st century. Indeed, across America, the Reagan revolution a generation earlier had emboldened corporate leaders — none more than GE’s Jack Welch, who was admired by both Condit and Stonecipher. The business titans embraced the idea that any company has a single social responsibility: to increase its profits.
posted by armoir from antproof case at 10:17 PM on January 24 [11 favorites]


There's just nothing worse than Welch's corporatism. It's insane to take any engineering-heavy company and focus on quarterly profits at the expense of reputation. It's burning seed-corn on an industrial scale. Wall Street doesn't care about reputations because they don't understand the idea: everyone is an equally low-life scumbag only out for money. They are fucking morons that caused this downfall.
posted by netowl at 11:40 PM on January 24 [42 favorites]


At this point, if it's a Boeing, I ain't going.
posted by Dysk at 2:04 AM on January 25 [8 favorites]


Another piece is buried in the Seattle Times article: that they lost machinists with 20 years of experience during the pandemic. Why exactly? I’m betting because Boeing laid them off, because they didn’t see the shareholder value of keeping them employed for later when the pandemic ended.
posted by corb at 2:28 AM on January 25 [17 favorites]




There is an evolution that sadly repeats and repeats and organizations seem absolutely immune to learning from it, probably because the financial incentives for not learning are so high:

1. Designers: We have designed a failsafe, foolproof system so that (horrible outcome X) simply can't happen

2. Management: That system is too slow/expensive/heavy. We will substitute for your system good training/documentation/QA

(time passes)

3. Management: Hey look at this, we could improve our Q3 profits by getting rid of this training/documentation/QA that's clearly extraneous, (horrible outcome X) never happens!

4. Horrible outcome X happens.

5. Management, testifying in Congress: Who could have possibly predicted this??
posted by range at 3:59 AM on January 25 [63 favorites]


Found this link in the comments, which is an article praising this reporting from a competitor. The big story has political dimensions beyond executive pay and cost-cutting.
posted by Brian B. at 4:32 AM on January 25 [7 favorites]


Brian B.: Found this link in the comments, which is an article praising this reporting from a competitor. The big story has political dimensions beyond executive pay and cost-cutting.

Great find. From the story:
The deal, which has long been criticized by crash victims’ families and recently challenged in court by air safety advocates, allowed Boeing to avoid criminal prosecution on fraud charges and shielded Boeing’s senior executives from such prosecution as well.

The agreement — which was announced just days before Trump left office...
posted by clawsoon at 4:59 AM on January 25 [10 favorites]


range, I was hanging out with some oil industry folks around the time of the Deepwater Horizon and they were talking about how it could of happened. One of the guys who worked in accounting at one of the supermajors explained it using more or less the exact same steps you have.
posted by nangua at 5:19 AM on January 25 [5 favorites]


I used to work with Boeing (as a "customer") quite closely, and this account does not surprise me at all. Not that any of the other big defense contractors are any better, mind you. I'm in development, not production, so the circumstances are a little different but the "Kill Bill" klaxon started going off in my head reading all of this.
posted by backseatpilot at 6:01 AM on January 25 [2 favorites]


I've been following that blog comment story. There's no way to tell if it's really well written fiction or if it's a real whistleblower account. I think it's probably genuine, but all the reasons I do are exactly what a good fake would have copied too.

It's fascinating the Seattle Times chose to run the article without confirming their primary source. They clearly disclose they're sourcing from an unverified blog post. And they did get a bunch of other people on the record saying the story was plausible. It's responsible journalism. But it's a bit of a leap. NPR re-reported the Seattle Times' story with no further attribution.

The NYTimes has a story with similar facts asserted. It looks to be their own reporting though, sourced as "according to a person familiar with the matter". That implies they verified the source. But they don't attribute the blog comment so it might be a different source.

Boeing's had a series of stories like this for years now about sloppy work. My favorite: Boeing reportedly investigating empty tequila bottles found on a future Air Force One. Not from a tipsy passenger, it's believed the Boeing employees working on the plane left them behind after some on-the-job drinking.
posted by Nelson at 6:17 AM on January 25 [5 favorites]


Another piece is buried in the Seattle Times article: that they lost machinists with 20 years of experience during the pandemic. Why exactly? I’m betting because Boeing laid them off, because they didn’t see the shareholder value of keeping them employed for later when the pandemic ended.

The idiotariat of the internet has been blaming this on DEI, saying the aircraft weren't being built as well because they had to hire a bunch of women and [insert ethnic group you would like to blame]. They also slagged off the pilot on the Alaska Airlines flight because her voice was shrill on the ATC recording. Mind you, she said exactly what she should have and reacted calmly, but she sounded wavery, which means everyone was about to die of wokeness.

Why do I know these things? Glutton for punishment -- I'm interested in aviation (despite having exactly no qualifications for it) and I read comments in hopes of finding knowledgeable ones. Sometimes you do get someone who knows what they're talking about, as you can see.
posted by Countess Elena at 6:37 AM on January 25 [18 favorites]


Haven't flown in over a decade.
posted by Windopaene
Eponysterrified
posted by wenestvedt at 6:38 AM on January 25 [6 favorites]


I'll drive anywhere, but flying, just no...

"From 2002 to 2020 ... the average annual injury rate for air travel was .01 injuries per 100 million passenger miles traveled, compared with 48 injuries for the same distance traveled in cars and trucks." Yes, being a safe and careful driver makes your risk of injury lower than average, but not thousands of times lower.

The Boeing processes are fucked up, but our acceptance of the carnage on the roads is far more fucked up. For example, manufacturers shouldn't be allowed to sell cars without speed-limiting systems.
posted by Mr.Know-it-some at 6:40 AM on January 25 [15 favorites]


since we were bought by McD

I know some folks who work at Boeing St Louis, a McDonnell Douglas former location, and they're all like “it's been terrible since Boeing took over”. Hmm.

I may only be hearing anecdotes and perhaps apocrypha, but it sounds like the most corrosively toxic workplace ever. They had a spate of human poop showing up in people's offices and workspaces a few years back. Dunno how that was resolved.
posted by scruss at 7:16 AM on January 25 [3 favorites]


As of a few years ago, Boeing was claiming to build ~42 737s every month, about 500 planes per year. Consider these numbers in that light:
...this check job that should find minimal defects has in the past 365 calendar days recorded 392 nonconforming findings on 737 mid fuselage door installations (so both actual doors for the high density configs, and plugs like the one that blew out).
~500 units per year producing ~400 nonconforming findings. ~400 times that re-work was required to make the parts that were already built look like the specifications. For these doors alone. That sounds like a system that's using QA as a kind of to-do list.

Maybe the world would be OK if something as big and as expensive and as complicated and as carbon-belching as a jetliner took more than 9 days to build.
posted by Western Infidels at 7:22 AM on January 25 [6 favorites]


Kinda reminiscent of the 2003 NOAA N-Prime tipover at Lockheed . . .
posted by torokunai at 7:46 AM on January 25 [1 favorite]


Why are these door plugs installed from the outside making them ultra reliant on the bolts, pins and pads to hold them in place

The hole is a doorway, and aircraft doors open outward. The plug has to conform with that design. Not saying it would be impossible to design a plug that could only be mounted from inside, but it doesn't sound obvious.
posted by insert.witticism.here at 8:17 AM on January 25 [1 favorite]


The thread doesn't have enough criticism directed towards this SAT system which I'll bet was introduced so they could trim the number of managers required to oversee everything. Why pay five experienced people with domain knowledge and institutional memory to maintain situational awareness of what's going on when you can pay some b-school dope to sit around watching a bunch of generic metrics fly across a "dashboard"?
posted by RonButNotStupid at 8:22 AM on January 25 [10 favorites]


They had a spate of human poop showing up in people's offices and workspaces a few years back. Dunno how that was resolved.

"That's not poop, that's a quality escapement log."
posted by swift at 8:23 AM on January 25 [3 favorites]


the "Kill Bill" klaxon

That's a sample from the title theme for _Ironside_ by Quincy Jones.
posted by GCU Sweet and Full of Grace at 9:09 AM on January 25 [2 favorites]


which means everyone was about to die of wokeness.

Countess Elena, that's a beautifully snarky phrase! Thanks for the hearty guffaw.
posted by Greg_Ace at 10:18 AM on January 25 [1 favorite]




Matt Stoller has a solution: It's Time to Nationalize and Then Break Up Boeing.
posted by rednikki at 12:18 PM on January 25 [2 favorites]


So one thing I learned in the wake of this incident is that Boeing still hasn't implemented fixes related to a 2018 incident in which a passenger was sucked out the window of a non-MAX 737.

Boeing has asked the FAA for exemptions that will allow it another four and a half years to implement those changes, by which time a full decade will have passed since the fatal incident, and coincidentally, many of the 737 NG planes involved will have been retired, potentially sparing airlines the cost of making the repairs.

In the meanwhile, around 6,000 737 NGs are in use worldwide, including about 2,000 in the US.

Today (Fri., Jan. 26) is the last day for public comment on the proposed changes. It feels like a fool's errand, but I'm half considering putting one in.
posted by evidenceofabsence at 9:27 PM on January 25 [6 favorites]


GE and it's consequences have been a disaster for corporate culture.

It's worth noting that Spirit Aerosystems got spun off because they were deemed as providing little added value; making them a supplier meant they could pressure them to keep costs down instead. Of course, if you're trying to squeeze water from a stone, you can't be surprised if the thing eventually just crumbles instead.

Organizations tend to develop competence in what they do, so if you offload a lot of your engineering and manufacturing work, over time you become less capable of it. And worse - you end up not being able to evaluate if the people you've hired to do it are doing a good job!
posted by ndr at 11:27 PM on January 27 [2 favorites]


« Older Future Sushi   |   The FDA Warned an Asthma Drug Could Induce Despair... Newer »


This thread has been archived and is closed to new comments