Boeing 737 Max Discussion

[wdolson]

I just noticed this thread. I didn't know it was out there. My first gig out of college was at Boeing's Flight Systems Lab which did all the testing on all the electronics going into new commercial aircraft. they also had flight deck simulators for each Boeing airliner.

I started just after the 747-400 rolled out and left just as the 777 was in flight test. Back then the company was a huge bureaucracy and there was a lot of waste, but the company was run by engineers and working out all the bugs before delivering the plane was the #1 priority on the engineering side. The testing regime was incredibly thorough and even minor glitches would be a big deal.

I had been there a year or two when a bug was found in the 747-400 navigation computer. SAS airlines started offering a certificate to passengers who flew from the US to Europe over the north pole. The first time they tried it in a 747-400, the plane did an 'S' around the pole because the navigation computer couldn't handle 0/0, but it safely compensated.

That got the entire 747-400 fleet grounded and was a level 1 excrement storm from management and the FAA. I went to Everett while the planes were grounded and Boeing couldn't even fly them to a holding airport. Paine Field was cheek to jowl in 747s. It was pretty impressive but also meant a lot of money tied up in inventory that wasn't going anywhere.

Boeing made a big deal of taking care of pilots. When the 777 became the first fly by wire plane in Boeing's fleet, a lot of effort went into making the controls behave like controls connected mechanically to the control surfaces. It was a quite involved system that cost a lot to develop. Airbus gave much less feedback and it contributed to the loss of the Air France Airbus over the Atlantic about 10 years ago.

My job was working on the simulator hardware and test instruments for the aircraft buses. Boeing took aircraft out of service seriously. One time a 747 was down for an unknown reason in Korea and I needed to add a feature to a test instrument going with the team heading out there to troubleshoot the problem. My manager told me in no uncertain terms that I couldn't go home until the feature was ready. Fortunately it was a quick fix and I was only about an hour late getting out of there, but that's how seriously they took fixing problems.

I don't have any contacts left in Boeing, but from what I can tell from the outside, the company has changed. The bean counters are running the show and cutting corners to try and make a quick buck. Just like the corner cutting that led to the Deepwater Horizon accident, things came back to bite them. I wonder how many other problems are lurking in recent Boeing designs that haven't surfaced yet?

 

[Cosmacelf]

Well, they made a bunch of changes in the Max, that’s for sure. I flew in one in January, and my 120v seat plug wasn’t working, so I told the stewardess. She hunted around her control panel, found a switch and flicked it on, and it worked, muttering, “oh, they changed things on the Max...”.

The thing that simply amazes me is the Mickey Mouse design of the MCAS anti stall system. You are going to nose down the plane based on ONE sensor? That is in disagreement with another supposedly identical sensor? And you’ve got lots of other sensors that tell you similar information that you don’t even use? It’s something a junior software programmer would do. It’s incompetence.

 

[wdolson]

Well, they made a bunch of changes in the Max, that’s for sure. I flew in one in January, and my 120v seat plug wasn’t working, so I told the stewardess. She hunted around her control panel, found a switch and flicked it on, and it worked, muttering, “oh, they changed things on the Max...”.

The thing that simply amazes me is the Mickey Mouse design of the MCAS anti stall system. You are going to nose down the plane based on ONE sensor? That is in disagreement with another supposedly identical sensor? And you’ve got lots of other sensors that tell you similar information that you don’t even use? It’s something a junior software programmer would do. It’s incompetence.

That would have been unheard of when I was there. Everything flight critical was designed with redundancies. The most critical flight systems often had three redundant data buses that ran different routes back to the flight computer.

 

[S'toon]

Weeks after the first fatal crash of the 737 Max, pilots from American Airlines pressed Boeing executives to work urgently on a fix. In a closed-door meeting, they even argued that Boeing should push authorities to take an emergency measure that would likely result in the grounding of the Max.

The Boeing executives resisted. They didn’t want to rush out a fix, and said they expected pilots to be able to handle problems.

<snip>

Full article at:
Before Ethiopian Crash, Boeing Resisted Pilots’ Calls for Aggressive Steps on 737 Max

 

[Brando]

737Max UPDATE 30 May 2019 FAA-737NG

 

[jbcarioca]

That would have been unheard of when I was there. Everything flight critical was designed with redundancies. The most critical flight systems often had three redundant data buses that ran different routes back to the flight computer.

Things have changed since I was involved two decades ago. I was not an engineer but a pilot, CFI, and frequent pilot tested of modifications (typically STC). Still the triple redundant sources have been typical for flight critical systems. As we all now know, MCAS was not considered to be critical and the 'flying side' of Boeing did not even know about the changes to increase authority, nor was anything disclosed outside Boeing.
"...Airbus gave much less feedback and it contributed to the loss of the Air France Airbus over the Atlantic about 10 years ago."
Actually FBW nor Airbus philosophy had nothing to do with this one at all. A pitot tube (shades of MCAS!) was clogged with ice and the low time pilots could not handle partial panel flying. Lack of basic IFR flying technique caused an inconvenient problem to become catastrophic.

Both cases share the common reliance on a damage/failure prone pitot tube that required better than average piloting skills to compensate. For all the MCAS controversy, the actual direct failure was a pitot tube. The 'weakest link' being such a mundane device is the appalling part, since several perfectly functional solutions have existed for decades which could have prevented numerous accidents in all categories of aircraft, not just airliners. All the alternatives are more expensive than pitot tubes and present issues of their own.

All this is reminiscent of the kind of short term traditional cost reduction as a source of flaws. It does not help when Being uses a 1960's era type certificate to avoid the time and expense of proper new design. That is not a perfect solution, but B777 and B787 share with Airbus since A320 the ability to be able to adapt to new versions, engines even wings without changing actual piloting skills in any material way.

Only Tesla among automakers has adopted that philosophy, so there are major advances from software alone. No surprise that highly technological platforms are easier to test, harder to get right the first time, and easier to use once they have been made correctly. Still, a software update can quickly fix the inevitable errors.

FWIW, as a pilot I choose the most advanced I can find, so long as I can be assured that the builder actually has tested everything.

Boeing, post McD acquisition has consistently been willing to delegate to suppliers and DER's often with minimal supervision. The B787 battery story was one of total abdication by doing to Saft, which delegated another layer. (I evaluated that mess on behalf of an industry client which made major changes as a result. Bizarrely Boeing seems not to have done that.) The NMAX story differs in detail, but not concept. The process of regulatory abdication to Boeing DER's and Boeing abdication to suppliers will change now to some degree.

We, as Tesla operators are fortunate that the management of TSLA is obsessively detail-oriented, driven in part by the experiences of SpaceX. We are indeed fortunate!

 

[wdolson]

Things have changed since I was involved two decades ago. I was not an engineer but a pilot, CFI, and frequent pilot tested of modifications (typically STC). Still the triple redundant sources have been typical for flight critical systems. As we all now know, MCAS was not considered to be critical and the 'flying side' of Boeing did not even know about the changes to increase authority, nor was anything disclosed outside Boeing.
"...Airbus gave much less feedback and it contributed to the loss of the Air France Airbus over the Atlantic about 10 years ago."
Actually FBW nor Airbus philosophy had nothing to do with this one at all. A pitot tube (shades of MCAS!) was clogged with ice and the low time pilots could not handle partial panel flying. Lack of basic IFR flying technique caused an inconvenient problem to become catastrophic.

Both cases share the common reliance on a damage/failure prone pitot tube that required better than average piloting skills to compensate. For all the MCAS controversy, the actual direct failure was a pitot tube. The 'weakest link' being such a mundane device is the appalling part, since several perfectly functional solutions have existed for decades which could have prevented numerous accidents in all categories of aircraft, not just airliners. All the alternatives are more expensive than pitot tubes and present issues of their own.

All this is reminiscent of the kind of short term traditional cost reduction as a source of flaws. It does not help when Being uses a 1960's era type certificate to avoid the time and expense of proper new design. That is not a perfect solution, but B777 and B787 share with Airbus since A320 the ability to be able to adapt to new versions, engines even wings without changing actual piloting skills in any material way.

Only Tesla among automakers has adopted that philosophy, so there are major advances from software alone. No surprise that highly technological platforms are easier to test, harder to get right the first time, and easier to use once they have been made correctly. Still, a software update can quickly fix the inevitable errors.

FWIW, as a pilot I choose the most advanced I can find, so long as I can be assured that the builder actually has tested everything.

Boeing, post McD acquisition has consistently been willing to delegate to suppliers and DER's often with minimal supervision. The B787 battery story was one of total abdication by doing to Saft, which delegated another layer. (I evaluated that mess on behalf of an industry client which made major changes as a result. Bizarrely Boeing seems not to have done that.) The NMAX story differs in detail, but not concept. The process of regulatory abdication to Boeing DER's and Boeing abdication to suppliers will change now to some degree.

We, as Tesla operators are fortunate that the management of TSLA is obsessively detail-oriented, driven in part by the experiences of SpaceX. We are indeed fortunate!

Tesla does fall down on the job a bit in non- "flight critical" areas. I've been nagging them about the mirrors unfolding on their own in the garage for over a year now and there is a thread on the forum about all the problems with the USB media player. For critical systems their software is pretty good. There have been many stories about AP issues, which is an evolving technology, but there have been few problems with other driving related systems. As a firmware programmer, I know there is a heck of a lot of code in there running all those systems, more than a regular ICE, and there are not that many glitches.

I left Boeing before the McDonnell Douglas merger, though I was still in Seattle and knew a lot of people affected. Since the late 1960s Boeing was sort of an ambivalent defense contractor. They were involved in the B-1 and B-2 projects and developed the cruise missile, but most of their defense work was repackaging commercial aircraft for the military. I had a class in the Renton factory building when they were building the last 707 bodies. I got a chance to look over one of the last ones built just before it flew off to Boeing Field. With little fanfare they soon after repurposed the line for more 737s.

With the merger overnight they were one of the biggest defense contractors. I think in merging the two cultures they got the worst of both worlds. There is a different mindset when your primary customer is a military with a fairly high tolerance for bugs as long as what they are getting is the latest whiz bang and commercial customers who want the most bug free product possible.

When I was there there was an attitude that every new airplane was a gamble on the very existence of the company. Management considered that there was not enough to fall back on from the defense side and the company could go under with a failed airliner roll out. As a result, they were very careful to hedge all bets. There were a lot of inefficiencies. I know a guy who was single handedly responsible for $100 million of the 777 cost overrun. He took an idea I had, took it away from me, blew it up into gargantuan proportions and ended up screwing up everything. He was aiming at upper management, but after that management asked him to leave the company. They couldn't fire him, but delegated him to a proverbial basement.

A few years later a software manager position opened up and only two people applied, the guy who was responsible for the one 777 mess and a friend of mine who had just gotten her degree in EE after starting there in the drafting department. They took her on as the
manager despite the lack of experience.

That guy was just the most extreme case of waste I saw. There were many others.

Because there is an engineer's union, it's pretty tough to fire anybody unless you make a pretty good case or catch them doing something extreme. As a result they ended up carrying along some people who had no business being there. They hired two sisters who had graduated together from Seattle U. I worked with both. One was very sharp and a pleasure to work with, I think the other got through school copying her sister. She didn't even know the basics of electrical engineering. As the software guy I had to save her first hardware design project. I spent a Saturday in the lab with a soldering iron rewiring the prototype board to get it working.

 

[jbcarioca]

Tesla does fall down on the job a bit in non- "flight critical" areas. I've been nagging them about the mirrors unfolding on their own in the garage for over a year now and there is a thread on the forum about all the problems with the USB media player. For critical systems their software is pretty good...

Thanks for this post. You helped give context to thoughts I have been trying to refine. My own contact has been with the MD side, where I interacted with many involved people at LGB during the C17 development.

Indeed the Tesla analogies include some laudatory elements coupled with support weaknesses that are hard to comprehend when coupled with such spectacular successes.

 

[S'toon]

Federal safety officials say Boeing should consider how cockpit confusion can slow the response of pilots who are dealing with the kind of problem that likely caused two airliners to crash in the past year.

They suggest that Boeing underestimated the time it takes — measured in precious seconds — for pilots to diagnose and react when they are being bombarded by multiple, cascading warning alerts.


<snip>


Boeing officials have said the pilots didn't follow known procedures for stopping a sudden nose-down pitch or, in the case of the Ethiopian crash, followed the procedures only briefly.

The safety board said Boeing assumed that pilots flying the Max would respond to an automated nose-down push by taking "immediate and appropriate" steps. Federal regulations allow manufacturers to make such assumptions, and Boeing even used test pilots in flight simulators to check its assumptions.


Boeing presented highly-trained test pilots only with a single alert indicating a condition known as runaway stabilizer trim, which can be triggered by the MCAS anti-stall system, safety board officials said. They said Boeing failed to consider that an underlying problem like sensor failures — which triggered MCAS in both Max crashes — would set off several alarms.

In the Lion Air and Ethiopian Airlines crashes, they said, the pilots' control columns would have shaken to warn of an impending stall. They would have also gotten several visual and sound alerts about things like altitude and speed.


"That's the actual scenario that never got evaluated in the simulator," said Dana Schulze, the board's director of aviation safety.

Schulze said years of research have shown that when multiple alarms compete for the attention of pilots, it can lead to a situation in which "pilots will not respond as perhaps you might have intended."

<snip>

Full article at:
Boeing underestimated pilot response time, cockpit confusion in responding to 737 Max problems, report says

 

[Brando]

Seems to me, the real start of the problem was management. Why? The focus on sales. IF you don't have to ask the buyer, airline, to train the pilots on the "new plane system" think of the "savings" (read cost advantage over Airbus). So Boeing tells the buyers, no need to train, just have pilots watch a video. Did the pilots "understand" how this new software and stall sensors actually worked? Did they under stand that this system would re-boot/reset it self after resetting the trim tab? Pilots thought they had "fixed" the problem - faulty system does it all over again and further reset the trim tab. Process repeats til plane flies into the ground.

Proper (read expensive in management eyes) training follow by simulator time to verify that the pilots understood what was going on. AND how you might disconnect this failing system (stall indicator was malfunctioning - no redundancy a big design flaw too.

Management had gotten (read captured regulator, FAA) to allow Boeing to certify them selves - (read no out side FAA inspectors needed).
So, no redundancy on stall sensor, system auto resets and pilots are confused.

So, sadly, I see @S'toon quoted article more of a cover up for management and FAA - they changed the rules to save money to increase sales margins and getting a cost advantage over Air Bus and both these groups want to make it appear it was just an "honest mistake" - Oh, how could they have known:

Federal safety officials say Boeing should consider how cockpit confusion can slow the response of pilots who are dealing with the kind of problem that likely caused two airliners to crash in the past year.

I call BS. Flying didn't become the safest method of transport IF the Boeing Engineers, test pilots, FAA inspectors had never dealt with confusion in the cockpit. NO, Boeing stopped the FAA inspectors, tried to save money on training. Poorly designed system that should have never been approved. Training might have helped pilots deal with this faulty system. But that back stop of training wasn't done either.
Verifying in simulators also avoided. These kinds of processes built the amazing safety of airline travel and these processes were avoided for Wall St. profits and management bonuses.

As W, Edward Deming said, he was wrong when he said 70% of company problems were from management - it was actually more like 90%.

GM ignition switches. air bags, VW clean diesels and the list goes on...

side note: who grounded the 737 MAX first ? China FAA and others followed and dead last was US FAA.
FAA used to lead the world in safety. Not so much any more. right??

This is a complex story - IF you have comments on my opinion, you should at least read this entire thread. And I am sorry about my poor writing, over simplifications and many details missing. Read this thread if you are really interested. A lot of good info.

 

[Brando]

And I love this part of the above article:

Boeing presented highly-trained test pilots only with a single alert indicating a condition known as runaway stabilizer trim, which can be triggered by the MCAS anti-stall system, safety board officials said. They said Boeing failed to consider that an underlying problem like sensor failures — which triggered MCAS in both Max crashes — would set off several alarms.

Failed parts/systems is exactly why there are often triple redundancy. Boeing failed to consider???? give me a break.
Just shows why this MCAS system would have never been approved IF normal procedures had been followed.

 

[user212_nr]

As the 737 Max situation never should deserve its own thread on TMC, yet has been brought up here a number of times, I'll use this thread to chime in.
...

6. His engineering-based summary is that no further action should be taken until the black box information is assessed; it would not take long to do so; he did, however, understand the political-based decisions not only of the rest of the world but also, as we were talking, Mr Trump's action.​

I like the theories, and I've read some things that indicate pilot incompetence was a factor, but don't you think that its up to Boeing to defend itself and to protest?

Their silence can mean one of two things:
a) they know something is terrible and want to work it out
b) they are so incompetent they can't figure out what to say or do

 

[wdolson]

It wasn't pilot incompetence, but poor pilot training was a factor. Boeing advertised the 737 MAX as having the same flight characteristics as the older 737s and retraining was minimal. Pilots were given iPads with some material on it instead of the normal training procedure for a new aircraft which involves pilots going to a training center and flying a simulator until they were confident they could handle emergency situations with the new plane.

Another factor was the anti-stall equipment on the MAX was poorly designed and even more poorly tested. Years ago I was involved in that process and the quality of development of the anti-stall system was engineering malpractice. It would have been completely impossible for such shoddy work to slip through when I was there.

I suspect the FAA has deteriorated too. The development and testing program involved the FAA throwing failure scenarios at the engineers and they had to come up with ways to handle the failure without losing the airplane. For example one of the scenarios on the 777 program was what would happen if the fan on one engine exploded, a fan blade went through the fuselage and embedded itself in the other engine. Could they ensure the other engine would keep running. The engineers had to put a layer of armor plate on the inside of the engine nacelles to prevent this very, very unlikely scenario. It has never happened to a jet airliner in over 60 years of jet travel, though it did happen once to a small 4 seat turbo-prop plane (that had significantly less fuselage to go through).

A couple of air crews had the stall prevention kick in after take off, but were skilled enough to figure out how to recover on their own, but the two losses came from less experienced crews who didn't have 20 or 30 years of flying under their belts.

 

[bxr140]

Speaking of poor pilot training, anyone else get this banner advertisement in this thread?

The internet really is an amazing place.

 

[S'toon]

Boeing staff texted about 737 Max issue in 2016

 

[Cosmacelf]

Dare I resurrect this 4 year old thread to dump on Boeing for not being able to install door plug bolts? Honestly, this company is a walking safety disaster.

 

[wdolson]

Dare I resurrect this 4 year old thread to dump on Boeing for not being able to install door plug bolts? Honestly, this company is a walking safety disaster.

The quality has gone downhill since I worked there (late 80s and early 90s).

One of the suspects being investigated is counterfeit Chinese bolts that were holding the door plug in place. Aftermarket Chinese bolts that turned out to be counterfeit were deemed responsible for an Alaska 727 that went down off the California coast about 20 years ago. It's still on Boeing though for sourcing counterfeit bolts if that is what happened.

 

[HVM]

Boeing 737 Max and Airbus Controls Discussion

This redesign scares me. Rather the new steering "wheel", not really bothered by the rest. I can't fathom what Tesla is thinking. There is a reason the steering wheel is mostly round. In fact, I thought it was required by law. The only way a yolk style steering wheel has any practicality is if...

teslamotorsclub.com

 

[SwedishAdvocate]

You guys beat me to it. I was busy cooking and then reading this thread...

Seems like 'quite' scary stuff...

This guy was a Manager at Boeing before turning whistleblower. He will not fly on any 737 MAX aircraft... I doesn't say for how long he has taken that stance, but I suspect it goes back a while...

 

[HVM]