Commercial Crew Transportation Capability (CCtCap) SpaceX and Boeing Developments

[bxr140]

It is much more likely that SpaceX does extensive end to end tests since they write all software in house and are much more vertically integrated compared to Boeing.

Two things

1. I think its important to align ‘ETE’ testing. There’s no way SpaceX is doing extensive ETE software testing at the launch processing facility because there’s minimal value in doing so. Since all of their other philosophies are forward leaning and generally based in first principals, its pretty clear their V&V approach is also on-brand. SpaceX will no doubt perform extensive software testing in the lab (as everyone does, apparently some more than others…); at the assembled launcher level testing will be designed to exercise representative touch points, not exhaustive functionality.
2. Other than the fact that SpaceX builds both the launcher and the crew capsule, the “much more vertically integrated” aspect is pretty non-applicable in context. Boeing is definitely writing all their code in house for Starliner; ULA—practically a wing of Boeing—has the launcher side.

And finally, they completely lost telemetry for quite a period with the rocket. Isn't that also bad engineering?

First order, no. That’s the whole point of software running a rocket and not some dude in a driver’s seat.

There’s plenty of situations where there’s no communications with a rocket (or satellite). For instance, one of my launches a few years ago the flight director was literally reading a timed script that said “satellite 1 should have just deployed”, “satellite 2 should have just deployed” etc, because we didn’t have tlm coverage in the region where the satellites were scheduled to deploy. We actually used successful satellite acquisition as each satellite flew over a ground station some minutes after scheduled separation to confirm successful deployment from the launcher. We could have put a boat out in the middle of the pacific to get tlm during the sep events but there was minimal value in spending a couple hundred grand on something that, at best, would have confirmed that there was in fact a launcher failure and the whole pile headed into the drink. Without tlm the best we could do is speculate that had happened, had we not received tlm from the deployed satellites.

That said it is certainly fair to suggest that, in context, a crewed mission have higher standards than a satellite launch (and thus demands more communication coverage), but a transient launch event still demands pre-determined anomaly identification and, in most cases, recovery. Even then, had the [fairly easy to check] clock offset been identified, the mission outcome would probably have been the same.

 

[Cosmacelf]

I think that losing telemetry is not bad engineering, it's a reality of what's available for telemetry communication from the ground. The bad engineering is not planning how to handle those known times when you won't have real time telemetry.

Does SpaceX routinely lose data telemetry?

 

[miimura]

Does SpaceX routinely lose data telemetry?

Yes, there are ground stations at known locations and there are areas that don't have coverage. This is obvious from the SpaceX webcasts because there are places that they can't get the video feed from the second stage. However, low bandwidth data telemetry could be a different situation because one would imagine that there might be satellite based communication links available if they have suitable antennas for communicating with vehicles heading to orbit.

 

[Cosmacelf]

Yes, there are ground stations at known locations and there are areas that don't have coverage. This is obvious from the SpaceX webcasts because there are places that they can't get the video feed from the second stage. However, low bandwidth data telemetry could be a different situation because one would imagine that there might be satellite based communication links available if they have suitable antennas for communicating with vehicles heading to orbit.

That's what I was asking when I said "data telemetry". I knew they lost high bandwidth video, but lower speed data telemetry maybe not?

 

[e-FTW]

I hate to drag-on the Boeing-is-terrible discussion (though we are low on Crew news) but Lee Hutchison really let loose on this one. See the featured comment: Companies are contracting out more jobs—that’s not great for workers

”Many tech companies solved this problem by having the lowest-paid workers not actually be employees. They’re contracted out. We can treat them differently, because we don’t really hire them. The person who’s cleaning the bathroom is not exactly the same sort of person. Which I find sort of offensive, but it is the way it’s done.”

I saw this happen first-hand at Boeing in the 2000s, mainly due to the cancerous influence of Harry Stonecipher. Starting in about 2004 under his "leadership," the company made a push to outsource any job that involved "turning a wrench," with the idea being that Boeing would employ only architects and designers and any kind of operational work would be done with sourced labor.

This model spread to every aspect of the company in both major divisions (Commercial Airplanes and IDS, though IDS these days is called "Space and Defense," I think) and all of the service organizations—everything from actual manufacturing to IT to HR. If you actually did anything, your job was on the chopping block. Only senior positions that made decisions about strategy or architecture were retained.

The idea, I guess, was to consolidate the company's OpEx into a series of line items that could be more easily forecast. What it actually did was to turn the company into an undead shambling corpse and taint every single product the company has developed since then—from 787 to 737 MAX to Starliner, everything has been screwed up by Boeing's braindead insistence on not employing anyone who isn't a senior-level person.

But Boeing also hasn't stopped its push for new hires, especially young folks and interns—because the company is facing severe aging workforce problems. (Somewhat hilariously, the company is still shedding people so quickly that it has had to make new employees "immune" to layoffs for at least one full year after their hire date.)

The push for new hires and interns, coupled with the complete lack of mid-level operational jobs (because they're all outsourced), puts the company in the utterly ridiculous position of being unable to offer a legitimate career path to, well, almost anyone. It's impossible to spend your career at Boeing because there's literally no way to grow from low-level employee to mid-level employee to senior employee for almost every single class of job the company has.

Before I left in 2010, our CIO Kim Hammonds was doing a series of telecons with each site. Knowing I was leaving soon and had nothing to lose, I asked Kim directly how Boeing planned on dealing with the fact that they're trying to entice a whole new generation of workers in but they have no career path to offer them—that they're effectively training up a bunch of mid-level employees for Boeing's competitors, since that's where all those people will have to go. Kim had some kind of MBA garbage answer about focusing on core competencies or whatever, but it was clear that she had no actual answer.

Boeing has shot itself in the foot, and now their airplanes kill people and their spaceships don't work either. It's sad that the place I used to take so much pride in working at is now such a massively incompetent clown show.

 

[Grendal]

One year anniversary video for DM-1:

 

[Nikxice]

This Friday, 3/6/20 at 11am EST, we'll possibly learn if NASA will require Boeing to conduct OFT-2 for Starliner. This event is scheduled as a media teleconference, which I think precludes it from being broadcast on NASA-TV. However the audio will be streamed online at the link below.
NASA, Boeing to Provide Outcome of Starliner OFT Review

 

[Grendal]

Vicious headline from The Register:
Boeing didn't run end-to-end test on Calamity Capsule, DSCOVR up and running, and NASA buys a Falcon Heavy

 

[e-FTW]

Vicious headline from The Register:
Boeing didn't run end-to-end test on Calamity Capsule, DSCOVR up and running, and NASA buys a Falcon Heavy

Ha! They never disappoint. Always true to form!

As for when a crew might ride the Calamity Capsule, "that is a NASA decision to make."

 

[Cosmacelf]

NASA teleconference about Boeing on going right now.

 

[Cosmacelf]

Formal part of the call was just everyone patting themselves on the back for being so awesome.

Q1: 60+ problems identified, and you also build SLS, how can Boeing assure that SLS will be OK?

Jim Chilten. Wasn’t 61 problems, just 61 recommendations. We built a learning package, and we’re gonna make sure The whole space ecosystem can learn from this. I’m not tracking 61 changes. Oops, now we hear a dial tone. Silence. Perfect.

 

[Cosmacelf]

Next Q: Ok, how many issues are Boeing tracking? And what about flying another unscrewed flight test.
A: Boeing is gone, silence. Too funny.

 

[Nikxice]

Technical difficulties as soon as the obvious OFT-2 question was asked. LOL!

 

[Cosmacelf]

A Doug Ladera (Boeing). First thing we did was to figure out where we had overlap to SLS. We looked at a non proprietary version. Of the report.

 

[Cosmacelf]

Q: How many issues were in the 61 recommendations. Did the IRT recommend OFT-2?

A: IRT found 3 issues, 2 software issues and 1 communication issue.
A: But there were also other process issues that we are looking at. We can't count them. Will we require a test flight? We don't know right now. Boeing will present NASA a plan and then NASA will evaluate, then Boeing will inspect NASA's evaluation, and then we (Boeing) will decide whether or not to do a second test flight.

 

[Nikxice]

NASA's Douglas Loverro on deciding on another OFT, "We're a ways away from that". It sounds like this decision could take months. Even if Boeing doesn't have to repeat the OFT, best guess is that Starliner might not get crewed until later this year or into 2021.

 

[ecarfan]

Boeing will present NASA a plan and then NASA will evaluate, then Boeing will inspect NASA's evaluation, and then we (Boeing) will decide whether or not to do a second test flight.

What? Boeing will make the determination as to whether or not they need to do a repeat mission to meet the objectives that they clearly did not meet during the previous mission?

That can’t be right.

 

[Cosmacelf]

Q: On the 61 corrective actions, can you single out a few significant ones
A: Three important areas. How to fix the specific software issues (mission timer, thrust mapping), and understanding why they weren't caught. Then we look at software testing. What were contributing causes of these coding errors. So we look at how we do our testing. A bunch of suggestions on that. Then systems engineering. Making sure HW and SW teams are working together properly. Making sure what is being tested is done properly. Then finally how we write code. She summarized by pointing out they had four major areas to look at. Kinda confusing.
A: Doug: Specific thing: We had a routine with 4 logical conditions. We didn't test all four logical conditions. It isn't unusual to not test all logical branch. We now are going to test all logical branches.
We had delegated too much authority to the software board to make changes to the software. They should have been brought up all the way to the overall engineering review board. Um, so a lack of communications.

 

[Cosmacelf]

Q: What does it mean when this is called a "High visibility close call"
A: No contract implications.
Q: Any impacts on SpaceX's flight?
A: No known cross over at this point. If NASA sees something that is replicated, then we'll address, but no impact forseen.

 

[Cosmacelf]

Q: Spaceflight Now: If OFT-2 is required, based on firm fixed price contract, is cost totally on Boeing? And how long will this take?
A: I don't want to get into contractual implications. Will still have to negotiate (hmmm).
A: Doug: I don't know timescale.
A: Cathy: shooting for end of month to see initial plan.