Firmware 6.1

[FlasherZ]

Right. And this only applies to newer models, correct? Our older VINs don't support tire matched TPMS I assume?

The older system does have "slots" as to which sensor is on which wheel location. Tesla's programming tool starts at right-front, I believe, and goes clockwise around the car. It's just that the system doesn't have the ability to send the alert location to the master computer, or the master computer just ignores it.

The button on the touchscreen that programs the system in my car on tire size change appears to randomly assign sensors it "hears" to slots in the TPMS system.

See a discussion here:
TPMS - Low Pressure - Why Not Tell Us Which Tire?

 

[Bet TSLA]

Well, that's an interesting theory. It doesn't appear to be what they do, though....

I've been speculating and describing how I might do things if I were in Tesla's position. I'm doing this to help people here understand some of the decisions and difficulties involved in software updates, since from the postings here such understanding is lacking. None of what I have written is from inside information, so it's entirely speculation. I'm sorry if you got a different impression.

Your data may be correct, I question your conclusion, though.

I have no data.

For example, over the first couple of days it seemed that .167 was mostly pushed out to AWD models, and the failure rate among those seemed quite high.

Really? I saw reports of a few serious problems. Lots of warnings that soon went away. A couple of power reductions. But maybe two or three actual cases where the car lost power and required some sort of action to get it going again. Maybe I wasn't paying close enough attention. Were there more?

TMC does not provide a statistically useful sample (self selection bias) plus there is the reporting bias (people are an order of magnitude more likely to report a problem than the absence of a problem), but still I'd say that more than 10% of the people who received .167 in AWD models experienced problems.

Agreed on the biases. And agreed it looked like only P85Ds (at least I didn't notice any S85Ds in the early reports). But I was considering only serious problems, and that number was much lower.

And BTW, this also contradicts your earlier assumption that they roll things out to "a reasonable size population, probably delivered to a wide variety of configurations".

No, it looks like a reasonable size population. As to whether they deliver to a wide variety of configurations before delivering to everybody, I think we're seeing that now. But it's all speculation without hard data.

Again, is there any proof to that assumption or are you just making things up that sound good?

I'm making stuff up, guessing how things might be done. No hard data, like everybody else here. But I do have some understanding of how these things are done.

Ignoring the tone ("whine and scream"? really?)

Oh, well that I have hard data on. Just look back on the posts here and on the Tesla forums every time a release is made. There are always a few people talking about how their consumer rights have been violated and how they're going to sue the company if they don't fix their problems RIGHT NOW!!!! Yup. Easily verified. On the other hand, I doubt that Tesla puts such people at the bottom of the list. I would though. Only prudent.

And this is where you drift from optimistic speculation into wishful thinking. There are quite a few people here who have done software for most of their lives. And no, the last few releases are anything but a "good job".

Great! Nice to have other people's opinions. In particular it sounds like your opinion is different from mine. It would be nice to hear the details on why you think the last few releases left something to be desired. Clearly you would be speculating about the issues too, but I don't see any reason not to do so. Oh, wait. Do you have any knowledge or experience in the area of creation and release of software updates? I notice you don't claim any, but that was probably an oversight.

And their handling of the issues that owners have reported does not impress me at all.

Well, that's customer service, an area in which I claim no expertise. As a customer, I'm mostly very pleased with Tesla, but I can think of some things they could easily do to reduce the number of disappointed people. The first thing I would suggest is that they entirely stop talking about future features and products.

 

[kabirakhtar]

Yes. The new, wider dropdown for the driver profile selection is working flawlessly, with range mode on or off.

also, the RESET buttons for trip odometers (Controls -> Trips) have been made wider in 167. clearly this was the reason for the update.

 

[ArtInCT]

If your S was built on or after the September 2014 time frame you probably have the new TPMS sensors or V2 sensors as they have be called.
Prior to September 2014 and perhaps even the early weeks of 2014 the V1 sensors were used. It is not likely that the V1 sensors have the
necessary bits to integrate into the specific wheel that is low on pressure warning function.

As an aside, it is not reliable but has been mentioned that VINs prior to around 50900 have V1 and after that V2.
As you all know, VINs are not sequentially assigned at build time and those VIN numbers are simply a guide posed by some here, not a reliable marker according to others.

 

[techiesjc]

I received .167 on my P85+ Autopilot HW on Monday night this week. This was and update from .140. The things I have noticed:
- Hill Brake is now working better on slight inclines - this was "broken" starting in .115 and they have now resolved the issue
- TACC driving is much smoother and really gives a nice ride - even in tough "stop-n-go" traffic
- Much better reliability on fetching music art, even over .140

That's all I have noted during my last two days of travel!

 

[drsaab]

I think the keyboard input looks updated when I typed in a Web address. Not sure about that though. I don't use it too often.

 

[dirkhh]

Oh, wait. Do you have any knowledge or experience in the area of creation and release of software updates? I notice you don't claim any, but that was probably an oversight.

I've been directly involved in creating software, releasing software and updating software in the field for more than 20 years.

 

[Andyw2100]

Oh, wait. Do you have any knowledge or experience in the area of creation and release of software updates? I notice you don't claim any, but that was probably an oversight.

I've been directly involved in creating software, releasing software and updating software in the field for more than 20 years.

Heh.

So it was, in fact, an oversight.

 

[Bet TSLA]

I've been directly involved in creating software, releasing software and updating software in the field for more than 20 years.

Excellent! Then I would be very interested in hearing your opinion on how Tesla's engineers did an unsatisfactory job on the last few releases of the software.

 

[AmpedRealtor]

It is not likely that the V1 sensors have the necessary bits to integrate into the specific wheel that is low on pressure warning function.

The Roadster displayed individual tire pressures and that car is a lot older than the oldest Model S. Would Tesla use a TPMS system inferior to the Roadster's? Also, at least one owner has reported that the service center accessed his logs and provided him with his individual tire pressures over the phone. This was not on a newer car.

 

[billarnett]

Excellent! Then I would be very interested in hearing your opinion on how Tesla's engineers did an unsatisfactory job on the last few releases of the software.

Well, I've been involved with software for more than 30 years, too. And if I had ever been responsible for releasing a bug that potentially put someone's life at risk I would be pretty damn embarrassed. (Did you see 4us2bev's posting on the other thread?) And if I was managing a group that let something like that slip thru I would institute a thorough review of my quality control procedures. Starting with a stern lecture about how a whole different level of care is needed in this industry than what they may have been used to in former jobs.

 

[breser]

Since there's a sensor in each wheel, I wonder how it knows which wheel has the problem if this is the case?

Probably could be something the system could discover since I believe the sensors can provide additional information on things like rotational speed. Combine that information with independent control of the breaks on all 4 wheels and you could pretty easily discover this information over some time.

 

[brkaus]

If there are rotation sensors in the TPMS sensors, at least on the D models, the system could lock up three wheels and smoke the fourth to decide which sensor is in which location

 

[Andyw2100]

Well, I've been involved with software for more than 30 years, too. And if I had ever been responsible for releasing a bug that potentially put someone's life at risk I would be pretty damn embarrassed. (Did you see 4us2bev's posting on the other thread?) And if I was managing a group that let something like that slip thru I would institute a thorough review of my quality control procedures. Starting with a stern lecture about how a whole different level of care is needed in this industry than what they may have been used to in former jobs.

Certainly there are things Tesla should do to attempt to insure that problems like this don't happen.

But I'm just as concerned about how Tesla is, in my opinion, mishandling the situation once it was brought to light. They could be doing so much more to minimize the negative impact on their customers than what they have been doing. The mistake happened and there's nothing that Tesla can do to undo that particular mistake. Sure, they can work on correcting it, but that mistake has happened, this firmware is out, and it has had an impact. What Tesla can do is make sure they are doing everything reasonable within their power to minimize the negative impact that mistake has on their customers. And Tesla just is not doing that.

I am bothered by this as much as I am by the bug.

 

[dirkhh]

Excellent! Then I would be very interested in hearing your opinion on how Tesla's engineers did an unsatisfactory job on the last few releases of the software.

I'm in Europe this week, spending time away from not having a Tesla (and wearing out the 'R' key on my keyboard, refreshing my Dashboard...) - that's why it sometimes takes me a while to respond...

When I look at software deliverables, there are a few criteria to measure them by. I usually look at three and it's one of those "pick two" situations: correctness, feature richness, time. There is no way to do great on all three.
If we are talking about an app for your phone, time is most important (first to market is critical). Feature richness is second (which is why many apps have 274 configuration options). Correctness is dead last (read reviews of any random app in the app store).
If we are talking about the firmware for a 4500 pounds machine that can go 0-60 in 3.2 seconds, I think we can agree that correctness should be first. To the point that I'd say that in this triangle it becomes almost a "pick one" situation. I.e., what ever you do, you cannot afford to be wrong.

So how does this translate into my statement that I think that

the last few releases are anything but a "good job"

(let's quote correctly, shall we?)
Very simple. They failed on the single most important issue.
We had the little issue that you could get the car out of park before the breaks worked (that was around .113 I believe?). Hmm.
We have power reduction or power loss. If I am on a steep uphill, passing a car with oncoming traffic and suddenly have reduced power, that's more than just an inconvenience.
And I will bet you a hundred bucks that you will find no one inside Tesla's firmware engineering group who would state that they were doing a "good job". They might state that they did the best job they could, given constraints they were given. I.e., if (and that is pure speculation), top management told them "I promised torque sleep by the end of January, I don't care if you are done with testing", then that might explain what was released. But it still doesn't excuse it. And it certainly doesn't make it a "good job".

- - - Updated - - -

I hate it that TMC combines answers to different posts...

But I'm just as concerned about how Tesla is, in my opinion, mishandling the situation once it was brought to light. They could be doing so much more to minimize the negative impact on their customers than what they have been doing. The mistake happened and there's nothing that Tesla can do to undo that particular mistake. Sure, they can work on correcting it, but that mistake has happened, this firmware is out, and it has had an impact. What Tesla can do is make sure they are doing everything reasonable within their power to minimize the negative impact that mistake has on their customers. And Tesla just is not doing that.

I am bothered by this as much as I am by the bug.

Same here. But I think this is simply a reflection of the lawyers preventing them from acknowledging a mistake. There are significant legal consequences at risk.

What I really don't understand (having done firmware releases) - why don't they have the ability to roll back to a known good version? This happens to a few people and you force push .116 (or whatever was the latest release that has working breaks AND working motors) to everyone who has received .167.

 

[Andyw2100]

I hate it that TMC combines answers to different posts...

I do too. Occasionally I'll even wait until someone else posts in a thread if I really want my posts to appear properly, but I shouldn't have to do that.

Same here. But I think this is simply a reflection of the lawyers preventing them from acknowledging a mistake. There are significant legal consequences at risk.

You may well be right. But if you are, then while I am not a lawyer, I think Tesla should really think about finding some better ones at this point. Because if anything really dire were to happen now it would not be difficult for the parties that wanted to do so to make a pretty strong case that Tesla knew about the situation and chose to do nothing. And by doing nothing, they are increasing significantly the likelihood that something dire does happen. The case against Tesla could be made easily with or without Tesla actually having said "here's the issue and here's how to make sure you're not affected." Since that's where we're at now, why not do everything possible to make sure that there isn't anyone who would be trying to make a case? I guess what I'm saying boils down to the cat is out of the bag.

What I really don't understand (having done firmware releases) - why don't they have the ability to roll back to a known good version? This happens to a few people and you force push .116 (or whatever was the latest release that has working breaks AND working motors) to everyone who has received .167.

I think you probably answered your own question to that with your answer to me. Pushing an earlier version of the firmware would be just as much an acknowledgment that there was a problem with the later version as telling people about the problem, if lawyers were trying to make a case against Tesla.

 

[DavidB]

That's good news, toto. How about you, Andyw2100: weren't you one of the D owners not seeing any benefit under .139/.140?

The question is still open as to why some cars saw significant efficiency improvements (mine is one of them) under .139/.140, and others did not.

I think it is a blend of two reasons:
1) Some driving habits could show significant efficiency improvements with the upgrade, whereas other driving habits might not have any efficiency improvement with the upgrade.
2) Expectation bias; people that expect to see an improvement are likely to interpret the data as improving and vice versa.

 

[dirkhh]

I do too. Occasionally I'll even wait until someone else posts in a thread if I really want my posts to appear properly, but I shouldn't have to do that.

THAT!

You may well be right. But if you are, then while I am not a lawyer, I think Tesla should really think about finding some better ones at this point. Because if anything really dire were to happen now it would not be difficult for the parties that wanted to do so to make a pretty strong case that Tesla knew about the situation and chose to do nothing. And by doing nothing, they are increasing significantly the likelihood that something dire does happen. The case against Tesla could be made easily with or without Tesla actually having said "here's the issue and here's how to make sure you're not affected." Since that's where we're at now, why not do everything possible to make sure that there isn't anyone who would be trying to make a case? I guess what I'm saying boils down to the cat is out of the bag.

I've had this argument with corporate lawyers in a different industry and in somewhat different circumstances and they lectured me pretty much "whatever happens, never, ever, ever admit that you did something wrong, released something buggy, etc. You've seen all these settlements were companies pay billions of dollars of fines but the settlement explicitly states that the companies "did not admit any wrongdoing". IANAL - I just observe the insanity

I think you probably answered your own question to that with your answer to me. Pushing an earlier version of the firmware would be just as much an acknowledgment that there was a problem with the later version as telling people about the problem, if lawyers were trying to make a case against Tesla.

Again, based on my experience in an unrelated industry that appears NOT to be the case. Providing a fix (and it doesn't matter if that's a new version or a rollback is NOT an acknowledgement of wrongdoing.
There's a reason why I'm in software and not in law...
Anyway, while I understand (grudgingly) that they are not admitting the bug, I absolutely cannot understand why they are not able to fix it and why they are letting a fair number of drivers drive around with the issue. There are what, 2000 P85D or so out in the wild. And maybe one in ten Tesla owner is here on the forums? And we keep reading about people who ARE on the forum and didn't notice the issue being discussed until they experienced it.
I love Tesla and I just don't want to think that the company that I admire so much would do such a thing. The "Seat Gate" thing that people are upset about I fully understand and I can't fault their logic for what they are doing. This one? I don't get it.

 

[Bet TSLA]

When I look at software deliverables, there are a few criteria to measure them by. I usually look at three and it's one of those "pick two" situations: correctness, feature richness, time. There is no way to do great on all three.

Fair enough.

If we are talking about the firmware for a 4500 pounds machine that can go 0-60 in 3.2 seconds, I think we can agree that correctness should be first. To the point that I'd say that in this triangle it becomes almost a "pick one" situation. I.e., what ever you do, you cannot afford to be wrong.

The place where we fundamentally disagree is that you present this as a binary situation, whereas I see it in shades of grey. I don't believe there is any such thing as a release that is not wrong -- it's just a question of how wrong it is. To get it very close to right takes months of QA -- this is just not going to happen. It would mean that nobody sees torque sleep until summer at the earliest.

They failed on the single most important issue.

I agree, but I think the failure was fairly minor, certainly in the same league with the cars of many competitors. That means that for all practical purposes, in court they could call it pretty much industry standard quality. Probably better. I'm fairly sure that when management gives their software teams guidance it sounds a lot like "We'd love it if you were perfect but we know that won't happen, so just make sure we're better than the competition. Preferably a lot better."

We had the little issue that you could get the car out of park before the breaks worked (that was around .113 I believe?). Hmm.

Yes, that wasn't good. Scary, but not a major risk for deaths.

We have power reduction or power loss. If I am on a steep uphill, passing a car with oncoming traffic and suddenly have reduced power, that's more than just an inconvenience.

Yup, also bad. Under some circumstances it could be a major problem, and possibly even cause deaths. I'm sure they never would have released the software had they known the problem existed. On the other hand, it's roughly equivalent to the known situation of cars with water pumps that fail and cause complete loss of power; that has happened enough that presumably there is standard practice and liability case law that informs the proper course of action.

I'm pretty sure that anybody who gets into a car and drives it on public roads is quite aware that they might die due to no fault of their own. This is reality, and is demonstrated every day. It is regularly on television so it's hard for anybody to claim they don't understand. Tesla's responsibility is to make their car reasonably safe, not as safe as they possibly can. How safe is reasonable is pretty much defined by what's customary for other cars on the road. My point (you knew I'd get around to it eventually) is that if Tesla's buggy software does not take their safety level below what's usual and expected, then I don't think they have an obligation to remove it from the road as soon as possible. They do have an obligation to the their customers to fix serious problems as soon as they reasonably can, but that's not nearly as high a standard.

And I will bet you a hundred bucks that you will find no one inside Tesla's firmware engineering group who would state that they were doing a "good job". They might state that they did the best job they could, given constraints they were given. I.e., if (and that is pure speculation), top management told them "I promised torque sleep by the end of January, I don't care if you are done with testing", then that might explain what was released. But it still doesn't excuse it. And it certainly doesn't make it a "good job".

I disagree. A "good job" takes all the constraints into consideration. When I worked for Apple, I was never able to release a bug free set of compilers. In fact I was never able to release software that was anywhere close to my personal standards for quality. Business realities did not permit it. Customers demanded new features now, not when good engineering practices deemed them ready. Management demanded that things ship in a reasonable timeframe. In the parlance of the time "they wanted it bad, so they got it bad." But it was good enough, especially so in that time of delivery is a critical aspect of quality.

Now, was I doing a bad job or a good job? The reality is that compiler bugs (and every compiler has many bugs) are serious business -- they can cause bugs in any software anywhere that developers using them write. So that meant all of OS X, and iOS, and all apps written for those systems. Including safety critical software which people chose to write using our tools. Yowza! Scary stuff. And yet each release had, I believe, fewer bugs than the last one. So things were getting better in some absolute sense. At least I thought so. And I believed I was doing a good job given the constraints. And I can see where the Tesla guys probably have similar feelings.

I hope I've made it clear why I see things in shades of grey. The commercial world is like that. Perhaps military software is different, with rigorous configuration management and formal V&V processes. I don't know. What I do know is that such things cost enormous amounts of money and take lots of time, luxuries not available in the commercial software world.

 

[dirkhh]

The place where we fundamentally disagree is that you present this as a binary situation, whereas I see it in shades of grey. I don't believe there is any such thing as a release that is not wrong -- it's just a question of how wrong it is. To get it very close to right takes months of QA -- this is just not going to happen. It would mean that nobody sees torque sleep until summer at the earliest.

Since you so kindly asked me for my qualifications, my I ask you about yours? I invite you to talk to people in the embedded software industry about the process to release firmware for commercial aircraft. And no, it does NOT take months to do a release. I actually know this space reasonably well (and just had a conversation with senior manager in a company that has their software in most current commercial airliners) and from code freeze to release it tends to be about three weeks in their case.

I agree, but I think the failure was fairly minor, certainly in the same league with the cars of many competitors.

Breaks not working.
Fairly minor.
You live in a different reality than I do. That's for sure.

I disagree. A "good job" takes all the constraints into consideration. When I worked for Apple, I was never able to release a bug free set of compilers. In fact I was never able to release software that was anywhere close to my personal standards for quality. Business realities did not permit it. Customers demanded new features now, not when good engineering practices deemed them ready. Management demanded that things ship in a reasonable timeframe. In the parlance of the time "they wanted it bad, so they got it bad." But it was good enough, especially so in that time of delivery is a critical aspect of quality.

Yes, Apple software sucks. The number of bugs that Apple releases has been going up quite steadily. But Apple isn't making cars. Apple also shipped tens of thousands of laptops with badly manufactured batteries that would swell and catch fire (ask me how I know, go ahead, you know you want to).

Now, was I doing a bad job or a good job?

As a customer of Apple and as a person who uses the Apple compilers quite frequently (and who has long since given up on XCode and uses upstream clang for his projects instead) I will give you an unqualified YES - you and your team were doing a <word censored by the forum overlords> job. The rest of your team (since you are speaking in past tense) still are doing a <censored word> job.
But again, that's a completely irrelevant comparison.
If you had done firmware for an airplane and the plane fell out of the sky, would you have done a good job?

The reality is that compiler bugs (and every compiler has many bugs) are serious business -- they can cause bugs in any software anywhere that developers using them write. So that meant all of OS X, and iOS, and all apps written for those systems. Including safety critical software which people chose to write using our tools. Yowza! Scary stuff

Yes, all those 4000 pound machines, going 0 to 60 in 3.2 seconds, running Apple software for their safety critical systems. All the nuclear power plants, all the trains, all the ships, all the elevators running on Apple software.
Oh, there are none you say?
BINGO.
I would like to state for the record that I question your qualification as an expert witness...

And yet each release had, I believe, fewer bugs than the last one.

(off topic, but no, absolutely not. most definitely, observably not.)

So things were getting better in some absolute sense. At least I thought so. And I believed I was doing a good job given the constraints. And I can see where the Tesla guys probably have similar feelings.

Funny you should say this. A member of the firmware team who is reading this thread contacted me privately (it's really easy to figure out who I am based on my user name - it's a bit harder with "Bet TSLA" - except that it's a fair guess WHY you are cheer-leading so desperately) and pointed out that they are under massive pressure from management, that people are really upset and struggling and trying to prevent disaster, and that the only reason they haven't pushed something new is that management told them in no uncertain terms that their heads are on the line if there's yet another bug in the next version.
So no. You are wrong. The firmware team does, in fact, not at all think they are doing a good job. Not. One. Bit.
And they aren't.

I hope I've made it clear why I see things in shades of grey. The commercial world is like that. Perhaps military software is different, with rigorous configuration management and formal V&V processes. I don't know. What I do know is that such things cost enormous amounts of money and take lots of time, luxuries not available in the commercial software world.

Yes. Apple and Microsoft and many others have a long history of releasing buggy software. You keep talking about "commercial world". This is NOT the commercial world. This is embedded firmware for an object that very much could be classified as a weapon.
But why believe me. If you are bored, read up on on ISO 26262 and IEC 61508...
ISO 26262 defines functional safety for automotive equipment applicable throughout the lifecycle of all automotive electronic and electrical safety-related systems. IEC 61508 is an overarching industry standard on Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems.
I know it's much more fun to talk about things without being bothered by facts and reality, and cheer-leading and saying "Weeeeeee, isn't that great, let's all bet on TSLA" is much easier than understanding the reality of firmware development. But It's really getting annoying to find you spout the same unsubstantiated "uhh, aren't they wonderful" nonsense in various threads.

Updated to mention: uhh, and just for kicks and giggles, I guess I should mention where I am right now... I'm at Embedded World in Nürnberg, Germany, attending a track on automotive embedded software development with fun session topics like "Ensuring Safety and Security within E/E Automotive Systems" and "How the world first SIL4 Certification of a MultiCore RTOS is revolutionizing Automotive Safety". Or how about "Satisfying Automotive SPICE Requirements with Model Driven Development Techniques". So yes, this is a topic that I'm not entirely unfamiliar with.