Warning, rebooting v10 on MCU1 while driving

[whitex]

This is just warn folks used to rebooting their MCU1 while driving (I used to do that every few drives, when the browser would stop working or some other artifact - Tesla software is not famous for its stability). After applying v10 I was driving with the family to dinner tonight. Browser (now taking up most of the screen ) was dead as usual so I went for the thumbwheel reset. What a disaster!

First, it took 5+ minutes to reboot, WTF? The front windshield started fogging up, so I had to open windows to continue to drive, which of course was uncomfortable for everyone since it's fall and we don't live in California. Second, Thumbwheel reboot used to only reboot the big screen (MCU1), however this time, after few minutes of dark main screen, my instrument cluster went dark too! So now I'm driving a car without HVAC or any screens at all, not even a speed indicator. Now, the cluster was only off for a short time (30-60 seconds maybe) but it seems like forever when you don't expect it. When it came back on it has messed up controls but at least you could see most indicators. Even once the main screen came back on, it still took a bit before it was responsive enough to turn on windshield defrost.

I so wish Tesla would keep MCU1 on v8 with security patches only. My car gets worse every update.

 

[Battpower]

This focuses your mind on a huge issue. It's one thing to be offered the choice of a fully tested system update with a clearly defined set of features.

But.... something very different to find yourself an unwilling participant in the middle of an ongoing beta (or alpha?) test program that is using you and your car as the testbed.

There must be an 'Opt Out' to give you the chance to stick with the car you originally purchased.

 

[whitex]

This focuses your mind on a huge issue. It's one thing to be offered the choice of a fully tested system update with a clearly defined set of features.

But.... something very different to find yourself an unwilling participant in the middle of an ongoing beta (or alpha?) test program that is using you and your car as the testbed.

There must be an 'Opt Out' to give you the chance to stick with the car you originally purchased.

There can't be with Tesla, because:

1. They sell vaporware, meaning they have to give you updates to deliver what they sold you (e.g. my wife's 2017 Model S didn't have auto headlights or auto-wipers for many months after delivery)
2. Their cars require internet connectivity to function, which means they have to be regularly patched for security vulnerabilities. Without patching, the cars become a joke to hack as security vulnerabilities for old software are freely available on the internet. With a 10-15 year lifecycle, there is no way Tesla can afford "security patch" only model (one big reason - after 2-4 years they lost the free support from the open source community, as the community simply moves to newer versions of the software).

 

[Battpower]

meaning they have to give you updates to deliver what they sold you

That's may be ok as long as it clear what they sold you.... which is often not at all clear (thinks EAP to FSD upgrade).... or if what you were actually sold was your own vision / expectation / interpretation of...... something Elon said... once upon a time...? (thinks FSD, robo-taxi,..... house on Mars....)

 

[dannycamps]

This happened to me once when the MCU in my 2019 became unresponsive while driving. It's nerve-wracking when everything goes dark and you are driving in complete silence during the restart.

 

[Barry]

I so wish Tesla would keep MCU1 on v8 with security patches only. My car gets worse every update.

Well said

 

[Ande]

Just tried to reboot 32.11.1 prior to installing the new 32.12.2 I've got today. Rebooted of just becouse of your post.
Rebooted while driving on autopilot, 92 seconds later MCU was up, the main display did never go black, AP stayed on just fine.

 

[pgkevet]

my 2018 2.5 EAP S updated to 12.1 a ew days ago. Day after when driving back from shopping suddeny the main display just went black.. didnt come back by waiting several minutes so i did the 2 wheel reset - binnacle went black too... took longer than usul to coem back and then promtly blacked out twice more before coming back and the main screen rebooting shortly after. The process took way longer than ever before. It wasn't particularly an issue from the driving viewpoint since we all know everything still works - just a PITA if you happen to be depending on nav instructions.

 

[whitex]

It wasn't particularly an issue from the driving viewpoint since we all know everything still works - just a PITA if you happen to be depending on nav instructions.

Unless you are dependent on HVAC to keep the car windshield from fogging up, or warm.

 

[pgkevet]

Unless you are dependent on HVAC to keep the car windshield from fogging up, or warm.

My first car had no such refinements but to be fair even the brakes didn't always work on that one and I had to drill holes in the floor pan to let the rain back out that came through the 'bulkhead' on every puddle. That and banging on the B pillar to release the indicator arm kept drivers busy. We had the quarter-lights to keep the screen demisted (or our sleeves). We didnlt worry too much about what was behind - someone elses problem cos the rear window was fogged and there weren't any wing mirrors..

 

[Battpower]

My first car had no such refinements but to be fair even the brakes didn't always work on that one and I had to drill holes in the floor pan to let the rain back out that came through the 'bulkhead' on every puddle. That and banging on the B pillar to release the indicator arm kept drivers busy. We had the quarter-lights to keep the screen demisted (or our sleeves). We didnlt worry too much about what was behind - someone elses problem cos the rear window was fogged and there weren't any wing mirrors..

I had one just like that.... And another that I had to have the passenger get out to hold the wing in before I could open the driver's door..... And another that I had to rebuild the engine cuz of a blown head gasket if I drove over 60mph for longer than 30 minutes.

I don't think I paid more than £100 for any of them! Thankfully times have moved on, but some of the apparent new norms are far more expensive and far harder for owners to fix themselves and the safety implications less obvious.

 

[Battpower]

my 2018 2.5 EAP S updated to 12.1 a ew days ago. Day after when driving back from shopping suddeny the main display just went black.. didnt come back by waiting several minutes so i did the 2 wheel reset - binnacle went black too... took longer than usul to coem back and then promtly blacked out twice more before coming back and the main screen rebooting shortly after. The process took way longer than ever before. It wasn't particularly an issue from the driving viewpoint since we all know everything still works - just a PITA if you happen to be depending on nav instructions.

Well, that answers the question if 'maybe I could deal with this by getting a new MS with v2.5 and MCU v2'. Evidently not a solution.

From a technical view point, I have a genuine concern that maybe someone far more experienced in the technical depths of Tesla hardware could answer.

It sounds as though the fundamental design of these systems is good. There are three problem areas:

1) Normal huge challenge of system development / maintenance over a range of platforms and time.

2) Hardware / software platform issues (eMMC, processing capacity, security, ) and other specific intrinsic design factors.

3) Developing new brand new technologies like AP / FSD, partly on live customer cars while maintaining safety and meeting customer needs & expectations.

Battery, motor and breaking are also huge development areas, but less so in the customer space.

My concern is in section 2. If hardware has certain chacteristics / problems such as its behavior being unpredictable, then it becomes very difficult or impossible to predict any system behavior. It also becomes impossible to diagnose faults, as so many could be attributable to the underlying hardware unreliability / unpredictability.

IF updates are stored first in eMMC before being applied to the system, or if functional code resides in eMMC, then it must become increasingly hard to carry out any function (upgrade or operational) with certainty.

Memory typically works on algorithms that try to allow normal operation even with a degree of error / error correction, but when you have a high enough level of failure / error, these algorithms reach their limits and who knows what the implications could be? Just as Elon pointed out about using simulation to test FSD being like marking your own exam paper, you can't design system testing to deal with every (impossible) combination of factors that you can dream up. Because you can't dream them up!

So how big of a risk is it in reality having cars perform updates when the hardware becomes unstable?

How well ring fenced are the different areas of car systems to ensure you can't brick vital safety and command controls (especially while driving) during a system update that goes impossibly off the rails?

 

[pgkevet]

Well, that answers the question if 'maybe I could deal with this by getting a new MS with v2.5 and MCU v2'. Evidently not a solution.

From a technical view point, I have a genuine concern that maybe someone far more experienced in the technical depths of Tesla hardware could answer.

It sounds as though the fundamental design of these systems is good. There are three problem areas:

1) Normal huge challenge of system development / maintenance over a range of platforms and time.

2) Hardware / software platform issues (eMMC, processing capacity, security, ) and other specific intrinsic design factors.

3) Developing new brand new technologies like AP / FSD, partly on live customer cars while maintaining safety and meeting customer needs & expectations.

Battery, motor and breaking are also huge development areas, but less so in the customer space.

My concern is in section 2. If hardware has certain chacteristics / problems such as its behavior being unpredictable, then it becomes very difficult or impossible to predict any system behavior. It also becomes impossible to diagnose faults, as so many could be attributable to the underlying hardware unreliability / unpredictability.

IF updates are stored first in eMMC before being applied to the system, or if functional code resides in eMMC, then it must become increasingly hard to carry out any function (upgrade or operational) with certainty.

Memory typically works on algorithms that try to allow normal operation even with a degree of error / error correction, but when you have a high enough level of failure / error, these algorithms reach their limits and who knows what the implications could be? Just as Elon pointed out about using simulation to test FSD being like marking your own exam paper, you can't design system testing to deal with every (impossible) combination of factors that you can dream up. Because you can't dream them up!

So how big of a risk is it in reality having cars perform updates when the hardware becomes unstable?

How well ring fenced are the different areas of car systems to ensure you can't brick vital safety and command controls (especially while driving) during a system update that goes impossibly off the rails?

I'm not a tesla techy but understand that the fundamental ability to drive is ringfenced. In fact it's probably safer than the bloody unpredictable automation - given a competent driver (didn't someone mention that in a survey 93% of drivers rated themselves as above average?)

@Battpower:

I had one just like that.... And another that I had to have the passenger get out to hold the wing in before I could open the driver's door..... And another that I had to rebuild the engine cuz of a blown head gasket if I drove over 60mph for longer than 30 minutes.

I don't think I paid more than £100 for any of them! Thankfully times have moved on, but some of the apparent new norms are far more expensive and far harder for owners to fix themselves and the safety implications less obvious.

You were spoiled. My first car cost £27.50p and my second £40

 

[Battpower]

the fundamental ability to drive is ringfenced. In fact it's probably safer than the bloody unpredictable automation

I wish there was a way of being sure just how deep and invasive ota updates can be. Obviously safety interlocking and control systems are secured and pretty inaccessible, but I have (not surpringly) seen some evidence that Tesla is not immune to the realities that other tech developers have to work with.

It feels like there should be some system level that is physically not accessible or modifiable remotely or basic electro-mechanical components that fail safe if all else fails.

Give the bafoon at the wheel a fighting chance when all else fails, and know what functions of your car are 100% inaccessible and off limits to well meaning manufactures wanting to 'improve' your car.

 

[Az_Rael]

This is just warn folks used to rebooting their MCU1 while driving (I used to do that every few drives, when the browser would stop working or some other artifact - Tesla software is not famous for its stability). After applying v10 I was driving with the family to dinner tonight. Browser (now taking up most of the screen ) was dead as usual so I went for the thumbwheel reset. What a disaster!

First, it took 5+ minutes to reboot, WTF? The front windshield started fogging up, so I had to open windows to continue to drive, which of course was uncomfortable for everyone since it's fall and we don't live in California. Second, Thumbwheel reboot used to only reboot the big screen (MCU1), however this time, after few minutes of dark main screen, my instrument cluster went dark too! So now I'm driving a car without HVAC or any screens at all, not even a speed indicator. Now, the cluster was only off for a short time (30-60 seconds maybe) but it seems like forever when you don't expect it. When it came back on it has messed up controls but at least you could see most indicators. Even once the main screen came back on, it still took a bit before it was responsive enough to turn on windshield defrost.

I so wish Tesla would keep MCU1 on v8 with security patches only. My car gets worse every update.

Man, that sucks. I once had a driving thumb wheel reset go wrong - after the reset I had all sorts of warnings about stability control unavailable, no regen, etc. Was on my old classic P85. Had to pull over and do the Power Off option. I no longer reset while driving anymore.

 

[pgkevet]

I wish there was a way of being sure just how deep and invasive ota updates can be. Obviously safety interlocking and control systems are secured and pretty inaccessible, but I have (not surpringly) seen some evidence that Tesla is not immune to the realities that other tech developers have to work with.

It feels like there should be some system level that is physically not accessible or modifiable remotely or basic electro-mechanical components that fail safe if all else fails.

Give the bafoon at the wheel a fighting chance when all else fails, and know what functions of your car are 100% inaccessible and off limits to well meaning manufactures wanting to 'improve' your car.

That bodes the fundamental problem of how to remote control steering and brakes but never have a rogue program affect them. You'ld need the big red button that disconnects it all and a secondary system takeover for basic servo assist. It'd be interesting to try driving an S without power steering or power brakes. I can recall an emergency stop when I was driving a '52 R type Bently (wasn't my car) when power assist and hydraulics failed and I had just the cables - both feet on the pedal and pulling on the wheel for extra pressure and stopped a fag paper from the guy in front. The cost of duplication would be the biggie

 

[whitex]

Man, that sucks. I once had a driving thumb wheel reset go wrong - after the reset I had all sorts of warnings about stability control unavailable, no regen, etc. Was on my old classic P85. Had to pull over and do the Power Off option. I no longer reset while driving anymore.

Maybe Elon should add it to his saving calculator which adds up time saved not having to go to a gas station (it used to ask you how much you value your time per hour) - now offset that with "before you start driving each time, spent 10 minutes rebooting the car".

 

[whitex]

My first car had no such refinements but to be fair even the brakes didn't always work on that one and I had to drill holes in the floor pan to let the rain back out that came through the 'bulkhead' on every puddle. That and banging on the B pillar to release the indicator arm kept drivers busy. We had the quarter-lights to keep the screen demisted (or our sleeves). We didnlt worry too much about what was behind - someone elses problem cos the rear window was fogged and there weren't any wing mirrors..

Did that car cost $100K?

 

[Feathermerchan]

So I rebooted my phone while on an important (life or death) call and it dropped the call. WTH?


Seriously, rebooting while driving? With the family? What makes you think that NONE of the safety systems (like airbags and seat belt tensioners) are not affected or disabled during a reboot?
Even if I did that I wouldn't publish it.

 

[whitex]

So I rebooted my phone while on an important (life or death) call and it dropped the call. WTH?


Seriously, rebooting while driving? With the family? What makes you think that NONE of the safety systems (like airbags and seat belt tensioners) are not affected or disabled during a reboot?
Even if I did that I wouldn't publish it.

1. Tesla has long maintained that an MCU reboot was safe while driving.
2. MCU reboots can happen automatically while driving, without any imput from the driver (according to my wife it happens regularly on her MCU2 car, it happens more rarely on my car, probably because I reboot manually as the MCU slows down or things like the browser lock up completely.

If it was dangerous Tesla would have a massive safety recall on their hands - all S/X and probably 3 as well ever made (see #2). Rebooting while driving is part of normal (and sadly necessary) driving experience with Teslas. Even if one was to add 10 minutes before each drive to reboot the car, there is no guarantee the car won't reboot itself while driving - if it did and Tesla saw it as a safety issue, everyone getting into a Tesla would be looking at blank screens for minutes before they can drive (forced reboot before each shifting ind Drive). Now imagine if you have an emergency and have to drive to a hospital, want a car with a 10 minute delay to drive?