NHTSA Investigates Failing MCUs

[viper2ko]

NHTSA it still going down their normal path and have upgraded from a PE, preliminary evaluation, to an EA, engineering analysis, investigation: U.S. upgrades safety probe into nearly 159,000 Tesla vehicles

Interesting that overall it only appears to be a ~9% failure rate so far. (Though they say that the failure rate is over 30 percent in certain build months.)

It sounds like Tesla has made software changes to try to avoid a recall. If it detects it is near the end of life the MCU goes into limited functionality mode that displays only the rear view camera, sets HVAC to auto, so defrost works, and has changed how the turn signals work so that they should continue to operate. (Though that might require a recall to make sure everyone is on the software version that supports the limited functionality mode.)

I have no sympathy for the tesla fan boys or stock holders who always downplayed the mcu1 problem and said to suck it up. I hope NHTSA rains hell on them. Tesla has known about this issue well over a year now and continues to skate around it and deny. Even the latest warranty upgrade is a slap in the face for cars over 100k miles. I even have started to suspect they have decreased updates and are holding back for MCU1 cars to avoid frying some emmcs.

 

[besnova]

I have no sympathy for the tesla fan boys or stock holders who always downplayed the mcu1 problem. I hope NHTSA rains hell on them. Tesla has known about this issue well over a year now and continues to skate around it and deny. Even the latest warranty upgrade is a slap in the face for cars over 100k miles. I even have started to suspect they have decreased updates and are holding back for MCU1 cars to avoid frying some emmcs.

I am a 2014 with 110K miles. It sucks.

 

[bhorn]

More news coverage:
After 12,523 replacements, Feds investigate Tesla Media Control Unit failures

 

[Wratran]

More news coverage:
After 12,523 replacements, Feds investigate Tesla Media Control Unit failures

There’s hope for a recall after reading this article.
At the moment Tesla still not replacing the chip proactively....until your car is dead in the middle of no where

 

[MP3Mike]

At the moment Tesla still not replacing the chip proactively.

We have seen at least one case of a person getting a proactive warning to schedule a service appointment to get the eMMC replaced prior to failure. It might not be as early as you would like to see it, but they are doing it.

until your car is dead in the middle of no where

That just isn't true. It might die in the middle of nowhere, or it might die in a Tesla service center parking lot. They don't wait until you are in the middle of nowhere.

They have also introduced a "limited functionality" mode so that, hopefully, even once the eMMC is dead you can still drive the car with HVAC on auto, rear camera on, and turn signals functioning. (I assume they put charging fixes in to that as well but we don't have any confirmation for that.)

 

[Barry]

There’s hope for a recall after reading this article.
At the moment Tesla still not replacing the chip proactively....until your car is dead in the middle of no where

And then you have to wait a month due to part unavailability. Such pathetic customer service on the corporate level.

 

[morrisdl]

I am a 2014 with 110K miles. It sucks.

Have to disagree: I am also 2014 w/110K and a dead MCU, but ~$400 seems like a reasonable repair price. Less than $2k for a new upgraded computer also seems fair. I think there were was definitely a programming design flaw (logging level) that abused the eMMc and also a little under spec'd at 8G capacity, but those are still less than I have expected for such a technically complex beast the Tesla S/X are.

I think Tesla did a remarkable job for 1st computer-on-wheels consumer product. The multiple over-the-air updates to try to correct and preserve the eMMC are also car industry 1st's Compare that to GM's recent recall of just limiting Bolt's to 90% charge capacity to mitigate fire risk. Apple caps old phone batteries too. Maybe its just my last car Cayenne Turbo was also a overly complicated beast and broke down way more frequently and stupid expensive parts to fix, that I am quicker to forgive Tesla.

Stay positive!

 

[BigNick]

Tesla should have known of this impending failure and had a sufficient supply of parts to replace the daughterboard.
But a concept of "spare parts inventory" seems to be alien to them.

 

[besnova]

Have to disagree: I am also 2014 w/110K and a dead MCU, but ~$400 seems like a reasonable repair price. Less than $2k for a new upgraded computer also seems fair. I think there were was definitely a programming design flaw (logging level) that abused the eMMc and also a little under spec'd at 8G capacity, but those are still less than I have expected for such a technically complex beast the Tesla S/X are.

I think Tesla did a remarkable job for 1st computer-on-wheels consumer product. The multiple over-the-air updates to try to correct and preserve the eMMC are also car industry 1st's Compare that to GM's recent recall of just limiting Bolt's to 90% charge capacity to mitigate fire risk. Apple caps old phone batteries too. Maybe its just my last car Cayenne Turbo was also a overly complicated beast and broke down way more frequently and stupid expensive parts to fix, that I am quicker to forgive Tesla.

Stay positive!

At least we have the option to repair the MCU1 issue for only $500. However, this is shortsightedness by Tesla and it looks like people will be getting it repaired for free or reimbursed. However, I did not get any messages and most people will need to wait for the MCU to die before getting the free repair.

The $2500 should be $2000 for MCU2 upgrade on MCU1 cars with symptoms of a failing MCU. It fixes the problem and gives the upgrade.

 

[Tezzie]

Thanks for the source link. I was able to get the facts of the investigation:

NHTSA Action Number: PE20010

Components ELECTRICAL SYSTEM, BACK OVER PREVENTION

Opened From: June 22, 2020–Present

Summary

The Office of Defects Investigation (ODI) has received eleven (11) complaints (VOQs) alleging failures of the touchscreen/Media Control Unit (MCU) in 2013 through 2015 Tesla Model S vehicles. The reports have been received over the past 13 months at service intervals ranging from 3.9 to 6.3 years.

The display control unit subassembly of the subject MCU is an NVIDIA Tegra 3 processor with an integrated 8GB eMMC NAND flash memory device. EMMC NAND flash devices have a finite lifespan based upon the number of program/erase (P/E) cycles. The subject MCU allegedly fails prematurely due to memory wear-out of the eMMC NAND flash. Tesla used the same MCU with the Tegra 3 processor in approximately 159 thousand 2012-2018 Model S and 2016-2018 Model X vehicles built by Tesla through early-2018.

MCU failures resulting from eMMC memory wear-out are likely to occur after periods of progressively degraded performance (e.g., longer power-up times, more frequent touchscreen resets, intermittent loss of cellular connectivity, loss of navigation). Final MCU failure results in loss of audible and visual features provided by the touchscreen, such as infotainment, navigation, and web browsing. This includes loss of rear camera image display when reverse gear is selected, resulting in reduced rear visibility when backing. Other effects of MCU failure include climate control defaulting to Auto mode and limits on battery charging current and maximum state of charge when recharging. MCU failure does not affect vehicle control systems (e.g., braking, steering, speed control) or supplemental restraint systems.

A Preliminary Evaluation has been opened to assess the scope, frequency, and safety consequences of the alleged defect.

The following VOQ numbers were referenced in the defect petition: 11317875, 11317499, 11315374, 11312177, 11288980, 11281197, 11279959, 11257209, 11232852, 11207819, 11302674

Thanks for this. I have a 2018 Tesla model S 100D, and since 2020 I've been having issues with connectivity, navigation, touchscreen. Many times I tried rebooting (scroll wheel, 4-finger) but nothing seemed to be a permanent fix. I had it checked out by the Service Center 2 or 3 times but they told me they couldn't find anything, and at the most, they just did a factory reset. Back in 2020, I called to ask them if the MCU could be an issue. If that was the case, I wanted to get it taken care of before the warranty expired. But they said no.
The other day, as I was navigating, my car did not know where it was. It thought it was somewhere else and navigation failed. I had to use my phone for Navigation. The Service Center just called to tell me it's a failed MCU, which had also stacked up a bunch of updates. Not sure what that means, but I have done all the updates! The cost to replace the MCU? $2231.10. :-(

 

[kg7jo-]

Thanks for this. I have a 2018 Tesla model S 100D, and since 2020 I've been having issues with connectivity, navigation, touchscreen. Many times I tried rebooting (scroll wheel, 4-finger) but nothing seemed to be a permanent fix. I had it checked out by the Service Center 2 or 3 times but they told me they couldn't find anything, and at the most, they just did a factory reset. Back in 2020, I called to ask them if the MCU could be an issue. If that was the case, I wanted to get it taken care of before the warranty expired. But they said no.
The other day, as I was navigating, my car did not know where it was. It thought it was somewhere else and navigation failed. I had to use my phone for Navigation. The Service Center just called to tell me it's a failed MCU, which had also stacked up a bunch of updates. Not sure what that means, but I have done all the updates! The cost to replace the MCU? $2231.10. :-(

That's the cost to upgrade the MCU. You could've signed up for the 8GB emmc recall, and it would've been free. You got a better system, but you came to rant on a 3 years old post. :-(

 

[SO16]

I don’t think the 2018 had the old MCU memory issue. I think MCU2 came standard in late 2017.

 

[aerodyne]

Thanks for this. I have a 2018 Tesla model S 100D, and since 2020 I've been having issues with connectivity, navigation, touchscreen. Many times I tried rebooting (scroll wheel, 4-finger) but nothing seemed to be a permanent fix. I had it checked out by the Service Center 2 or 3 times but they told me they couldn't find anything, and at the most, they just did a factory reset. Back in 2020, I called to ask them if the MCU could be an issue. If that was the case, I wanted to get it taken care of before the warranty expired. But they said no.
The other day, as I was navigating, my car did not know where it was. It thought it was somewhere else and navigation failed. I had to use my phone for Navigation. The Service Center just called to tell me it's a failed MCU, which had also stacked up a bunch of updates. Not sure what that means, but I have done all the updates! The cost to replace the MCU? $2231.10. :-(

I had similar problems for over 18 months on the old 85D. Never mind this is a recall that they must fix. Never mind no update will fix the failing chip that Tesla broke with excessive logging.

A poster here gave me a tip. I used the "Secret Word". They scheduled a replacement eMMC chip immediately.

PM me.

 

[dannycamps]

I don’t think the 2018 had the old MCU memory issue. I think MCU2 came standard in late 2017.

Sometime in March of 2018. Anything built before that would have come with MCU 1.

 

[whitex]

I had similar problems for over 18 months on the old 85D. Never mind this is a recall that they must fix. Never mind no update will fix the failing chip that Tesla broke with excessive logging.

A poster here gave me a tip. I used the "Secret Word". They scheduled a replacement eMMC chip immediately.

PM me.

No secret word was needed for me. My 2015 MS went in for an unrelated fix beginning of this year, Tesla Service automatically added a free replacement of the MCU Tegra board. It didn't exhibit any symptoms, since I replaced the eMMC myself years earlier, but my VIN showed up as part of the recall so they automatically added that to the service. At first I was thinking of removing it, since I know the eMMC part I put in was faster and more robust than any of the ones Tesla uses (about 6x the price too), but since I was about to sell the car, I figured it would show up as an outstanding recall for the new owner, so might as well get it done. No noticeable change after the replacement.

 

[aesculus]

No secret word was needed for me. My 2015 MS went in for an unrelated fix beginning of this year, Tesla Service automatically added a free replacement of the MCU Tegra board. It didn't exhibit any symptoms, since I replaced the eMMC myself years earlier, but my VIN showed up as part of the recall so they automatically added that to the service. At first I was thinking of removing it, since I know the eMMC part I put in was faster and more robust than any of the ones Tesla uses (about 6x the price too), but since I was about to sell the car, I figured it would show up as an outstanding recall for the new owner, so might as well get it done. No noticeable change after the replacement.

Same for me. I went in for the pyro fuse replacement (covered under warranty) and they did the eMMC chip without me even asking. It seemed a tiny bit more responsive to me but that is subjective. I never really had issues before.

 

[IndyToronto]

Same for me. I went in for the pyro fuse replacement (covered under warranty) and they did the eMMC chip without me even asking. It seemed a tiny bit more responsive to me but that is subjective. I never really had issues before.

Is there really anyone who had anything else other than the emmc (flash memory failure) issue when it comes to the MCU.

Basically the real fix is to replace the tiny flash emmc module with something much bigger and it should take >20+ yrs of recording before failed blocks can show up. It's basically like the TBW on an SSD. If your storage is less than 1/2 full, it's exponentially more difficult to write so many times to same block - as long as the file system management rotates the freed blocks within the pool next time a unit needs allocation to be written.

I can artificially replicate it by writing an app to destroy an SSD drive by repeatedly filling it and then repeatedly deleting a tiny file and creating it - the blocks used by that tiny file would go bad very quickly. The key is to try to never fill up your flash drive (important lesson for laptops as well).