Preventive eMMC replacement on MCU1

[Ulmo]

I remain optimistic. Jason was quite specific it was about MCU1.

I'm pretty sure he also said it is a problem on Model 3, just not as bad? Anyway, let's see what happens. Get those backups! (The day your warranty expires, I suppose, since before that, it is a warranty repair, and you don't want to void your warranty.)

 

[CuriousG]

Are you having any issues with your MCU such as random restarts while driving or the odometer/trip meters show --- for mileage or crazy alien fonts?

I recently had issues with my MCU1 where I would have continuous random reboots, the alien fonts, can't login to reset to factory defaults, wiper service would take ~5 minutes before it would engage, can't connect to Tesla app, WiFi and Bluetooth not working. After having Tesla mobile attempt assess problem twice (he didn't spend much time on it since the majority of repair was for Model 3), said I needed to bring to service center as it wasn't connecting to the VPN and certificates were corrupted or missing. Found out that my reboots were related to USB drive.

When I did make the appointment, I was told there would be a $97.50 diagnostic fee. After keeping my car for 3 days, I got it back in working order and was told that they had to call an engineer for assistance to upload a custom firmware just to update with two additional firmwares. Since I had complained about the issue of not being able to connect via the app for over a month and a half, they waived my fee although I think it was probably a bad firmware that caused the issue. All the issues mentioned above have been resolved and the same USB drive that was causing constant rebooting when it did happen, hasn't reoccurred.

Hopefully my issue was just firmware and not related to eMMC failing.

 

[LuckyLuke]

@CuriousG I don’t want to alarm you too much but that sounds exactly like your emmc is getting corrupted. If that is indeed the case then similar issues will occur soon (within weeks possibly).
When I had my first “black screen” earlier this year I took the car in to service. They were able to revive it by reinstalling the firmware twice. But about four weeks later the screen went black&unresponsive again so I had the emmc replaced which solved the issue.

 

[Dameon]

For my own car i made clone of tegra with changed eMMC , if one dies on the road, another prepared in glovebox

Since you've taken on the task of creating the clone daughterboard, are you going to offer copies of the board to others? If not the board, full schematics, parts list and CAD files?

I'd be interested.

Thanks.

 

[MP3Mike]

Since you've taken on the task of creating the clone daughterboard, are you going to offer copies of the board to others? If not the board, full schematics, parts list and CAD files?

I assumed that he meant he bought a dead board from eBay and put a cloned eMMC on it so it was ready to swap out when the one in the car fails.

 

[widodh]

Preventive eMMC change is best option, every 2-3 years. Either way its risky even to desolder working eMMC there is small posibility to change/alter/damage original data during the process. Proces of desolder/solder is risky.

Tesla is not to blame, this tegra is nvidia “of the shelf” embed system which can be found i Volvo, audi multimedia systems and some industrial tools.

They should use something like this, plug&play emmc and this aggony could be much easier.

For my own car i made clone of tegra with changed eMMC , if one dies on the road, another prepared in glovebox

Do keep in mind that the VPN certificates on the eMMC have an expire date. They need to be refreshed. The MCU does this when online.

 

[swegman]

Do keep in mind that the VPN certificates on the eMMC have an expire date. They need to be refreshed. The MCU does this when online.

How often do they need to be refreshed? Every month? Every quarter? Something else?

I was thinking of plugging a cable into the debug port of the MCU to read the contents of the eMMC and burn a new eMMC for future installation. But that idea won’t work if the certificates have to be periodically renewed.

 

[widodh]

How often do they need to be refreshed? Every month? Every quarter? Something else?

I was thinking of plugging a cable into the debug port of the MCU to read the contents of the eMMC and burn a new eMMC for future installation. But that idea won’t work if the certificates have to be periodically renewed.

It depends when they were last refreshed. They can have a expire date which is 2 years into the future.

 

[David99]

Who can I go to for replacement of the chip? Preferably west coast.

 

[CuriousG]

@CuriousG I don’t want to alarm you too much but that sounds exactly like your emmc is getting corrupted. If that is indeed the case then similar issues will occur soon (within weeks possibly).
When I had my first “black screen” earlier this year I took the car in to service. They were able to revive it by reinstalling the firmware twice. But about four weeks later the screen went black&unresponsive again so I had the emmc replaced which solved the issue.

It's been just a tad over a month so I'll have to monitor the progress.

Who can I go to for replacement of the chip? Preferably west coast.

This appears to be a closely guarded secret. I would think @Ingineer can do it but these days you have to know a guy that knows a guy type of deal to get a hold of him.

 

[TonyT]

Who can I go to for replacement of the chip? Preferably west coast.

Hi David. We can help you, we are based out of the OC. I'll contact you in PM.

 

[ACA Man]

Hi David. We can help you, we are based out of the OC. I'll contact you in PM.

TonyT,
I'd be interested too please.
Thanks,

 

[TonyT]

It's been just a tad over a month so I'll have to monitor the progress.


This appears to be a closely guarded secret. I would think @Ingineer can do it but these days you have to know a guy that knows a guy type of deal to get a hold of him.

Hi CuriousG, it's more that this is a complicated operation and if done incorrectly the MCU can be bricked and leaving Tesla MCU replacement at 2-3k as the only option. Preventive is much cheaper and will last longer. We can help people in the SoCal area.

 

[psyched]

I think it would be helpful if someone could post the pinout for MCU0 and MCU1 boards for inline eMMC reading.
Already damaged eMMC's don't like the unsoldering procedure.

 

[S100D]

Hi CuriousG, it's more that this is a complicated operation and if done incorrectly the MCU can be bricked and leaving Tesla MCU replacement at 2-3k as the only option. Preventive is much cheaper and will last longer. We can help people in the SoCal area.

@TonyT, have you tried installing a socket so the eMMC would be easier to install when it fails the next time?

 

[LuckyLuke]

Besides introducing a possible extra point of failure what is the use for adding a socket?
If you use a good quality 16GB (or bigger) chip it will probably outlive your car (or you )

 

[TonyT]

@TonyT, have you tried installing a socket so the eMMC would be easier to install when it fails the next time?

Besides introducing a possible extra point of failure what is the use for adding a socket?
If you use a good quality 16GB (or bigger) chip it will probably outlive your car (or you )

Correct, we consider this fix permanent. Original chip lasted 3-X years. By decreasing the amount of useless logs being written we easily extend the longevity of the chip by 10-100x, and once you factor in the extra space to wear out depending on the size of the chip you get the factor goes up another 100-1000x I would expect after several thousand years you will want a new car since this car is only rated for 1 million miles.

Personally on my car I still have the original chip/mcu (Dec 13). I was planning to replace it, however I have taken all the precautions already and now I'm going to see how long it last.

 

[S100D]

Besides introducing a possible extra point of failure what is the use for adding a socket?
If you use a good quality 16GB (or bigger) chip it will probably outlive your car (or you )

I asked because I could not de-solder, read the eMMC, reprogram the new eMMC, and re-solder it into the car. If it had a socket, I would keep a spare and I could install it myself. That is the only reason.

 

[David99]

If the new chips last 4 times longer, it will definitely outlive the car.

 

[Target]

General guidance: Unlike an old spinning hard drive, flash storage only holds data so long after power is removed from the device. As it turns out the more worn out the flash is the less time you have from removal of power until data loss. "Worn out" enterprise SSD's only have about 2 years of data longevity without power (per their specification with worn out as defined as after so many TB of data written). Considering these are being driven to the edge of dead before you try to read them time is definitely not on your side. Flash stores data in "capacitors" and worn out "capacitors" leak down faster than good ones.

In general if you are dealing with mostly dead flash storage you want to limit periods of no power as much as you can as cells can start to lose their charge and hence their data. I don't have any data on how long you can go, but just keep in mind how flash works when you are shipping stuff around. The data loss may be less related to the heat of removal, and more related to the removal of a constant source of power.