Battery Imbalance after 8 years

[EstebanP212]

Seems I need to talk to my PR team...

To many people think of gruber first and they mostly busy with roadsters
Ya upload on YouTube of customer repairs to get exposure

 

[rowdy]

I'm curious what the voltage difference that is considered an imbalance? I have a 2014 P85+ with 90k miles and my average cell difference is .005v.

Yeah I'm interested in this too. I haven't seen anyone mention what size imbalance becomes a problem. Or how fast it can progress from small imbalance to large imbalance.

I have a 2014 P85 and battery warranty is up next month. One module is off by about .02V so charging will stop around 98% (with modules between 4.18V to 4.20V).

I've only done 69,000 km on the car and want to get a rough idea of how much life is left in the battery.

 

[wk057]

Voltage imbalance is only about 1/4 of the picture. You can have a massive voltage imbalance at various SoC, but as long as other variables line up it doesn't matter. What matters more is CAC imbalance, or actual capacity of the cell group. The BMS calculates this per group, so it can estimate where voltages will land at various SoC. The BMS also keeps track of cell impedance, and per-group load profiles, among other things. It will calculate the max safe load/charge and allow it to continue to operate provided no safety issues.

A voltage imbalance itself might not cause issues under load/charge. While there are hard limits on this as a single metric (usually between 60-100mV imbalance while avg pack voltage is in the 3.5-3.8V range, depending on pack type and a few other factors), these are less common to hit than things related to the overall picture. For example, if a cell group is higher than the rest, under load it will likely sag to within a small % of the rest of the pack, and thus not really an issue. It'll also be likely to end up flipping to a lower voltage than the rest near lower SoC, indicating a weaker group. The BMS is able to calculate the window of usable power and energy based on everything it knows, show that on the dash as your range number, and generally you're none the wiser even if it's locking out a small portion of energy to keep things useful and safe.

The issue becomes when that window shrinks to a point where the BMS feels that a normal driving load could cause a safety concern in pushing a cell group out of spec of pulling too much power from a particular group, for example. Many of the "Maximum Charge Level Reduced" errors are the result of one or two cell groups just finally crossing a threshold that makes that operating window too tight for normal operation. A lot of them also end up being physical issues with modules that have more quickly pushed them out of the safe window, such as corrosion from moisture ingress... in which case the pack will very quickly lose more and more usable capacity with every cycle.

Overall, while a persistent imbalance can likely indicate a weak or strong cell group relative to the rest of the pack, it's not necessarily an indication of a problem. Can be, but not always. You really need the rest of the data, which the BMS unfortunately isn't normally happily divulge on CAN.

The 057 Tech Battery Monitoring Device that goes along with our extended service plans tries to gather more of the picture and come up with a better overall health estimate for the cell groups and pack as a whole by using some proprietary methods for coaxing more information than normal from the BMS. The vast majority of packs are perfectly fine, even with some voltage imbalance, as long as its within the limits of the BMS. It's the trends on this that tend to matter. If things are trending towards being outside the BMS's abilities, then this would be an issue. Most all batteries are trending this way, but at different rates. Some at a rate where issues won't occur for decades, others are a rate where you're going to be stuck in a few months or less. (We're actually considering exposing an estimated date range for this on the BMD dashboard for customers once enough data is collected to assess the trends.) There's just no way to really know based on passive CAN monitoring of group voltages alone.

Funny enough, it seems Tesla really doesn't like us gathering this data from customer vehicles and exposing it for them. We know for sure that Tesla themselves has at least one of our BMDs, and they've actually released a BMS OTA for some older S's somewhat recently that appears to have the sole purpose of obfuscating some of this data and the method we were using to gather it. Fortunately, the S/X BMS is an open book to me nowadays, so tracking these changes and updating the BMDs accordingly isn't a huge issue and there's not much they can do long term on this without hardware changes to older cars (not happening).

Anyway, bottom line is that voltage is only one small piece of the puzzle.

 

[David99]

Voltage imbalance is only about 1/4 of the picture. You can have a massive voltage imbalance at various SoC, but as long as other variables line up it doesn't matter. What matters more is CAC imbalance, or actual capacity of the cell group. The BMS calculates this per group, so it can estimate where voltages will land at various SoC. The BMS also keeps track of cell impedance, and per-group load profiles, among other things. It will calculate the max safe load/charge and allow it to continue to operate provided no safety issues.

A voltage imbalance itself might not cause issues under load/charge. While there are hard limits on this as a single metric (usually between 60-100mV imbalance while avg pack voltage is in the 3.5-3.8V range, depending on pack type and a few other factors), these are less common to hit than things related to the overall picture. For example, if a cell group is higher than the rest, under load it will likely sag to within a small % of the rest of the pack, and thus not really an issue. It'll also be likely to end up flipping to a lower voltage than the rest near lower SoC, indicating a weaker group. The BMS is able to calculate the window of usable power and energy based on everything it knows, show that on the dash as your range number, and generally you're none the wiser even if it's locking out a small portion of energy to keep things useful and safe.

The issue becomes when that window shrinks to a point where the BMS feels that a normal driving load could cause a safety concern in pushing a cell group out of spec of pulling too much power from a particular group, for example. Many of the "Maximum Charge Level Reduced" errors are the result of one or two cell groups just finally crossing a threshold that makes that operating window too tight for normal operation. A lot of them also end up being physical issues with modules that have more quickly pushed them out of the safe window, such as corrosion from moisture ingress... in which case the pack will very quickly lose more and more usable capacity with every cycle.

Overall, while a persistent imbalance can likely indicate a weak or strong cell group relative to the rest of the pack, it's not necessarily an indication of a problem. Can be, but not always. You really need the rest of the data, which the BMS unfortunately isn't normally happily divulge on CAN.

The 057 Tech Battery Monitoring Device that goes along with our extended service plans tries to gather more of the picture and come up with a better overall health estimate for the cell groups and pack as a whole by using some proprietary methods for coaxing more information than normal from the BMS. The vast majority of packs are perfectly fine, even with some voltage imbalance, as long as its within the limits of the BMS. It's the trends on this that tend to matter. If things are trending towards being outside the BMS's abilities, then this would be an issue. Most all batteries are trending this way, but at different rates. Some at a rate where issues won't occur for decades, others are a rate where you're going to be stuck in a few months or less. (We're actually considering exposing an estimated date range for this on the BMD dashboard for customers once enough data is collected to assess the trends.) There's just no way to really know based on passive CAN monitoring of group voltages alone.

Funny enough, it seems Tesla really doesn't like us gathering this data from customer vehicles and exposing it for them. We know for sure that Tesla themselves has at least one of our BMDs, and they've actually released a BMS OTA for some older S's somewhat recently that appears to have the sole purpose of obfuscating some of this data and the method we were using to gather it. Fortunately, the S/X BMS is an open book to me nowadays, so tracking these changes and updating the BMDs accordingly isn't a huge issue and there's not much they can do long term on this without hardware changes to older cars (not happening).

Anyway, bottom line is that voltage is only one small piece of the puzzle.

Thanks so much for your insight! That's very helpful.

 

[dark cloud]

Funny enough, it seems Tesla really doesn't like us gathering this data from customer vehicles and exposing it for them. We know for sure that Tesla themselves has at least one of our BMDs, and they've actually released a BMS OTA for some older S's somewhat recently that appears to have the sole purpose of obfuscating some of this data and the method we were using to gather it. Fortunately, the S/X BMS is an open book to me nowadays, so tracking these changes and updating the BMDs accordingly isn't a huge issue and there's not much they can do long term on this without hardware changes to older cars (not happening).

Oh, that is the reason for the "bug fixes" we keep getting lately. You're the bug, and they are trying to squash you.

edit: Or is this just a hidden update not included with firmware?

 

[samppa]

Voltage imbalance is only about 1/4 of the picture. You can have a massive voltage imbalance at various SoC, but as long as other variables line up it doesn't matter. What matters more is CAC imbalance, or actual capacity of the cell group. The BMS calculates this per group, so it can estimate where voltages will land at various SoC. The BMS also keeps track of cell impedance, and per-group load profiles, among other things. It will calculate the max safe load/charge and allow it to continue to operate provided no safety issues.

A voltage imbalance itself might not cause issues under load/charge. While there are hard limits on this as a single metric (usually between 60-100mV imbalance while avg pack voltage is in the 3.5-3.8V range, depending on pack type and a few other factors), these are less common to hit than things related to the overall picture. For example, if a cell group is higher than the rest, under load it will likely sag to within a small % of the rest of the pack, and thus not really an issue. It'll also be likely to end up flipping to a lower voltage than the rest near lower SoC, indicating a weaker group. The BMS is able to calculate the window of usable power and energy based on everything it knows, show that on the dash as your range number, and generally you're none the wiser even if it's locking out a small portion of energy to keep things useful and safe.

The issue becomes when that window shrinks to a point where the BMS feels that a normal driving load could cause a safety concern in pushing a cell group out of spec of pulling too much power from a particular group, for example. Many of the "Maximum Charge Level Reduced" errors are the result of one or two cell groups just finally crossing a threshold that makes that operating window too tight for normal operation. A lot of them also end up being physical issues with modules that have more quickly pushed them out of the safe window, such as corrosion from moisture ingress... in which case the pack will very quickly lose more and more usable capacity with every cycle.

Overall, while a persistent imbalance can likely indicate a weak or strong cell group relative to the rest of the pack, it's not necessarily an indication of a problem. Can be, but not always. You really need the rest of the data, which the BMS unfortunately isn't normally happily divulge on CAN.

The 057 Tech Battery Monitoring Device that goes along with our extended service plans tries to gather more of the picture and come up with a better overall health estimate for the cell groups and pack as a whole by using some proprietary methods for coaxing more information than normal from the BMS. The vast majority of packs are perfectly fine, even with some voltage imbalance, as long as its within the limits of the BMS. It's the trends on this that tend to matter. If things are trending towards being outside the BMS's abilities, then this would be an issue. Most all batteries are trending this way, but at different rates. Some at a rate where issues won't occur for decades, others are a rate where you're going to be stuck in a few months or less. (We're actually considering exposing an estimated date range for this on the BMD dashboard for customers once enough data is collected to assess the trends.) There's just no way to really know based on passive CAN monitoring of group voltages alone.

Funny enough, it seems Tesla really doesn't like us gathering this data from customer vehicles and exposing it for them. We know for sure that Tesla themselves has at least one of our BMDs, and they've actually released a BMS OTA for some older S's somewhat recently that appears to have the sole purpose of obfuscating some of this data and the method we were using to gather it. Fortunately, the S/X BMS is an open book to me nowadays, so tracking these changes and updating the BMDs accordingly isn't a huge issue and there's not much they can do long term on this without hardware changes to older cars (not happening).

Anyway, bottom line is that voltage is only one small piece of the puzzle.

We have a 2013 S here with "Maximum Charge Level Reduced" error. HV Battery had some issues and has been fixed now, but can't figure out a way to reset that alert.
Would you help us out here? For payment of course. I'm in Europe, so no way to get the car to you.

 

[wk057]

Oh, that is the reason for the "bug fixes" we keep getting lately. You're the bug, and they are trying to squash you.

edit: Or is this just a hidden update not included with firmware?

Normal OTA. There were virtually no changes to the BMS other than this stuff to hide data.

We have a 2013 S here with "Maximum Charge Level Reduced" error. HV Battery had some issues and has been fixed now, but can't figure out a way to reset that alert.
Would you help us out here? For payment of course. I'm in Europe, so no way to get the car to you.

"Fixed" is unlikely in this case, so no probably not. Unless the fault is with hardware and not related to a CAC imbalance (this has been less than 5% of cases in my experience), then nothing has actually been "fixed" and no reset will solve the issue.

 

[howardc64]

@wk057

Reading your various post seems to show water ingress usually at the front of the pack. Been going through my 13 MS85 in detail post recent LDU seal leak rebuild and disabled HV (likely corrosion on coolant heater plug on gen1 DCDC as replug fixed it) Here is my general understanding

1. AC runoff dumps directly behind the penthouse on earlier cars. While my climate (PNW) doesn't demand AC, this car has no natural air venting requiring low level AC more often to defog the windshield. For hotter climates with heavy AC usage, its probably always wet behind the penthouse during the summer.

2. Water run off path below the windshield is really poorly designed. The plastic cowl leaks both at passenger wiper spindle hole directly onto top of penthouse and dumps ample water over the gen1 DCDC area (behind it, front of it, directly onto the AC and 2 heater connectors) For cars parked outside frequently in wet weather, I'd imagine quite a problem. Wheel liners also have huge holes to for tire spray to enter as well. I park indoor at home and work but the car does see rain periodically in PNW. Incidentally the coolant overflow also dumps onto the penthouse and battery near the driver side but this is unlikely a common event (happened to me after filling up coolant post LDU install)

You also mentioned breathing vents. I think these are along the entire side of the battery? I guess front tire must spray water onto these vents driving on wet roads?

I have a 22mV imbalance in module 10 and 12 (parked, plugged in L2, 70% SOC, seems similar at higher SOC). Module 10 is the worst. Diagram shows it is directly behind the front passenger tire ( Temperature-of-18650-Cell.jpg (385×500) (circuitdigest.com) ) where tire spray and windshield runoff near gen1 DCDC likely dump to.

Anyway, any evidence majority of the ingress is near the penthouse towards the passenger side? and are imbalanced modules in this area a possible early warning?

 

[wk057]

Out of the many packs we've done, the common places seem to be the side vents on either side of the car, usually near the front and front center, suggesting front tire road spray near the failed vents is a likely culprit. Next to that is failed fasteners on top of the pack, usually from AC condensate, rain, or road salt infused moisture. Road salt areas tend to really accelerate this type of failure on older cars, too. I feel kinda bad for anyone up north who's driven regularly with a pre-2015 S in winter conditions.

We're working on custom replacement side vents (prototypes should be ready soon), but it's probably too little too late for a lot of folks.

 

[Krash]

…We're actually considering exposing an estimated date range for this on the BMD dashboard for customers once enough data is collected to assess the trends…

Would love this

 

[samppa]

Normal OTA. There were virtually no changes to the BMS other than this stuff to hide data.



"Fixed" is unlikely in this case, so no probably not. Unless the fault is with hardware and not related to a CAC imbalance (this has been less than 5% of cases in my experience), then nothing has actually been "fixed" and no reset will solve the issue.

Few sense wires were loose and some capacitors in bms boards had failed. Capacitors have been replaced, sense wires reattached.

Granted it's possible there's still something else, but for now all modules are balanced. Well if you're not willing to help, I'll have to find someone else.

 

[howardc64]

Out of the many packs we've done, the common places seem to be the side vents on either side of the car, usually near the front and front center, suggesting front tire road spray near the failed vents is a likely culprit. Next to that is failed fasteners on top of the pack, usually from AC condensate, rain, or road salt infused moisture. Road salt areas tend to really accelerate this type of failure on older cars, too. I feel kinda bad for anyone up north who's driven regularly with a pre-2015 S in winter conditions.

We're working on custom replacement side vents (prototypes should be ready soon), but it's probably too little too late for a lot of folks.

Took a look on my car... can see why moisture ingress on side vents are rampant. Following pics are from my 6k mi/year car garage parked at home and work with no salted roads In rainy/wet PNW.

- The vent/breathing slats and battery pack mounting bolt holes are completely or mostly clogged behind front tire to about 2/3 way down the front doors.
- Closer it gets to the tire, the more clogged.
- Debris on the slats packs more towards front of the car (shows clearly when getting to partial clogged slats starting at the middle of the front door) One might have expected the opposite but who knows how debris, water, air flows behind the slats.
- Slats in the inside are all clear on my pack. These slats are bigger but far fewer than the many exterior facing slats.
- Surely there are further moisture barrier behind these slats but clogged slats are trapping moisture.

My car has been parked for few days in the garage and all removed dirt/debris are still moist. So I assume water gets trapped behind these clogged holes/vents causing corrosion. Salted roads debris surely way more aggressive.

Seems highly desirable to clean these vents frequently. Probably don't want to dig too deep with a pick or blow it all out with compressed air and likely get moisture dirt deeper into the next moisture barrier. Will be checking mine every few weeks to see how long does it take to clog in my usage pattern (Now consciously avoiding the rain/wet roads after seeing the water intrusion from the windshield cowl/frunk side/wheel liner design). This is first time I've looked at these holes/slats for a nearly 10 year old battery so perhaps only need an annual cleaning... no idea how much it will help. Probably should be called out as a frequent maintenance item if the slat openings has a functional purpose (drain?, vent?, breathe?)



Since water and electricity don't mix, I'd imagine all EVs with battery pack on bottom of the car faces some challenging moisture ingress issues. Toyota hybrids don't have this issue because they tuck the small battery (compared to EV) in the center rear of the car above the floor board in away from any water under the hood/tire area.

 

[wk057]

Few sense wires were loose and some capacitors in bms boards had failed. Capacitors have been replaced, sense wires reattached.

Granted it's possible there's still something else, but for now all modules are balanced. Well if you're not willing to help, I'll have to find someone else.

It's not as much anything to do with a lack of willingness to help as it is with this being a self correcting error/alert. If the pack is actually fixed, the error will go away on its own after some use and the BMS recognizes the hardware fix. If it's not actually fixed, the error won't go away. There's only a handful of things that need manual resets on modern firmwares, and this isn't one of them.

Took a look on my car... can see why moisture ingress on side vents are rampant. Following pics are from my 6k mi/year car garage parked at home and work with no salted roads In rainy/wet PNW.

[...]

Since water and electricity don't mix, I'd imagine all EVs with battery pack on bottom of the car faces some challenging moisture ingress issues. Toyota hybrids don't have this issue because they tuck the small battery (compared to EV) in the center rear of the car above the floor board in away from any water under the hood/tire area.

Yeah, it can definitely get bad. I wouldn't suggest the compressed air route for sure, since you'll likely open up some of the likely-already-degraded pressure vents.

Tesla's design theory is pretty sound, it's just the longevity of the implementation on the earlier S/X packs just isn't there.

 

[tmyl21]

I, very recently, replaced an ‘A’ pack on my 2013 MS for over $12K. Would you say in your experience and expertise that the new ‘C’ pack I had Tesla install is more durable over time and less suspect to moisture ingress and ultimately failing?

 

[Wackelkontakt]

... The 057 Tech Battery Monitoring Device that goes along with our extended service plans tries to gather more of the picture and come up with a better overall health estimate for the cell groups and pack as a whole by using some proprietary methods for coaxing more information than normal from the BMS. The vast majority of packs are perfectly fine, even with some voltage imbalance, as long as its within the limits of the BMS. It's the trends on this that tend to matter. If things are trending towards being outside the BMS's abilities, then this would be an issue. Most all batteries are trending this way, but at different rates. Some at a rate where issues won't occur for decades, others are a rate where you're going to be stuck in a few months or less. (We're actually considering exposing an estimated date range for this on the BMD dashboard for customers once enough data is collected to assess the trends.) There's just no way to really know based on passive CAN monitoring of group voltages alone.

Any news on the monitoring only BMD plan for us overseas classic S owners that are curious about their battery's well-being?

 

[samppa]

It's not as much anything to do with a lack of willingness to help as it is with this being a self correcting error/alert. If the pack is actually fixed, the error will go away on its own after some use and the BMS recognizes the hardware fix. If it's not actually fixed, the error won't go away. There's only a handful of things that need manual resets on modern firmwares, and this isn't one of them.

Had a guy from Ukraine help us out, he managed to reset the alert with a remote connection, once I had the tokens to the car. Everything seems to be in order now. We'll see if it lasts, customer knows that nothing is for certain and is comfortable with it.
Ukrainian fellow was of the opinion that "charge level reduced" alert used to be selfcorrecting, but is not anymore in latest softwares.

 

[wk057]

Any news on the monitoring only BMD plan for us overseas classic S owners that are curious about their battery's well-being?

We're still working through some hardware iterations needed to catch up completely with everyone else, so won't happen until that's completed. We have a handful of people who have been waiting quite a while for the units, unfortunately.

Had a guy from Ukraine help us out, he managed to reset the alert with a remote connection, once I had the tokens to the car. Everything seems to be in order now. We'll see if it lasts, customer knows that nothing is for certain and is comfortable with it.
Ukrainian fellow was of the opinion that "charge level reduced" alert used to be selfcorrecting, but is not anymore in latest softwares.

Interesting.

There really isn't even an external method to "reset" this particular error, so this seems pretty strange to me overall. There's ways to factory-reset a BMS back to day 0 that not many people know about, but that's a terrible plan in pretty much all cases. Sure, it'll make all latched errors go away for a bit, but the BMS will be quite unhappy soon after it realizes it's been duped and none of its readings make sense.

Best of luck!

 

[raphy3]

Tesla just replaced the HV battery in my 2013 P85+ with 308,402 miles on it. They installed a 90kw pack, new 12v battery, upgraded rear control arms and did a four wheel alignment for $15,500. When the car was new I got 265 miles of range. Now it gets 298 miles of range.

this is awesome. this sounds like an OK price for this.

 

[raphy3]

Funny enough, it seems Tesla really doesn't like us gathering this data from customer vehicles and exposing it for them. We know for sure that Tesla themselves has at least one of our BMDs, and they've actually released a BMS OTA for some older S's somewhat recently that appears to have the sole purpose of obfuscating some of this data and the method we were using to gather it. Fortunately, the S/X BMS is an open book to me nowadays, so tracking these changes and updating the BMDs accordingly isn't a huge issue and there's not much they can do long term on this without hardware changes to older cars (not happening)

this is annoying that they try to hinder an effort just to keep old cars on the road. I guess it makes sense, since it might prevent one or two of us from breaking down and buying a new Tesla instead of trying to keep our current ones off the junk heap.
BTW I am one of the folks who's been waiting for a while for a BMD.

We're working on custom replacement side vents (prototypes should be ready soon), but it's probably too little too late for a lot of folks.

I'm interested. even though my car/battery is old, it will probably help.

 

[howardc64]

Here is my 2013 MS85 original RevB pack @ 75k miles. Car spends most of its time at 70% SOC setting plugged in at home. Charged above 93% a couple of times within last month in trying to trigger re-balancing. Seems to have made no difference.

Here is a plot of imbalance @ various SOC % for all 96 brick voltage against average

First, just the single imbalance number for the whole pack (highest brick voltage - lowest brick voltage) for 30-80% SOC. Didn't go above 80% SOC because Tesla turns on coolant pump + chiller persistently above 80% SOC for aged packs ( according to this post by @David99 )



Quite interesting imbalance is not linear to SOC. ~22mV is best my pack can do at 70% SOC and gets nearly doubled at 60% SOC. I've captured data at these SOCs multiples times. Its always the same/similar. Cell temp seems to not matter (don't know for sure)

Here is the plot of all 96 bricks. Summary is

• Bricks in module 10 is the biggest gap. Bricks in module 12 not quite as bad. All other modules similar to each other. 10 is right behind front passenger tire and fuse door. 12 is just 2 modules further down. These are high water intrusion locations but don't know if that explains this.
• Bricks within a module all close together

Module 10 and Module 12 bricks are similar to the whole pack imbalance/SOC graph above. Of course, they are the main contributors to the whole pack data (module 10 is always the lowest brick voltage)



Per @wk057 explanation in post #43 Imbalance is an incomplete picture. Presumably imbalance over SOC sweep is also incomplete picture. However, getting closer to the hard limits ( 40mV at 60% SOC) is perhaps easier to understand. @wk057 mentions 60-100mV in post #43 and this youtube video says 45mV @ 3.88V which is about 65-70 SOC on my pack where my imbalance is 22mV.

As @wk057 noted above, BMS is recording all of this to try and predict the voltage say under load and charge. Imbalance is hard to evaluate with staggered latency to read 96 bricks on ScanMyTesla.

====

The fact that my pack has the lowest imbalance at 70 SOC where it lives most of its live in last few years is interesting. Perhaps this is explainable?