Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Gen3 HPWC disassembly, with overheating issues explained!

This site may earn commission on affiliate links.

Sophias_dad

Active Member
Supporting Member
Jul 29, 2018
3,215
3,875
Massachusetts
Caution: Wall of text ahead!

So, I got a secondhand Gen3 HPWC (TPN:1457768-01-F) that had known issues at full power and of course ran it right up to 48 amps to see what she'd do(note it had updated firmware). For a little while it did 48 amps, but after a half hour in my 20f garage, it throttled back to 38-42 amps and stayed oscillating around there for a few hours until I turned it off when the outside of the case just above the identification plate reached 120f(still in my 20f garage!) Seeing a chance for a little electronic adventure, I opened it up to see what made it tick.

First, we have the familiar front. It is removed by removing two screws from the backside near the bottom, and then carefully sliding the glass downwards maybe a half inch.

Next picture is the backside, where the teeny screws(for the front) are seen at the bottom, the ground connector is the lower one, and L1/L2 are the upper ones. The circle near the middle is an optical temperature sensor, watching for the input lugs to overheat.

After the decorative glass is removed, we see eight screws holding a clear plastic plate(five were already removed in the picture!). It is gasketed and fairly weathertight, which is why there are eight screws. Remove them and the clear plastic comes off easily.

Now note the three connections at the bottom for the power and ground lines going out the cable to the handle. On mine, the screws for the power lines weren't as tight as I'd expect. Also note the small bundle of wires that clips into a black socket halfway up, they are signaling wires and a temperature sensor for the handle, and undoubtedly the button in the handle as well. Finally, note the wifi wire at the top. Be careful about pulling that off, in that it pops off perpendicular to the board, it doesn't slide off like a spade lug.

So here's where the overheating bits come into play... it gets ugly fast...
Those two big dark brown things are each two-pole relays, one relay for L2, and the other relay for L1. Tesla's hope seems to be that the two poles(called A and B in the pictures, in small blue font) of each of the two relays will take 24 of the 48 amps of that 120V leg. That hope is more of a dream, but I was surprised by how it happened. The incoming L2(I did all my analysis on L2 because it was closer to the edge of the board) spade is split into two smaller 'bars' that have a cross-sectional area between 9 and 10 AWG. I don't know exactly what the material is, but its nonferrous, I'm guessing tin coated copper or the like. I'm really surprised at this, since it'd be running 'hot'ish even with a perfect 24 amp distribution. Notably, the longer of the legs is around 4" long, the shorter is probably around 2.75" long. Right there, there's an imbalance that might throw things off, especially running the conductor at such a high current.


I removed the board from the case and was somewhat shocked(pun intended) that the PCB board was getting scorched on one of the two input poles on each of the relays. The immediate suspicion was that that pole was for some reason taking more than the desired 24 amps. The surprising bit is that for the left relay(L2), the pole connected to the LONGER input bar was the one that was discoloring, even though it should have higher resistance and therefore be carrying slightly less current.

Also note on the back of the board the optical temperature sensor, constantly looking at the input wires(from the house) to see if they are overheating due to being too small or not tightly attached. Luckily(or not), they also are checking the heat of the rest of the HPWC, even if only indirectly. This is a good thing, since it saves the entire HPWC from becoming a flaming decoration on your garage wall.

So, after some deliberation, I soldered five test leads to the underside of the board and reassembled it. The five leads were:

Arrival of L2 bus bar on the circuit board for pole A
Departure of L2 pole A relay connection(like a quarter inch of PCB board away!)
Arrival of L2 bus bar on the circuit board for pole B
Departure of L2 pole B relay connection(like 3/8 inch of PCB board away!)
Return of L2 pole A from relay.

With all these connections(and the metal bars that are relatively easily accessible from the front) and a fairly precise multimeter, I can see pretty clearly what the losses are along the way, although there's a decent chance my signals are a little polluted by the electric field of the relays. It turns out that the A pole of the L2 relay is taking around 44(!) amps of the 48 I'm charging with, and this is causing just stupid levels of heating in the box even though its not really all THAT much power being dissipated. I see around 0.21V AC is lost from the input metal bar to the output lug of the L2 side, which means that its dumping around 10 watts(0.21*48)(. Sadly, those watts have nowhere but heat to go, in a small brown package and the metal bus bar going into it.

I didn't have detailed measurements on the second(L1) relay, but saw it was dropping even MORE voltage, like 0.285V.

As an idea of temperatures in this region, the space between the two relays was reading at 250f. The relays themselves were 200f in spots, after an hour or so of running with all the covers OFF. On the bright side, it kept charging at 48 amps for around four hours, with no throttling at all.

Some might notice the rogoski coils on three of the four output lines from the relays. I assume that the software doesn't actively read them, or I'd hope something would actively shut down the system when it sees one pole of each relay is running 91% of the current. This HPWC shouldn't have been allowed out of the factory the way it is, IMHO. I also assume that the reason there are only three coils is that the HPWC could(if it cared to) see the current leaving on L1(cores 2 and 3) and assume its coming back in the same amount on L2(core not-there and 1), and thereby know the unknown not-there current individually.

So, the reason for the mismatch:

At such high current levels, even the negligible resistance of wires and relays becomes significant. Initially, I expected the longer bus bar(A) to be the issue, since longer means more resistance. Two identical resistances in parallel will cause half the current to flow each resistor, and the apparent resistance of the pair will be half each individual resistance. When one of the resistances is higher(like when a bus bar is longer) it should cause an increase in current to the lower resistance path, but clearly the longer bus bar is actually taking much MORE current than it should. This leads me to believe that either the other pole has excess resistance in the relay contacts themselves, or the very small bits of PCB board involved are just too resistant and inconsistent from one pole to the other. There may be something to this, since the PCB traces are TINY compared to AWG 9/10, and the PCB distance for the not-lots-of-current pole is somewhat longer, and the 'landing pad' for solder around the metal connection bars seems a bit small.. If I believe the 44/4 amp split of current, the itty-bitty chunk of PCB for the B side of L2 is showing about four times the resistance of the A side. It is unclear why they didn't link the A and B poles to each other at both the input and output sides for each relay, which would reduce the effect of the not-same-length feed bars. Maybe they were concerned about heating and cooling cycles cracking the PCB in between the poles.

IMHO, the design is flawed because it relies too heavily on balanced currents with nothing to really balance them, and it seems that even though there's enough electronics to monitor the imbalance, the logic doesn't seem to do much with that information. As a result, while its great to have redundancy, both paths through each relay should have been designed to be able to handle the full 48 amp load continuously without overheating.

I hope you have enjoyed this trip down Gen3 lane... and yes, it still works, but I won't be running it past 32 amps until I improve the B side connections.

front_glass.JPGfirst-off.JPGrear_outer.JPGlight-off.JPGMain-front.jpgRelays.jpgBack_full.jpgBrowned PCB.jpg
 
Last edited by a moderator:
Maybe I missed something in your wall of text, or the voltage measurements from your observation points somehow rules it out, but what makes you think it is not a bad relay? Seems more likely to be the reason for a large voltage drop (aside from a cold-soldered joint or something). I know there are points on the board that are brown, but that could be due to heat from the relay. I have no idea, just asking how you know.

Also the L1 relay seems to be dissipating a fair amount of power as well. Maybe they are both bad?

Would be interesting to know what is “normal.”
 
  • Like
Reactions: thor_2019
It had indeed occurred to me that its a bad relay, where one pole has a better connection than the other. I saw that a good portion(0.17 of the 0.21 overall) was across the terminals of the relay. But I would think that a bad relay would have zero flow through the other leg, not ~four amps.

And yes they'd have to both be bad, could be a bad batch for sure. I'd love to pop the relay off the board and check the conductivity, but even if its a mediocre relay I suspect it won't show its weakness until its pushed quite hard. Its also going to be quite an adventure, desoldering the relay will be challenging given the size of its conductors. I could desolder the input metal bars and then activate the relay by feeding it 12v to pull the coil in, but that would endanger the drive circuitry which was never expecting to be powered by the relay.

Side note, the relay is not publicly available.
 
Caution: Wall of text ahead!

So, I got a secondhand Gen3 HPWC (TPN:1457768-01-F) that had known issues at full power and of course ran it right up to 48 amps to see what she'd do(note it had updated firmware). For a little while it did 48 amps, but after a half hour in my 20f garage, it throttled back to 38-42 amps and stayed oscillating around there for a few hours until I turned it off when the outside of the case just above the identification plate reached 120f(still in my 20f garage!) Seeing a chance for a little electronic adventure, I opened it up to see what made it tick.

First, we have the familiar front. It is removed by removing two screws from the backside near the bottom, and then carefully sliding the glass downwards maybe a half inch.

Next picture is the backside, where the teeny screws(for the front) are seen at the bottom, the ground connector is the lower one, and L1/L2 are the upper ones. The circle near the middle is an optical temperature sensor, watching for the input lugs to overheat.

After the decorative glass is removed, we see eight screws holding a clear plastic plate(five were already removed in the picture!). It is gasketed and fairly weathertight, which is why there are eight screws. Remove them and the clear plastic comes off easily.

Now note the three connections at the bottom for the power and ground lines going out the cable to the handle. On mine, the screws for the power lines weren't as tight as I'd expect. Also note the small bundle of wires that clips into a black socket halfway up, they are signaling wires and a temperature sensor for the handle, and undoubtedly the button in the handle as well. Finally, note the wifi wire at the top. Be careful about pulling that off, in that it pops off perpendicular to the board, it doesn't slide off like a spade lug.

So here's where the overheating bits come into play... it gets ugly fast...
Those two big dark brown things are each two-pole relays, one relay for L2, and the other relay for L1. Tesla's hope seems to be that the two poles(called A and B in the pictures, in small blue font) of each of the two relays will take 24 of the 48 amps of that 120V leg. That hope is more of a dream, but I was surprised by how it happened. The incoming L2(I did all my analysis on L2 because it was closer to the edge of the board) spade is split into two smaller 'bars' that have a cross-sectional area between 9 and 10 AWG. I don't know exactly what the material is, but its nonferrous, I'm guessing tin coated copper or the like. I'm really surprised at this, since it'd be running 'hot'ish even with a perfect 24 amp distribution. Notably, the longer of the legs is around 4" long, the shorter is probably around 2.75" long. Right there, there's an imbalance that might throw things off, especially running the conductor at such a high current.


I removed the board from the case and was somewhat shocked(pun intended) that the PCB board was getting scorched on one of the two input poles on each of the relays. The immediate suspicion was that that pole was for some reason taking more than the desired 24 amps. The surprising bit is that for the left relay(L2), the pole connected to the LONGER input bar was the one that was discoloring, even though it should have higher resistance and therefore be carrying slightly less current.

Also note on the back of the board the optical temperature sensor, constantly looking at the input wires(from the house) to see if they are overheating due to being too small or not tightly attached. Luckily(or not), they also are checking the heat of the rest of the HPWC, even if only indirectly. This is a good thing, since it saves the entire HPWC from becoming a flaming decoration on your garage wall.

So, after some deliberation, I soldered five test leads to the underside of the board and reassembled it. The five leads were:

Arrival of L2 bus bar on the circuit board for pole A
Departure of L2 pole A relay connection(like a quarter inch of PCB board away!)
Arrival of L2 bus bar on the circuit board for pole B
Departure of L2 pole B relay connection(like 3/8 inch of PCB board away!)
Return of L2 pole A from relay.

With all these connections(and the metal bars that are relatively easily accessible from the front) and a fairly precise multimeter, I can see pretty clearly what the losses are along the way, although there's a decent chance my signals are a little polluted by the electric field of the relays. It turns out that the A pole of the L2 relay is taking around 44(!) amps of the 48 I'm charging with, and this is causing just stupid levels of heating in the box even though its not really all THAT much power being dissipated. I see around 0.21V AC is lost from the input metal bar to the output lug of the L2 side, which means that its dumping around 10 watts(0.21*48)(. Sadly, those watts have nowhere but heat to go, in a small brown package and the metal bus bar going into it.

I didn't have detailed measurements on the second(L1) relay, but saw it was dropping even MORE voltage, like 0.285V.

As an idea of temperatures in this region, the space between the two relays was reading at 250f. The relays themselves were 200f in spots, after an hour or so of running with all the covers OFF. On the bright side, it kept charging at 48 amps for around four hours, with no throttling at all.

Some might notice the rogoski coils on three of the four output lines from the relays. I assume that the software doesn't actively read them, or I'd hope something would actively shut down the system when it sees one pole of each relay is running 91% of the current. This HPWC shouldn't have been allowed out of the factory the way it is, IMHO. I also assume that the reason there are only three coils is that the HPWC could(if it cared to) see the current leaving on L1(cores 2 and 3) and assume its coming back in the same amount on L2(core not-there and 1), and thereby know the unknown not-there current individually.

So, the reason for the mismatch:

At such high current levels, even the negligible resistance of wires and relays becomes significant. Initially, I expected the longer bus bar(A) to be the issue, since longer means more resistance. Two identical resistances in parallel will cause half the current to flow each resistor, and the apparent resistance of the pair will be half each individual resistance. When one of the resistances is higher(like when a bus bar is longer) it should cause an increase in current to the lower resistance path, but clearly the longer bus bar is actually taking much MORE current than it should. This leads me to believe that either the other pole has excess resistance in the relay contacts themselves, or the very small bits of PCB board involved are just too resistant and inconsistent from one pole to the other. There may be something to this, since the PCB traces are TINY compared to AWG 9/10, and the PCB distance for the not-lots-of-current pole is somewhat longer, and the 'landing pad' for solder around the metal connection bars seems a bit small.. If I believe the 44/4 amp split of current, the itty-bitty chunk of PCB for the B side of L2 is showing about four times the resistance of the A side. It is unclear why they didn't link the A and B poles to each other at both the input and output sides for each relay, which would reduce the effect of the not-same-length feed bars. Maybe they were concerned about heating and cooling cycles cracking the PCB in between the poles.

IMHO, the design is flawed because it relies too heavily on balanced currents with nothing to really balance them, and it seems that even though there's enough electronics to monitor the imbalance, the logic doesn't seem to do much with that information. As a result, while its great to have redundancy, both paths through each relay should have been designed to be able to handle the full 48 amp load continuously without overheating.

I hope you have enjoyed this trip down Gen3 lane... and yes, it still works, but I won't be running it past 32 amps until I improve the B side connections.
How did they fix this issue then? I got a new unit that doesn't overheat anymore.
 
How did they fix this issue then?

I gotta assume they either put in new not-bad relays, increased/changed the size of the bars delivering power, or added a slab of copper to the back of the PCB, or some combination thereof. It would be interesting to pop the cover off a new one and see if relay part numbers changed.
 
I gotta assume they either put in new not-bad relays, increased/changed the size of the bars delivering power, or added a slab of copper to the back of the PCB, or some combination thereof. It would be interesting to pop the cover off a new one and see if relay part numbers changed.
If the part numbers can be seen just from popping the cover off, i might be will to do that for you. If it requires more disassembly, then no .:D
 
  • Helpful
Reactions: jjrandorin
@Sophias_dad Thank you for the extremely informative post!

That answered a lot of the questions a lot of us have had regarding the mysterious problems with the Gen 3 Wall Connector.

I'd love to see you do a detailed analysis of the Gen 2 unit, but not quite bad enough to rip mine off the wall and send it to you. ;)
 
I'd love to see you do a detailed analysis of the Gen 2 unit
I actually have already had one torn all the way down to its individual boards(3) a few weeks ago and repaired the main logic board where one of the power lines(during install) had damaged the SMD inductor just left of the signal wire connection block. It had gotten insufficiently coated in epoxy at the factory and the wires in the coil are just a bit bigger than a human hair. Sorry I didn't think to get pics at the time, although I have a few of the inductor I fixed.

The Gen2 has largely the same stuff, but bigger. One board near the top is strictly a relay board, housing just two MASSIVE relays, one rather simple-looking one near the rear of the unit is for power entry and dipswitches/current-dial(didn't look real hard), and the main logic board is the one you see the bottom edge of, closest to you when you take off the cover.
 
The car uses L1 and L2. That's how it gets 240V. It's not using two 120V circuits. It is physically impossible for there to be a current mismatch on L1 and L2. Any additional resistance in circuit would result in additional power usage on both L1 AND L2.

(moderator note: removed inflammatory comment)

Yes, you misunderstood... There's a big relay for L1, and there's a big relay for L2. EACH of those relays has two poles, an A side, and a B side. In my case, L2's A side is conducting a calculated 44 amps, and L2's B side is conducting 4 amps....Last time I checked, 44+4 is 48, so there's 48 amps at 120 V.

And YES, the car uses L1 and L2, and they are both 120V circuits, but 180 degrees out of phase, so that if you compare them using the AC range of a multimeter, they are 240 volts.

Here's a relevant wikipedia page: Split-phase electric power - Wikipedia

And a handy quote from the second paragraph ...."Two 120 V AC lines are supplied to the premises which are out of phase by 180 degrees with each other (when both measured with respect to the neutral), along with a common neutral."

And here's the schematic of EACH of the big relays. Sadly, What I've been calling pole A would be A1/B1(pole 1), and what I've been calling pole B would be A2/B2(pole 2), and the orientation of the HV connections on the actual physical relay are rotated 90 degrees compared to the diagram, so the poles follow the long-sides of the relay.
dpst_diagram.png


Here's a nice zoomed-in shot of the back of the L2 relay, followed by the output side where the two poles are MERGED for a single L2 presented to the car.
 

Attachments

  • Relay-closeup.jpg
    Relay-closeup.jpg
    210.8 KB · Views: 1,277
  • relay-output.jpg
    relay-output.jpg
    142.5 KB · Views: 1,237
Last edited:
Maybe I missed something in your wall of text, or the voltage measurements from your observation points somehow rules it out, but what makes you think it is not a bad relay? Seems more likely to be the reason for a large voltage drop (aside from a cold-soldered joint or something). I know there are points on the board that are brown, but that could be due to heat from the relay. I have no idea, just asking how you know.

Also the L1 relay seems to be dissipating a fair amount of power as well. Maybe they are both bad?

Would be interesting to know what is “normal.”
💯 Bad relay or bad connection (cold solder)

Bad things happen quickly to wires, connections and PCBs when they are well engineered.... IE just big enough to handle design load/heat dissipation
 
Bad things happen quickly to wires, connections and PCBs when they are well engineered.... IE just big enough to handle design load/heat dissipation
Oh man, you happened upon one of my trigger points. There was a time.... you know what, forget it. This is one of those subjects that is just way too much for me to discuss in a rational manner. Probably has to do with my career choice.

So instead, let's just leave it with something like, "Gee, I wonder how much money this well-engineered part is going to save us in the long run." Well, if you subtract the warranty replacement costs and customer ill will (and possible liability law suits), maybe a touch of over-engineering is still warranted.
 
As an engineer you have to design for all components to be within spec and installed properly.... I am sure it's some MBAs fault on the assembly line.... Labour cuts everywhere 🤣.

I am sure Tesla HPWC is well within acceptable ranges of failure rate, and it does its job to derate when it detects overheating. It sucks as an end user to get a stinker.... These issues should be caught by QC on the production line somewhere.
 
Just to wrap this up... pretty sure its just a crappy relay batch. I improved the connections dramatically(IMHO), and although it was doing a bit better temperature and voltage-drop wise before the covers went on, it still started to rollback the current after a half hour in my 50f garage once all the covers were back on. I guess that's a little better overall, since the last time it had that behavior in a 20f garage.

The relays are not available to commoners, which is just as well because they are essentially not removable(in one piece) from the board due to the size of the conductors. I guess if you had spares on hand you could destroy the old relay from the top down and then remove the conductors one by one (eww).
 
Relays are Song Chuan's 118-2AH - F-C. I'm not familiar with the 118 series, but 108 is a SPNO (single pole normally open) 25A rated relay for automotive usage; I'd imagine that the 118 has to be a series with similar characteristics; "2A" is DPNO (double pole normally open, "H" is AgSnO contact type (silver alloy), " " after that is contact gap>1.85mm, " " after "-" means standard type, "F" is Class F insulation for the coil (up to 155C or 311F), "C" is flux tight.

Not a common relay but very likely a "made for customer" part number that only Tesla can buy. Other than that, there is no way for this kind of imbalance between the 2 poles: cold solder in one of the legs is extremely likely; the heat is not in the relay or it would have melted already, their standard parts are rated for 85C or 185F only.

That said, 25A rating is suspect without doubt, I'd imagine that the 118 series is at least a 60A per leg to switch 48A comfortable ...

Nice write-up thanks,

Hope it helps
 
Sooo... more developments afoot.. as an exercise, I added some custom-tailored aluminum heat sinks to the top of the two relays, in hopes that they would get some cooling inside that intentionally-sealed box. Needless to say it was ineffective, partly because there was not much space available for the heatsinks(10mm tall for the right relay, 4-10mm for the left relay).

Not willing to let this Gen3 get the better of me, I went back to the soldering iron and solder wick and was able to remove the more-lossy right-hand relay, I chose it just because it had a greater voltage drop and would likely be more interesting. Initially, the resistance measured for the two legs(with the relay pulled in) was 0.02ohm and 0.04ohm respectively, but I don't really trust the meter I have for such low resistances.

So I built a test-harness, soldered the two input legs together with a pair of 10awg so there would be no meaningful voltage loss there, and equal-length(3") 12awg for the two output legs. Connecting my handy-standard 11amp 120vac load to the two output legs(one at a time), with wire nuts and such(presumed to be 'perfect' connections). I found voltage drops of 0.0128V and 0.365V respectively at the actual relay lugs, leading me to find the A leg's contacts had 0.0116 ohms resistance, while the B leg had 0.0332ohms resistance.

The combined parallel perceived resistance of this pair would be 0.0086 ohms, and there would be 0.412 volts of loss at 48 amps, dissipating almost 20 watts. Its not quite as good as I'd like compared to my earlier actual measurement of 0.285v but maybe the contacts are a bit non-linear . The power through the A leg(14.6w) is only three times that of the B leg(5w), but that's plenty of difference to explain the scorched PCB.

If both contacts were 'good', the flow through them would be even-ish, the overall resistance would be 0.0058, and power dissipation would be a much more manageable 13.4 watts. This isn't really as good as I'd like, but given there's really nowhere for that power to go, having 6-7 less watts to dispose of probably makes a big difference.
 
Last edited:
When I read all this, what I basically see is "jjrandorin, if your wife gets a tesla when the BMW goes back after its lease is over, and you want another wall connector, move heaven and earth to buy a gen2 as you likely dont want the gen3."

lol... thats what I get out of it, anyway. I should have never sold the (unopened) spare gen 2 I got from the referral program.
 
  • Like
Reactions: Nenarek
I wonder if the reason they didn't go with a really beefy single pole breaker per leg was the difficulty in moving 48 amps through one single circuit board trace. The life of a Tesla HPWC relay has to be pretty easy, other than the power being moved, since they don't have to worry about arcing or other awful effects during connection/disconnection because the car can be told to not draw power until the circuit is deemed 'good'.

They might have an issue with contacts getting dirty/oxidized, because apparently some relay contacts are designed to be self-cleaning when engaged/disengaged with at least SOME reasonable amount of current flowing.