Powerwall 2 + UPS Connundrum - and solution

[mspohr]

The Enphase IQ series microinverters communicate over powerline. CANBus is a wired standard but not power line. Whatever protocol they use is almost certainly not something that a Powerwall speaks. I don't think a Powerwall uses any power line communication. Anyone know what protocols either use, or if proprietary, on what either are based?

communication kit
The wireless communication kit enables direct communication between Encharge, Enpower, and IQ Envoy. It uses 2.4 GHz and 915 MHz frequencies in parallel for maximum reliability. The kit will be connected to one of the USB ports on the IQ Envoy.

No idea if Powerwall can talk to Enphase. My guess is probably not.

 

[cwied]

I might not be the most informed person about this..
But why isn't possible for (micro)inverters to just communicate via a Canbus or something with the powerwall/backupgateway.
So that when they need to shutdown they can just send such a message via the can instead of increasing the frequency and causing all kinds of problems?

Sure, and even better, they could hook the batteries up to the DC directly as well to avoid the double-conversion losses from AC charging. This was the premise of the DC coupled Powerwall. I believe it was a serial line integration for that. The problem is that you have to do an integration with every inverter that you want to support directly. AC coupling requires almost no incremental engineering effort to support additional inverter brands. This is probably why the DC coupled Powerwall was canceled.

 

[mspohr]

Sure, and even better, they could hook the batteries up to the DC directly as well to avoid the double-conversion losses from AC charging. This was the premise of the DC coupled Powerwall. I believe it was a serial line integration for that. The problem is that you have to do an integration with every inverter that you want to support directly. AC coupling requires almost no incremental engineering effort to support additional inverter brands. This is probably why the DC coupled Powerwall was canceled.

You can get more power through DC connections. I have an SMA inverter connected to a series of panels which put out 420v (nominal... often goes up to 500v) through 10 gauge wire and puts out up to 2800 W which gives about 6.5 amps. If this was AC, it would be 240v (rms) and 11.5 amps. Less current, lower power loss with DC. (This is why utilities sometimes use DC for high power transmission lines.)

 

[cali8484]

I might not be the most informed person about this..
But why isn't possible for (micro)inverters to just communicate via a Canbus or something with the powerwall/backupgateway.
So that when they need to shutdown they can just send such a message via the can instead of increasing the frequency and causing all kinds of problems?

Of course it's possible but Tesla (and many other hybrid inverter manufacturers) don't want to invest the money to integrate with all the different and proprietary string and micro grid-tied inverter communication protocols. Instead they chose a shortcut and expects buyers to live with the inherent problems of the shortcut.

 

[Doom Man]

Just an FYI for those similarly situated: I have Sunpower 370 panels with Sunpower microinverters that are just re-branded Enphase IQ 7X. The microinverters default to shut off above 61hz, per Enphase customer support, but installers have the option of customizing that default setting. My third-party installers did not change the default. Tesla changed my full-PW frequency from 65 to 62hz, and it took them a while to do it, but everything seems to work fine now. Everyone I spoke to in Tesla customer service (three calls, and this was in April 2020) was knowledgeable about the power frequency issue.

Problems I was having at 65hz: an older heat pump with a Surestart (soft start) would not start up all the way -- the compressor would make a sound and the blades would spin, but the compressor itself would not kick on. The Surestart seemed to recognize the "bad" power and would not let the unit run until the PW's dropped to 95 or 96% and the power frequency dropped back to 60. A newer heat pump with a modern brain also recognized the bad power -- there was an audible sound of something tripping/clunking about 20 seconds after switching to battery, and the compressor would not start at all until battery power dropped. Both of those work fine now. I also had some of the UPS and flickering-LED issues that others have described, all fixed by going to 62 hz.

 

[gpez]

Of course it's possible but Tesla (and many other hybrid inverter manufacturers) don't want to invest the money to integrate with all the different and proprietary string and micro grid-tied inverter communication protocols. Instead they chose a shortcut and expects buyers to live with the inherent problems of the shortcut.

Not just a shortcut - the Powerwall already had to do this to be UL 1741 SA certified so it likely was almost no additional work.

 

[Cirrus MS100D]

Thanks in advance to all the contributors on this thread!

I believe we've been bitten by this "feature" and I finally made the call today to Tesla to ask them to fix it. I noticed it for the first time during our first real, lengthy outage since our system was installed in Oct 2019. The 2x Powerwalls were at about 98% charged on a very sunny day. When power went out, I was surprised to see our solar drop to zero. It came back on after about 30 minutes, but I was curious about this issue as I remembered seeing this thread a while back.

I decided to conduct my own test (which I should have done ages ago...) and killed the main breaker from the street. Sure enough, I realized my UPSes were beeping while solar dropped to zero. I even let the Powerwalls run down to 90% and only then did they briefly start to take solar (we only have a tiny 4kW system!) and even then, it would run for a little while, then shut it down again for ~10 minutes, then solar would come back for a little bit, then shut down again.

Today, we had another "real" power outage, but this time NO solar production (dark as night storm clouds overhead) and I was able to snag the reading from http://my.internal.powerwall.ip/api/meters/aggregates and sure enough, I saw "frequency":65.19xxx and my computer UPSes were beeping as if power was fully out.

I only had to wait on hold for about a minute and spoke to a helpful Tesla Energy tech who understood what I was talking about. He actually asked for the name/model number of UPSes (I also verified by pulling them up on the APC and CyberPower site that they would only take inputs from 57-63hz). He confirmed they knew we had a SolarEdge HD Wave inverter (if I'm reading the documentation correctly, it seems to show 60.5hz?) He said he had to pass on all this info to a L2 tech who would investigate and, if possible, make the change. I got the impression this was standard procedure at this point. I'm crossing my fingers that I'll hear back in the next week or so and can then conduct another power-off test to see if it solves my issues.

 

[jjrandorin]

Thanks in advance to all the contributors on this thread!

I believe we've been bitten by this "feature" and I finally made the call today to Tesla to ask them to fix it. I noticed it for the first time during our first real, lengthy outage since our system was installed in Oct 2019. The 2x Powerwalls were at about 98% charged on a very sunny day. When power went out, I was surprised to see our solar drop to zero. It came back on after about 30 minutes, but I was curious about this issue as I remembered seeing this thread a while back.

I decided to conduct my own test (which I should have done ages ago...) and killed the main breaker from the street. Sure enough, I realized my UPSes were beeping while solar dropped to zero. I even let the Powerwalls run down to 90% and only then did they briefly start to take solar (we only have a tiny 4kW system!) and even then, it would run for a little while, then shut it down again for ~10 minutes, then solar would come back for a little bit, then shut down again.

Today, we had another "real" power outage, but this time NO solar production (dark as night storm clouds overhead) and I was able to snag the reading from http://my.internal.powerwall.ip/api/meters/aggregates and sure enough, I saw "frequency":65.19xxx and my computer UPSes were beeping as if power was fully out.

I only had to wait on hold for about a minute and spoke to a helpful Tesla Energy tech who understood what I was talking about. He actually asked for the name/model number of UPSes (I also verified by pulling them up on the APC and CyberPower site that they would only take inputs from 57-63hz). He confirmed they knew we had a SolarEdge HD Wave inverter (if I'm reading the documentation correctly, it seems to show 60.5hz?) He said he had to pass on all this info to a L2 tech who would investigate and, if possible, make the change. I got the impression this was standard procedure at this point. I'm crossing my fingers that I'll hear back in the next week or so and can then conduct another power-off test to see if it solves my issues.

My guess is, they will set you to 62hz, after the level 2 person reviews it and looks at the inverter specs etc.

 

[aesculus]

My guess is, they will set you to 62hz, after the level 2 person reviews it and looks at the inverter specs etc.

I am betting they will set him to 63 Hz, but it should be 62Hz IMHP. I had to fight for that extra (less) Hz.

But maybe by now they have learned a bit more.

 

[power.saver]

And I think it should be 61Hz if the inverters shut off at 60.5Hz. No need for anything higher.

That's what I got, after I specifically requested it.

 

[aesculus]

And I think it should be 61Hz if the inverters shut off at 60.5Hz. No need for anything higher.

That's what I got, after I specifically requested it.

That's what I wanted and asked for too. They even sent a guy out at the height of the COVID-19 event to run tests. I had to of course report on effects because they were not coming into the house. They lowered it to 63 Hz but I was still having issues with the UPS. They then lowered it to 62 Hz and that fixed the UPS problem but my Insteon stuff still would not work. They would not budge from there.

My interters also trip at 60.5 Hz and they knew that.

 

[jjrandorin]

And I think it should be 61Hz if the inverters shut off at 60.5Hz. No need for anything higher.

That's what I got, after I specifically requested it.

I think you would find that we all agree with you on what it should be, but I my guess is, tesla is being extra cautious with this. When the trailblazers in this thread started reporting the issue to tesla, they probably were handling things on a more customized, case by case basis (more likely escalated to level 3 / engineering type folks).

Now that it seems enough people have reported the issue (and gotten changes from tesla) that they, in general, understand the issue, I bet there is some guidance for the "level 2" techs to follow. They are not going to let level 1 (call center /help desk) resolve the issue, even though its probably just a few settings on their end, due to the liability or "what could happen if something goes wrong"..... but they also probably try to set the number as high as possible that does not impact the customers reported equipment (UPS, microwaves, etc).

TL ; DR .... Its much easier to get it resolved now, but much less customized to the persons actual inverter specifications etc.

At least, this is my opinion anyway, from my experience knowing how support organizations tend to work.

 

[mspohr]

I think you would find that we all agree with you on what it should be, but I my guess is, tesla is being extra cautious with this. When the trailblazers in this thread started reporting the issue to tesla, they probably were handling things on a more customized, case by case basis (more likely escalated to level 3 / engineering type folks).

Now that it seems enough people have reported the issue (and gotten changes from tesla) that they, in general, understand the issue, I bet there is some guidance for the "level 2" techs to follow. They are not going to level level 1 (call center /help desk) resolve the issue, even though its probably just a few settings on their end, due to the liability or "what could happen if something goes wrong"..... but they also probably try to set the number as high as possible that does not impact the customers reported equipment (UPS, microwaves, etc).

TL ; DR .... Its much easier to get it resolved now, but much less customized to the persons actual inverter specifications etc.

At least, this is my opinion anyway, from my experience knowing how support organizations tend to work.

I still think this whole situation could be eliminated by making the software smarter.
Instead of having a fixed frequency offset to shut off the solar inverters, the software could change the frequency in steps and monitor the inverter output for a response. Over time, it could even "learn" the correct frequency to shut down the inverters.

 

[HankLloydRight]

Slightly OT Powerwall event: Last night my wife and I were watching Netflix (using Tivo). When I suddently realized our neighbor's gas generator had been running for a while. Strange, I though, they do test run it every Tuesday morning for 30 minutes, but why is it running now? Oh! We must have a power outage! Sure enough, our power had been out about 20 minutes at that point and nobody in our house even noticed. I added a consumer UPS to my cable modem about a month ago, so even our internet connection was completely uninterrupted. I did remember our incandescent lights flickering, but that's normal due to our old A/C units. By the time I opened my apps up to monitor the situation, our power was restored.

 

[power.saver]

That's what I wanted and asked for too. They even sent a guy out at the height of the COVID-19 event to run tests. I had to of course report on effects because they were not coming into the house. They lowered it to 63 Hz but I was still having issues with the UPS. They then lowered it to 62 Hz and that fixed the UPS problem but my Insteon stuff still would not work. They would not budge from there.

Really odd that they wouldn't help you after the adjustment didn't solve the problem. Maybe 62Hz is the new limit, but for me, I had to run multiple simulations with grid off and solar producing to demonstrate the inverters shutoff at 60.5Hz. After that, it was changed. No site visit.

TL ; DR .... Its much easier to get it resolved now, but much less customized to the persons actual inverter specifications etc.

That's probably right, unfortunately. And if your inverter offers any possibility of changing the shutoff frequency, that too may be a contributing factor. My inverters old enough that there are no adjustments, and no curtailment ramp either.

 

[aesculus]

I still think this whole situation could be eliminated by making the software smarter.
Instead of having a fixed frequency offset to shut off the solar inverters, the software could change the frequency in steps and monitor the inverter output for a response. Over time, it could even "learn" the correct frequency to shut down the inverters.

Are we sure it does not operate this way today? The only tests I have done were at or near 100% charge. In that case they wanted no solar at all so going to the max frequency was the thing to do.

Has anyone monitored their units (who also have IEEE 1547 inverters) to see if Tesla gradually starts to increase the frequency when they get above 80% SoC?

 

[power.saver]

Has anyone monitored their units (who also have IEEE 1547 inverters) to see if Tesla gradually starts to increase the frequency when they get above 80% SoC?

Yes, I have, and they do. At about 96%, the frequency begins to ramp up and stops going up once the inverters are shutoff. So in that situation you will not get to 62Hz. It's only when the batteries are full, your solar is producing, and the grid goes down that they jump to the max set frequency. No ramping up.

 

[aesculus]

@power.saver an additional question: If in your scenario about the solar system is not producing either by time of day or cloud cover etc., does the PWs all sense this and not ramp up either? So are they actually monitoring the current from the solar input and reacting appropriately vs just some setting based on PW SoC?

 

[power.saver]

@power.saver an additional question: If in your scenario about the solar system is not producing either by time of day or cloud cover etc., does the PWs all sense this and not ramp up either? So are they actually monitoring the current from the solar input and reacting appropriately vs just some setting based on PW SoC?

It's a combination of the two. Even if solar is not producing (say at night) but the batteries are at 100%, the frequency will go up (probably to prevent the solar from starting if it could). Once the battery drops below 98% the frequency will come down.

 

[aesculus]

So it sounds like it's all based on SoC of the PWs. If it truly was measuring the output of the solar it would even bother adjusting the frequency if there was no output. And the amount of time it would take to respond if the inverters miraculously came on would seem to be negliable.

It's going to be a rare occasion when this happens but nice to be aware of it none the less.