Gateway 2 and Smart Meter accuracy

[woferry]

I am in the US and I'm sure there are lots of brands of meters out there, plus I don't know what the power factor of my house loads are (I do have very few actual motors and lots of electronics), but our utility replaced the mechanical meters with Smart Meters a long time ago (to make their lives easier, so they could read the meters remotely as well as making TOU easier), and it did not affect my bill at all, my reported monthly consumption for the last years on the analog meter were no different than my first years with the Smart Meter. There was lots of FUD going around at the time telling people to refuse the switch, that it was just a way for the utility to start fudging the numbers or the like, but I never saw any evidence of that personally, I've been able to account for any increases or decreases in my reported consumption by changes I've made in devices/appliances I use, personal habits, etc.

I realize that doesn't help with the issue you're seeing, just saying that it's not necessarily the simple fact that it's a Smart Meter (and I'd hope there would have been huge negative press if it actually was a ploy to increase what people pay).

 

[mikemillar]

Thanks @woferry. That's useful and interesting. Apparently fridge and freezer motors are among the appliances that have a power factor of less than one so one would have thought that your meter might show a difference if it was recording different parameters. Similarly, air con works like a fridge in reverse it would affect power consumption in a similar way. Like most houses in UK we don't have air con, but I imagine you do and it probably makes a heavy demand.

I only noticed it because with the Powerwall operating there should be zero consumption during the day and it's definitely clocking up 1.5kWh/day or about 200kWh per year. Given the typical annual variation on my 12000kWh per year, I would probably not noticed an increase of 200kWh just by looking at monthly or annual meter readings.

It may still be the Powerwall taking more than it should do, it appears to be well in excess of the Tesla specification. As you say, one would have thought that there would have been a big stink in the press if the power companies were using it to bring in extra income.

According to the forum article I posted, commercial companies are metered and billed that way anyway. If everyone is being measured and billed the same way it would be a level playing field and should not be an issue. One would hope that the increase in measured and billed kWh would eb offset by a reduction in price....maybe.

 

[Merrill]

I also fought the change to a smart meter because I was worried there would be an increase in my bill. I logged my usage for several years before they finally made me switch and there was no change.

 

[woferry]

No AC here, so I don't have that demand (my house typically runs 12-13kWh/day year-round, mostly due to computers and other electronics, so 350-400kWh/month on average). As I had remarked earlier in this thread, my system certainly seems skewed to result in a net ~20W average draw from the grid. So that only works out to ~0.5kWh/day max, and since roughly half of the day I'm exporting solar it's really more like half of that, so it's a lot less than what you're seeing. It seems to me that some skew is probably intentional, as I suspect they'd be in trouble if the steady state was 'leaking' PW energy back out to the grid, and there always has to be some tolerance so it's biased to ensure there's never a net output when the consumption is supposed to be "zero". But as to how much variance there is I really don't know (is it 20W nominal and my system is average, or 20W minimum and my system is on the 'good side' of the distribution while others will be worse, etc). It wasn't clear to me looking back on the thread whether it was established that you're using a v1 Gateway (simple metal box) or the v2 (mini Powerwall enclosure), and if v2 I've always wondered just what it looks like on the inside (is it still using a Neurio or something else), I'd guess if there might be something wrong with a CT (perhaps even picking up noise from some adjacent circuit wire, etc) it might affect things enough.

Looking back, post #27 seems like a red flag to me. As far as I can tell, the Gateway achieves my ~20W average by sometimes importing >20W, and sometimes exporting power to compensate. I.e. a load in the house turns on, the instant demand comes partly from the grid because the PW is ramping-up to deal with the new load, and then it lets some of the PW energy leak out onto the grid to make up for (most, but not all) of the surge (similar behavior in reverse when a load suddenly turns off). If you're meter is only recording the import and ignoring the export then it would not work with this approach as it wouldn't see the 'make up' periods. So if that's all true (that the meter is working one-way and the Gateway works the way my data suggests it does) then that might explain why things aren't lining up (and would, effectively, mean that your Smart Meter is 'cheating').

The way I first determined the 20W average of my system was by looking at all of its reported grid consumption from the Gateway's API over multiple evening periods where solar was off and the house was 100% powered from the PW (never hit the Reserve). But this wasn't just a constant 20W number, it was a bunch of numbers all over the place, some positive and some negative. When you averaged all of them the answer was 20W. But if you threw out all of the negative numbers the average obviously would have been quite a bit higher than 20W. So it seems to me that is what your meter is effectively doing, averaging the readings without considering any of the negative numbers. Since I also get readings from my meter, it showed the same sometimes-positive, sometimes-negative readings my Gateway was reporting, and the same overall average of ~20W.

 

[JohnRatsey]

My smart meter records the eletricity export data but I do not think it sends the data anywhere - definitely not to the in-home read-out which only displays usage. I can see the export total by pressing the '9' button on the meter.

I had no reservations about having a smart meter installed when it was offered as it eliminated the hassle of making manual readings. I would hope that a box of electronics would be at least as accurate as a mechanical meter and didn't notice any significant change in the bills before/after smart meter installation. [My meter is actually a pair - one for electricity and one for gas. The gas meter transmits it's usage data to the electricity meter which uploads both via the cell phone network. I hope that the gas meter is programmed to shout for a new battery before it needs it.]

 

[charlesj]

.... There was lots of FUD going around at the time telling people to refuse the switch, that it was just a way for the utility to start fudging the numbers or the like, but I never saw any evidence of that personally, I've been able to account for any increases or decreases in my reported consumption by changes I've made in devices/appliances I use, personal habits, etc.

I realize that doesn't help with the issue you're seeing, just saying that it's not necessarily the simple fact that it's a Smart Meter (and I'd hope there would have been huge negative press if it actually was a ploy to increase what people pay).

And, some people protested against them that it is a health hazard.

These revenue meters are covered by regulations the same way as gas pumps, store scale accuracy, ets are. The utility would pay a hefty fine if they were caught cheating.

 

[mikemillar]

I'm sure there are many pay-for services which will calculate your solar yield for the following day but I am still struggling up a steep learning path trying to get an approximate figure using the various free weather services that are out there.

Sorry @sashton, I missed your post. You're right, there' a whole industry devoted to forecasting solar radiation. However, for the likes of you and me, here's a cheap possibility. I use www.topmeteo.eu. It's a paid for weather forecasting service for aviators. I subscribe to the gold plated all Europe package because I fly a light aircraft to France/Italy/germany in the summer and it's invaluable or that. But it looks like the Smart Spender Package for one region (e.g.UK) with 2 hourly forecast for €57.90, per annum would be your best option. It used to be possible to buy a 2 week package to try it out but I don't see that now.You'll get a lot more useful weather info than solar radiation for that. Here's an example of the solar radiation forecast for monday for my locality.

The cloud cover bars are quite useful too. Only a piffling 230w/m2 max on Monday, but the lack of cloud cover should get me 4-5kWh of solar generation. I used it quite a lot in the summer when it forecast up to about 700 watts/m2, to try and forecast when the Powerwall might get it wrong and run out of power, which to did. It's quite accurate enough for our purposes. I ran a spreadsheet of tomorrow's forecast solar radiation, compared to actual solar generation and also, in the evening, downloaded what I assume was the close to the actual radiation. They were't usually that much different. You can look 5-6 days ahead and watch how the forecast refines itself. If you're prepared to invest about £50 to investigate it and get a lot of other useful weather forecasting info it would be worth your while exploring. My Europe wide hourly 7 day forecast package costs about twice that and I use it in the winter for general forecasting as well as for aviation in the summer.

It ought to be quite simple for Tesla to integrate that sort of data from a paid service into the algorithm, but although it's been mentioned before it seems to have been work in progress for a long time. I'd be prepared to pay a little extra to avoid running out of power during the peak rate period.

 

[mikemillar]

PS - The Weather Special package gives you a 3 day or 2 week subscription to a one region hourly forecast for s much reduced price if you want to try it out. Suggest the 3 day - by the time you've paid for 2 weeks it costs one third of the annual 2 hour package.

 

[sashton]

Thanks for the pointer @mikemillar . It looks like a very useful tool but appears to lack a critical component for my needs: - An API interface. My ultimate goal is an integrated system that manages the power, car charging, heating, DHW etc. without any manual intervention. Some key pieces of the jigsaw already work well. It reads our calendars to determine when and how much charge is needed in the cars. It reads the car's SoC while it is plugged in (ODB2 via Bluetooth). Charge management is achieved by the Myenergi API. Powerwalls are managed by a combination of the local and server APIs. Appliances (as much as they can be controlled) are managed by the Home Connect API. Minor devices, lighting, the alarms and the AGA are covered by a Z-wave controller. Heating - Solar collectors, mixing pumps for heating circuits, destratification functions in the stores, woodburner air control, backup oil boiler and over temperature controls are all directly controlled.
Above all the toys, the most critical component is an understanding wife!

 

[JohnRatsey]

Above all the toys, the most critical component is an understanding wife!

Too true! Cooking is probably the biggest load in my house and with substantial day-to-day variability which I can never predict accurately and I doubt if any software can do better. The only certainty is that the highest electricity demand will be on 25th December but how high remains to be seen.

 

[mikemillar]

It looks like a very useful tool but appears to lack a critical component for my needs: - An API interface. My ultimate goal is an integrated system that manages the power, car charging, heating, DHW etc. without any manual intervention.

Wow! What you are developing sounds really impressive! Are you actually able to control the Powerwall behaviour? I've just emailed Topmeteo to as about an API interface.

 

[sashton]

Yes, there are two methods for talking to a Powerwall. There is a local API with endpoints on the Gateway. Most of the calls result in status information: - the State of Charge, flows and cumulative totals etc. You can emulate the configuration settings there but I am wary of going down that route. Alternatively you can emulate the APP and talk to the Tesla servers. Authentication is more complicated and until recently there was a significant drawback, the changes were not applied back to your Powerwall immediately, sometimes it took up to 3/4 hour. This delay appears to have been removed recently (hooray!).
The local API is very simple and requires no authentication. While it is probably unofficial there are loads of people who have documented the endpoints. You need no real programming skills to use this as you can enter the URIs directly in your browser, save the response as a file. This is an example of graphing the State of Charge over time https://www.bowmill.net/xmenu/powersoc.shtml
If you want just a gauge-like display https://www.bowmill.net/rad3.htm
There are loads of other examples there. Some are broken, others just useless and many need attention but the whole site is just a play-pit.
A lot of the data comes from a Davis weather station which as soon as I fix one sensor another breaks. I am at the point of giving up with it.

 

[mikemillar]

I'm sure there are many pay-for services which will calculate your solar yield for the following day but I am still struggling up a steep learning path trying to get an approximate figure using the various free weather services that are out there.

@sashton - I got a very prompt reply from Stefan Goretzki. at the Topmeteo team, who said he'd have to discuss it with his team and wrote:

"We usually don't release our data API to "private" customers. This is very difficult to control, because the data is then really open for download.
Is there any form of collaboration that TopMeteo could benefit from here? That would make the decision easier.
In any case, your project sounds very interesting!"

I replied, telling him about how the Powerwall charges and the problem with solar and UK weather. I suspect they would want to charge for access, if it's possible to provide restricted user only access to api's. It might be worth your while contacting Stephan and explaining your project. You can mention my contact with him. The email address is: [email protected]

 

[mikemillar]

Looking back, post #27 seems like a red flag to me. As far as I can tell, the Gateway achieves my ~20W average by sometimes importing >20W, and sometimes exporting power to compensate. I.e. a load in the house turns on, the instant demand comes partly from the grid because the PW is ramping-up to deal with the new load, and then it lets some of the PW energy leak out onto the grid to make up for (most, but not all) of the surge (similar behavior in reverse when a load suddenly turns off). If you're meter is only recording the import and ignoring the export then it would not work with this approach as it wouldn't see the 'make up' periods. So if that's all true (that the meter is working one-way and the Gateway works the way my data suggests it does) then that might explain why things aren't lining up (and would, effectively, mean that your Smart Meter is 'cheating').

The way I first determined the 20W average of my system was by looking at all of its reported grid consumption from the Gateway's API over multiple evening periods where solar was off and the house was 100% powered from the PW (never hit the Reserve). But this wasn't just a constant 20W number, it was a bunch of numbers all over the place, some positive and some negative. When you averaged all of them the answer was 20W. But if you threw out all of the negative numbers the average obviously would have been quite a bit higher than 20W. So it seems to me that is what your meter is effectively doing, averaging the readings without considering any of the negative numbers. Since I also get readings from my meter, it showed the same sometimes-positive, sometimes-negative readings my Gateway was reporting, and the same overall average of ~20W.

If you are only being charged for an average of 20W that's just within the Tesla specification of max 500Wh per day. However, as you say, import and export appears to alternate and the peaks may well be much higher. I'm not sure that one can necessarily equate the api power figures with what the meter is reporting. I've only downloaded individual api data sets manually so I can't see the true cycle at present. I'm looking into writing something for Raspberry Pi or Arduino to log a constant stream of api data, but, not being an experienced programmer, that's going to take a while. At present I can't relate the Powerwall api instant_power figures with what the Smart Meter is telling me. I'm still trying to get to grips with real, apparent and reactive AC power and its not clear to me which of these should relate to the Smart Meter figures. I would have assumed that the api "site" instant_power figure would be what I'm actually consuming but that figure is usually less than 10W and occasionally negative so it doesn't tie in with the Smart Meter, even ignoring the negative values. From what I've read about how Smart Meters work I think they continuously sample the distorted sine wave power flow and average out the power demand. As I understand it, the old analogue meters only measured 'real' power, which, I think, is the instant_power figure that the api reports. However, smart meters are more complex and I think reactive power may come into the equation, which is probably why the meter is reporting a much higher figure.

I don't think the Smart Meter is 'cheating' by ignoring the export, not deliberately anyway. If it logged export it would record solar export as well as the small amounts of export/import that the Gateway is contributing and there would be no way of separating them out. I get paid a solar Feed-in Tariff based on an assumed 50% of my separate solar meter readings, which is very good with the Powerwall as in reality I hardly export any power. Nearly all domestic UK solar export agreements are based that way because until the recent appearance of smart meters it hasn't been possible to record export with the existing domestic meters. That may well change in the future, but I have a legal agreement to pay the FIT based on the assumed 50% export for 20 years. In that sense I'm winning. However, it niggles me that the Powerwall appears to be drawing more power than it should be, if my assumptions are correct.

 

[sashton]

@mikemillar if you're looking at a Pi I can heartily recommend playing with Node-Red. Admittedly I come from a programming background but a few months back I was persuaded to try Node-Red on a Pi and it is surprisingly easy to pick up. There is a Dashboard optional component with which you can very quickly knock together an interactive web page. If you do go down that route I can send you some samples which will give you a head start.
Thank you for contacting topmeteo. Currently I am in discussion with a lady at the MetOffice / Exeter Uni regarding insolation prediction so should stick with them for now.
If you do try a Pi and get stuck with Node-Red I regularly drive up to Sussex to catch up with my children and can always be persuaded to help out for a pint of Harvey's (one of the few reasons I miss Sussex).

 

[mikemillar]

Thanks very much @sashton, I'll take a look at Node-Red. My programming experience was limited to Basic at university at a time when PC's were only just becoming available, and, more recently, Python for the Pi. I can make intelligent changes to existing code but coding from scratch is a bit challenging, and I'm trying to improve my french as well! If I get anywhere with it I'll keep your offer in mind, many thanks for that.

 

[mikemillar]

There is a local API with endpoints on the Gateway. Most of the calls result in status information: - the State of Charge, flows and cumulative totals etc. You can emulate the configuration settings there but I am wary of going down that route. Alternatively you can emulate the APP and talk to the Tesla servers.

Could that be a way of forcing it to charge to 100% when a rainy cloudy day is forecast after lots of sun? What we need is an emergency boost button - for a one off charge, like the 1 hour over-ride you can fit to immersion heaters.

 

[mikemillar]

It wasn't clear to me looking back on the thread whether it was established that you're using a v1 Gateway (simple metal box) or the v2 (mini Powerwall enclosure), and if v2 I've always wondered just what it looks like on the inside

Here's your answer @woferry. Taken during installation, but if you're handy with a Torx screwdriver, there's a single screw near top centre that holds the black cover on, no seals....

The busbar at the bottom has the 100amp fuse (left) and 2 CB's that I can see and access with the back cover on (removed in this photo). Left CB is to solar inverter from the backup side and the right CB is for the car charger, wired on the non-backup side to avoid the Model S emptying the Powerwall during a power outage. The orange cover in the middle contains the main contractor I think and has connections for mains in (bottom left) from the 100amp fuse; non-backup side output above; and backup side power out to the home (bottom right) that's the white label strip bottom right. The unused terminals are for 3 phase installations. The electronic control seems to be at the top and has a connection from my solar inverter (signalling?) via the black plastic concertina conduit entering bottom left with the grey wire from it feeding up the right hand side to the top connector at top right with blue/white wires for something else. There is a solar connection from thre home backed up positive output via the CB to the Powerwall - to run the inverter during power outage? But these's still a connection (main solar output?) from the solar inverter direct to the consumer unit. That's the white cable to the left of the powerful, going to the solar meter above the white square conduit with a couple of CT clamps that supply data to a separate home and solar monitor.

 

[mikemillar]

ERRATUM!! Sorry folks , misidentified! The left hand of the 2 CB's is the Powerwall CB, NOT the Solar CB, that's in the consumer unit. So the black crinkly concertina conduit bottom left comes from the Powerwall itself and carries the supply cable from the battery to the backup side of the orange central busbar and stuff. The other cable from the Powerwall which goes to the top right must be the signal and control cable from the Gateway to the Powerwall. There are only two cables from the Solar inverter, a white power supply cable left of the Gateway, to the consumer unit, and an ethernet cable to my powerline wifi adapter, neither of which are connected to the Gateway. There is also an ethernet cable from the Gateway to my powerline wifi and a mysterious small capacity twin cable covered in black woven fabric which goes to the consumer unit, maybe a supply to the Gateway control circuit.

 

[miimura]

There is also an ethernet cable from the Gateway to my powerline wifi and a mysterious small capacity twin cable covered in black woven fabric which goes to the consumer unit, maybe a supply to the Gateway control circuit.

The small wire that goes to the consumer unit is probably from the CT's measuring your total grid flow.