Adjust Tesla charge rate from mobile app, API & PW2 to allow use of excess solar

[EcoCloudIT]

It's not plug and play you have to do some configuration of TWCmanager
This fork mentions support for Solar Edge API and Powerwall 2

ngardiner/TWCManager

Sure, thanks...however I cannot find anywhere notes on what it is I am "meant" to do in order to connnect to my PW2 etc.

Hopefully the link you've proivded sheds some light on it....otherwise it's all good but the main reason for using the TWC Manager is undocumented as such....

Question 1:
Since I have a PW2 would I really only need to connect the EMS to the PW2 and not bother attempting to connect to the SolarEdge? As it is the Tesla GW connects everything "together" so I would assume it would be the definitive source of information for the TWC Manager/EMS....no?

Question 2:
We have 2x Tesla cars, however, both are on their own account. Wonder does anyone know if the TWC Manager is capable of saving 2x Tesla API's tokens in order to issue a "stop" charging command when required?

 

[Myfirstone]

Happy new year!

are there more people that have strange issues with TWCManger an the latest software update for MCU1 AP1 that the car charge settings overruled the TWCManagers settings?

I have update the raspberry pi software and I have the 2020.48.12.1 mcu update.

 

[Hairyman]

The Zappi chargers will manage some of this for those less familiar with adding electronics and code.

I’ve attached a screenshot from a Victorian user (not me) to show how it manages the system.

 

[EVOutback]

Sure, thanks...however I cannot find anywhere notes on what it is I am "meant" to do in order to connnect to my PW2 etc.

Hopefully the link you've proivded sheds some light on it....otherwise it's all good but the main reason for using the TWC Manager is undocumented as such....

Question 1:
Since I have a PW2 would I really only need to connect the EMS to the PW2 and not bother attempting to connect to the SolarEdge? As it is the Tesla GW connects everything "together" so I would assume it would be the definitive source of information for the TWC Manager/EMS....no?

Question 2:
We have 2x Tesla cars, however, both are on their own account. Wonder does anyone know if the TWC Manager is capable of saving 2x Tesla API's tokens in order to issue a "stop" charging command when required?

Q1.. TPW2 setting are there ~/TWCManager/etc/twcmanager $ sudo nano config.json
Line 485 ish...
Q2, can't test as both my cars are on same account however a suggestion is to share 1 Car via www.tesla.com > login .go to your Car Access (Give access to up to five additional drivers) then add driver and add one car to perhaps a common (sudo) shared account then go to the other and do the same..or just share to one account if you trust each other?? (
Then put then shared account into config.json (above)....tell us how you go and test for loopholes (no warranty remember!)

 

[Crustyman]

Hey if anyone is looking for the kit to use TWC Manager hit me up as I bought the Raspberry pi, adapters and usb-serial converter etc with the intent of doing this but then decided to replace our other ICE with an MG so had to swap the charger over to a Zappi

 

[Hairyman]

Hey if anyone is looking for the kit to use TWC Manager hit me up as I bought the Raspberry pi, adapters and usb-serial converter etc with the intent of doing this but then decided to replace our other ICE with an MG so had to swap the charger over to a Zappi

How are you finding the Zappi?

 

[lonertic]

only took like 10 years... but looks like it's finally here.. just waiting for 2021.36 to drop OTA.

ps. not my pic. repost off FB post. I dont think the app is on Android yet, not that I care, I just want the API.

 

[Vostok]

only took like 10 years... but looks like it's finally here.. just waiting for 2021.36 to drop OTA.

I dont think the app is on Android yet, not that I care, I just want the API.

Good luck with the API… because the endpoints file doesn’t tell you everything. It tells you what endpoints exist, but not what data field names or values to pass to it.

I’ve only just started to delve into the API because I want to automatically pull down historical PW2 data including the historical battery charge % values, which are now shown in the new v4 of the Tesla App. Also, the “download my data” button in the App returns a CSV file with only 100W resolution (completely inadequate), whereas the unfiltered API returns the same data to fractions of a Watt (heaven!).

The historical battery % values must be extracted through one of the existing API endpoints (likely ‘calendar_history’) because there is no new command in the 4.0.2 endpoints file that looks like it returns historical battery percentage. But what ‘kind’ parameter is passed to the ‘calendar_history’ endpoint to return those values? No idea. And no-one seems to know.

But your concept is sound. If you did know what parameters to send, it would not be that hard to have a Python script running that, say, every 5 minutes, extracts the excess solar data, current charge state of the car, and tells your car to start charging at whatever rate you want for whatever time period or set of criteria that you want.

 

[Vostok]

I dont think the app is on Android yet, not that I care, I just want the API.

And here's the new endpoints in the 4.1.0 endpoints file... 'set_charging_amps' would be what does it, but no further info on the params to pass to it.

  "SCHEDULED_DEPARTURE": {
    "TYPE": "POST",
    "URI": "api/1/vehicles/{vehicle_id}/command/set_scheduled_departure",
    "AUTH": true
  },
  "SCHEDULED_CHARGING": {
    "TYPE": "POST",
    "URI": "api/1/vehicles/{vehicle_id}/command/set_scheduled_charging",
    "AUTH": true
  },
  "CHARGING_AMPS": {
    "TYPE": "POST",
    "URI": "api/1/vehicles/{vehicle_id}/command/set_charging_amps",
    "AUTH": true

 

[lonertic]

And here's the new endpoints in the 4.1.0 endpoints file...

Thanks heaps! That is a lot closer than I got, I'm an android user and the Android version isnt even up yet. I'd imagine the parameter to be something straightforward like ?amps=xx or ?charging_amps=xx etc... would only take a minute or two to guess it now that we have the exact endpoint, assuming someone doesn't get a data capture of it first. We'll need 2021.36+ on the cars too before we can test it and nobody's got that yet, I think 2021.32.22 is the latest GA as of right now?.. But this gives me a head start so i can start coding and jump straight into QA once .36 drops.

This is gonna be so exciting and just in time too! We're getting into the peak solar seasons and some of us can end up with a negative FiT during the day, such as yesterday, FiT went down to -7c/kWh, with this API i can now carefully calibrate the exact amount of excess in my PV during these times and send it to the car instead, at the same time, without any imports from the grid!

 

[Vostok]

Thanks heaps! That is a lot closer than I got, I'm an android user and the Android version isnt even up yet. I'd imagine the parameter to be something straightforward like ?amps=xx or ?charging_amps=xx etc... would only take a minute or two to guess it now that we have the exact endpoint, assuming someone doesn't get a data capture of it first.

I’d be impressed if you guess the right parameter name in a few minutes. I’ve had no luck “guessing” the parameter name to retrieve historical battery percent charged values from the `calendar_history` endpoint. The known `kind`s for this endpoint are:

/api/1/energy_sites/{site_id}/calendar_history?kind=power
/api/1/energy_sites/{site_id}/calendar_history?kind=energy
/api/1/energy_sites/{site_id}/calendar_history?kind=self_consumption
/api/1/energy_sites/{site_id}/calendar_history?kind=time_of_use_energy
/api/1/energy_sites/{site_id}/calendar_history?kind=savings

I‘ve tried kind=battery_percent, kind=battery_charge, kind=percentage_charged etc none work so far. You quickly realise that there so many possible names they could have called this that it’s throwing darts in the dark to guess the right parameter name. The odds are close to zero!

assuming someone doesn't get a data capture of it first.

Yeah I’ve tried that too. The Tesla App uses SSL Certificate Pinning to verify it is talking to owner-api.teslamotors.com and not some intermediate server pretending to be owner-api.teslamotors.com, so it’s not possible as far as I know to sniff the commands it sends. Otherwise this would be trivial to solve.

We'll need 2021.36+ on the cars too before we can test it and nobody's got that yet, I think 2021.32.22 is the latest GA as of right now?.. But this gives me a head start so i can start coding and jump straight into QA once .36 drops.

Having vehicle-PW2 integration has been on Tesla’s ‘to do’ list for a couple of years. It’s within the realms of possibility that 2021.36 will pave the way for Tesla to build charging from excess solar into their App (saving you the need to roll your own ). There’s no need for v3 HPWCs either, Tesla talks directly to the PW2 and vehicle to work out what to do.

We're getting into the peak solar seasons and some of us can end up with a negative FiT during the day, such as yesterday, FiT went down to -7c/kWh, with this API i can now carefully calibrate the exact amount of excess in my PV during these times and send it to the car instead, at the same time, without any imports from the grid!

You’re in Sydney, who’s giving you negative FITs? You still potentially have an issue with this if your PW2 is full and your car is full and your house is using what it can. You’d need an additional script to turn your inverter off in that situation, and turn it back on again when you can use the power.

 

[atj777]

You’re in Sydney, who’s giving you negative FITs? You still potentially have an issue with this if your PW2 is full and your car is full and your house is using what it can. You’d need an additional script to turn your inverter off in that situation, and turn it back on again when you can use the power.

That's the exact 'dilemma' I have right now. While I don't have negative FIT, it is only $0.05/kWh and the Powerwalls and car are full.

That's the problem with the stay at home orders and barely driving. I'll be out and about from next Monday, but it looks like it is going to be overcast and rainy all next week so I'll probably no longer have spare solar.

 

[GrimRe]

Does anyone know if TWCManager can retrieve the current state of charge of attached vehicles? If so, can you stop charging without sending the api call? (I.e just stop charging from the charger itself)

The reason is that my car spot has zero cellular connection so I can’t wake up the car. There is wifi however so would be amazing to control charging start/stop by only controlling the charger.

 

[lonertic]

@Vostok I'm with Amber which just passes wholesale prices directly... we've had negative FiT for a few days this week now, but typically around noon... those on a different retailer probably wont see negative FiT's for a while (that's because they make it up their margin off you elsewhere). It's not always bad news having market rates, back in July there were a few price peaks where Solar FiT spiked to over $20/kWh (yes, in dollars! not cents!). I automated my system to export from batteries whenever those spikes were detected so, I'm almost $1k ahead on my energy bill this year.

Also I'm not using PW2 if it weren't already obvious... I have a custom 30kWh lithium system based on 3-phase commercial hybrid inverters. PW2 doesn't meet my requirements unfortunately.

As for current-matching the car's charging to PV excess, my old way of doing this is to set charging current > PV output, so then I'm guarenteed not to export the excess, and any shortfalls comes from my home batteries. But now that charging current can be API'ed, I'd rather throttle the car's charging in real-time, allowing the batteries to have a rest during the day so it'll have a few hours to balance the cells, it's just better for its health. And eventually when the car's also fully charged, I'll just issue command for inverter to disconnect from grid, so excess PV simply isn't collected, and i'll have a fully charged car/home battery in the garage and PV will go towards domestic load.

So yeh this is gonna require a few API's.. Amber's realtime pricing API, inverter and battery BMS API's (which I wrote myself), and Tesla's ownerapi. I probably won't try and "guess" the api params then, but it'll only be a matter of time before those are discovered. Maybe when the Android app is released? It's a lot easier to decompile an APK than an IPA, someone will just read it straight from source code soon enough.

 

[cafz]

@Vostok I'm with Amber which just passes wholesale prices directly... we've had negative FiT for a few days this week now, but typically around noon...

Although the negative FIT is shown in the app, Amber doesn't actually pass it on when they bill you so effectively there's a floor of zero at the moment.

 

[whiten0ise]

I’d be impressed if you guess the right parameter name in a few minutes. I’ve had no luck “guessing” the parameter name to retrieve historical battery percent charged values from the `calendar_history` endpoint. The known `kind`s for this endpoint are:

/api/1/energy_sites/{site_id}/calendar_history?kind=power
/api/1/energy_sites/{site_id}/calendar_history?kind=energy
/api/1/energy_sites/{site_id}/calendar_history?kind=self_consumption
/api/1/energy_sites/{site_id}/calendar_history?kind=time_of_use_energy
/api/1/energy_sites/{site_id}/calendar_history?kind=savings

I‘ve tried kind=battery_percent, kind=battery_charge, kind=percentage_charged etc none work so far. You quickly realise that there so many possible names they could have called this that it’s throwing darts in the dark to guess the right parameter name. The odds are close to zero!


Yeah I’ve tried that too. The Tesla App uses SSL Certificate Pinning to verify it is talking to owner-api.teslamotors.com and not some intermediate server pretending to be owner-api.teslamotors.com, so it’s not possible as far as I know to sniff the commands it sends. Otherwise this would be trivial to solve.


Having vehicle-PW2 integration has been on Tesla’s ‘to do’ list for a couple of years. It’s within the realms of possibility that 2021.36 will pave the way for Tesla to build charging from excess solar into their App (saving you the need to roll your own ). There’s no need for v3 HPWCs either, Tesla talks directly to the PW2 and vehicle to work out what to do.


You’re in Sydney, who’s giving you negative FITs? You still potentially have an issue with this if your PW2 is full and your car is full and your house is using what it can. You’d need an additional script to turn your inverter off in that situation, and turn it back on again when you can use the power.

Are the CA baked into the app? otherwise you can use a MITM http proxy with a resigning intermediate CA to capture all the http calls to the tesla api

guide below:

Certificates

docs.mitmproxy.org

 

[Vostok]

Are the CA baked into the app? otherwise you can use a MITM http proxy with a resigning intermediate CA to capture all the http calls to the tesla api

I’m pretty sure the CA is baked into the App and it uses “Certificate Pinning” (which is described in your link).

Putting a HTTPS proxy inbetween the App and owner-api.teslamotors.com doesn’t work because I tried it - the App repeatedly tries to authenticate and fails, and no data is exchanged. You are not asked ”do you want to trust this server?” as you are in a browser, for example, so you can’t force the App to accept the SSL certificate of the proxy.

 

[whiten0ise]

I’m pretty sure the CA is baked into the App and it uses “Certificate Pinning” (which is described in your link).

Putting a HTTPS proxy inbetween the App and owner-api.teslamotors.com doesn’t work because I tried it - the App repeatedly tries to authenticate and fails, and no data is exchanged. You are not asked ”do you want to trust this server?” as you are in a browser, for example, so you can’t force the App to accept the SSL certificate of the proxy.

ah ok I've had success with other apps that uses the Android/IOS trust store rather than its own so I'd add the Intermediary CA to the OS trust store. shame it doesnt work that way.. i guess there is always the option of decompiling the APK... but that is usually alot of work

Edit: just did a quick decompile the parameter looks to be named "chargingAmps". I dont have the update yet so can't check

2nd edit:
this might be of interest for you to try Vostok:

 def get_calendar_history_data(
            self, kind='savings', period='day', start_date=None,
            end_date=time.strftime('%Y-%m-%dT%H:%M:%S.000Z'),
            installation_timezone=None, timezone=None, tariff=None):
        """ Retrieve live status of product
        kind: A telemetry type of 'backup', 'energy', 'power',
              'self_consumption', 'time_of_use_energy',
              'time_of_use_self_consumption' and 'savings'
        period: 'day', 'month', 'year', or 'lifetime'
        end_date: The final day in the data requested in the json format
                  '2021-02-28T07:59:59.999Z'
        time_zone: Timezone in the json timezone format. eg. Europe/Brussels
        start_date: The state date in the data requested in the json format
                    '2021-02-27T07:59:59.999Z'
        installation_timezone: Timezone of installation location for 'savings'
        tariff: Unclear format use in 'savings' only
        """
        return self.api('CALENDAR_HISTORY_DATA', kind=kind, period=period,
                        start_date=start_date, end_date=end_date,
                        timezone=timezone,
                        installation_timezone=installation_timezone,
                        tariff=tariff)['response']

 

[Vostok]

ah ok I've had success with other apps that uses the Android/IOS trust store rather than its own so I'd add the Intermediary CA to the OS trust store. shame it doesnt work that way.. i guess there is always the option of decompiling the APK... but that is usually alot of work

There are ways in iOS to get around this - but it first involves ‘jailbreaking’ the device and then directly patching the relevant App to not do the certificate checking or accept an alternative certificate. It’s a complete hack that makes the entire device insecure and a path I ain‘t going down.

Edit: just did a quick decompile the parameter looks to be named "chargingAmps". I dont have the update yet so can't check

2nd edit:
this might be of interest for you to try Vostok:

Thanks for that @whiten0ise - is that from a decompile? It doesn’t show any new ‘kinds’ that I don’t already know about. So it’s either from a v3 decompile (not v4) - or the battery charge state history doesn’t come from `calendar_history` but some other endpoint. Which then makes it even more mysterious, since it’s not obvious at all what other endpoint that data could come from (and I’ve done a complete search of the 4.1.0 JSON endpoints file).

If you can get a full v4 decompile then I could trawl it… PM me. Thanks.

 

[whiten0ise]

There are ways in iOS to get around this - but it first involves ‘jailbreaking’ the device and then directly patching the relevant App to not do the certificate checking or accept an alternative certificate. It’s a complete hack that makes the entire device insecure and a path I ain‘t going down.

ok on android you dont need to jailbreak you can upload ca certs to the trust store...

Thanks for that @whiten0ise - is that from a decompile? It doesn’t show any new ‘kinds’ that I don’t already know about. So it’s either from a v3 decompile (not v4) - or the battery charge state history doesn’t come from `calendar_history` but some other endpoint. Which then makes it even more mysterious, since it’s not obvious at all what other endpoint that data could come from (and I’ve done a complete search of the 4.1.0 JSON endpoints file).

ah sorry that is the older version i didn't get that from my decompile I just searched github to see how others were using the endpoint.


its quite a massive file(over 0.5gb) and they produce smali files which are enough for searching strings etc.[/QUOTE]