TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker or making a Paypal contribution here: paypal.me/SupportTMC

Model S telemetry best practices

Discussion in 'Model S: User Interface' started by brianman, Apr 14, 2015.

  1. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    #1 brianman, Apr 14, 2015
    Last edited: Apr 16, 2015
    I'm aware of the following:
    1. We have a gigantic thread about the REST API.
    2. We have a Wiki that has links to some implementations.
    3. The telemetry API changed/extended at some point and I never updated my logger.
    4. I have a logger that I've been running for approximately two years with very little changes that mostly works, but sometimes "upsets" the server (such that it starts rejecting my logins for 24hr or so).
    5. Running my logger in parallel for two different cars triggers the upset of #4 almost consistently (meaning it's more rejected logins than accepted ones).
    6. I'd like to revisit my logger (and perhaps update it for the API changes).
    7. I'm hopeful the community might have some best practices to recommend.

    Goals:
    • 24x7 logging for multiple vehicles as allowed by general internet stability, uptime for my Azure VM, uptime for Tesla's servers, and 3G/wifi reliability for the vehicles
    • Follow expressed or implied "rules" Tesla has for its telemetry API, as a "well behaved" client app.
    • Obtain "snapshot" command data (see step 8 below) at least a few times an hour.
    • Obtain streaming data continuously (when possible) while the vehicle is not parked.
    • (Bonus) Obtain streaming data continuously (when possible) while supercharging.

    Here's what I'm currently doing for each instance of my logger:
    1. fetch the login info and vehicle index from command-line args
    2. start a new log file; post via https to portal.vn.teslamotors.com/login; look for user_credentials in the cookies
    3. if step 2 fails, wait 15 seconds and return to step 2
    4. get via https from portal.vn.teslamotors.com/vehicles; fetch the id, vehicle_id, and tokens for the index-th vehicle
    5. if id is missing from step 4, wait 15 seconds and return to step 2
    6. get via https from portal.vn.teslamotors.com/vehicles/<id>/mobile_enabled
    7. if step 6 fails, wait 15 seconds and return to step 2
    8. invoke 5 commands using get via https from portal.vn.teslamotors.com/vehicles/<id>/command/<command> for each of the following: charge_state, climate_state, drive_state, gui_settings, and vehicle_state
    9. if step 4 found a token, then get via https from streaming.vn.teslamotors.com/stream/<token> in a while loop until it fails
    10. if step 4 found a vehicle_id, then get from https from portal.vn.teslamotors.com/vehicles/<id>/command/wake_up [Sidenote: not sure why my code cares about _vehicleId here rather than _id, that seems like a bug -- as it is using id not vehicle_id for the URL.]
    11. if step 10 fails, wait 15 seconds
    12. go to step 2


    As indicated earlier, I'm trying to run multiple of the above and I think combined they are making the server unhappy by making paired requests at step 2 (one for each running logger) rather than sharing the same login attempt. As I restructure for that presumed issue, I want to see if others have some suggestions about additional tuning I might want to do.

    Thanks in advance for assistance.
     
  2. mochouinard

    mochouinard Member

    Joined:
    Aug 3, 2014
    Messages:
    339
    Location:
    Montréal, Quebec, Canada
    Hi,

    I've made a hosted data logger that about 13 Tesla Owner use so far. You can see a demo of the UI and function here :
    Tesla Logger - YouTube

    Let me know if you would like to give it a try.

    Moc
     
  3. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    Thanks but that doesn't help at all with the issues in the OP. :(
     
  4. mochouinard

    mochouinard Member

    Joined:
    Aug 3, 2014
    Messages:
    339
    Location:
    Montréal, Quebec, Canada
    I just wanted to offer you a solution that already work to save you time.

    New or old API, principle is the same. If your 2 car are under the same account, a single authentication should be done for both of them. If not, then you should authenticate once for both of them.

    Tesla API is actually pretty stable. There have been a few down time but not often. The biggest problem I've seen is poor wifi reception when at home (in garage or driveway), and when your switching country (USA/Canada border take a hell of a long time for the 3g service to become functional again. Not great especially that you lose your maps download...

    You can stay connected to the Streaming API most of the time but you will have to handle timeout when the car is idle especially.

    Make sure you do some caching operation so you don't hit Tesla server all the time. For example, unless you query really often vehicle_state, most of it information is already out of date by the time you actually look at it. So some API query less often than other as it data might be used for different purposes.
     
  5. rdrcrmatt

    rdrcrmatt Member

    Joined:
    Jun 27, 2013
    Messages:
    551
    Location:
    Milwaukee
    Moc, would you be willing to share the source so I can host it myself for my use?
     
  6. jepicken

    jepicken VIN: 80322

    Joined:
    Feb 21, 2015
    Messages:
    26
    Location:
    Chandler, AZ
    I'm interested. Is it open source?
     
  7. mochouinard

    mochouinard Member

    Joined:
    Aug 3, 2014
    Messages:
    339
    Location:
    Montréal, Quebec, Canada
    I started with the idea to have it open source, but I had a single user running it on his end it and I was already tired of dealing with issue with his environment :) It wasn't that bad, but it made everything more complicated and I didn't want to deal with everyone different level of knowledge. So make that lot of different installation and I don't want to deal with that at this stage.

    It not out of the question in the long term, but right now I don't I can't support different people installation and putting the extra work dealing with upgrades.
     
  8. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    Teach thyself...
    I hadn't looked at this part of my logger in a while. It turns out I had completely forgotten that the login timeout is like 3 months whereas the token timeout for streaming is on a much shorter leash (15/30/45min). Whenever the streaming token was dying, I'd repeat the login (as well as the token fetch). Whenever car was out of 3G range, logins would start spinning every 15sec or so.

    This is likely most of the problems I was running into. So the two big changes I'm trying for the server 429 error are:
    1. cache the login cookies to a file
    2. when launching the logger, use the file-based login cookies if they haven't expired (acc to the timestamp in the cookie set from 1)

    On another machine (not the Azure VM), that seems to dramatically improve things. On my VM it's still in "sit in the corner with the dunce cap on" mode, but should get out of that mode in ~30min if it was a 24hr penalty box.
     
  9. spc

    spc Member

    Joined:
    Jan 28, 2015
    Messages:
    57
    Location:
    Edinburg, TX
    Moc, can i get more info on it please. I should get my car in a couple of weeks and would love to be able to log the telemetry. I have done some code in the past. Would be glad to help if I can.
     
  10. mochouinard

    mochouinard Member

    Joined:
    Aug 3, 2014
    Messages:
    339
    Location:
    Montréal, Quebec, Canada
    I'm using the Google Cloud service with their cheapest machine. It really doesn't take much to run it even when your monitoring 14 car driving around. I never tried the azure cloud.

    I noticed with the new API, that the key seem to change less often for the car, and I never had to do a wakeup on the car with it. The key value are always present.

    If you are to play more with your code, I suggest you switch over as soon as you can, especially if your not caching the cookie already so you can implement the new authentication system (oauth 2.0).

    You need to make sure you don't send command too often or else you will get the booth. I setup a throttling system early on, but now I just have a 2 second delay between query allowing me to do about 30 query per minute which is fine for my need right now. If your monitoring a single car(or 2), I would make it 5 second minimum between query.

    Everyone else asking about the tracker, just private MSG me and I'll send you the link to register... Let keep this thread about brian effort. Thanks
     
  11. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    What's the best reference (link) for the new API (and/or the delta to the new API)? Thanks.
     
  12. rolosrevenge

    rolosrevenge Dr. EVS

    Joined:
    Feb 7, 2009
    Messages:
    1,851
    Location:
    Right behind you...
    Your goals aren't big enough. Add to it the ability to automatically dispatch charging of the Teslas, and then go corner the market for V2G services.
     
  13. mochouinard

    mochouinard Member

    Joined:
    Aug 3, 2014
    Messages:
    339
    Location:
    Montréal, Quebec, Canada
    I've used the info on this forum. Here is the post : Model S REST API - Page 104

    Here is sample command to get your started :
    Authentication : curl -H "Content-Type: application/json" -d '{"grant_type" : "password", "client_id" : "CLIENT_ID_YOU_MUST_FIND", "client_secret" : "SECRET_CLIENT_ID_YOU_MUST_ALSO_FIND", "email" : "YOUR TESLA EMAIL", "password" : "YOUR PASSWORD"}' https://owner-api.teslamotors.com/oauth/token

    curl -H "Authorization: Bearer YOURBERRER_CODE_FROM_THE_FIRST_COMMAND" \
    -H "Content-Type: application/json" \
    -i https://owner-api.teslamotors.com/api/1/vehicles
    This will return the list of all the cars including it ID

    curl -H "Authorization: Bearer YOURBERRER_CODE_FROM_THE_FIRST_COMMAND" \
    -H "Content-Type: application/json" \
    -i https://owner-api.teslamotors.com/api/1/vehicles/YOUR_CAR_ID/data_request/vehicle_state
     
  14. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    So far so good with the login change.

    Two car querying x5 works. Second car is streaming. First car isn't for some reason.
     

Share This Page