TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker or making a Paypal contribution here: paypal.me/SupportTMC

TeslaMS tools for telemetry data visualization

Discussion in 'Model S: User Interface' started by dirkhh, Aug 12, 2013.

  1. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    Here are a couple of examples of the latest you can do with visualize.js application that's part of the teslams project (it's in the examples folder of the teslams project on github). Not quite as fancy (needs some love from someone better with UIs than I am), but fairly functional and easy to run on your own system.

    All you need is Node.js, MongoDB and a web browser. You can then stream your telemetry data into a database and connect to that database with two small web applications that do some analysis of the data. One view tracks your car on a map, visualizing speed through color, the other view tracks energy used / regeneratted over speed + SOC. They are linked to each other (so they cover the same time range) and the time range can be entered on the web pages.
    MapView.png
    Energy.png
     
  2. hans

    hans P631

    Joined:
    Sep 27, 2012
    Messages:
    1,123
    Location:
    Menlo Park
    When polling or retrying requests in JavaScript I use setTimeout() to introduce a small delay (see Window setTimeout() Method)
    That way if your code enters a tight loop, it won't fire off thousands of requests a second due it the non-blocking nature of JS in Node.js
     
  3. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    Is that in the latest version of teslams?
    I'd love to see this as part of the core infrastructure...
     
  4. hans

    hans P631

    Joined:
    Sep 27, 2012
    Messages:
    1,123
    Location:
    Menlo Park
    It's used in the streaming.js because that app has to poll often. I will look into how it might make sense to add some throttling into the main teslams.js library.
     
  5. jpasqua

    jpasqua P19325

    Joined:
    Feb 19, 2013
    Messages:
    696
    My use case is a little different. For one thing, I need to issue several requests in succession to display a single tab of the application. As a result, I took a slightly different approach.

    I established a max_request_rate and before issuing any request, I ensure that I am below that rate. Right now I have it set to 20 requests / minute.

    I keep a count of total_requests and the start_time when the library was initialized.

    if (total_requests / (now()-start_time)) > max_request_rate then I wait until enough time elapses to get the rate down to acceptable levels.

    It seems to be working OK and should save me from future bouts of buggy code.
     
  6. dsmith2189

    dsmith2189 Active Member

    Joined:
    Aug 18, 2012
    Messages:
    1,303
    Location:
    Honolulu, HI
    I don't see the answer to my question so I hope someone can answer it.
    if an update gets pushed to me:
    is there any way I can tell without physically being in the car?
    Is there any way to start an update that has been pushed to me remotely?
     
  7. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    I'm currently aiming at no more than 10 requests a minute... Most of the time it will be fewer than 4 (just at the start I allow more)
     
  8. jpasqua

    jpasqua P19325

    Joined:
    Feb 19, 2013
    Messages:
    696
    I need to monitor the network while the iPhone app and see what rate it produces in normal usage. I don't want to be any worse than that.

    I don't suppose anyone has done that already...
     
  9. hans

    hans P631

    Joined:
    Sep 27, 2012
    Messages:
    1,123
    Location:
    Menlo Park
    No, you can't tell that an update has been pushed nor can you start the update.

    However, once the update had been started (including via delayed start) you can monitor the upgrade process, and see the new firmware version once complete, all remotely.
     
  10. FlasherZ

    FlasherZ Sig Model S + Sig Model X + Model 3 Resv

    Joined:
    Jun 21, 2012
    Messages:
    7,019
    I was seeing ~7 requests per minute when I watched the updates to the app (8 sec between updates).
     
  11. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    Thanks for that data point. I will make sure not to exceed that...
     
  12. moviemaker2000

    Joined:
    Jan 18, 2013
    Messages:
    45
    Location:
    midwest
    OK,
    I need a bit of help here. I have gotten all the tools on the GitHub site to work except for the streaming+MongoDB+visualize.js combination. Streaming works fine when output to the text file, but I am missing something when I try to utilize the -db option.

    I have installed MongoDB per the website and fire it up after creating a c:\data\db folder for the data. I then set streaming to work with the -db option and things appear to be connecting on both the output screen and the Mongo server.

    After collecting some data I close out of streaming and fire up visualize.js. Pointing a web browser to the localhost just gives me a timeout. Pointing it to http://localhost:8766/energy?from=2013-08-18-21-08&to=2013-08-18-21-20 will get me a blank "Tesla Energy Chart" with two blank boxes for start and end time.

    On the server side it just keeps repeating:
    /energy?from=2013-08-18-21-08&to=2013-08-18-21-20
    /jquery-1.9.1.js
    /jquery-ui-1.10.3.custom.min.js
    /jquery-ui-timepicker-addon.js
    /jquery-ui-timepicker-addon.css
    /jquery-ui.css
    /url.min.js


    I feel like I am close, but just can't get it to return anything.

    Does anyone have a "step-by-step" guide of what needs entered from the time you fire up Mongodb through getting data out of visualize?

    Thanks in advance for all the incredible work here. It is truly impressive.
     
  13. hans

    hans P631

    Joined:
    Sep 27, 2012
    Messages:
    1,123
    Location:
    Menlo Park
    #13 hans, Aug 18, 2013
    Last edited: Aug 18, 2013
    Did you install by downloading the code from github or by using the "npm" commands?

    If you installed using npm then try again and you should get the latest version 0.6.5, which I just updated to includes all of dirkhh's latest visualize.js features including from/to energy charting.
     
  14. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    Ummm. I wasn't ready to push the latest of that just yet. It has a few rough edges...

    Or are you talking about the last code that I sent you a pull request for?
     
  15. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    Are you using "-db" or "--db" (two dashes)?

    If you just start mongo and issue the following commands
    Code:
    use tesla
    db.tesla_stream.count()
    Are there data in the data base?

    What's the console log in the browser showing?
     
  16. moviemaker2000

    Joined:
    Jan 18, 2013
    Messages:
    45
    Location:
    midwest
    I have been using the -db option.

    When I try db.tesla_stream.count() the count comes back at 0

    Below is what I am seeing on the console log.

    C:\mongodb\bin>mongod
    mongod --help for help and startup options
    Mon Aug 19 06:57:56.746 [initandlisten] MongoDB starting : pid=9636 port=27017 d
    bpath=\data\db\ 64-bit host=xxxxxx
    Mon Aug 19 06:57:56.747 [initandlisten] db version v2.4.6-rc1
    Mon Aug 19 06:57:56.747 [initandlisten] git version: 5dfa2980fcb1783c8dd005a4316
    44d1e3d676d76
    Mon Aug 19 06:57:56.748 [initandlisten] build info: windows sys.getwindowsversio
    n(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST
    _LIB_VERSION=1_49
    Mon Aug 19 06:57:56.749 [initandlisten] allocator: system
    Mon Aug 19 06:57:56.750 [initandlisten] options: {}
    Mon Aug 19 06:57:56.769 [initandlisten] journal dir=\data\db\journal
    Mon Aug 19 06:57:56.769 [initandlisten] recover : no journal files present, no r
    ecovery needed
    Mon Aug 19 06:57:56.813 [FileAllocator] allocating new datafile \data\db\local.n
    s, filling with zeroes...
    Mon Aug 19 06:57:56.814 [FileAllocator] creating directory \data\db\_tmp
    Mon Aug 19 06:57:56.848 [FileAllocator] done allocating datafile \data\db\local.
    ns, size: 16MB, took 0.033 secs
    Mon Aug 19 06:57:56.849 [FileAllocator] allocating new datafile \data\db\local.0
    , filling with zeroes...
    Mon Aug 19 06:57:56.981 [FileAllocator] done allocating datafile \data\db\local.
    0, size: 64MB, took 0.131 secs
    Mon Aug 19 06:57:56.983 [initandlisten] command local.$cmd command: { create: "s
    tartup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 reslen:37
    170ms
    Mon Aug 19 06:57:56.985 [initandlisten] waiting for connections on port 27017
    Mon Aug 19 06:57:56.985 [websvr] admin web console waiting for connections on po
    rt 28017
    Mon Aug 19 06:58:26.652 [initandlisten] connection accepted from 127.0.0.1:57325
    #1 (1 connection now open)
    Mon Aug 19 06:58:26.855 [conn1] end connection 127.0.0.1:57325 (0 connections no
    w open)
    Mon Aug 19 06:58:26.857 [initandlisten] connection accepted from 127.0.0.1:57327
    #2 (1 connection now open)
    Mon Aug 19 06:58:26.859 [initandlisten] connection accepted from 127.0.0.1:57328
    #3 (2 connections now open)
    Mon Aug 19 06:58:26.860 [initandlisten] connection accepted from 127.0.0.1:57329
    #4 (3 connections now open)
    Mon Aug 19 06:58:26.861 [initandlisten] connection accepted from 127.0.0.1:57330
    #5 (4 connections now open)
    Mon Aug 19 06:58:26.862 [initandlisten] connection accepted from 127.0.0.1:57331
    #6 (5 connections now open)
    Mon Aug 19 06:58:28.411 [conn3] end connection 127.0.0.1:57328 (4 connections no
    w open)
    Mon Aug 19 06:58:28.411 [conn2] end connection 127.0.0.1:57327 (4 connections no
    w open)
    Mon Aug 19 06:58:28.411 [conn4] end connection 127.0.0.1:57329 (4 connections no
    w open)
    Mon Aug 19 06:58:28.411 [conn5] end connection 127.0.0.1:57330 (4 connections no
    w open)
    Mon Aug 19 06:58:28.412 [conn6] end connection 127.0.0.1:57331 (4 connections no
    w open)
    Mon Aug 19 07:00:48.179 [initandlisten] connection accepted from 127.0.0.1:57356
    #7 (1 connection now open)
    Mon Aug 19 07:00:48.389 [conn7] end connection 127.0.0.1:57356 (0 connections no
    w open)
    Mon Aug 19 07:00:48.391 [initandlisten] connection accepted from 127.0.0.1:57358
    #8 (1 connection now open)
    Mon Aug 19 07:00:48.392 [initandlisten] connection accepted from 127.0.0.1:57359
    #9 (2 connections now open)
    Mon Aug 19 07:00:48.393 [initandlisten] connection accepted from 127.0.0.1:57360
    #10 (3 connections now open)
    Mon Aug 19 07:00:48.394 [initandlisten] connection accepted from 127.0.0.1:57361
    #11 (4 connections now open)
    Mon Aug 19 07:00:48.395 [initandlisten] connection accepted from 127.0.0.1:57362
    #12 (5 connections now open)
    Mon Aug 19 07:00:51.353 [FileAllocator] allocating new datafile \data\db\true.ns
    , filling with zeroes...
    Mon Aug 19 07:00:51.390 [FileAllocator] done allocating datafile \data\db\true.n
    s, size: 16MB, took 0.036 secs
    Mon Aug 19 07:00:51.391 [FileAllocator] allocating new datafile \data\db\true.0,
    filling with zeroes...
    Mon Aug 19 07:00:51.540 [FileAllocator] done allocating datafile \data\db\true.0
    , size: 64MB, took 0.148 secs
    Mon Aug 19 07:00:51.542 [FileAllocator] allocating new datafile \data\db\true.1,
    filling with zeroes...
    Mon Aug 19 07:00:51.543 [conn10] build index true.tesla_stream { _id: 1 }
    Mon Aug 19 07:00:51.545 [conn10] build index done. scanned 0 total records. 0.0
    01 secs
    Mon Aug 19 07:00:51.545 [conn10] insert true.tesla_stream ninserted:1 keyUpdates
    :0 locks(micros) w:192640 192ms
    Mon Aug 19 07:00:52.198 [FileAllocator] done allocating datafile \data\db\true.1
    , size: 128MB, took 0.656 secs
    Mon Aug 19 07:01:22.744 [initandlisten] connection accepted from 127.0.0.1:57369
    #13 (6 connections now open)
     
  17. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    That's what I was guessing. There's a bug in the code that I haven't tracked down yet. If you use "-db tesla" instead of "--db tesla" then the database name is 'true' instead of 'tesla'.

    Can you run this again, with a different db name?
    Code:
    use true
    db.tesla_stream.count()
    If that shows you data present, then just run the visualize.js script with "--db true" and things should work (I'll help you later to move this into the "tesla" database... I'm about to leave on an all day road trip...)
    You posted the console log of your PC. What I meant was if you are in your browser, can you show the javascript console log of your browser. With Chrome you can right-click on the page and pick "Inspect Element". Same with Firefox. If you are using Internet Explorer then calmly walk away from the computer and wait until the urge stops. Then install a real browser.
     
  18. moviemaker2000

    Joined:
    Jan 18, 2013
    Messages:
    45
    Location:
    midwest
    I installed the latest version 0.6.5 as suggested and it works now.
    Thank you

    - - - Updated - - -

    Your guess was correct. I changed to the "true" db and all the data was there.
    I knew it had to be something small :wink: (one extra -)
     
  19. hans

    hans P631

    Joined:
    Sep 27, 2012
    Messages:
    1,123
    Location:
    Menlo Park
    This is an artifact of the way the "optimist" argument parsing module works. I can enhance it to check for a proper db name.

    The way it is working now is that when you type -db instead of --db, it is treating the arguments as a list of option like "-d -b" and so it's setting both -d and -b flags to be the boolean value "true". I will add a .check() function that verifies that the -d or --db value is set to something other than "true" or else throw an error and print the usage information.
     
  20. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    We should be precise in our language regarding the underlined: "... that we know of."

    There may be an "update_pending_state" or "initiate_update" command that we just haven't discovered yet.
     

Share This Page