Vendor Scan My Tesla, a CANBUS reader for Android

[joltheadq]

Would the PLX kiwi adaptors work? Like the version 3 or 4?

 

[jaitch]

I said I'd share my powershell script to reduce the data, so here it is. I'm not a wonderful programmer (hobby), so I'm certain there are better ways to do things, but this script works for me and I thought I'd share since SMT CSV files can be SO BIG!

Example of size reduction:
NOTE: SMT on 'ALL' tab recording for 90 minutes. Resulted in CSV file with more than 1.3 Million rows
Original file size: 181.3 MB
no switch (combine only): 147.1 MB
-ms 1000 switch: 2.7 MB
-ms 60000 switch: 52 KB

In a nutshell, since SMT spits out data that might be the SAME millisecond timestamp, but for different measures, the CSV files always have extra rows and this script was born to combine the data of different rows that share the same exact timestamp. This did result in a smaller file, but not by very much. Then I added 2 different ways to reduce the data by throwing out data:

Option 1 is using the '-skipevery' switch. This simply combines N rows into one keeping the last value. Combining vs tossing is important because different measures have different poll rates. So if you used '-skipevery 3', the script would combing 3 rows into 1 row. If any measure has a value in any row, they are kept. The most recent measures win.

Option 2 is using the '-ms' switch. This does the same thing as '-skipevery', only you specify the number of millisceconds to combine. So if you used '-ms 1000' you'd 'combine' the data measured for 1 second, again, most recent wins. If you used '-ms 60000', you'd combine the data for 1 minute.

The intent is to use option 1 OR option 2, not at the same time.

Final example of use, you'd type this in powershell:
.\fixcsv.ps1 -filein "MyData.csv" -ms 1000

The output file would be in the same folder called "MyData_out_1000ms.csv"

Hope you find the script useful
......

I think what you have done sounds really useful. My current way of shrinking the huge files involves filling all blanks with data from cells above, so all data rows have all data columns in, and then deleting 99 rows out of 100 (or so). It is a little tedious so your script thing seems more straightforward and does the same thing from what I can tell.

However I am a code dumb@ss so don't really understand how I get your script to work. I can open powershell but that is it! Are there any idiot guides - ie I have a csv file then what do I do?

If you could spare a few minutes to explain I would be most grateful.

James

 

[insaneoctane]

I think what you have done sounds really useful. My current way of shrinking the huge files involves filling all blanks with data from cells above, so all data rows have all data columns in, and then deleting 99 rows out of 100 (or so). It is a little tedious so your script thing seems more straightforward and does the same thing from what I can tell.

However I am a code dumb@ss so don't really understand how I get your script to work. I can open powershell but that is it! Are there any idiot guides - ie I have a csv file then what do I do?

If you could spare a few minutes to explain I would be most grateful.

James

Google can help with most of the steps. But, in a nutshell:

1. copy-paste my code into notepad (or your favorite text editor)
2. review that it doesn't contain anything malicious (it doesn't)
3. Save it as "csvfix.ps1"
4. Now run powershell and navigate to the "csvfix.ps1" script (ie cd "C:\the\location\of your script")
5. Now type the command like I suggested (ie .\csvfix.ps1 -infile "C:\path\to\your\file\file.csv" -ms 1000 ).

Note: If you've NEVER run powershell scripts, you may have to "enable" that on your computer. Just google for details about the error you receive when you try to run the ps1 file.... HERE is a good overview. Basically Microsoft set the execution policy to NOT allow any powershell scripts for security purposes since many users will NEVER run them. But if you want to run scripts, you'll need to enable the option. You'll need to be admin when you update your policy, so run cmd as administrator before setting your execution policy. I'd recommend "set-executionpolicy remotesigned" as it seems like a good compromise- Any ps1 files you download directly from the internet will need to be digitally-signed, but you can write your own without any problem- so, when you copy-paste my code into notepad and save, you just became the author on your local machine and it will run. That's also why you should always check that it's not doing anything malicious (and not trust it if you can't tell!).

Good luck!

 

[jaitch]

Google can help with most of the steps. But, in a nutshell:

1. copy-paste my code into notepad (or your favorite text editor)
2. review that it doesn't contain anything malicious (it doesn't)
3. Save it as "csvfix.ps1"
4. Now run powershell and navigate to the "csvfix.ps1" script (ie cd "C:\the\location\of your script")
5. Now type the command like I suggested (ie .\csvfix.ps1 -infile "C:\path\to\your\file\file.csv" -ms 1000 ).

Note: If you've NEVER run powershell scripts, you may have to "enable" that on your computer. Just google for details about the error you receive when you try to run the ps1 file.... HERE is a good overview. Basically Microsoft set the execution policy to NOT allow any powershell scripts for security purposes since many users will NEVER run them. But if you want to run scripts, you'll need to enable the option. You'll need to be admin when you update your policy, so run cmd as administrator before setting your execution policy. I'd recommend "set-executionpolicy remotesigned" as it seems like a good compromise- Any ps1 files you download directly from the internet will need to be digitally-signed, but you can write your own without any problem- so, when you copy-paste my code into notepad and save, you just became the author on your local machine and it will run. That's also why you should always check that it's not doing anything malicious (and not trust it if you can't tell!).

Good luck!

Brilliant thanks. I should be able to follow that perfect explanation.

James

 

[derotam]

I said I'd share my powershell script to reduce the data, so here it is. I'm not a wonderful programmer (hobby), so I'm certain there are better ways to do things, but this script works for me and I thought I'd share since SMT CSV files can be SO BIG!

Example of size reduction:
NOTE: SMT on 'ALL' tab recording for 90 minutes. Resulted in CSV file with more than 1.3 Million rows
Original file size: 181.3 MB
no switch (combine only): 147.1 MB
-ms 1000 switch: 2.7 MB
-ms 60000 switch: 52 KB

In a nutshell, since SMT spits out data that might be the SAME millisecond timestamp, but for different measures, the CSV files always have extra rows and this script was born to combine the data of different rows that share the same exact timestamp. This did result in a smaller file, but not by very much. Then I added 2 different ways to reduce the data by throwing out data:

Option 1 is using the '-skipevery' switch. This simply combines N rows into one keeping the last value. Combining vs tossing is important because different measures have different poll rates. So if you used '-skipevery 3', the script would combing 3 rows into 1 row. If any measure has a value in any row, they are kept. The most recent measures win.

Option 2 is using the '-ms' switch. This does the same thing as '-skipevery', only you specify the number of millisceconds to combine. So if you used '-ms 1000' you'd 'combine' the data measured for 1 second, again, most recent wins. If you used '-ms 60000', you'd combine the data for 1 minute.

The intent is to use option 1 OR option 2, not at the same time.

Final example of use, you'd type this in powershell:
.\fixcsv.ps1 -filein "MyData.csv" -ms 1000

The output file would be in the same folder called "MyData_out_1000ms.csv"

Hope you find the script useful

I'm just going to address you but really it could be for anyone that is smarter than me. Another thing that would be nice to do would be to dump duplicate adjacent values in every column. For example, if a given column stays at one value for a "long" time before it changes, getting rid of the repeating same value till it changes could also help reduce the datafile size. If I am trying to plot vs time, I only need to save the rows where the data isn't repeating.

I of course don't know how to do this.

 

[insaneoctane]

I'm just going to address you but really it could be for anyone that is smarter than me. Another thing that would be nice to do would be to dump duplicate adjacent values in every column. For example, if a given column stays at one value for a "long" time before it changes, getting rid of the repeating same value till it changes could also help reduce the datafile size. If I am trying to plot vs time, I only need to save the rows where the data isn't repeating.

I of course don't know how to do this.

Hmmmm. Valid suggestion for reducing data size. I don't think I'll bother adding that feature though, because I'm finding the current tool is working very well to reduce data to very manageable / workable sizes....and while your suggestion would reduce size even further, it wouldn't plot the same unless you added in the "previous" value back in right before a change (think of the same value for 20 entries in a row, then it goes up a little. You'd have to have keep the first value, skip 18 of them, then plot the first value again just before the change up, otherwise it would plot like a slope vs a straight line that then went up a little) . So, while clever and size-reducing, I think I'll pass on the opportunity to include for my script. But, thanks for the suggestion- maybe someone else will want to implement it.

 

[R1Fast]

I'm using SMT with an OBDLink LX and after 2-3 days of normal use, the BT connection to the LX seems to fail (stalls on "Initializing Adapter" in the app). It works fine again if I physically hold the pairing button in on the LX and re-pair to phone, but that's a pain since I stow the OBDLink behind the plastic trim above the OBD plug (e.g. rear bottom of center console). Every time it stops working I have to pull the trim off and reset. I've already had Amazon send a replacement OBDLink LX and it suffers from this as well.

It looks like this was an issue for a few others last year but search didn't return anything recent. Has anyone else experienced this issue with the LX recently or other OBD modules? And/or are there any solutions? I've already flashed the most recent firmware.

 

[amund7]

I'm using SMT with an OBDLink LX and after 2-3 days of normal use, the BT connection to the LX seems to fail (stalls on "Initializing Adapter" in the app). It works fine again if I physically hold the pairing button in on the LX and re-pair to phone, but that's a pain since I stow the OBDLink behind the plastic trim above the OBD plug (e.g. rear bottom of center console). Every time it stops working I have to pull the trim off and reset. I've already had Amazon send a replacement OBDLink LX and it suffers from this as well.

It looks like this was an issue for a few others last year but search didn't return anything recent. Has anyone else experienced this issue with the LX recently or other OBD modules? And/or are there any solutions? I've already flashed the most recent firmware.

If it's the same bug I am experiencing, what works is to unplug the OBDLink, then re-plug it, just to reset it. I have been on OBDLink for at least 2 years, if not 3, feeding them bug reports and test results to get this fixed in their firmware. But they never do anything. Please bug them about it!

BTW interesting that a re-pairing fixes it ?

To make this happen less often, don't use CAN DUMP recording (that always hangs mine), and use any other tab than the All tab. Less traffic seems to make this happen more rarely, or not at all.

 

[R1Fast]

If it's the same bug I am experiencing, what works is to unplug the OBDLink, then re-plug it, just to reset it. I have been on OBDLink for at least 2 years, if not 3, feeding them bug reports and test results to get this fixed in their firmware. But they never do anything. Please bug them about it!

BTW interesting that a re-pairing fixes it ?

To make this happen less often, don't use CAN DUMP recording (that always hangs mine), and use any other tab than the All tab. Less traffic seems to make this happen more rarely, or not at all.

Thanks, yes unplugging fixes it but I'd like to keep it a stealth install tucked behind the console trim ( right above where the plug sits). I guess either I can get one of those in line toggles or a different adapter.

 

[R1Fast]

Update and question for amund7

I'm going to try the commands posted earlier on the LX here to see if this helps with the LX not waking from sleep.

Ok, my OBDLink LX worked flawlessly for one week, after disabling sleep mode. What I did was to install Bluetooth Terminal by Qwerty (check Append newline in the Setup), and send the following commands to the adapter.
stslu off,off
stslcs
atz

Hope this helps.

@amund7, did you already implement sleep commands for the LX in the software? Would this theoretically address the issue with the LX not waking up?

I can confirm that the current version of Scan My Tesla, the filters work correctly with all the pictured adapters in the 'reccommended adapters' info page.

I hear what you are saying, and will try to find a way to allow this. Problem is, there are so many wierd adapters now, I've even seen some that don't support the ATMA command.

Will try to implement something for you soon.
Also need to implement the OBDLink LX sleep commands.
BTW does anyone know if new OBDLink MX suffers from that same problem? Mine does not, but it's old (although has latest firmware).

 

[amund7]

Update and question for amund7

I'm going to try the commands posted earlier on the LX here to see if this helps with the LX not waking from sleep.



@amund7, did you already implement sleep commands for the LX in the software? Would this theoretically address the issue with the LX not waking up?

No, the app never implemented this, as it is a problem that has been reported only by 2 or 3 users I believe, and I concluded OBDLink had made some non-standard settings to a very small batch of the adapters.

If it is sleeping, it should wake up by a quick push of the button, that's what the OBDLink manual says. And if that's what happens, the sleep settings could fix it, yes.

 

[R1Fast]

I'm using SMT with an OBDLink LX and after 2-3 days of normal use, the BT connection to the LX seems to fail....Every time it stops working I have to pull the trim off and reset.


Edit: not sure why I didn't find results the first time I searched the thread but I see now this (or similar) is a common issue with the OBDLink LX.


What OBD module in the $40-50 range works WELL without these frequent hangs/disconnects?

So the sleep commands didn't work. 3 days later and the LX stopped connecting again.


So back to my question. What other OBD module has the same packet rates and DOESN'T suffer from sleep issues?

 

[amund7]

So the sleep commands didn't work. 3 days later and the LX stopped connecting again.


So back to my question. What other OBD module has the same packet rates and DOESN'T suffer from sleep issues?

NONE

Which is why, AFAIK, OBDLink are still the best option, if you want fast and smooth that is.

Note that you then don't have a sleep issue, but an OBDLink firmware issue. Please complain to them, if enough people complain, maybe they will fix this bug that ruins the worlds most awesome canbus adapter!

 

[R1Fast]

NONE

Which is why, AFAIK, OBDLink are still the best option, if you want fast and smooth that is.

Note that you then don't have a sleep issue, but an OBDLink firmware issue. Please complain to them, if enough people complain, maybe they will fix this bug that ruins the worlds most awesome canbus adapter!

Ok thanks. Hmmph. I did post on their LX forum and sent an e-mail to support with the issue. There's a handful of other posts even a few Tesla specific w/similar issues.... you're right they just deflect with "try re-pairing or a firmware update bla bla" and never circle back on these threads to admit theres a problem.

So the options are A) performance w/loss of connection stability or B) vice-versa or C) shell out another $40 for an MX? (would the latter even solve the issue?)

BTW, re: firmware problem, I did update to the newest firmware as well and no dice so still in the underlying code.

Also, I'm only really using a custom tab with ~10 parameters, never the other tabs.

Would a cheapo elm be able to handle packets in real-time (ish) if limited to these same 10?

 

[MarcG]

Also, I'm only really using a custom tab with ~10 parameters, never the other tabs.

Would a cheapo elm be able to handle packets in real-time (ish) if limited to these same 10?

I bought this cheapo device and it seems to be able to handle 40 to 60+ packets per second fine on my custom tab with ~15 dynamic variables (based on the Perf tab):

https://www.amazon.com/gp/product/B00UV7G7LI/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1

 

[R1Fast]

I bought this cheapo device and it seems to be able to handle 40 to 60+ packets per second fine on my custom tab with ~15 dynamic variables (based on the Perf tab):

https://www.amazon.com/gp/product/B00UV7G7LI/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1

Cool, might give that one a shot. And no issues with initialization or pairing? How many days straight (ballpark) has it worked for you?

 

[MarcG]

Cool, might give that one a shot. And no issues with initialization or pairing? How many days straight (ballpark) has it worked for you?

No issues initializing or pairing so far.

Not sure about keeping it on for days straight, as I just recently got my adapter from GPS Tracking America (took nearly 3 weeks to ship from.. Canada - go figure).

The other thing is, I have been and plan on always turning off the adapter when I’m done driving, to avoid any potential vampire drain. I bought this extension cord with a switch:

https://www.amazon.com/gp/aw/d/B079NN3Z7B?psc=1&ref=ppx_pop_mob_b_asin_title

Only downside I noticed so far is that, after turning the car on and flipping that switch back on, I still have to hit the button on the Konnwei to have it wake up. Strange..

 

[amund7]

Ok thanks. Hmmph. I did post on their LX forum and sent an e-mail to support with the issue. There's a handful of other posts even a few Tesla specific w/similar issues.... you're right they just deflect with "try re-pairing or a firmware update bla bla" and never circle back on these threads to admit theres a problem.

So the options are A) performance w/loss of connection stability or B) vice-versa or C) shell out another $40 for an MX? (would the latter even solve the issue?)

BTW, re: firmware problem, I did update to the newest firmware as well and no dice so still in the underlying code.

Also, I'm only really using a custom tab with ~10 parameters, never the other tabs.

Would a cheapo elm be able to handle packets in real-time (ish) if limited to these same 10?

I've been on their developers, with logs, test cases and instructions to reproduce. They recieve my feedback very politely, then never do anything with it.

MX and MX+ behave exactly the same from what I've seen. MAYBE the MX+ lasts a bit longer before hanging, but not sure if it's just placebo.

I bought this cheapo device and it seems to be able to handle 40 to 60+ packets per second fine on my custom tab with ~15 dynamic variables (based on the Perf tab):

https://www.amazon.com/gp/product/B00UV7G7LI/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1

60 packets per second, that's cute! OBDLink LX does 750 packets per second if you add enough signals to the Perf tab

I suggest looking at some of the more expensive OBD2 competitors to OBDLink. But I don't think any of them are nowhere near the speed of OBDLink, but still many of them are near the price!

 

[insaneoctane]

I have random disconnects with my LX. PITA honestly.

 

[R1Fast]

Yeah I guess fast doesn't always mean superior. if Scantool would would just fix their firmware it'd be worth every 1 of the 50 bucks!

@amund7 I found the Firmware rev history for the LX on their website. Wonder if reverting to any of these would improve this issue (though I don't know if was an introduced bug or just underlying code issue from v4.0.0).

Thoughts? Interestingly v4.2.3 was specific to hanging when exiting sleep mode (though no download avail for it)

This minor release fixed an issue where some devices became unresponsive upon exiting sleep mode. This release also changed the button functionality when in sleep mode.

I wonder if 4.0.0 is worth a shot since it wouldn't include this fix (and also does still have the download avail).