Chassis CAN Logging To ASCII Text Plus Graphing

[MikeBur]

So, have few things to share re. excel files whilst we await Smac's automagic web site importer (see what I did there smac? ;-)).

I've hosted my initial captured data, as promised, as well as the excel source files in my attempt to help quickly create your own charts and visualisations. This Google Drive link is read only to all, and read/write to anyone wishing to copy these up (ie. just ask via. PM). The previous Lolachampcar sessions and mine are in sub directories.

In terms of using these, the battery one I created (MikeBur 02-16-2015 Battery Data.xlsm) should be the easiest. It presumes the ASCII output format of Bill's logger, just copy the data from the battery logging session to the first sheet (SourceData) and then you should have a static line graph for pack info and an animated graph potential to watch charging over time. Just press the animate button and the histogram will animate over each row showing cell voltage and battery module temp (and you can manually change the animation step rate for speed ). Note: to do this requires enabling macros in excel. Anyone can look at how dull my VBA code is, though if you'd prefer me to drop/post the source here and manually enter it just lemme know.

The second xls (MikeBur 02-16-2015-2230) is a little more involved, and is for the performance runs. It requires manually copying/pasting the data from each run, as there can be many captured in one session) into it's own sheet and then fixing up the range information in the calculated cells (after column "i").
The way I do this is copy a "Run" tab and then overwrite the first 8 columns from the logger session text file. Column 'K' appears to suffer the most as I decided to base everything on relative time and first entry being overwritten causes excel to go #ref tastic. Easy fix: cell K1 should read "=A2-$A$2" (without the quotes) and then sweep fill the column (select bottom-right corner with left mouse held down and move mouse down until you hit the end of data, then let go). This will generate the column data.
Next manual step is to enter your graph heading data and SoC, etc. from the log file. Currently this is at the end of each run, so I manual move this to the top right and paste the values in the appropriate spots (ie. Q2 to S2). After this I edit O5 and R5 for manual info - the graph title uses this with the min/max speed and Soc/Bat data to generate the titles.
There's a few other areas of manual prettifying, so don't hesitate to ask, make suggestions, or just edit and share back - no rights, guarantees, yada yada. Would love to see / hear what others can do with these, excel is not my forte and likely there should have been pivottables and the like, though they scare me ;-). Have fun!

Cheers, Mike

 

[MikeBur]

Special request for Andy, comparison between CAN data and TeslaLog. 2 Runs, captured simultaneously. One is 0-60 fast foot start, other is 30-60 and both in wet conditions. CAN data in white, TeslaLog in Black.

I'll leave judgment to each other, though I will note the general shape of the curves and data ranges are ok, though there does appear to be latency concerns for me in both the steepness of the gradient and also time to respond to large changes (eg. up to 1/2 second by my eyeball).

Hope this helps, Mike

 

[kennybobby]

...
I played with this chart a little more just. It makes sense to calculate BatW offline (always best to capture less data if possible). I also note now which sets of values against which axis.
View attachment 111639

btw - 2 questions back at you:
1) Where's the best place for us all to share this data? Dropbox, or shall I just share out folder via. link on Google Drive?
2) How did you create that awesome battery cell animation? I'm guessing some wizardry involving PivotCharts?

Wow that BatW power is a dead flat line--so this is some sort of constant power charging, not CC-CV, but managed C and V such that the product is a constant. i would never have expected that since cc-cv is the de facto biblio everywhere you look.

re: questions
1. i have no clue, your ideas sound good.

2. i used the developer tab to add a slider to the chart that points to the 'time' column, so clicking the slider would step thru the data over time. Then i just recorded the chart while clicking the slider. i thought it was in the xls i sent you, but if not i can send it if you want. i don't know about pivotcharts they seemed too complicated for what i wanted to do.

You got some nice runs in with good data captures, and the charts look great. You don't seem to have the big torque dip like we saw in Bill's 7 runs.

 

[MikeBur]

Wow that BatW power is a dead flat line--so this is some sort of constant power charging, not CC-CV, but managed C and V such that the product is a constant. i would never have expected that since cc-cv is the de facto biblio everywhere you look.

re: questions
1. i have no clue, your ideas sound good.

2. i used the developer tab to add a slider to the chart that points to the 'time' column, so clicking the slider would step thru the data over time. Then i just recorded the chart while clicking the slider. i thought it was in the xls i sent you, but if not i can send it if you want. i don't know about pivotcharts they seemed too complicated for what i wanted to do.

You got some nice runs in with good data captures, and the charts look great. You don't seem to have the big torque dip like we saw in Bill's 7 runs.

Just calling it the way it's recorded, though this didn't surprise me too much. I interpreted this as constant output from the dual-chargers at ~70A, though remember this is I*V and although you cannot easily compare the battery voltage with battery current (as they're on different axis on the graph) you can see that the general trend of each makes reasonable sense imo.

Yep, just ran with Google Drive. Works well for me elsewhere and permissions are pretty flexible and controllable.

Talking of which you should check out the way I hacked the animation on the excel I did. It's pretty geeky and I added both a slider as well as programmatically being able to animate after pressing a button (with ability to vary update speed). Now all we need is an upload service that can do this as Google Docs barfs on the graphs we're generating and Office365 is too confusing to actually use Excel online and the programmatic access is too high a barrier for my time availability at the moment. Smac?

 

[Andyw2100]

Special request for Andy, comparison between CAN data and TeslaLog. 2 Runs, captured simultaneously. One is 0-60 fast foot start, other is 30-60 and both in wet conditions. CAN data in white, TeslaLog in Black.

I'll leave judgment to each other, though I will note the general shape of the curves and data ranges are ok, though there does appear to be latency concerns for me in both the steepness of the gradient and also time to respond to large changes (eg. up to 1/2 second by my eyeball).

Hope this helps, Mike

I appreciate your posting these, Mike.

The reason I was interested in seeing the comparison, (as you know but I'm now sharing with everyone else,) is that I wanted to see how close TeslaLog comes to accurately representing what's going on as compared to the more sophisticated logging some of you are and will be doing.

It seems, based on the above, that for people like me who just want something really simple, TeslaLog is probably capable of providing pretty decent data.

Thanks again!

 

[lolachampcar]

Now this is why I'm providing loggers for free

Better data... Better graphing... More learning....

​Thank you to all!!

 

[smac]

So, have few things to share re. excel files whilst we await Smac's automagic web site importer (see what I did there smac? ;-)).

lol, yep I get the hint

 

[kennybobby]

Simon is too busy cutting and welding metal to monkey with any code...

 

[MikeBur]

Now this is why I'm providing loggers for free

Better data... Better graphing... More learning....

​Thank you to all!!

lol, yep I get the hint

Simon is too busy cutting and welding metal to monkey with any code...

heh, well in the spirit of progress, we're waiting for noone, eh?

Bill, some additional feedback. You asked I share publicly, and again this is a great logger and wanted to highlight in the spirit of continuous, transparent, improvement.

1) Can you "read" a few initialization variables from the storage for more flexibility? ie. sometimes I want TPS at 30% so I can record some range mode motor torque data. Sometimes I want it just the way it is. It varies and reading once on startup initialization of the device would be a great compromise. I can also see this being using for battery refresh time, eg. I would like more accurate data on a Sc or ChAdeMo session than 80A than 40A - a number representing number of seconds before update poll would be awesome. If these numbers don't exist than fail back to your current values as defaults perhaps?
2) Can you put Pack Soc, etc from end of log to start - Though if this impacts the "just too late" starting mechanism of TPS at 50%, keep it the way it is.
3) Is it feasible to setup a ring buffer so you can store the data *before* the trigger point? The way I've done this before, in other perf software, is continuously output to a fixed size buffer and then start dumping from the start of the buffer as the trigger occurs - hopefully this makes sense? - reason why is to record from start of the activity - ie. if we can capture from TPS==0 we may notice other preparation activity
4) Can you not output the Pwr field please? ;-)
I noticed a bug with your data file. The Pwr field is both truncating to first decimal place, rather than rounding to it. Additionally it appears to be abs'ing the result. I hadn't noticed until just comparing some more data and using kW vs. Amps (ie. I prefer plotting both, rather than the product though others don't ;-)).
I'm presuming this is a calculated field anyway and not taken from the bus? If so, my recommendation would be to stop exporting this (see earlier opine on only store the data you need to, and then offline process). If you need this, then we should be aware that the data exported looks different when comparing, ie. truncating:

src Pwr	Calc Pwr(kW)
17.50	17.57
22.10	22.19
24.00	24.08
25.70	25.79
26.40	26.50
26.10	26.14
27.50	27.59

and (abs'd data vs. signed)

24.30	-24.35
25.30	-25.30
24.10	-24.12
27.70	-27.80 (it's actually -27.999)
26.30	-26.32
29.20	-29.27
31.70	-31.79

Offline, I just do a simple P=I*V/-1000 if that helps as graphing power output shows well as a +ve, and power regen as a -ve in my mind. Of course, in fixed point math this might be a little more complex
Though again, calculated after the fact gives more accuracy and less data storage requirements - so large amount of justification for a hopefully simple conclusion. Don't export power ;-)

Any of the above would be great, and beggars can't be choosers, though if I can help let me know. I'm already asking around on the FAT file system stuff, though there appears to be some copyright concerns with MS-DOS floppy disk formats. Don't know if others know way 'round this, though it would appear solved in the Linux space as it reads floppies just fine

Again, thanks.

Cheers, Mike

Side effect of #4 is the power graphs now look even more similar to the Teslalog info, and this is what sent me on the quest for why. Original here. Corrected one below:

 

[lolachampcar]

Initial thoughts (some more reflection on my part required)
1)
Possible but then we are maintaining a separate file on the "disk". It would be much easier to have a few versions of the firmware with different TPS thresholds IF TPS threshold was the only issue. Update rate could be made a function of max experience charge current. If BatI > X then log at Y.
2)
The delay you see at the end of logging is partially due to waiting for a SoC and BatODO message. I'll look at the report rate for each to see if either can be captured before starting logging without impacting logging itself. There may also be an issue of delay in changing CAN filters. If that is quick enough, perhaps I can constantly update a local SoC and BatODO value then place it at the start of a log file before switching over CAN filtering for the performance logging. I'll look into it. Doing this may also interfere with the ring buffer you are looking for in 3. I have four filters available so it would not be possible to wait in SoC update filtering while simultaneously maintaining a ring buffer of performance logging data.
3)
I considered this briefly.
Using flash requires the management of erasure along the way.... It might be possible to devote a number of flash pages to a ring buffer allowing enough time to erase first out pages before needing them again. The "usable" pages could be processed (save raw data, not engineering units) and written to the start of the log file once logging is complete. The last option I considered was to push SRAM usage down to a bare minimum then use the remaining as a buffer, again, raw data only and post process to the start of the log file once logging is complete. I'll look at the timing for these options again in more detail.
4)
No problem and yes it was truncated and I did not bother with polarity.... I was just looking for a qualitative feel for what was going on and not exact values.

Thanks for looking into the 4 meg FAT thing. What a convoluted POS system.

 

[MikeBur]

Initial thoughts (some more reflection on my part required)
1)
Possible but then we are maintaining a separate file on the "disk". It would be much easier to have a few versions of the firmware with different TPS thresholds IF TPS threshold was the only issue. Update rate could be made a function of max experience charge current. If BatI > X then log at Y.
2)
The delay you see at the end of logging is partially due to waiting for a SoC and BatODO message. I'll look at the report rate for each to see if either can be captured before starting logging without impacting logging itself. There may also be an issue of delay in changing CAN filters. If that is quick enough, perhaps I can constantly update a local SoC and BatODO value then place it at the start of a log file before switching over CAN filtering for the performance logging. I'll look into it. Doing this may also interfere with the ring buffer you are looking for in 3. I have four filters available so it would not be possible to wait in SoC update filtering while simultaneously maintaining a ring buffer of performance logging data.
3)
I considered this briefly.
Using flash requires the management of erasure along the way.... It might be possible to devote a number of flash pages to a ring buffer allowing enough time to erase first out pages before needing them again. The "usable" pages could be processed (save raw data, not engineering units) and written to the start of the log file once logging is complete. The last option I considered was to push SRAM usage down to a bare minimum then use the remaining as a buffer, again, raw data only and post process to the start of the log file once logging is complete. I'll look at the timing for these options again in more detail.
4)
No problem and yes it was truncated and I did not bother with polarity.... I was just looking for a qualitative feel for what was going on and not exact values.

Thanks for looking into the 4 meg FAT thing. What a convoluted POS system.

Great, thanks.

1) adds flexibility, though if we're writing anything to the flash then you're correct, an image update is likely just as easy as editing a file.
2) I would ignore this then. It's just a convenience for moving data around - takes about 5 seconds, so manual labor cost of editing is the better option here. On reflection, just writing it as SoC<space>BatT<Space>BatOdo<Space>\n<Soc float><space><BatT float><space><BatOdo float> would be very useful at the end of file so cut'n'paste would be trivial as excels text into columns will do the "hard" work .
3) Writing ring buffer into SRAM is going to be the way if possible. Erasing/writing flash is slow (relatively speaking), right?
4) Thanks

Can I add a couple more suggestions?
5) Can you obtain the front/rear motor rpm? Thinking behind this was to calculate HP (at the motor, before anyone starts ). Might also be useful to determine to what extend torque vectoring happens and how much is braking vs gearing (if that makes sense).
6) Can you obtain some battery data in performance mode without slowing down capture rate? I'm getting limited a few times now due to battery heat and it'd be interesting to investigate that and determine if this gets worse with ludicrous upgrade. It appears to be worse when Max Battery mode is on, ie. seems a little ironic and I can't imagine that would be the intended result.

Another couple of data graphs as well as something I tried and it appeared to work re. battery mode whilst driving.

0-95 launch with max battery, heated at > 90% SoC

Interesting things are afoot, note the drop in available current at 6secs post launch. BatTemp at end was 45.7C, so shouldn't have been battery reduced, though something odd.

Started to get more battery messages, so ran battery gather mode on logger. In normal use, everything looks as expected:

Though did manage to capture one session whilst under load. As you can see these are pretty amazing battery packs to deal with this load:

Additional Excel spreadsheets (incl. 100% SoC charge data on 80amp, and a lot more runs) are on the share.

 

[lolachampcar]

No problem to alter storage of Soc, etc. at the end of each record. Do please provide an example of what you are thinking using real data just to make sure I get exactly what you are looking for here.

Available CAN filters (only four) determine what can be captured at any one time. If 50 samples per second was sufficient, I may be able to actively alternate between two different sets of CAN filters to allow for more to be captured. I'll investigate. My micro is bloody fast so it may be possible to simply capture all frames and sort the ones I'm interested in. I'll try that as a side project.

I'll do up some alternate TPS threshold updates for now. Have you given any thought to Battery Health sample rate versus a/multiple data variable(s) (like charge current) so as to dynamically choose a more appropriate sample rate (AC v. DC charging)?

The battery health stuff I've been capturing is contained within 32 CAN frames which take three seconds for a full set of 32 to appear on the bus. The last ten or so have the module temps. The difficulty I found with trying to include battery health in the performance logging was that I would need to interject a full battery health frame every three seconds into an ongoing performance log or only include temps or battery sheet voltages from a single CAN frame (whatever one happened to be available at the time). Neither solution seemed worth doing.

On the limiting side, I suspect performance limiting is a function of any one battery module getting warm and/or motor/inverter temperatures. I was likely exceeding motor/inverter limits at the end of my five 0-60 runs as I was nowhere near max performance bat temp at the end of my last run.

 

[okashira]

The individual cell voltages are not updating in real time:
Green should be cell voltage at 11,00 amps.

 

[lolachampcar]

The individual cell voltages are provided by the car in multiple CAN messages over a three second time frame. It would not seem possible to compare data provided at this slow a rate with data provided at 100 Hz (if that is what you are trying to do).

 

[MikeBur]

Good ideas Bill and Good catch both on battery update. Wasn't trying to correlate with perf data, though was trying to correlate within one data row. Good to know that's non-sensical, thanks both.

Was actually hailing as well as raining here today, so no new data today

 

[kennybobby]

...
0-95 launch with max battery, heated at > 90% SoC
View attachment 111848

It appears that you were using LM but i don't see the steps up in torque like we did in LCC's runs--did you release the brake at the same time as pressing the TPS, so maybe the car didn't load up the tires doing a squat? Maybe you can do a run with a pronounced squat to see how it compares to Bill's car.

At ~6 sec the current pulls back at around 82 mph, but the voltage had a slight step down about .4 sec before that occurred. Maybe the voltage sagged to a limit that triggered power limiting to keep the voltage from dropping further, or there is an 80 mph speed threshold that triggers power limiting? It is all very interesting to see and i'm glad you were able to capture this data.

 

[MikeBur]

It appears that you were using LM but i don't see the steps up in torque like we did in LCC's runs--did you release the brake at the same time as pressing the TPS, so maybe the car didn't load up the tires doing a squat? Maybe you can do a run with a pronounced squat to see how it compares to Bill's car.

At ~6 sec the current pulls back at around 82 mph, but the voltage had a slight step down about .4 sec before that occurred. Maybe the voltage sagged to a limit that triggered power limiting to keep the voltage from dropping further, or there is an 80 mph speed threshold that triggers power limiting? It is all very interesting to see and i'm glad you were able to capture this data.

definitely strange, here's another run in dry conditions from slightly earlier in the same day (before it started to rain a little). I didn't show it here as I eased off the throttle too early and the start is messed up - I don't remember rolling after launch mode prep, though I see this in the data. Anywho, this does show the launch mode prep better. I'll be out again soon and try to get another run - though, unsurprisingly for PacNW, it's wet again

 

[lolachampcar]

Ah, there are now three modes of operation-
Launch Mode
Fast Foot
Floppy Foot

 

[kennybobby]

Looks like some dancing on the pedals with that run.

Interesting that the current pulled back at 81 mph on that run while the voltage was actually slightly higher.

Now i'm thinking this is a speed-triggered pull-back rather than pack voltage.

Wonder about catching some data like high-speed passing, e.g. 50 or 60 -> 90, to see if the current pull-back occurs?

And just so nobody condones the wrong idea, but i am encouraging him to go too fast...

 

[lolachampcar]

Head getting sore on ring buffer pre-trigger data capture.... I'm remembering why I jumped past this on the first iteration.

Now that time allows for coding this up-
13 bytes per frame at 100 frames per second makes for 1300 bytes per second. I've got enough known sram to do about 1/3 a second of ring buffer for pre-trigger event data capture which is not really worth doing.
Given that sram is not an option, how much pre-trigger data do we want? Implementing a one flash page buffer or twenty flash page buffer is the same amount of work (512 bytes per page).

On another note, is 100 Hz data worth keeping or should we drop down to 50 Hz? Although I can not explain why, my gut had me wanting the fastest rate the car could supply.