Guys, it may not have been covered all together in this particular thread, but there are several things that (at least I continue to believe) are fact related to USB scan times through 2.42.40...
Scan Tests Continued - Part II (2.42.40)
For those that have no clue what I'm talking about, reference
Post 589 and read forward. If you don’t care about detail, please just skip this post or perhaps read the bold summary points.
Discussion in this thread on Thursday with
@aesculus and
@msnow got me thinking. I was going to delete my last batch of test data yesterday morning from my Mac, but decided to embark on a few more tests before I sent it all into the sunset. Here's what I did:
Tesla has implemented ”Album Art" a bit differently from the ID3 standard. See
Post #604 for detail.
Key to subsequent discussion here is that Tesla appears to be retaining some amount of Album Art across full CID reboots, and there is one piece of art for each unique Album.
TEST 5) Determine where my new Sweet Spot* really is for the number of tracks I maintain in my MS based on Scan Time (YMMV!)
* “Sweet Spot” for this exercise is a rather subjective max number of tracks on the USB stick that can both be successfully scanned, and before avg scan time per track takes a nose dive becoming increasingly long — especially given MS is still doing unnecessary rescans upon occasion.
I last did this work in Fall 2015 with V7.x firmware, timing increments of 1K tracks (and related albums) up to 15K tracks. The net at that time was scan duration started to dramatically drop-off for me somewhere between 6-7K tracks. It’s why I landed ever since housing about 6K tracks in my MS. Note that my tracks back then were from the same source as today and each track contained an original size Album Art (some in excess of 1500x1500) even if it was not displayed by Media Player — the only difference being the tracks were encoded in mostly M4A Lossy format (vs FLAC today and in these tests).
Method: I created a series of 6 actual tests, incrementally adding groups of albums & tracks to the same USB stick to finally reach the same total as TEST 2 data composed of 6829 Songs, 430 Artists, 1196 Albums, 13 Genre and 1216 Folders. Every track has a JPG of 300x300 or smaller. As before, I performed a full CID Reboot and allowed a minimum 5 minute settle time before beginning each scenario with my MS off.
Test Summary. I won't bore you with all the detail:
- First four sets of data (932, 1887, 2773, and 3738 tracks respectively) all had similar avg scan times per track within reasonable variation
- Beginning with the 5th set (4594 tracks from 835 albums), avg scan time per track increased 25-30% than any of the first four sets
- The 6th set (6829 tracks from 1197 albums) was as slow as set 5
My Observation:
As previously described in my Premise #3, the actual mix and number of subdirectories, files, tracks, albums and each of the tag contents itself influence scan time. We can’t compare e.g. 1K tracks from one of us to another for that reason, as our library contents are completely different, and so scan times are gonna vary.
For me with 8.0, if I’m willing to wait long enough, I can achieve perhaps half the number of tracks being available in my MS compared to 7.1. Realistically, I need to bring the number of tracks on my USB device down by about 15% to achieve similar scan times and not worry with scans restarting.
- I cannot achieve even half the number of tracks I was able to scan if I were willing to wait long enough with 7.x. Tesla’s algorithms have changed and/or memory usage is different. Today, at somewhere around 6600 tracks, the probability of my scan restarting or not completing increases.
- Because of that, and where the knee in my scan time curve appears to be, my new Sweet Spot (before scan time begins to substantially increase) for number of tracks in my MS is now closer to 5K tracks vs 6K with 7.x.
- While I can’t prove it, circumstantial observations lead me to believe that Tesla’s design decision to cache a single piece of art for each Album is consuming memory that could previously be used for track access. For someone with perhaps proportionally more Albums to Tracks, that could be another part of the reason for some of us seeing less tracks being able to be scanned with 8.0 than 7.x, as well as longer general scan times for a similar number of tracks. It all depends how Tesla codes their scanning routine. (FWIW I suspect I’m a fairly hefty 1:5.7 ratio of albums to tracks compared to some that place full albums on their USB sticks, as tracks on my USB device are for the most part my 4-5 Star tracks across my larger music library.)
TEST 6) Validate my previous assumption that eliminating all unused ID3 tags will have no effect on scan time.
Method: Same TEST 2 data: 6829 Songs, composed of 430 Artists, 1196 Albums, 13 Genre and 1216 Folders, EXCEPT that I removed all ID3 Tags except for ALBUM, ALBUMARTIST, DISCNUMBER, GENRE, TRACKARTIST, TRACKNUMBER, TRACKTITLE. Album Art remained embedded in every individual track.
My Observation: Scan Time remained effectively the same with extra tags and their data in place, and when they did not exist.
TEST 7) Validate the effect of eliminating all unused ID3 tags AND ALL ALBUM ART on scan time.
Method: Same TEST 6 data: 6829 Songs, composed of 430 Artists, 1196 Albums, 13 Genre and 1216 Folders, where I removed all ID3 Tags except for ALBUM, ALBUMARTIST, DISCNUMBER, GENRE, TRACKARTIST, TRACKNUMBER, TRACKTITLE,
and Album Art was also removed from every individual track. In other words, these tracks in a 2-level directory structure have only the basic tag data needed to play and use them within Media Player (assuming Tesla fixes other previously described bugs and requests.)
My Observation: Scan Time was approximately 8% faster without any embedded Album Art. Of note though: Tracks I randomly checked still showed Album Art after the scan was complete. This was how I found and starting investigating the oddities associated with
Tesla’s Implementation of Album Art.
GENERAL NOTES:
Twice in my 11 times attempting to scan my full 6829 library, I had instances where the scanning process went berserk. I’ve never encountered symptoms like this. Perhaps you have. My assumption is I've hit some sort of wall with memory usage, be that physically, or how it's used during scan -- or this whole Album Art thing can create another type of bug:
- Scanning began as normal. The "Loading (_%)..." indicator typically shows numbers incrementing over time, sometimes bouncing between 1 digit up/down I suspect because of odd rounding issues. Fine. In this case though during the initial scan, the Loading indicator constantly flipped between two numbers as their spread increased over time, e.g. 1% & 10%, 22% & 35%, 54% & 82%.
- Somewhere after 92% complete (when I last physically checked it), and then came back, I found the scanning process had restarted. It processed normally to completion.
- The scan reached 99%, then Loading started alternating with 100%, and within a few seconds, Media Player restarted the scan from zero. No CID reboot took place, but the scanning process was normal in the subsequent run.