Summary: The scatter plot I made below illustrates that Tesla has steadily shrunk the time intervals between AP2 software updates by almost 45% during 2017. In other words the mean time between autopilot updates was 80% greater during the first four months of 2017 than it has been during the most recent four week time period ending September 27 with the release of build 17.38. Scroll down for my more detailed explanation of the data and method after the charts.
Intro
As Mr. Twain said, "There are three kinds of lies: lies, damned lies and statistics." With that disclaimer out of the way I see good news here and this is fun to look at. While making an entry today for 17.38 in my AP2 tracker spreadsheet I noticed that the time between releases seems to be dropping. So for fun I averaged the time between releases (measured in weeks) for five time periods - starting with the first limited release on 12.13.16 up til now. The intervals were chosen by me to coincide with what seemed like significant dates or developments.
Method
These are obviously not contiguous time periods. The first graphed time period only contains releases prior to the video upload program. The last four graphed periods each start at the date mentioned and include all releases up til now. I did it this way to show that after Karpathy was hired the interval until the first release under his watch was a full four weeks - longer than the average intervals prior to his hiring, even though he was running the program under new conditions of a data firehose Tesla had never had before. Perhaps he was putting his own touches on method? All we know is that the mean intervals between release after Karpathy fired off the first update under his watch have dropped significantly - from four weeks for the first one to less than 2.5 for all releases after that - and that the last month has seen a further increase in update frequency - with the last three updates coming only 2 weeks apart. What this may indicate I don't know - but of course the fleet size is growing significantly each quarter. The AP2 total fleet size is now over 33% larger than when Karpathy first came on board - presumably the training data Tesla is receiving from the cars is coming in over 33% faster as well. Perhaps Tesla's progress rate is improving and so they are pushing out updates more rapidly. Perhaps they are simply working harder to meet some internal time goal for an announcement we don't know about yet. And of course the Model 3 production ramp is happening now. It seems reasonably that by the end of 2018 the AP2.X total fleet size will be 300-400% greater than it is now - and the data Tesla is receiving for reinforcement training will be coming in at a correspondingly increased rate. Or - perhaps nothing is happening and these are just meaningless data points.
Intro
As Mr. Twain said, "There are three kinds of lies: lies, damned lies and statistics." With that disclaimer out of the way I see good news here and this is fun to look at. While making an entry today for 17.38 in my AP2 tracker spreadsheet I noticed that the time between releases seems to be dropping. So for fun I averaged the time between releases (measured in weeks) for five time periods - starting with the first limited release on 12.13.16 up til now. The intervals were chosen by me to coincide with what seemed like significant dates or developments.
Method
- Data: My AP2 update tracker spreadsheet. I set dates for AP2 releases by looking at the earliest date I can find mention online of a particular build - either a thread on TMC or a media article.
- Sample size: Admittedly small - 12 total releases from 12/31/16 until now. However the confidence level as measured by an r squared calculation is reasonably high. I can make another chart showing every single release and then do a linear regression but I made this one using mean-interval-between-releases because it seemed fun to compare the frequency of AP2 releases against what might be significant developments in the program.
- Description of Intervals: The five periods in the chart are described in detail below as periods 1-5:
- Pd 1: 12/31 - 5/6/17: All five AP2 releases prior to the commencement of the video upload deep learning program that was announced on May 6. The first release of this period was Tesla's limited, low speed initial roll out and the period ended with the release of 17.17.4, speed now up to 90 mph - and the announcement that Tesla had begun uploading large video files from cars in the wild to aid in deep learning.
- Pd 2: 5/7 - now: All six releases after video uploads began until now - including the very first "silky smooth" June 10 (which was still prior to Karpathy joining the team as AP head).
- Pd 3: 6/21 - now: All releases after the hiring of Andrej Karpathy. About a month passed after his hiring on 6/21 until the first release under his watch on July 20 - "silky smooth V2" - or 17.28
- Pd 4: 7/20 - now: All releases after the first Karpathy release until now. If you remove the first release under his watch you
- Pd 5: 8/17 - 9/27: There have been four releases from 8/17 until now. What is interesting about the three intervals between these four releases is that each has been only two weeks apart.
These are obviously not contiguous time periods. The first graphed time period only contains releases prior to the video upload program. The last four graphed periods each start at the date mentioned and include all releases up til now. I did it this way to show that after Karpathy was hired the interval until the first release under his watch was a full four weeks - longer than the average intervals prior to his hiring, even though he was running the program under new conditions of a data firehose Tesla had never had before. Perhaps he was putting his own touches on method? All we know is that the mean intervals between release after Karpathy fired off the first update under his watch have dropped significantly - from four weeks for the first one to less than 2.5 for all releases after that - and that the last month has seen a further increase in update frequency - with the last three updates coming only 2 weeks apart. What this may indicate I don't know - but of course the fleet size is growing significantly each quarter. The AP2 total fleet size is now over 33% larger than when Karpathy first came on board - presumably the training data Tesla is receiving from the cars is coming in over 33% faster as well. Perhaps Tesla's progress rate is improving and so they are pushing out updates more rapidly. Perhaps they are simply working harder to meet some internal time goal for an announcement we don't know about yet. And of course the Model 3 production ramp is happening now. It seems reasonably that by the end of 2018 the AP2.X total fleet size will be 300-400% greater than it is now - and the data Tesla is receiving for reinforcement training will be coming in at a correspondingly increased rate. Or - perhaps nothing is happening and these are just meaningless data points.
