2020.40.x and Autopilot Rewrite

[MrBadger]

Some contradictions that I am trying to get to the bottom of. Its not helped by me having not test driven any variant of 2020.40.x so don't have my own reference point.

The FSD Beta Release is 2020.40.8.10 - believed to be based upon the Autopilot rewrite ('the rewrite').

Yesterday took 2020.48.8 (not to be confused with FSD beta 2020.40.8.10), up from 2020.40.4 a week or so before. The download seems to be exceptionally quick, so the belief is that 2020.40.8 is a very minor update from 2020.40.4.

I'm trying to find at what version 'the rewrite' landed, if it has indeed landed. Why, because reports of phantom braking are still rife.

The evidence for the rewrite being in 2020.40.x
Oct 8 - EM says that phantom braking should be fixed in this release https://twitter.com/EVHQ2/status/1314287584105779200 On this date, 2020.40.x was already rolling. Either fixed by some targeted solution, or as a consequence of the rewrite?
Oct 20 - FSD beta started to roll to select users
Oct 21 - Andre Karpathy (Tesla software) in response to being asked how the rewrite was going said that "many months of hard work finally land in production builds!" https://twitter.com/alexhorner2002/status/1318984153031974914
Oct 22 - @verygreen suggested that "recent dev builds that were same nn-wise as current prod builds which leads me to believe this is likely no NN change," in other words, the FSD beta is same as current production (2020.40,8?) except for some additional features and not a completely different code base (the rewrite) imho, you would expect the rewrite to need a different NN.
Various - AP is smoother - but this has been the case over many builds

The evidence against the rewrite being in 2020.40.x
Lots of reports of phantom braking - similar to 2020.36.x which I rated poor on this count
Surely there would have been more of a fanfare?
Not the step change improvement that I/we were expecting?
@verygreen could be wrong on this occasion
Could we have had the rewrite before 2020.40? Some early speculation that 2020.36 was it, but soon dispelled.

 

[NorfolkMustard]

Personally I think there’s a big if-then-else right at the top of the code. If you can flick the FSD switch which is visible in 2020.40.8.10 settings then you get the rewrite. Otherwise you get the entirely old branch.

I don’t think 2020.40.8 is the rewrite with parts disabled to look like pre-rewrite, it just is the pre-rewrite code base entirely, with the rewrite branch sitting there in parallel but entirely hidden behind an option switch which is only visible in 2020.40.8.10

 

[MrBadger]

That makes sense, although I would have thought that having two completely different systems (even if a functional toggle) would be somewhat overkill - maybe its to give a fallback if needed? I guess with 2 FSD chips, its possible to do 2 different things and toggle between the two.

I know its only just conjecture, but do you think the switch is only for city driving or it includes highways too? From what I have seen, highways and city present driver visualisation very different (old vs new/developer mode). I'm yet to see any significant non city driving footage for 2020.40.8.10 to be able to see if any improvements there (I have seen what is touted as an improved summon) and it is that which I can currently relate to most. Trying to gauge phantom braking, cornering, on/off ramp etc. The sort of stuff we are use to seeing.

 

[Glan gluaisne]

Would it make sense to have the old FSD visualisation preview for all non-FSD cars? I got the feeling that the FSD visualisation preview was really a bit of an advert for FSD, to show those who'd not opted to buy it, what they were missing. If non-FSD cars are going to show the old visualisation, then that doesn't seem to be much of an enticement to fork out north of £8k for the new FSD.

 

[verygreen]

Could we have had the rewrite before 2020.40? Some early speculation that 2020.36 was it, but soon dispelled.

step zero is you need to clearly define what do you mean by "the rewrite"?

2020.40.8.10 is just the stuff that firstappeared in 2019.40.50 with "fsd preview", it just added BEV shim NN layer under hydranet NNs (that got direct input from cameras) It still works on a single frame at a time of camera input at the hydranet layer. hydranet itself first came into prod firmwares in September 2018. AK did presentation in Februrary about this hydranet+BEV nets and what we see in that presentation today 100% matches what's in the firmware code.
the EAP added conventional c++ code module named city_streets that also appeared in dev firmwares around 2019.40 but so far was absent in prod firmwares - the responsibility of this module is to drive on city streets, track priority of other cars around you and so on.

So NN-wise 2020.40.8.10 differences are very minimal from 2020.40.8

Because all this stuff appeared way before Elon mentioned the "fundamental rewrite" in January 2020 and there was supposed to be a March/April hackathon to actually drive the rewrite and also a brief appearance of a new NN architecture called "plaidnet" in 2020.8-2020.12 (after which it disappeared aftere I tweeted about it - a good sign they did not want it to be known) - I am assuming THIS was the fundamental rewrite Elon spoke about and this is what I mean by "the fundamental rewrite". 2020.40.8.10 does not have any of the plaidnet stuff.

 

[Dilly]

I drove a fair bit on 36.12, had 40.4 for a week and 40.8 a couple of days ago.
NoA on 40.4 seemed much smoother and off ramps better anticipated. Slowing for bends and junctions much improved.
Under 36.12 auto lane change worked on any duel carriageway if the wheel was nudged first. In 40.4 this has gone and is now restricted to where the screen shows it as useable. Haven’t had the opportunity to drive much under 40.8.
It’s quite conceivable that common coding is hidden at present and will be unleashed at the appropriate time

 

[MrBadger]

step zero is you need to clearly define what do you mean by "the rewrite"?

Thanks for your response. 'The rewrite' is "the fundamental rewrite" EM mentioned earlier in the year, ie '4D' labelling from all cameras for HW3 which I think is what you refer to as 'plaidnet' below.

the rewrite and also a brief appearance of a new NN architecture called "plaidnet" in 2020.8-2020.12 (after which it disappeared aftere I tweeted about it - a good sign they did not want it to be known) - I am assuming THIS was the fundamental rewrite Elon spoke about and this is what I mean by "the fundamental rewrite". 2020.40.8.10 does not have any of the plaidnet stuff.

Unless I am mistaken from this, I take it that 'the (fundamental) rewrite' is still to come and not yet playing a part in the behaviour of the FSD beta/city streets? That would be even more intriguing if that was the case. Was it ever actively involved in production behaviour in 2020.8-2020.12 ? Any suggestion when/if it will/did become involved in production vehicle behaviour?

Just trying to see if some recent behaviour regression, in particular increased occurrences of Phantom braking (getting bad), or poor behaviour of new functionality, in particular misunderstanding the relevance of speed signs (picking up speed signs in adjacent streets etc) et al. is likely to be dramatically improved upon by a future step change coming from 'the (fundamental) rewrite'.

 

[mikes_fsd]

The evidence for the rewrite being in 2020.40.x
Oct 8 - EM says that phantom braking should be fixed in this release https://twitter.com/EVHQ2/status/1314287584105779200 On this date, 2020.40.x was already rolling. Either fixed by some targeted solution, or as a consequence of the rewrite?
Oct 20 - FSD beta started to roll to select users
Oct 21 - Andre Karpathy (Tesla software) in response to being asked how the rewrite was going said that "many months of hard work finally land in production builds!" https://twitter.com/alexhorner2002/status/1318984153031974914
Oct 22 - @verygreen suggested that "recent dev builds that were same nn-wise as current prod builds which leads me to believe this is likely no NN change," in other words, the FSD beta is same as current production (2020.40,8?) except for some additional features and not a completely different code base (the rewrite) imho, you would expect the rewrite to need a different NN.
Various - AP is smoother - but this has been the case over many builds

Contrary to what green or others might say about the rewrite.
In 1/2020 Musk on 3rd Row Podcast said "foundational rewrite of the Autopilot system that is almost complete" I detailed in a post why HydraNet and BEV Net with Temporal Layer and Fusion Layer is the culmination of the rewrite in this thread Blog - Musk Touts 'Quantum Leap" in Full Self-Driving Performance

Think, how long would a "foundational rewrite of the Autopilot system that is almost complete" take? Then if it was almost complete in Jan 2020 (via Musk) what would be the likely timeline? HydraNet appeared in 2018 and BEV Net in 2019.

 

[NewbieT]

Having the core components of a new operating structure is different to having them production ready. Elon is notorious for being slightly optimistic on timescales. ‘Nearly ready’ in January does not mean they rolled it out in say March.

The other thing here is the question about how much of the system got re-written. If you think of FSD being a load of building blocks that plug together, how much is a fundamental re-write, replacing half of them? All of them?

 

[mikes_fsd]

The other thing here is the question about how much of the system got re-written. If you think of FSD being a load of building blocks that plug together, how much is a fundamental re-write, replacing half of them? All of them?

The fundamental rewrite is more about Karpathy's vision of Software 2.0
The point is that the fundamental rewrite really started with Karpathy taking over the AI team at Tesla, first step was HydraNet (2018), then BEV Net (2019) then Temporal Module...
That is also why Musk can say "almost complete" in Jan 2020.

NN's absorbing more and more of the functionality that was in the Software 1.0 stack.
I talk about that in another post here: Blog - Musk Touts 'Quantum Leap" in Full Self-Driving Performance