FSD v12.x (end to end AI)

[hobertg]

Not installing a version ahead of FSD Beta release has helped get FSD Beta in the past:

2022.40.4.1 am I screwed out of FSD beta?

Right, but presumably it has *some* version of FSD? I'm not aware of FSD versions being published - only FSD beta.

teslamotorsclub.com

A vehicle on 2022.40.4.2 had downloaded but not installed 2022.44.2. After delaying the install many times, the vehicle was eventually pushed 2022.40.4.10 as part of FSD Beta wide release. Installing a version "further ahead" (2023.44.1) of the expected end-to-end version (2023.38.10) likely restricts eligibility until 12.x is updated to 2023.44+. This could be a wait or might end up being not too long if part of this year's holiday update.

That is what I understood from past comments on forums which I why I will wait and not upgrade even if prompted for the 44 update.
Question - Is there any rhyme or reason to cars that get specific versions of FSD...like subscription based FSD vs fully paid for FSD? Or does it solely have to do with model/make/year, etc. because of hardware?

REALLY hoping V12 includes (and fixes) bringing back summon (or actually smart summon) and autopark which has never been available on my 2023 M3P without USS.

 

[jebinc]

Too Long? Don't Read!

Besides, don't you have any new software Vibrations (er, Versions) to test this evening?

Nope, I’m on the six month behind flagship FSDj branch. Won’t see a .44 branch for FSDj for another six months. Remember, flagship member benefit only!

 

[JHCCAZ]

Nope, I’m on the six month behind flagship FSDj branch. Won’t see a .44 branch for FSDj for another six months. Remember, flagship member benefit only!

May your flagship ever wave.

 

[Mardak]

I think that demo just shows their “end-to-end” planner in action. Which is an incremental change. They’ve rolled things in like this before - their lanes network, their occupancy network, etc. This just adds another facet (and gets rid of a lot of code).

Do you believe Tesla has gotten rid of most of the previous 300k+ lines of control code for the demo? Elon Musk said it's closer to 3k lines.

I can understand the desire of an incremental modular end-to-end approach of building off 11.x neural networks as practically it should allow for faster training of "just" another neural network on top of the existing perception outputs as opposed to learning from raw inputs. If this is the case and ignoring what Tesla has presented about vision foundation models, a trainable planner/control module learning from human examples should still drive quite differently from the existing rules-based approach.

This should still result in much smoother driving even if depending on mispredictions from existing perception networks such as false positive pedestrian mailboxes or incorrect lane guidance / counts / connectivites as instead of strictly following heuristics causing jerky movement or undesired lane changes, the end network could learn nuanced behaviors for when to ignore unreliable inputs.

 

[AlanSubie4Life]

It seems to me that to believe this, you have to specifically deny and disbelieve what Elon and Ashok explained: that there is no code for recognition of stop signs, traffic lights, lanes and so on.

No it would not. If I had to guess, I don’t think that planning network contains any of that. I guess maybe someday we will see.

Do you believe Tesla has gotten rid of most of the previous 300k+ lines of control code for the demo?

Yes. I think it was a demo of the planner!

I assume there are still plenty of lines of code! Maybe it’s only 3k for the planner. I have no idea whether Elon meant what he said. It’s very difficult to interpret it.

He said v12 would not be beta, yet it might well be v12 beta - and that’s completely consistent. Yet somehow people think it is not. So anyway probably the statement about 3k lines was accurate at the time.

What it most certainly does not mean is that there are only 3k lines of codes (obviously!).

I think everything is reasonably consistent. Obviously things may have changed a little over the last few months but overall probably pretty consistent.

 

[drtimhill]

Whole link here:

Tesla FSD v12 Rolls Out to Employees With Update 2023.38.10 (Update: Elon Confirms)

Tesla's FSD v12 rollout begins for employees, featuring AI enhancements, real-world navigation prowess, and a glimpse into autonomous driving's future.

www.notateslaapp.com

So what happens to v11.4.8?

V12 has no beta, so how long testing by employees will take?

hmm Elon didnt really say that. He said V12 was the one that would go out of beta, but that doesnt necessarily mean the first public release (12.1.1 ???). Indeed it would be absurd to do so given the magnitude of the change.

 

[drtimhill]

Do you believe Tesla has gotten rid of most of the previous 300k+ lines of control code for the demo? Elon Musk said it's closer to 3k lines.

I can understand the desire of an incremental modular end-to-end approach of building off 11.x neural networks as practically it should allow for faster training of "just" another neural network on top of the existing perception outputs as opposed to learning from raw inputs. If this is the case and ignoring what Tesla has presented about vision foundation models, a trainable planner/control module learning from human examples should still drive quite differently from the existing rules-based approach.

This should still result in much smoother driving even if depending on mispredictions from existing perception networks such as false positive pedestrian mailboxes or incorrect lane guidance / counts / connectivites as instead of strictly following heuristics causing jerky movement or undesired lane changes, the end network could learn nuanced behaviors for when to ignore unreliable inputs.

I suspect there is something else going on here. 300,000 lines of code is not a lot these days (one project I work on has 11 MILLION lines). It's not the quantity, it will be the quality. My guess is that the code base has been hacked on for so long and is now such a mess that making changes without regressions and unexpected changes to the cars driving has become incredibly hard. Witness some of the odd changes we've seen in the 11.x cycle of releases.

So the project reached a fork in the road. Either re-factor the C++ code (major work, possibly another dead-end), or migrate the driving effort to to NNs, reducing the C++ code to manageable proportions and the stuff it HAS to do (interfaces to the hardware, the math and calculus needed for smooth driving etc). Since Tesla has built up massive experience in NNs, but (presumably) less so in large-scale C++ OO projects, they chose a path that was more comfortable to them (and more sexy for Elon).

 

[Todd Burch]

I suspect there is something else going on here. 300,000 lines of code is not a lot these days (one project I work on has 11 MILLION lines). It's not the quantity, it will be the quality. My guess is that the code base has been hacked on for so long and is now such a mess that making changes without regressions and unexpected changes to the cars driving has become incredibly hard. Witness some of the odd changes we've seen in the 11.x cycle of releases.

So the project reached a fork in the road. Either re-factor the C++ code (major work, possibly another dead-end), or migrate the driving effort to to NNs, reducing the C++ code to manageable proportions and the stuff it HAS to do (interfaces to the hardware, the math and calculus needed for smooth driving etc). Since Tesla has built up massive experience in NNs, but (presumably) less so in large-scale C++ OO projects, they chose a path that was more comfortable to them (and more sexy for Elon).

Watch videos from Andre Karpathy from many years ago where he talked about “Software 2.0” eating away at “Software 1.0”. The switch away from C++ code to the v12 approach was in Tesla’s longer-term game plan many years ago. This wasn’t because of some frustration reached trying to maintain the existing codebase.

 

[MP3Mike]

Do you believe Tesla has gotten rid of most of the previous 300k+ lines of control code for the demo? Elon Musk said it's closer to 3k lines.

Huh?

No, he says they will remove 300k:

Where are you seeing 3k?

Or are you trying to say that they dropped it from 300k to 3k?

 

[jebinc]

Huh?

No, he says they will remove 300k:

View attachment 994077

Where are you seeing 3k?

Or are you trying to say that they dropped it from 300k to 3k?

Remove 300k of old code and add 500k of new, perhaps….

 

[Grifon]

Remove 300k of old code and add 500k of new, perhaps….

If you take Elon at his word, two orders of magnitude reduced from a starting point of ~300,000 lines of code is about 3,000 lines of code (300,000 / 10 / 10). The LLM’s like ChatGPT have a very small chunk of control code, the bulk of the size of the entire package is the parameters.

It will be interesting to see how Tesla achieves their version of a ‘LLM’ that results in FSD.

 

[AlanSubie4Life]

Or are you trying to say that they dropped it from 300k to 3k?

This. He was asking me whether I thought this was what was happening.

I think it is. It’s the big incremental change for v12 Beta - replacing this code with the NN planner as described in the video! Similarly significant to the massive occupancy network enhancement, which nearly led to reaching the first 9 on ULTs, and probably a bit more significant than the lanes network (though the lanes network may have been an underrated improvement!).

Of course, that is just for that piece of vehicle control code, as Elon says.

There are still certainly many thousands of lines of code elsewhere (no idea how many - Elon has never indicated the quantity there). There are very likely even other pieces of “vehicle control” code.

 

[Krakatoa]

They can code all they want but it won't fix the hardware issues/limitations with vison based FSD.

 

[Captkerosene]

Best to manage and set expectations low, given Elon's track record here....

Why wouldn't you release V12 if it's better than V11? People seem to want to wait until V12 is fully autonomous to see it released. Elon has repeatedly released FSD that is far from perfect because it's still better than nothing. V12 won't take away the requirement that the driver closely monitor the car and keep their hands on the wheel ... at first.

 

[jebinc]

Why wouldn't you release V12 if it's better than V11? People seem to want to wait until V12 is fully autonomous to see it released. Elon has repeatedly released FSD that is far from perfect because it's still better than nothing. V12 won't take away the requirement that the driver closely monitor the car and keep their hands on the wheel ... at first.

Well, as the v11 releases have shown us, subsequent releases are actually worse than prior ones....

 

[stopcrazypp]

Huh?

No, he says they will remove 300k:

View attachment 994077

Where are you seeing 3k?

Or are you trying to say that they dropped it from 300k to 3k?

I think he is referring to the part about ~2 orders of magnitude which means 100x reduction or about ~3k.

 

[Tronguy]

I think he is referring to the part about ~2 orders of magnitude which means 100x reduction or about ~3k.

That is, a reduction from 300k lines about 3k lines. That's a 99% reduction or thereabouts, nominally 297k lines gone.

 

[EVNow]

I think that demo just shows their “end-to-end” planner in action. Which is an incremental change.

Noway it's incremental. It is fundamentally different

 

[Phlier]

Too Long? Don't Read!

Besides, don't you have any new software Vibrations (er, Versions) to test this evening?

Aye, it was a long read, but well worth the time.

Sorry... the family is on a Pirates of the Caribbean movie marathon. Only three nights remaining.

Seriously, though, that was a very well done post.

 

[Supcom]

Noway it's incremental. It is fundamentally different

One person's rewrite is another's increment?

I am curious how visualization works with E2E. If the car is taking in photons and outputting vehicle control, with no teaching for identifying specific things like stop signs, other vehicles, etc, how will the system create the visualization? Perhaps some of the legacy perception NNs are kept to run alongside the E2E stuff?