FSD v12.x (end to end AI)

[diplomat33]

https://twitter.com/x/status/1697037944387981628

I remember when occupancy networks were first announced, they were touted as the next big game changer in FSD. Now E2E is the next big game changer and occupancy networks get tossed aside like yesterday's news.

Does anything think Tesla will release V12 within the next 5 years?

Yes. Tesla will likely releaseV12 as "beta" first, like they've done with every release. My guess is V12 will probably go to "early access" release some time next year.

 

[jeewee3000]

I remember when occupancy networks were first announced, they were touted as the next big game changer in FSD. Now E2E is the next big game changer and occupancy networks get tossed aside like yesterday's news.



Yes. Tesla will likely releaseV12 as "beta" first, like they've done with every release. My guess is V12 will probably go to "early access" release some time next year.

You are misinterpreting what Ashok says regarding Occupancy Network.

Whilst it is "not really Occupancy Network" anymore, it is now an entire world model that they want to base the planner on. This world model includes some form of what traditionally was the Occupancy Network.

In other words: the term "Occupancy Network" does not do justice to the world model since the Occupancy Network is a very purpose-built NN that takes the camera input and turns it into a 3d voxel representation of the world in order to allow distance perception.

The world model they train with v12/E2E includes distance perception and therefore the Occupancy Network is no longer needed.

It is therefore not "tossed aside", it has grown into something bigger and better and was an essential step in the learning process.

 

[JB47394]

Is there anywhere where the roadmap for this to happen is actually agreed yet? Presumably it would be on a state by state basis in the US, potentially with different criteria in each state?

Heh. Imagine a cop seeing someone on a highway eating their lunch in the driver's seat. Do you pull them over or not? Is that particular car operating on L3+ or are they using their knees?

 

[jeewee3000]

Heh. Imagine a cop seeing someone on a highway eating their lunch in the driver's seat. Do you pull them over or not? Is that particular car operating on L3+ or are they using their knees?

Belgian attorney chiming in.

Over here they couldn't give you a ticket for that, unless you were swerving in your lane.

The rule isn't "you must hold the wheel", the rule is "you must have full control over your vehicle at all times".

When I'm on autopilot and AP does a good job, I am (indirectly) in control. (of course the perception of the cop is different in this day and age, so I do think it's better to at least appear like you're holding the wheel even with an L3 system.)

 

[EVNow]

Running red lights is a subset of 'watch out for other cars'. If collision avoidance is implemented, then the only additional step might be reduced speed or increased vehicle spacing in higher risk areas (crossing intersections).

True, but, its more complicated.

"Watchout" for other cars and collision avoidance are the reason why we have "phantom braking" and lots of hesitation. In general, NN should be able to figure out when to expect the other car to brake and stop (stop lights, red lights, yields, roundabouts etc) so ego doesn't need to reduce speed. But in all those cases there is a small chance that the other car will not stop (ie.e. when the other car is going to run red lights, stop signs etc) when the ego needs to defensively stop.

To be able to differentiate these conditions, NN needs a lot of training on the normal case (other cars stopping) and cars doing the wrong thing (may be oversampled).

Doing this for pedestrians would be even more difficult - since their action depends on what the car does.

 

[mongo]

True, but, its more complicated.

"Watchout" for other cars and collision avoidance are the reason why we have "phantom braking" and lots of hesitation. In general, NN should be able to figure out when to expect the other car to brake and stop (stop lights, red lights, yields, roundabouts etc) so ego doesn't need to reduce speed. But in all those cases there is a small chance that the other car will not stop (ie.e. when the other car is going to run red lights, stop signs etc) when the ego needs to defensively stop.

To be able to differentiate these conditions, NN needs a lot of training on the normal case (other cars stopping) and cars doing the wrong thing (may be oversampled).

Doing this for pedestrians would be even more difficult - since their action depends on what the car does.

Sure, but safety/ defensive driving is based on not expecting others to do the right thing. However, excessively high levels of safety yeild low levels of satisfaction. There's always a chance.

Relatedly, I recall Tesla training against vehicles that changed lanes without signalling.

 

[EVNow]

Sure, but safety/ defensive driving is based on not expecting others to do the right thing. However, excessively high levels of safety yeild low levels of satisfaction. There's always a chance.

Relatedly, I recall Tesla training against vehicles that changed lanes without signaling.

I think training to strike the right balance is going to be very difficult. Most humans ignore low probability events (and thus) get into a lot of accidents. Hopefully AVs can eventually get better without being annoyingly conservative.

Its one thing for everyone to be excited about how NN can actually drive without hand coded planning code - but entirely different when it comes to ensuring no more than 1 mistake per year of driving.

 

[Knightshade]

How does the software even become >L2, in practice? I don't mean its technical capabilities, I'm talking regulation.

Let's say Tesla suddenly decide it's ready and they'd be prepared to take liability for any accidents that happen while the software is engaged. For that to happen there'd have to be modifications to the legal framework - which will currently be written such that the meat in the seat is always in charge of the vehicle and accountable for its actions - and presumably there needs to be some sort of certification process that some regulatory authority would need to go through, much like how Cruise/Waymo etc keep renegotiating the parameters of their permission in SF.

Is there anywhere where the roadmap for this to happen is actually agreed yet? Presumably it would be on a state by state basis in the US, potentially with different criteria in each state?

There's already a BUNCH of states where what you wrote is not true and you can put an L3, or L4, or L5 vehicle on the road today without "approval" from anyone.

In Nevada for example you just have to file a 2 page paper that essentially says "Our system can self drive and we have insurance for it. Trust me bro" and that's it.

 

[Knightshade]

I feel like the NHTSA will have a field day with non-beta full self driving software that can’t full self drive. It can hide behind the beta tag now, but removing that tag is asking for trouble in my opinion as it means it’s finished software working as promised.

Beta has no legal meaning to "hide" behind.

Multiple people (GM and Ford) have entirely hands-off L2 systems already.

So not sure what you're talking about here.

 

[Buckminster]

I think training to strike the right balance is going to be very difficult.

Are you suggesting more training is not necessarily better?

 

[fasteddie7]

Beta has no legal meaning to "hide" behind.

Multiple people (GM and Ford) have entirely hands-off L2 systems already.

So not sure what you're talking about here.

Ford and GM don't call their system Full Self Driving. Would you not agree the name implies that the vehicle will drive itself, and with musks promises since 2016, removing the beta tag would lead the general public to believe that that promise is now fulfilled, heck, people that don't follow Tesla and obsess over the software think that now. I'm simply saying that musk might be inviting more investigations for himself and Tesla when the inevitable headlines "Full Self Driving out of beta" hit the web and YouTube.

 

[hooty]

I'd fully expect it WAY before then.

The far more relevant question is does anyone think V12 will offer >L2 operation on HW3- and I'm pretty firmly in the no camp on that one.

How the heck is Musk going to get it out of beta at such a low SAE level?

 

[JB47394]

The rule isn't "you must hold the wheel", the rule is "you must have full control over your vehicle at all times".

The assumption is that if I'm in an L3+ car then I can legally be preoccupied with my meal. So how is a police officer going to know that my obvious lack of control is, in fact, legal?

 

[Mullermn]

There's already a BUNCH of states where what you wrote is not true and you can put an L3, or L4, or L5 vehicle on the road today without "approval" from anyone.

In Nevada for example you just have to file a 2 page paper that essentially says "Our system can self drive and we have insurance for it. Trust me bro" and that's it.

Really? So they have, for example, modified their drink drive laws to say that if you’re in an L5 vehicle then you’re not drink driving?

 

[jeewee3000]

The assumption is that if I'm in an L3+ car then I can legally be preoccupied with my meal. So how is a police officer going to know that my obvious lack of control is, in fact, legal?

Maybe L3 will have a bumper sticker

 

[Knightshade]

Really? So they have, for example, modified their drink drive laws to say that if you’re in an L5 vehicle then you’re not drink driving?

Careful you're gonna hurt your back moving the goalpost that far


You just went from wrongly claiming "presumably there needs to be some sort of certification process that some regulatory authority would need to go through" to "BUT WHAT ABOUT DRUNK DRIVING??"

But also you're still wrong even then

Here's Nevada state law 482A.200 for example that makes it clear the CAR is driving, not the human.

 Application of motor vehicle and traffic laws to certain fully automated vehicles; human driver not required under certain circumstances.  No motor vehicle laws or traffic laws of this State shall be construed to require a human driver to operate a fully autonomous vehicle which is being operated by an automated driving system. The automated driving system of a fully autonomous vehicle shall, when engaged, be deemed to fulfill any physical acts which would otherwise be required of a human driver except those acts which by their nature can have no application to such a system.

So the human is not driving when such a system is engaged, and could not be charged with any crime that involves them driving- since they aren't.

 

[Knightshade]

How the heck is Musk going to get it out of beta at such a low SAE level?

Beta has no legal meaning, nor any regarding SAE driving level- so your question does not make any sense.

Ford and GM don't call their system Full Self Driving. Would you not agree the name implies that the vehicle will drive itself,

Does "diaper genie" imply it grants wishes?

Does "Happy Meal" imply it cures depression?

Exactly what FSD gets you is spelling out with a list of specific features right on the purchase page.

All of which tons of folks have been enjoying in their car in fsdb for years now.

Drop the b and nothing else needs to change at all.

 

[fasteddie7]

Beta has no legal meaning, nor any regarding SAE driving level- so your question does not make any sense.





Does "diaper genie" imply it grants wishes?

Does "Happy Meal" imply it cures depression?

Exactly what FSD gets you is spelling out with a list of specific features right on the purchase page.

All of which tons of folks have been enjoying in their car in fsdb for years now.

Drop the b and nothing else needs to change at all.

The average person isn’t going to see it that way. If I walk up to 20 people on the street and ask what does the “Full self driving” mode on a Tesla do, I’ll bet they have a different thing in mind. As far as happy meals and diaper genies are concerned, the name does have meaning, when the happy meal was introduced in 1979 its name came from the parents in the family being happy that the kids had something to do and play with while the family had a meal. And the diaper genie was renamed by playtex for the form factor and implied a “magical” solution that played on words and “granted the wishes” of parents to not have a stinky trash can. A product name, although not always literal, can certainly generate thoughts and ideas, regardless of how true, of what that product will do for you.

And as far as its capabilities being listed on the product page. I remember when “pending regulatory approval” your car would go out and make money for you after it dropped you off being listed on there as a feature. It wasn’t until many years passed that they got more skiddish on their description, but who reads that beside people like us anyway? tmc, Reddit, twitter are all bombarded with posts that ask the same question over and over with things that are not only on the page, but in some cases on the screen. how many “what percentage should I charge to?” and “no regen when it’s cold, weird dotted line on dash, what is it?” posts are on this site alone?

 

[Mullermn]

Careful you're gonna hurt your back moving the goalpost that far

I don’t think I am, my original question was about whether a L4/5 Tesla system could actually be used as such in practice if the software became ready.

Perhaps you’re right, maybe Nevada is very progressive and has very simple motoring laws to start with. In the UK for example we have a law on the books that makes it an offence to be in charge of a vehicle if you’re drunk - people get prosecuted for sleeping in their car if they are in possession of the keys, for example (I imagine in practice there are aggravating factors too). Such a law would definitely still apply without modification.

 

[hooty]

Beta has no legal meaning, nor any regarding SAE driving level- so your question does not make any sense.

So what is the purpose of it?
Why designate it as such?
Why use it at all?