FSD v12.x (end to end AI)

[Mardak]

Teslascope states that FSD V12 went to less than 100 managers and execs

If these numbers are somewhat accurate of initial smaller "true early access" group of 100 employees most likely not breaking NDA testing 12.0, and now around 3000 with 12.1, there's still around another 3x for "Wave1" employees that are probably included in what we've been able to notice as a vehicle on TeslaFi getting updates earlier. It sounds like this is a perk for Tesla employees to opt-in to earlier vehicle software updates, so presumably still under NDA but not as strictly enforced.

Customer vehicles that still have the FSD Beta video snapshot button would probably be in the subsequent rollout phase if no blocking issues are found.

 

[Mardak]

The issue is - sometimes we don’t know what FSD might do and brake to prevent an accident. Happens with me a lot at roundabouts- FSD will happily try to enter the roundabout even when a can is coming from the left.

Presumably Tesla has a map of which locations tend to have more disengagements even if caused by driver not feeling comfortable with FSD Beta 11.x as opposed to actual safety issue. These could then be aggregated from specific locations to types of situations, e.g., roundabout nearby, no traffic, pedestrian detected; and as 12.x rolls out wider, they can reuse that same analysis to compare where things are getting better or regressing. I suppose even similar to geofencing to a mapped area, could Tesla define specific thousands (millions?) of road segments as those that have seen under X disengagements per Y vehicles as a map of where they know FSD Beta is safest?

 

[Nakk]

Newer builds do have camera and interior illumination.
However, hands on the wheel will always be better reaction time wise than off wheel.
Model Y:
View attachment 1002872

Interesting, but I only see the camera part numbers there? My nephew just bought a new build Y, I see no sign of an infrared light... Of course, I could have overlooked it.

 

[Nakk]

Perhaps, but that doesn’t explain why they can’t use the regular camera during the daylight.

An IR light during daylight hours would illuminate the driver's eyes through sunglasses. This whole thing is simply a guess on my part based on other hands free ADAS systems having an IR light and camera.

 

[Supcom]

Interesting, but I only see the camera part numbers there? My nephew just bought a new build Y, I see no sign of an infrared light... Of course, I could have overlooked it.

When in sentry mode at night, simply use the phone app to access the cabin camera. If the car has an IR illuminator, you will see a well illuminated interior.

 

[EVNow]

Presumably Tesla has a map of which locations tend to have more disengagements even if caused by driver not feeling comfortable with FSD Beta 11.x as opposed to actual safety issue. These could then be aggregated from specific locations to types of situations, e.g., roundabout nearby, no traffic, pedestrian detected; and as 12.x rolls out wider, they can reuse that same analysis to compare where things are getting better or regressing. I suppose even similar to geofencing to a mapped area, could Tesla define specific thousands (millions?) of road segments as those that have seen under X disengagements per Y vehicles as a map of where they know FSD Beta is safest?

Yes they should - but do they … no idea.

Anyway, my point was - everyone should disengage when they feel unsafe - irrespective of whether the car would have actually gotten into an accident or not. All drivers around you and VRU should feel safe - otherwise FSD won’t work. Human drivers leave quite a bit of tolerance and FSD should too, to feel safe.

 

[DrChaos]

That is what we are seeing with robotaxis. I think for consumer L4, we are likely to see ODDs based on road types rather than geographical areas. Shashua argues that consumer "eyes off" needs to be "everywhere" since consumers likely want to go everywhere in their personal car. So Mobileye has defined some standard ODD for consumer "eyes off" based on road types, rather than geographical locations:

Obviously, this list of ODDs is not exhaustive but I think it gives us a nice starting point. I do think that weather and speed limits will also be ODD limits. So I think we will see consumer L4 that is limited to certain road types, weather and speed limits.

I think in practice Rural is easier than Arterial --- it's complex city intersections with lots of traffic and multiple moving streams which seems to confuse FSDb for me the most, even if some of them are technically divided. But full winding roads where there isn't much choice other than 'stay on the road' works well.

 

[diplomat33]

I think in practice Rural is easier than Arterial --- it's complex city intersections with lots of traffic and multiple moving streams which seems to confuse FSDb for me the most, even if some of them are technically divided. But full winding roads where there isn't much choice other than 'stay on the road' works well.

I agree. I suspect Mobileye's decision to put urban driving as the last ODD is more of a business/marketing decision. There are more rural roads than urban streets. So by doing rural before urban, they can offer driving in more areas sooner and tackle the harder ODD (urban) last. It is an ongoing debate whether it is better to go wide on the easier ODD first and do the hardest ODD last or tackle the hardest ODD first so that you are left with the easiest ODD afterwards. We've seen robotaxi companies like Waymo and Cruise tackle the hardest urban first, figuring that if they can solve it first, the rest will be easier. Plus, the majority of the potential customers for robotaxis live in urban areas. So there is logic in robotaxi companies focusing on urban first. Companies focusing on consumer cars tend to focus on the easier first because then they can offer a useful ADAS to more consumers sooner.

 

[Mardak]

Human drivers leave quite a bit of tolerance and FSD should too, to feel safe

Yeah, it'll be interesting how end-to-end handles safety in similar but different low and high complexity situations. For example, merging into a roundabout when there's just one other car with right of way, FSD Beta doesn't need to follow very closely, but I'm sure there's probably some training examples where human drivers become impatient at a very busy roundabout squeeze into a small gap, and will 12.x behave differently? Even if end-to-end learns to wait, there will likely be situations where the driver presses the accelerator to get in and the neural networks need to have enough training to complete the maneuver even if it otherwise wouldn't have gone ahead. Or even more extreme, the driver made a mistake pressing the accelerator not realizing there's a pedestrian approaching, so will end-to-end handle automatic emergency braking?

Generally, this has also been an issue for large language models and image generation where only providing "safe" content during training could limit the model's general understanding, so does end-to-end want to have safe and unsafe examples so that it can prefer safe behaviors and understand how to get out of unsafe situations?

 

[DrChaos]

I agree. I suspect Mobileye's decision to put urban driving as the last ODD is more of a business/marketing decision. There are more rural roads than urban streets. So by doing rural before urban, they can offer driving in more areas sooner and tackle the harder ODD (urban) last. It is an ongoing debate whether it is better to go wide on the easier ODD first and do the hardest ODD last or tackle the hardest ODD first so that you are left with the easiest ODD afterwards. We've seen robotaxi companies like Waymo and Cruise tackle the hardest urban first, figuring that if they can solve it first, the rest will be easier. Plus, the majority of the potential customers for robotaxis live in urban areas. So there is logic in robotaxi companies focusing on urban first. Companies focusing on consumer cars tend to focus on the easier first because then they can offer a useful ADAS to more consumers sooner.

I agree with you but I disagree with the ranking of Arterial as being easier than Rural, as was implied by the MobileEye figure. Arterial blends in with Urban, and often has higher speeds. I don't think there is a strong distinction there---unless they are intentionally geofenced manually to easy parts.

At slow speeds it's probably easier to be don't-crash-safe but hard to be don't-be-a-dick-while-driving in complex urban.

 

[EVNow]

For example, merging into a roundabout when there's just one other car with right of way, FSD Beta doesn't need to follow very closely, but I'm sure there's probably some training examples where human drivers become impatient at a very busy roundabout squeeze into a small gap, and will 12.x behave differently?

I think this is one area where FSD / manual coding is bad. There are a lot of situations with varying degrees of traffic in which people behave differently. If the roundabout is very busy you have to squeeze in. If it's mostly empty you wait. You know which ones are busy not just by scanning at the traffic but by experience. V12 can't get the experience part .. only scanning at best.

This kind of thing applies in many conditions.. like unmarked roads. If there are a lot of parked cars, you want to stay in the middle and if there is a lot of traffic you want to stay at the edge ... when you have both, you need to figure out the best method opportunistically. May be this one V12 can work out.

Applies also to merging to the exit lane on freeways .. too early you get stuck, too late you may miss the exit. Experienced taxi drivers tend to merge quite late ... I might merge earlier. V12 have to work it out based on traffic info ?

 

[clydeiii]

https://twitter.com/x/status/1740119464870555830

 

[Mardak]

Looks like TeslaInfo had previously detected 2023.44.30.10 on Sunday while StatsApp newly detected it today, so maybe that was newly detecting ~3k employees expanding rollout to ~15k. Nothing yet on TeslaFi nor Teslascope publicly.

 

[Scott Chapman]

Looks like TeslaInfo had previously detected 2023.44.30.10 on Sunday while StatsApp newly detected it today, so maybe that was newly detecting ~3k employees expanding rollout to ~15k. Nothing yet on TeslaFi nor Teslascope publicly.

feels like it would need to be a 2024.x.x release at this point?

 

[powertoold]

If V12 is better than 11.4.9 at release, my mind will be blown

When Tesla finally released single stack, it was already better than regular NoA

 

[AlanSubie4Life]

If V12 is better than 11.4.9 at release, my mind will be blown

If it can handle stop signs, my mind will be blown. Specifically if: it can come to a sensible brisk complete stop without using the brakes except possibly below 1-2mph. And 2) actually start moving immediately after stooping, rather than WTFing.

These things seem easy, but many years of engineering and physics-based models have not managed it. Will AI blow our minds?

How will they train this, though?

 

[JHCCAZ]

...Specifically if: it can come to a sensible brisk complete stop without using the brakes except possibly below 1-2mph. And 2) actually start moving immediately after stooping, rather than WTFing.
...
How will they train this, though?

Originally they implied that they would have to seek out rare clips of people coming to a full stop, but pointed out the paradox that such an outlier group isn't representative of the desired "good driver" cohort.

My opinion is that in practice they'll need to get most of this training, for a targeted but rare behavior, from simulated clips rather than real recorded ones.

(For me, 11.4.9 still dawdles too much, especially when it stops had to stop sign placed well back of the actual intersection line. However, it seems to be a little quicker off the stop.)

 

[MP3Mike]

And 2) actually start moving immediately after stooping, rather than WTFing.

You know that won't happen as the NHTSA made Tesla add the delay in a recall.

 

[AlanSubie4Life]

You know that won't happen as the NHTSA made Tesla add the delay in a recall.

They just required the vehicle to stop, didn’t they? I don’t think any insane delay was required. But maybe I am not fully up to speed on this.

I absolutely think the vehicle should stop. It’s not hard to do it right. Braking to bring to a complete halt executed at about 1mph may well be needed. Because regen only actually takes a while (though it is very low jerk - jerk increase from optimal will be required, though will be lower than current FSD stopping implementation ).

Of course, nothing about the current stopping behavior resembles this.

 

[stopcrazypp]

They just required the vehicle to stop, didn’t they? I don’t think any insane delay was required. But maybe I am not fully up to speed on this.

I absolutely think the vehicle should stop. It’s not hard to do it right. Braking to bring to a complete halt executed at about 1mph may well be needed. Because regen only actually takes a while (though it is very low jerk - jerk increase from optimal will be required, though will be lower than current FSD stopping implementation ).

Of course, nothing about the current stopping behavior resembles this.

What you perceive (or most people perceive) as an "insane" delay may just be what NHTSA feels is necessary to count as a "complete stop".

My understanding is the previous method, even when rolling stop mode was off, is that the car reaches 0mph literally for a split second (like the car reaches 0mph, then immediately moves). This may not be good enough for some law enforcement (as they may miss the exact moment the car is stationary). The common myth is the stop must be for 3 seconds, and for safe driving (to make sure car is fully settled) up to 5 seconds. A lot of people would feel even 1 second is insanely long.