FSD v12.x (end to end AI)

[JulienW]

This geographically segmented rollout is new for FSD, right?

The installs on Teslafi show a pretty clear focus on California (of course), Texas (and neighbors), and the east coast (Virginia, North Carolina, Georgia). I thought it was silly of Tesla to send V12 to new inexperienced testers, but doing a rolling geographical rollout seems like a pretty good idea for risk management.

Sad Colorado isn't included yet... but we also just got a big snow storm so smart to this region for FSD ....

Yea, looks to be very Sunbelty.

 

[Tronguy]

This geographically segmented rollout is new for FSD, right?

The installs on Teslafi show a pretty clear focus on California (of course), Texas (and neighbors), and the east coast (Virginia, North Carolina, Georgia). I thought it was silly of Tesla to send V12 to new inexperienced testers, but doing a rolling geographical rollout seems like a pretty good idea for risk management.

Sad Colorado isn't included yet... but we also just got a big snow storm so smart to this region for FSD

Wonder what exactly Tesla's targeting criteria is. Target areas where initial internal testing shows the best results? Target areas with the most Teslas? Target areas where driving is easiest? Avoid bad weather? The lack of installs in the north in any of larger states with a decent number of Teslas like Washington, Utah, Colorado, Wisconsin and New York is conspicuous.

(Teslafi does show one single install in Alaska funnily enough)

Kind of wondering about the mid-Atlantic and Northeast, myself. But I have a hypothesis.

While I'm sure that every State in the Union has its own scary-as-heck bunch of roadways and roadway intersections, I suspect that the current high population areas with road layout and construction dating back to the Colonial Area is, generally, a nightmare for the FSD-b developers.

Roads changing names every 1.5 miles or 500 yards? We got 'em. Roads laid out by drunken cows? Boston has those. Intersections with five roads coming in and you don't even want to think what the traffic lights look like? All over the place.

Generally, a somewhat larger level of sanity on road construction became evident as various settlers moved West; hence, less squiggly lines on a map, more grid-shaped as surveyors put their collective feet down.

Further, while North Carolina, Virginia, and Georgia do have their dense bits in Tidewater, Raleigh-Durham, Atlanta, and No. VA, most of those places are a lot more rural with better laid-out roads than one gets in the mid-Atlantic to the Northeast. Similar arguments could be made for Texas and Arizona, I guess.

So, may be that Tesla is sneaking up on the hard parts.

 

[Supcom]

This geographically segmented rollout is new for FSD, right?

The installs on Teslafi show a pretty clear focus on California (of course), Texas (and neighbors), and the east coast (Virginia, North Carolina, Georgia). I thought it was silly of Tesla to send V12 to new inexperienced testers, but doing a rolling geographical rollout seems like a pretty good idea for risk management.

Sad Colorado isn't included yet... but we also just got a big snow storm so smart to this region for FSD

Wonder what exactly Tesla's targeting criteria is. Target areas where initial internal testing shows the best results? Target areas with the most Teslas? Target areas where driving is easiest? Avoid bad weather? The lack of installs in the north in any of larger states with a decent number of Teslas like Washington, Utah, Colorado, Wisconsin and New York is conspicuous.

(Teslafi does show one single install in Alaska funnily enough)

Still very small numbers. Assuming no major issues, hopefully we will see much larger expansion soon!

 

[drtimhill]

It’s very possible to make a sound argument or decision based on the available information and still be wrong.

Kind of wondering about the mid-Atlantic and Northeast, myself. But I have a hypothesis.

While I'm sure that every State in the Union has its own scary-as-heck bunch of roadways and roadway intersections, I suspect that the current high population areas with road layout and construction dating back to the Colonial Area is, generally, a nightmare for the FSD-b developers.

Roads changing names every 1.5 miles or 500 yards? We got 'em. Roads laid out by drunken cows? Boston has those. Intersections with five roads coming in and you don't even want to think what the traffic lights look like? All over the place.

Generally, a somewhat larger level of sanity on road construction became evident as various settlers moved West; hence, less squiggly lines on a map, more grid-shaped as surveyors put their collective feet down.

Further, while North Carolina, Virginia, and Georgia do have their dense bits in Tidewater, Raleigh-Durham, Atlanta, and No. VA, most of those places are a lot more rural with better laid-out roads than one gets in the mid-Atlantic to the Northeast. Similar arguments could be made for Texas and Arizona, I guess.

So, may be that Tesla is sneaking up on the hard parts.

Not really, what the car finds hard and what a human finds hard are not coincident. it's more convenience for Tesla .. testers are close to home base where they have in-house testing and so lower risk.

 

[JB47394]

Chuck was clear it was not perfect but nobody expects that.

For me, an ADAS has to be reliable. I found what worked reliably with V11 and that's where I used it. The rest of the time, I drove. With Chuck's left turn, I would drive that by hand because finding out if V12 could do it today just isn't worth the aggravation.

That said, I expect to use V12 in many more scenarios than V11.

 

[sleepydoc]

While I'm sure that every State in the Union has its own scary-as-heck bunch of roadways and roadway intersections, I suspect that the current high population areas with road layout and construction dating back to the Colonial Area is, generally, a nightmare for the FSD-b developers.

Those roads are scary-as-heck to me!

 

[gsmith123]

Not really, what the car finds hard and what a human finds hard are not coincident.

I'd argue areas where humans find driving harder are harder to supervise FSD and other drivers are more likely to make mistakes.

Both of these increase risk - which may be motivating Tesla to avoid these areas. It doesn't necessarily have anything to do with how "hard" it is for the car, or even how good or badly the car performs.

 

[lzolman]

Kind of wondering about the mid-Atlantic and Northeast, myself. But I have a hypothesis.

While I'm sure that every State in the Union has its own scary-as-heck bunch of roadways and roadway intersections, I suspect that the current high population areas with road layout and construction dating back to the Colonial Area is, generally, a nightmare for the FSD-b developers.

Roads changing names every 1.5 miles or 500 yards? We got 'em. Roads laid out by drunken cows? Boston has those. Intersections with five roads coming in and you don't even want to think what the traffic lights look like? All over the place.

Generally, a somewhat larger level of sanity on road construction became evident as various settlers moved West; hence, less squiggly lines on a map, more grid-shaped as surveyors put their collective feet down.

Further, while North Carolina, Virginia, and Georgia do have their dense bits in Tidewater, Raleigh-Durham, Atlanta, and No. VA, most of those places are a lot more rural with better laid-out roads than one gets in the mid-Atlantic to the Northeast. Similar arguments could be made for Texas and Arizona, I guess.

So, may be that Tesla is sneaking up on the hard parts.

I was having similar thoughts about New England. While, as I understand the process, 12.3 wouldn't improve any from a few weeks' "experience" in the usual suspects states, one scenario that makes sense to me is that due to the many issues of this area, the team would prefer to wait until all "low-hanging fruit" has been picked. Each update they aborted and reworked to this point has been spared the travails of New England.

Now, the rubber meets the (New England) road.

 

[TK211X]

Lots of hope current pending values on TeslaFi drop sub 30 and new wave pops tonight.

I'm going to be brutally honest. Once this hits my car I'm going to download it and could care less if there are delays afterwords. I just need to get it first.

I'm not even ashamed.

 

[powertoold]

Fsd just needs to
1) handle closed roads gracefully
2) not squeeze into bad situations

Then it's golden FSD ADAS

It feels so close (for the bay area anyway)

Just drove through SF during st paddy's traffic and peds, it was incredible except for a closed road (I didn't want to let it try )

Btw, I think u-turns on 12.3 are a bit worse than 12.2.1, goes too slow

 

[AlanSubie4Life]

Alright everybody, here we go!

Awesome! Chuck is a machine.

I haven't watched all of Chuck's v12 videos, but I gather from his first video (not unprotected left specific) he had an easy roller on his UPL. So we'll call that 1/1.

From the new UPL-focused video:

This was a medium to light traffic situation. There was a decent amount of traffic (but plenty of gaps) in the close lanes, but very very little in the far lanes.

Hopefully he does another one with more traffic soon! Plenty of easy rollers here, where it just had to wait for a break in near-side traffic and then it was all clear.

Overall: 6 out of 11. (Plus 1/1 for the initial video, so 7/12)

Chuck's UPL:

1 Pass Easy roller
2. Fail. Came to near stop in traffic lanes; this is incorrect and wrong. Fail!
3. Pass Easy roller
4. Pass Easy roller
5. Pass Easy roller
6. Pass Easy roller
7. Fail. Left butt out in lanes for a while. This is a failure, because it is wrong. Then had wrong pose in median. Not clear it could see, though it behaved correctly by not going, so perhaps it could see further than the view that is displayed.
8. Pass Easy roller

Other UPL:
1 Fail. Caused traffic to slow down to avoid near miss or collision (there was construction, but traffic had to slow for Chuck even though he said it was for the construction). I would definitely have disengaged, but this was an obvious failure. Chuck has nerves of steel; must be the Navy training!
2 Fail. Stopped in traffic lanes for left-turning truck in median. First Disengagement.
3 Fail. Missed a six-second gap in traffic (I would give this a pass...but then subsequently it paused in near-side traffic lanes again due to traffic on the far side). Anyway, the pause is incorrect. I think missing a six-second gap followed by a huge gap could be argued to be fine, but it's a close call. But it's a fail anyway.

So Chuck UPL alone:
6/8 + 1/1 = 7/9. (Have not reached minimum number of attempts on that specific turn, but would take 11/11 on the next video on 12.3 for him to get it up to 18/20, and for me to lose.)

Overall:
7/9 + 0/3 = 7/12. One disengagement.


For the Unprotected Lefts:

The NHTSA stop at the exact (legally required) stop line is annoying. But more annoying than that is how long it takes to resume and go to the creep limit. Tesla should fix this! There are big problems with pausing in traffic lanes. That to some extent has existed before but it is back with a vengeance here. I didn't really evaluate pace closely, but it seemed a bit slow to cross, and it still was taking a little bit of time to really push it when entering traffic on the far side.

For 12.3 I nearly have a lock (arguably I've already won), but I'd like to see more difficult situations tested. There's a serious regression here as Chuck said, with the stopping in traffic lanes, etc. And we saw no examples of left turns with significant traffic from the right, far-side lanes We really need to see that threading in - it's not clear it can handle it reliably without stopping in the near-side lanes.


I'm willing to bet again on the next version, as long as it's not a bunch of easy rollers. It really has to be tested with some traffic. It failed even the easy case this time, but next time with all the special training, we need to have situations where it usually has to wait for (or time the gaps) on far-side traffic.

Usually Chuck always tests higher traffic situations, so I expect unless there's another release very soon, he'll get a chance to try 12.3 in busier traffic. That'll be exciting.

But on 12.4 or whatever is next, I think there's a chance I'll lose. I still am betting against FSD though. I think it will not do 9/10 or better (with traffic).

Overall, it's good to hear that people are generally describing this as a "step change" in utility. That's a bit more than I expected, which was incremental improvement. I guess I'll see, when I get it. So far, a regression on unprotected lefts, but hopefully that is cleaned up soon!

 

[Todd Burch]

We are not “close” to Robotaxi yet. Still have to deal with:

-Emergency vehicles
-Crossing Guards
-Police directing traffic
-Closed Roads
-School buses
-no turn on red signs
-school zones
-no stopping on railroad tracks
-and hundreds or thousands of other edge cases that are not common but common enough to require handling before we get to driverless.

There’s a lot to do still.

HOWEVER, having said that, neural nets have shown the ability to handle all of these situations. (I remember watching a video years ago about Waymo recognizing hand signals from police officers directing traffic at intersections where the lights have gone out).

Moreover, the pace will accelerate from here. And I don’t think there’s a whole lot more of “how do we do this?” left. It feels like Tesla’s now at the point of continuing to execute their e2e plan just by more training data and bigger networks.

If 12.3 is now more than a gimmick for most, and will get significantly better from here, I could see us realistically being ready for Robotaxis (not necessarily approval) in a few years. In 2014 I predicted on this forum that we’d get there in 2024, best case scenario. Looks like we might be within a few years of that timeframe.

 

[AlanSubie4Life]

Still have to deal with:

Don't forget: being able to make left turns with cross traffic!

Will be exciting to see the progress to something usable.

I should add that I fear for Chuck’s safety. This is really quite hazardous stuff to test if you don’t disengage at the first hint of incorrect behavior. Hopefully all will be well.

 

[aronth5]

Kind of wondering about the mid-Atlantic and Northeast, myself. But I have a hypothesis.

While I'm sure that every State in the Union has its own scary-as-heck bunch of roadways and roadway intersections, I suspect that the current high population areas with road layout and construction dating back to the Colonial Area is, generally, a nightmare for the FSD-b developers.

Roads changing names every 1.5 miles or 500 yards? We got 'em. Roads laid out by drunken cows? Boston has those. Intersections with five roads coming in and you don't even want to think what the traffic lights look like? All over the place.

Thanks for brightening up my day . Driving in the land where many roads (i.e. paths) were designed by horses and cows in the 1600's and 1700's testing V12 will be interesting. Can't wait to try it at some of the obstructed view intersections where FSD is simply unsafe to use now. Probably best for FSD to have the ability to alert the mother ship that certain intersections should just not be used. I'd be fine with that.

I'm going to be the optimist though. Go V12!!

 

[gsmith123]

The NHTSA stop at the exact (legally required) stop line is annoying. But more annoying than that is how long it takes to resume and go to the creep limit. Tesla should fix this!

I noticed this as well.

Related: to me it looks like the car is stopping a few feet short of the stop line as well. Check out this screenshot from the video. The car is stopped (0 mph indicated!) and if you look at the drone view there's obviously space in front of the car to the stop line.

In situations where there's a pretty large creepable area, I'd like to see the car to make it's NHTSA mandated stop as late as is legally allowed (which at a minimum would be with the nose of the car at but not over the stop line). Use that space!

(That said, this is a pretty minor complaint and I'm pretty excited about V12 now. It just seems like the car could knock a couple hundred ms off these maneuvers and keep following vehicles happier)

 

[AlanSubie4Life]

I noticed this as well.

Related: to me it looks like the car is stopping a few feet short of the stop line as well. Check out this screenshot from the video. The car is stopped (0 mph indicated!) and if you look at the drone view there's obviously space in front of the car to the stop line.

View attachment 1028811

In situations where there's a pretty large creepable area, I'd like to see the car to make it's NHTSA mandated stop as late as is legally allowed (which at a minimum would be with the nose of the car at but not over the stop line). Use that space!

(That said, this is a pretty minor complaint and I'm pretty excited about V12 now. It just seems like the car could knock a couple hundred ms off these maneuvers and keep following vehicles happier)

Yeah it stops way early sometimes. We’ll see. At least that behavior should be easy to fix. The dynamic stuff while in traffic is potentially much more challenging. We’ll see if the NN approach suffers from the same problem as the “300k lines” approach: the inability to fix persistent problems.

 

[gsmith123]

We’ll see. At least that behavior should be easy to fix.

To me the "how fast can they iterate" question is the question about V12.

How did the car learn to stop so far back anyway? Humans typically err completely in the other direction stopping late, so doesn't seem like they'd accidentally end up with human sourced clips of it stopping early. Actually, humans don't stop at all most of the time...

Maybe they're leverage synthetic data pretty heavily for forcing the car to make a full stop and however they generate that synthetic training data stops early?

 

[arnolddeleon]

One word to describe V12.3 (coming from V11) is smooth.

I agree with an earlier post described how V12 doesn't seem to do wild wheel jerks.

Even with my first V12 disengagement was from the car smoothly entering a center lane to turn and then drifting into the double yellow towards the opposite lane of travel with cars in that lane.

BTW I believe it is super easy to see the switch been V11 and V12 if you have autospeed enabled. You can see Max Speed switch from "auto" to a number. As I previously noted, I believe there is a little of V12 logic for freeway speed but the actual driving is V11. The Max Speed that I'm seeing are 10 over the posted speed limit.

The first drive I had this morning had an amazing first. It chose the outside lane of a left turn even though there was another immediate left. With V11 I always have to disengage here. The outside lane is the "correct" choice for this intersection because the inside lane is almost always blocked from entering the intersection. Basically, anyone who lives around here knows to avoid the inside lane. It's not clear if this is an "overfit" or if it is used contextual clues, like all the other cars lined up on the outside lane. The challenge with this lane selection is now you have cut across other cars going straight to get to the next left turn lane. It executed that very well too. My drive to my destination was effectively curb to curb.

My drive home was more interesting. Autospeed couldn't overcome what is either bad map data or the car being "lost" while it comes out from an underground garage. The car was displaying a speed limit of 5 mph, in what should have been either 25 or 35 mph street.

Merges onto the freeway look like V12 to me and that was well done. I disengaged on the freeway due to car about change into us. The driver looked they assumed people would get out of their way. I guess they weren't wrong since we all got out of the idiot's way. I decided to change my route home to take more city streets versus the freeway.

Once we were on city streets it was pretty much the experience that has been described several times. Smooth, occasionally a little slow. I'm confident that one of the big improvements is modeling cars slowing down but turning away from lane. V12 handles them smoothly (unlike V11). Overall, it was pretty good with the notable exception of the disengagement I described at the top.

My observation is that one of the things that is making V12 be better is the control actions seemed to model an understanding of anticipation. I think is an emergent property of end-to-end training. I am cautiously optimistic that this will help overcome some of the compute limitations.

 

[scottf200]

Michael Dell - Super impressive, Tesla FSD v12.3 is. Like a human driver, it is.
Elon Musk - V12.4 is another big jump in capabilities. Our constraint in training compute is much improved.
10:09 PM · Mar 16, 2024

https://twitter.com/x/status/1769199345746735123

 

[FOOLSLFDRVNG]

Lots of hope current pending values on TeslaFi drop sub 30 and new wave pops tonight.

View attachment 1028762

I'm going to be brutally honest. Once this hits my car I'm going to download it and could care less if there are delays afterwords. I just need to get it first.
View attachment 1028773

I'm not even ashamed.

The same anxious feeling for every new version I don’t have yet is like this poor kid listening to mommy’s scream with another new boyfriend every night.