FSD v12.x (end to end AI)

[sleepydoc]

“They indicated the nag has been dialed up to ensure drivers ae paying extra attention.”

This sucks - I definitely noticed it on my last drive. any time it even things you’re looking away for more than a second it starts to nag you. (And that’s a literal second, not a figurative one.) The nag system went from being manageable to a royal PITA.

 

[spacecoin]

1) We know in the past, Tesla was overfitting the data, causing FSD beta to drive better in places like SF and not as well in other parts of the country. Will we see a similar pattern with V12? I look forward to trying V12 in my area where FSD Beta has struggled for a long time.

It's likely.

2) Will end-to-end get to 99.99999% reliability where Tesla can remove the driver supervision requirement and if yes, how quickly will that happen?

Likely never on existing hw. But if it happens, it will be at least ten years before I see an Optimus on the supermarket getting milk.

 

[rlsd]

I am not surprised if V12 ends up being better in every day, common driving. End-to-end is a very efficient way of training NN. And NN are very good at imitating patterns. Tesla says that they used millions of clips for V12. So if you plug in millions of clips of good driving, you can except V12 to be good at imitating what it sees in those millions of clips. And I am not surprised if V12's driving is more natural, more human-like. Tesla has trained it on millions of clips, presumably taken from good human drivers, so we would expect V12 to imitate that human behavior and therefore drive in a more natural, human way. We also saw this when Waymo changed to a ML-first planner. They also saw a big improvement in the driving being less robotic, more natural, more confident.

My 2 big questions right now (which I am sure will get answered in time):

1) We know in the past, Tesla was overfitting the data, causing FSD beta to drive better in places like SF and not as well in other parts of the country. Will we see a similar pattern with V12? I look forward to trying V12 in my area where FSD Beta has struggled for a long time.

2) Will end-to-end get to 99.99999% reliability where Tesla can remove the driver supervision requirement and if yes, how quickly will that happen?

I doubt that Tesla will remove supervision in V12. The regulations and greedy lawyers and reckless drivers won't let Tesla do that. Tesla has learned a lesson with the past lawsuits. Supervision is a good strategy to collect data and to keep FSD moving on. They don't want to kill it early like GM Blue Cruise.

 

[rlsd]

This sucks - I definitely noticed it on my last drive. any time it even things you’re looking away for more than a second it starts to nag you. (And that’s a literal second, not a figurative one.) The nag system went from being manageable to a royal PITA.

It's not desirable for careful drivers but it's a must to avoid lawsuits and regulator harassments with the initial releases. Maybe we will get less nags with 12.4.9 like with 11.4.9 now.

 

[AlanSubie4Life]

I'm not sure what you are reacting to in this clip.
The car paused while:

• passing a double parked car
• in the marked crosswalk
• whose passenger was actively opening doors and exiting
• with oncoming traffic.

And the car still caught up to traffic by the next stop sign

It’s hard to tell in this video. Can someone find the realtime location of this in the other videos?

Anyway the issue I saw is that it lagged behind the other vehicle. Not clear whether that was needed. Hard to know without a bit more detail.

I was just watching it on a tiny screen too. We’ll see. As I said, watched really quickly.

And the car still caught up to traffic by the next stop sign

It is this that was the issue specifically. The question is why. Looks like it was due to the oncoming car but hard to tell from wide angle sped up on a tiny screen whether it was necessary or not.

 

[diplomat33]

I doubt that Tesla will remove supervision in V12. The regulations and greedy lawyers and reckless drivers won't let Tesla do that. Tesla has learned a lesson with the past lawsuits. Supervision is a good strategy to collect data and to keep FSD moving on. They don't want to kill it early like GM Blue Cruise.

I get that keeping supervision makes a lot of sense now. But remember that the promise of FSD is to remove driver supervision. So Tesla needs to remove driver supervision at some point if they want to keep their promise.

 

[powertoold]

Possibly ran a red light here while turning right:

 

[DanCar]

Definitely trolling. ...

He has posted similar stuff for v8, v9, v10, v11. He is not interested in learning from his mistakes. I find it entertaining how overly optimistic he is, which at least half the posters here tend to be.

 

[Mardak]

I noticed the blue driving path visuals are extremely short on that V12 build. It doesn't show it's going to make a right turn until it's partway through it.

That might be the only part of the visualization that is newly "added" with 12.x so far. It seems like most of the visualization is reused from 11.x but at lower framerate with other things completely removed like the original FSD Visualization Preview (cones, trash cans, painted turn arrows, etc.) and crosswalks presumably to free up compute for the new neural networks. Outputs from the old control code are also removed, e.g., creep wall, blue highlighting of other vehicles, messages of what it's doing "stopping for stop sign," slowing down chevrons.

So the 12.x blue path visualization is probably now neural network prediction of where it's going to be in ~1.5 seconds. Previously 11.x control showed a path that can be both dark blue and light blue split with a stop line to indicate where it would go after stopping. Now the path can be extremely short when stopping or starting because it won't get very far at low speeds.

 

[powertoold]

Here what I see when I watch these V12 videos:

1) human-like understanding of speed control based on surroundings/occlusions
2) very stable build without any illogical wheel movements
3) accurate response to speed bumps, slows down on top and speeds up after
4) incredible scene understanding before and around the firetruck
5) incredible double-parked car performance with confident and human-like squeezes
6) great stop sign performance and speed control
7) you can see/feel V12's ability to reason about its environment in a human-like way, everything from the subtleties of speed control to how fast to turn the wheel, dealing with slopes and visibility, etc.
8) never once did it make a weird/scary maneuver
9) didn't pass any cars unnecessarily/mistaken as double-parked, etc.
10) didn't get stuck behind any double-parked cars mistaken as lead car

 

[AlanSubie4Life]

All that just from a single video of less than 5 miles by Omar of all ppl? Ask yourself how many serious safety disengagements from v11 has Omar posted?

However, looking carefully at the screen, it does appear smoother than prior iterations. It is hard to cherry pick that. And that would improve L2 usability (currently mostly unusable on City Streets with passengers remember!).

We’ll see is this holds up in further videos with sound (important for assessing smoothness since you can hear things moving around).

It's surprising how seemingly stable and coherent V12 is already.

Why? It’s an incremental change they have been working on for a while - I think at least a year?

any time it even things you’re looking away for more than a second it starts to nag you. (And that’s a literal second, not a figurative one.)

The nag is great in v11. Ensures you keep the eyes on the road. If you want to look away, wait until you are stopped and just disengage! You still have the green light chime. I don’t recommend due to possible loss of situational awareness, but just saying it is great and really wonderful at ensuring some engagement with the driving task assuming you are not wearing sunglasses - and even then it is pretty good at detecting head tilt, just loses glance detection.

Hopefully we get another level of rigor in v12.

One of the biggest improvements in FSD development so far. Very happy to have the NHTSA on the job, pushing this forwards, for sure. And to be fair, even Tesla has been improving it, prior to being forced to roll that improvement wide through recalls.

 

[willow_hiller]

Possibly ran a red light here while turning right:

Looks like the intersection of California St & Parker Ave in San Francisco, CA. Looking at Google Street View, once you turn left onto California St, you're already in the intersection and have passed the lights controlling Parker Ave. So I think V12 did the correct thing here and treated that immediate right as a part of the same intersection.

 

[powertoold]

Why? It’s an incremental change they have been working on for a while - I think at least a year?

Are we under-appreciating that V12 got rid of 300k lines of code and is now performing way way better than V11 in the first drives?

The first drive video is packed with subtleties if you analyze each second. For example, check out this pedestrian who crosses and then waits for V12 to pass:

And here is the most surprising decision in the first-drive video, incredible gray-area here: taking in account speed, intention, etc. etc. There's even 2 pedestrians walking fast from the left to right when V12 decided to commit:

 

[mongo]

Looks like the intersection of California St & Parker Ave in San Francisco, CA. Looking at Google Street View, once you turn left onto California St, you're already in the intersection and have passed the lights controlling Parker Ave. So I think V12 did the correct thing here and treated that immediate right as a part of the same intersection.

Indeed, all one big intersection

 

[DanCar]

Are we under-appreciating that V12 got rid of 300k lines of code and is now performing way way better than V11 in the first drives?

Are we over appreciating video from a known Shyster? Claims has zero interventions in the past when he was actually was hitting the gas pedal multiple times?
Wait till real people get it and you will learn about all the regressions.

 

[Todd Burch]

Are we over appreciating video from a known Shyster? Claims has zero interventions in the past when he was actually was hitting the gas pedal multiple times?
Wait till real people get it and you will learn about all the regressions.

Ignore Mars' comments and simply watch the 1x speed video, which shows the accelerator/brake pedal and steering wheel to show that he's not intervening. All else being the same, the video clearly shows areas of improvement vs V11. I get that Whole Mars thinks every version is ready for RoboTaxi. Most people do. Ignore his opinion and just watch the video. The evidence of significant improvement is there.

 

[AlanSubie4Life]

Are we under-appreciating that V12 got rid of 300k lines of code and is now performing way way better than V11 in the first drives?

No. This is not its first drive and it has to be better otherwise we would not see it.

The planner has been the source of no end of complaints for years so it’s not that surprising that getting rid of it helps.

These are early days though. We need full videos with sound and more of them with busier situations to see how it does. This was some really easy driving with minimal complexity.

We’ll see. The apparent smoothness in a very limited sample and the assertiveness are promising for more usable L2. But need a lot more samples.

 

[spacecoin]

The evidence of significant improvement is there.

How can you even tell the difference from v11 to v12 based on that drive. I wouldn't rule out a dude with a accelerator/breaking button wired up to the CAN-bus in the back seat when it comes to WholeMars...

If every drive in every state is like that drive, then we have "evidence of significant improvement". RIght now there is hope.

 

[hankjost@gmail]

Will FSD 12 make basic autopilot work better?

I currently am not happy enough with basic auto pilot to use it.

 

[SOULPEDL]

V12 Looks less iffy to me.
Congrats FSD team!