FSD v12.x (end to end AI)

[Todd Burch]

How much more do we need to see before concluding Tesla and team have basically done it?

The craziest thing is I have no idea how all of this is possible

I share your excitement and optimism, but let's stay grounded here before we start claiming they've done it:

1. This is San Francisco. Due to the other self-driving systems in use here, it probably has the most accurately-mapped city in the world. We don't yet know how reliant v12 is on mapping. In my area, there are tons of mapping issues, causing v11.4.9 to behave silly. Even regular navigation has some silly mapping issues. (Easy to fix, but I want to see FSD that *doesn't* rely much on mapping.

2. Omar's daytime drive to the Golden Gate Bridge still showed some issues. Excessive hesitancy at stop signs, etc. Nothing earth shattering or unfixable, and nothing dangerous. But we still have a ways to go.

3. This video, despite being in a city, was a fairly easy drive. common 4-way squared intersections, light traffic, wide roads. There are many much harder driving situations.

4. We haven't seen a lot of things like stopping for school buses, obeying crossing guards/police instructions, etc.

Clearly Tesla is ahead and E2E looks to be the right solution. And I'm betting the farm on TSLA. But it's still way too early to claim that they've done it.

 

[powertoold]

I share your excitement and optimism, but let's stay grounded here before we start claiming they've done it:

1. This is San Francisco. Due to the other self-driving systems in use here, it probably has the most accurately-mapped city in the world. We don't yet know how reliant v12 is on mapping. In my area, there are tons of mapping issues, causing v11.4.9 to behave silly. Even regular navigation has some silly mapping issues. (Easy to fix, but I want to see FSD that *doesn't* rely much on mapping.

2. Omar's daytime drive to the Golden Gate Bridge still showed some issues. Excessive hesitancy at stop signs, etc. Nothing earth shattering or unfixable, and nothing dangerous. But we still have a ways to go.

3. This video, despite being in a city, was a fairly easy drive. common 4-way squared intersections, light traffic, wide roads. There are many much harder driving situations.

4. We haven't seen a lot of things like stopping for school buses, obeying crossing guards/police instructions, etc.

Clearly Tesla is ahead and E2E looks to be the right solution. And I'm betting the farm on TSLA. But it's still way too early to claim that they've done it.

Here's my thoughts on Tesla having "done it"

1) V12 is trained on videos from all over the world
2) Elon mentioned V12 being tested in other countries like Thailand during the livestream
3) There's no more concept of overfitting wrt to V12, overfitting was regarding the heuristics in V11, where the engineers were more biased towards heuristics they were familiar with (Bay Area, CA)
4) The unintuitive thing about V12 is that a "simple" drive is just as "difficult" as a more complicated drive for something like V12, because of the diversity of the training set -- because of this, we have V12 struggling / hesitating on the most basic stop signs
5) With these V12 videos, we can see that the approach has been perfected, essentially it's all about the training set and compute now

Remaining questions:

1) How horrendous are V12's judgment failures
2) How often do significant failures occur
3) Is it possible to fit the world's data to run on HW3/HW4/HW5? Based on what we're seeing with HW4, there's a good chance V12 will be reliable enough to surpass human safety x5 (which is robotaxi territory imo)
4) How fast is the improvement for each point release
5) How long does it take to train the world model and how fast can Tesla iterate on architectures

 

[uscbucsfan]

3) There's no more concept of overfitting wrt to V12, overfitting was regarding the heuristics in V11, where the engineers were more biased towards heuristics they were familiar with (Bay Area, CA)

This doesn't track. They are obviously training individual areas as we see test drivers testing Chuck's UPL.

It's way too early to say "they did it". We are still seeing level 2 ADAS that still makes mistakes on curated drives. I don't see anything that doesn't suggest robotaxi is imminent. TBH, it appears to be another small step from 11.4.9 comparing his videos, but hopefully the backend will be easier to make substantial changes as the progress with FSD has been incredibly slow.

 

[sleepydoc]

It's actually the opposite, Elon was asked about using the cabin camera in place of the steering nag, which he agreed to do:
Tesla FSD Beta users will be able to turn off steering wheel nags this Jan
NHTSA clamped down and killed that idea pretty quickly:
Elon Musk might have tweeted himself into hot water — again — with a regulator

So why is Ford allowed to use a camera but Tesla isn’t?

 

[sleepydoc]

True there is a clear improvement on comfort and driving style. But the issue is that the Tesla faithful don't see it as that. As usual on the eve of every new version they proclaim it as a "Technological breakthrough on scale never seen before. A game, set, match!"

We need to get Alan’s opinion of the acceleration profile before we can say they’ve done it!

 

[powertoold]

This doesn't track. They are obviously training individual areas as we see test drivers testing Chuck's UPL.

This doesn't track. I think we've gone over this before. There's no reason for Tesla to be "training" anywhere when they have millions of cars on the road they can extract data from.

 

[uscbucsfan]

That is probably right, but he is also a paying tester.

I've never really seen Omar "test". He plans drives that will show a good response and posts them.

He also has an in with Tesla, more so than people like Chuck. He's never had nags prior to V12 and that's either because he has a device or Elon mode in the past.

 

[uscbucsfan]

This doesn't track. I think we've gone over this before. There's no reason for Tesla to be "training" anywhere when they have millions of cars on the road they can extract data from.

But they are. Chuck has many videos of testers in the last 2 months trying his turn over and over.

And by testers, they are Tesla employees.

 

[powertoold]

But they are. Chuck has many videos of testers in the last 2 months trying his turn over and over.

Training is different than testing, and we're just speculating if they're testing V11 or V12

 

[AlanSubie4Life]

Based on what we're seeing with HW4, there's a good chance V12 will be reliable enough to surpass human safety x5 (which is robotaxi territory imo)

What leads you to this conclusion? Is it the lack of safety interventions over tens of miles? What is the probability of Omar being able to repeat this tens of thousands of times without curation?

I don't see anything that doesn't suggest robotaxi is imminent.

Extra negative?

We need to get Alan’s opinion of the acceleration profile before we can say they’ve done it!

It looked ok so far except on one downhill to a stop which was awful. But all of this is way too low a speed to assess. All I can say is it appeared to be improved but sample size far too limited.

 

[uscbucsfan]

Training is different than testing, and we're just speculating if they're testing V11 or V12

Many of them were manual drives as he pointed out. That's more of a leap that they are doing manual drives to train V11 while V12 is being developed.

You are ignoring what has been said about V12. They need numerous videos of good driving to train the software and that's why Chuck has made the leap that the manual drives are for V12. Sure, it's still a leap, but a logical one.

 

[uscbucsfan]

Extra negative?

No. Just an error

 

[powertoold]

What leads you to this conclusion? Is it the lack of safety interventions over tens of miles? What is the probability of Omar being able to repeat this tens of thousands of times without curation?

I know that V12 is totally different than V11 and not really comparable, but computers do have a certain level of reliability baked in, because of how stable and predictable the drive videos have been, it's tempting to extrapolate that V12 will have similar performance in similar weather conditions

 

[powertoold]

Many of them were manual drives as he pointed out. That's more of a leap that they are doing manual drives to train V11 while V12 is being developed.

You are ignoring what has been said about V12. They need numerous videos of good driving to train the software and that's why Chuck has made the leap that the manual drives are for V12. Sure, it's still a leap, but a logical one.

So you're saying Chuck was able to prove they were driving manually AND testing / training V12?

V11.4.9 was only released about a month ago, it's not possible they were testing V11?

 

[stopcrazypp]

So why is Ford allowed to use a camera but Tesla isn’t?

I'm going to guess the position (rear view mirror vs behind steering wheel) and nature of the camera (no infrared except in newer versions). Basically Tesla didn't design the camera as a primary source of driver monitoring, so probably it is suboptimal for that and not ready to be such.

 

[uscbucsfan]

So you're saying Chuck was able to prove they were driving manually AND testing / training V12?

V11.4.9 was only released about a month ago, it's not possible they were testing V11?

He claims, in multiple instances, they are driving manually. From the actions, not going into the creep, etc.

You think Tesla is flying or hiring testers for 11.4.9 during it's release? It's unlikely...again, we know V12 will need that data. We know Tesla has people driving for Tesla to help train.

 

[uscbucsfan]

The point is that videos from WholeMars prove nothing...again, he said almost a year ago that FSD was solved. That's a provable lie.

He post videos to hype the product. That's his purpose.

 

[powertoold]

He claims, in multiple instances, they are driving manually. From the actions, not going into the creep, etc.

You think Tesla is flying or hiring testers for 11.4.9 during it's release? It's unlikely...again, we know V12 will need that data. We know Tesla has people driving for Tesla to help train.

What you're saying makes much less sense though. There's no reason for Tesla to be spending resources on manually training a single maneuver for V12 during daylight hours in decent weather as Chuck's posts have shown

Tesla has a fleet of millions of cars, there's no reason they can't collect complicated unprotected lefts from real drivers at all times of day, in any weather

And what you're saying is all speculation (whereas mine is based on what Elon and Ashok and etc have said)

 

[uscbucsfan]

And what you're saying is all speculation (whereas mine is based on what Elon and Ashok and etc have said)

No, it's not. You are making leaps from what they have said to what you believe.

Tesla absolutely hired test drivers in the 2nd half of last year. The positions were posted here. We've seen them on the roads in videos.

 

[EVNow]

How much more do we need to see before concluding Tesla and team have basically done it?

Full rollout to all customers and for us to use it for a couple of weeks and see the reactions ?

BTW - it is indeed very impressive, and it is a technological breakthrough if it is truly end to end. No doubt about that. In that sense "they have done it".

But actual FSD is still far away ... and we get there when we can drive with no interventions for a few months at a time ...