Elon: FSD Beta tweets

[Daniel in SD]

Yes - I'd put V11 in that ULT as better than average humans.

More importantly that made FSD actually be able to handle simpler ULTs that it could barely ever do earlier. Now I've > 90% success rate on all my (admittedly simpler) ULTs.

I asked Chuck to set up the cameras to measure a human baseline and he told me he's not aware of any collisions at that intersection. Of course our 90% success metric isn't the same because most of the time if you screw up other drivers will be able to avoid the collision.

 

[ItsNotAboutTheMoney]

Good thing Elon has never overhyped anything before.

Has anybody done a grading of FSD hype by day of quarter? We're getting to the end-of-quarter please-buy-FSD hype period..

Maybe it's just a set up for the follow up tweet on 4/1. "April Fools!" ?
13 ~ 1 + 3 = 4 = 4/1. Get it?

What I actually read in the attempted message is that v12 is really late, but that's OK because by the time it arrives, it'll really be v13, which is better than v12, honest. It's very sad.

 

[EVNow]

I asked Chuck to set up the cameras to measure a human baseline and he told me he's not aware of any collisions at that intersection. Of course our 90% success metric isn't the same because most of the time if you screw up other drivers will be able to avoid the collision.

Exactly.

For eg. you might count a ULT as unsuccessful because the oncoming car had to slow down, shift lanes - but if you take almost any busy intersections, by design or otherwise people in the way of oncoming cars all the time slowing down other traffic / cars.

Moreover - if it takes "too long" o turn left, people give up and turn right. I do it all the time at busy roads. People get antsy and honk if you wait a long time to turn left instead of choosing a different route and turn right quickly.

 

[Daniel in SD]

Exactly.

For eg. you might count a ULT as unsuccessful because the oncoming car had to slow down, shift lanes - but if you take almost any busy intersections, by design or otherwise people in the way of oncoming cars all the time slowing down other traffic / cars.

Moreover - if it takes "too long" o turn left, people give up and turn right. I do it all the time at busy roads. People get antsy and honk if you wait a long time to turn left instead of choosing a different route and turn right quickly.

So what is your estimate of the collision rate of unsupervised FSD for Chuck’s turn?

 

[willow_hiller]

Karpathy is talking about an AI-coder in this tweet, but he also lays out a greatly summarized FSD road map:

In my mind, automating software engineering will look similar to automating driving. E.g. in self-driving the progression of increasing autonomy and higher abstraction looks something like:
1. first the human performs all driving actions manually
2. then the AI helps keep the lane
3. then it slows for the car ahead
4. then it also does lane changes and takes forks
5. then it also stops at signs/lights and takes turns
6. eventually you take a feature complete solution and grind on the quality until you achieve full self-driving.

Interesting to see that we go from step 5, which was essentially the "Stop Light / Stop Sign" feature of NoA, to step 6 being "grinding" full autonomy.

Maybe that's why he left Tesla in the first place; if he thought all that was left was "grinding."

 

[spacecoin]

Karpathy is talking about an AI-coder in this tweet, but he also lays out a greatly summarized FSD road map:



Interesting to see that we go from step 5, which was essentially the "Stop Light / Stop Sign" feature of NoA, to step 6 being "grinding" full autonomy.

Maybe that's why he left Tesla in the first place; if he thought all that was left was "grinding."

1) That’s not how you build an autonomous system. It’s almost shocking that the former head of AP still doesn’t understand what goes into building a safety critical system.
2) AK understood that Tesla was 5-10 years away from autonomy when he left.
3) Waymo has been grinding for 10 years. Tesla hasn’t even started yet. The grinding starts when you drive without interventions for 20-40 hours. Then you need to get to tens of thousands of hours.

 

[kabin]

With respect, I don't think this is the correct summary. What we were told in the original Elon/Ashok video, and AFAIK is still true,
Was that the key to everything is lots of training.

They didn't say the NN didn't need to be trained and just knew what to do. They said they didn't explicitly program it to recognize and label objects like stop signs, traffic lights, pedestrians etc. Nor did they program it how to drive - all that ability comes out of the extensive E2E training.

In the video, when it did make a mistake, they immediately said that it would be solved by more training of similar scenarios. More of the same. So talking about needing more training today is not a case of "back to overfitting"; it's perfectly consistent with the original explanation of v12.

I should add that I haven't seen Elon deny that overfitting can occur. He said that the first selectively released version needed more training outside of California and more training in bad weather. That's essentially an admission that it was overfit for California.

For sure I left out details but it's a little of both. Elon overstated capabilities and raised expectations with comments I previously touched on. I think many of us suspected it was all nonsense at the time.

Overfitting has always been the last ditch effort and with repeated Tesla ADAS drivers at Chuck's unprotected left v12 appears to be more of the same. Overfitting tends to result in poor NN generalization - we've seen that over and over after Chuck's unprotected left updated was introduced. Overfitting also causes less response to new training data so it might be the same old story for those of us outside California with unique drives and edge cases.

I should add- As TSLA profits fall there's gonna be some new pressure on the team to make some real progress. These short cut, kick the can, kill time, solutions are convincing fewer and fewer people.

 

[EVNow]

So what is your estimate of the collision rate of unsupervised FSD for Chuck’s turn?

Zero if it bails like humans do.

I asked Chuck to set up the cameras to measure a human baseline and he told me he's not aware of any collisions at that intersection.

To get a proper baseline we should setup a camera and critique the ULTs just like we do with FSD

 

[Daniel in SD]

https://twitter.com/x/status/1768041413063663704

Are three nines possible?
@AlanSubie4Life

 

[AlanSubie4Life]

https://twitter.com/x/status/1768041413063663704

Are three nines possible?
@AlanSubie4Life

I’m old enough to remember they took months to get this “working” the first time and they still failed to get one nine.

As long as they are calling failures successes, beer will flow directly to my belly for free.

You don’t have FSD Beta. That makes this like taking candy from a baby.

If the adversarial conditions have to be generated in such small quantities of just thousands of clips (insanely small number!), there’s definitely no way it’s going to get to 90% (unless it’s light traffic and it is just rolling everything of course, but that is just silly). I just don’t understand why anyone (even super credulous Elon) would think such a tiny training set would be good enough - I must be some sort of complete idiot when it comes to this stuff, I guess.

As always, I hope I am wrong.

 

[AlanSubie4Life]

I’m old enough to remember they took months to get this “working” the first time and they still failed to get one nine.

As long as they are calling failures successes, beer will flow directly to my belly for free.

You don’t have FSD Beta. That makes this like taking candy from a baby.

If the adversarial conditions have to be generated in such small quantities of just thousands of clips (insanely small number!), there’s definitely no way it’s going to get to 90% (unless it’s light traffic and it is just rolling everything of course, but that is just silly). I just don’t understand why anyone (even super credulous Elon) would think such a tiny training set would be good enough - I must be some sort of complete idiot when it comes to this stuff, I guess.

As always, I hope I am wrong.

Honestly not sure what there is to disagree about, or whether you are just salty about the impending payment for a delicious IPA. It’s money well spent. You have to have convictions. After all, a Level 5 system has to be able to do this turn soon.

Perhaps someone can explain to me how it would be possible for a NN to be able to do this 90% of the time with decent amounts of traffic present. Because I can’t conceive of it. How many training clips would be needed? How do you calculate that number?

 

[Daniel in SD]

Honestly not sure what there is to disagree about, or whether you are just salty about the impending payment for a delicious IPA. It’s money well spent. You have to have convictions. After all, a Level 5 system has to be able to do this turn soon.

Perhaps someone can explain to me how it would be possible for a NN to be able to do this 90% of the time with decent amounts of traffic present. Because I can’t conceive of it. How many training clips would be needed? How do you calculate that number?

Alpha Go.

 

[Knightshade]

delicious IPA.

This is how we know this is a parody account

 

[kabin]

This is how we know this is a parody account

Yes, AlanSubie. I haven't had a delicious IPA. What do you recommend?

 

[Bouba]

Yes, AlanSubie. I haven't had a delicious IPA. What do you recommend?

Budweiser

 

[kabin]

Budweiser

Isn't that only good for target practice?

 

[Daniel in SD]

This is how we know this is a parody account

All we drink in San Diego is IPAs.

 

[Knightshade]

All we drink in San Diego is IPAs.
View attachment 1027884

The list of reasons I never intend to live in CA keeps growing

 

[EVNow]

If the adversarial conditions have to be generated in such small quantities of just thousands of clips (insanely small number!),

Not necessarily. A few thousand adversarial clips are quite good (for a given situation) is my understanding. You need millions of clips to train driving in general, though ...

 

[momo3605]

Not necessarily. A few thousand adversarial clips are quite good (for a given situation) is my understanding. You need millions of clips to train driving in general, though ...

How does millions of clips help when the low-res cameras can't even see far enough?