FSD Beta Videos (and questions for FSD Beta drivers)

[diplomat33]

I agree, but I really do think we'll know it when we see it in videos.

No. It is statistically impossible to judge when the car has reached "human level+ FSD" based on videos alone. You need about 2B miles to reliably validate if the FSD is safe or not. You can't get that from videos. Videos are basically anecdotal evidence. They are not quantitative data. Videos only give a snapshot in time. They can't tell you if the FSD can safely handle all the thousands of permutations that a single driving scenario will have. That's why you need data.

 

[gearchruncher]

Otherwise, most people won't even use city streets fsd (even some fsd beta testers admit that they don't use it much, other than to show it off),

I really, really appreciate and respect you for being honest about the mixed results you see from this release.

The above is very interesting, and I had not heard it. It seems to fly in the face of the idea that Tesla will be able show FSD is working due to miles used. I wonder how they are really testing then if this is more of a marketing campaign.

Right now, it's making many unnatural choices.

I agree. My bar for "mind blowing" is when someone shows me a video without telling me it's FSD, and I would have to ask if it was a human or a computer. Kind of a FSD Turing test. Right now it's so clearly not a human.

Otherwise, if we subtract the lane choice issues,

What about the cases where it tries to drive into a concrete pillar, or pull in front of an oncoming car?

So we are all just guessing wildly based on what videos we decide to cherry pick.

I generally agree, but we only have a few 10's of hours of video right now, and we can see lots of accidents if there was no intervention.
Per Tesla's own numbers, humans have an accident once every 2M hours. About every 50,000 hours. Until we have anything like 50,000 hours of videos with no failures, we're nowhere near Human FSD. It's not cherry picking if there are even two failures in those 50,000 hours, that's the way statistics work.

We're 9+ months in since v8, and this is the improvement Tesla was able to make. The idea we will improve by ~5 orders of magnitude in 3 months is not realistic.

 

[gearchruncher]

It is statistically impossible to judge when the car has reached "human level+ FSD" based on videos alone.

You can disprove it though.

 

[Dan D.]

Personally, I do not think Tesla is 3+ months away from "human level+ FSD". But the thing is that without real data like accidents per million miles, near misses per million miles or interventions per million miles, etc... it is impossible to really quantify how close or far Tesla might to be "human level+ FSD". So we are all just guessing wildly based on what videos we decide to cherry pick.

For all we know the NDA prohibits posting of videos showing an accident, the user isn't going to show that. If someone takes a bumper off and doesn't report the accident then there is no record.

 

[mikes_fsd]

Brandon's comments on smoothness of cars driving beside your car and also confirming Chuck's comment about other cars tail lights being rendered on the ICU visualizations.

https://twitter.com/x/status/1413886693858365441

The smoothness of the cars from B-pillar and repeater cam has been something I've been monitoring since last Thanksgiving. It is a visible gap in performance and distance/depth estimation on the transition from repeater to B-pillar cam feeds.

 

[mikes_fsd]

Another example of improvements that I can compare to my car...
Oncoming cars turning towards you from perpendicular intersection -- much smoother than what we have in production.

https://twitter.com/x/status/1413940153647828996

"must know your surroundings"

 

[Battpower]

Right now, it's making many unnatural choices. It's difficult for me to see Tesla doing a wide release until the car acts naturally

^^^^ yes.

software is showing all possible path predictions the car is considering.

Looks different from previous jitters. As you say, the route commit seems very late.

 

[rxlawdude]

I noticed several videos now where the path planning jumps around a lot when stopped at an intersection.

Here is one example where you can see the path planning is all zig zaggy to the left when the car needs to turn right. The path planning keeps jumping back and forth a lot. It is like the planning is not sure what to do.

Here is source video:

Yet it didn't stop dead in the road. You know, like Waymo.

 

[powertoold]

I have a feeling brandone got banned from posting actual FSD videos, lol.

Another example of improvements that I can compare to my car...
Oncoming cars turning towards you from perpendicular intersection -- much smoother than what we have in production.

https://twitter.com/x/status/1413940153647828996

"must know your surroundings"

This blows my mind x100; vision-only magic. Elon said it'll get a lot better in the coming releases. You compare it to something from Mobileye; Tesla is lightyears better at vision-only perception.

 

[gearchruncher]

Elon said it'll get a lot better in the coming releases. You compare it to something from Mobileye, Tesla is lightyears better at vision-only perception.

Is this where we're back to? Elon says the next one will be better, I promise, and what more do you want, it's way better than the competition? Who even cares how the competition is doing with vision only? Tesla's the one that took money for their product and described a feature set in 2016, not Mobileye. E

Elon is the one that said THIS ONE was going to be great. Elon said that this version was mind blowing. capable of intervention free drives most of the time, had fixed almost all known problems, and was on the road to generalized self driving. It's not, so nothing he says about the future carries any weight. We really need to stop using what he says about the future as any sort of data. What we have is what we can see now, after 8 months of development. An incremental improvement. There will be no mind blowing releases or magical accelerations up the S curve. Just a release like this, every 6 months, for the next 10+ years.

 

[powertoold]

Is this where we're back to? Elon says the next one will be better, I promise, and what more do you want, it's way better than the competition? Who even cares how the competition is doing with vision only? Tesla's the one that took money for their product and described a feature set in 2016, not Mobileye. E

Elon is the one that said THIS ONE was going to be great.

I'm wrong about V9 with regards to the driving performance, not as good as my high expectations. It's definitely much better than 8.2 but nowhere near "1 month" to wide release.

Elon isn't wrong about the "step change" or "mind-blowing"-ness that V9 represents. Like I said, I still think V9 is more convincing evidence for human level+ FSD, soon.

 

[Daniel in SD]

Yet it didn't stop dead in the road. You know, like Waymo.

Why doesn't Waymo just program the car not to stop? They're so dumb. Their current software actually stops way more often than their older software! What are they thinking? Maybe we can use our collective intelligence here to figure it out...

 

[gearchruncher]

Like I said, I still think V9 is more convincing evidence for human level+ FSD, soon.

I'm wondering, who would take me up on this deal:
Give me $5K for FSD now. (half current price)
I pay you whatever FSD costs on the day when a 2016-2021 Tesla drove you to work for 30 days without an intervention (assuming at least a 5 mile commute). This can still be L2 according to Tesla. Just zero interventions, door to door.
But I keep the $5K until that happens. Forever if it never does.
If "Human+ Level" FSD is only 30 days, or 90, or 180 days away, this is a great deal. You'll get $10K+, a huge ROI. $50K according to Elon given how much it would cost once this functional.

I'm serious. Someone do it.

 

[mikes_fsd]

Maybe we can use our collective intelligence here to figure it out...

That seems to be what you want to prescribe for Tesla... so, seems disingenuous to whine like that on your part.

 

[jeewee3000]

Is this where we're back to? Elon says the next one will be better, I promise, and what more do you want, it's way better than the competition? Who even cares how the competition is doing with vision only? Tesla's the one that took money for their product and described a feature set in 2016, not Mobileye. E

Elon is the one that said THIS ONE was going to be great. Elon said that this version was mind blowing. capable of intervention free drives most of the time, had fixed almost all known problems, and was on the road to generalized self driving. It's not, so nothing he says about the future carries any weight. We really need to stop using what he says about the future as any sort of data. What we have is what we can see now, after 8 months of development. An incremental improvement. There will be no mind blowing releases or magical accelerations up the S curve. Just a release like this, every 6 months, for the next 10+ years.

Once perception is solid, the rest is just a deep learning issue. Labeled data+compute+time = fast improvements. Dojo will help accelerate the trend. Personal guess: L5 by 2023.

 

[gearchruncher]

Elon isn't wrong about the "step change" or "mind-blowing"-ness that V9 represents.

Can you please expand on what is mind blowing about this release? I hear it's smoother and the visualizations are neat. It's mind blowing that it can use a camera to do this, exactly like our brains do? I'm missing what is so clearly better about this that it shows Tesla's development is accelerating massively.

 

[mikes_fsd]

Labeled data+compute+time = fast improvements. Dojo will help accelerate the trend.

Agreed, but this labeling thing seems to be a real challenge, Krapathy sound optimistic, but not without first pointing to the 4 years of what seems like "hell" in managing a massive data intensive hydra!

 

[gearchruncher]

Once perception is solid, the rest is just a deep learning issue. Labeled data+compute+time = fast improvements. Dojo will help accelerate the trend. Personal guess: L5 by 2023.

You cannot teach a car how to drive by just labeling data. You need reinforcement on behaviors- what is right, what is wrong. Humans do not do the "wrong" thing enough to teach a machine that it can't drive on the sidewalk. Seeing a sign does not mean you can read it. This is why path planning is hard coded.

This is why machine learning can be good at board games. It can play the game 1B times, and it can clearly see if it "won" or "lost". But it can't do that if all it gets fed is watching people drive, and it has no idea what the rules of the game are, or what is a "loss".

You seriously think a Tesla can drive down a snowy forest road at night with someone asleep in the back in the next 18 months!? Take me up on my bet. I'll quadruple it for full L5 vs city L3 in a 2016-2021 Tesla.

 

[powertoold]

Can you please expand on what is mind blowing about this release? I hear it's smoother and the visualizations are neat. It's mind blowing that it can use a camera to do this, exactly like our brains do? I'm missing what is so clearly better about this that it shows Tesla's development is accelerating massively.

 

[gearchruncher]

@powertoold - Yep, that's pretty cool. A nice incremental improvement, like the youtuber says. 100X mind blowing though because it manages to not slow down as much for a car in front? It seems so hyperbolic to say that a normal person watching that would say they never ever thought a computer could drive like that until they saw it, or that it means Tesla is only a few months away from 5 orders of magnitude improvement on disconnect rate.

Take me up on my bet.