FSD v12.x (end to end AI)

[Usain]

This approach would go against everything Elon seems to believe in. He has been very clear that he is against geofencing or training FSD to be specialized in an area. Rather, Elon has said that he wants FSD to be generalized to work everywhere. So I doubt that Tesla will start curating data to specialize FSD to be a robotaxi in a specific area. Rather, Tesla will continue to gather data everywhere until FSD no longer requires supervision anywhere. It would actually be easier for Tesla to do what you suggest. But that is how Elon wants to do things and I don't see things changing unless somehow Elon did leave Tesla.

I agree with you, but Elon has been known to change his mind. Elon and others thought universal FSD would be a lot easier than it has turned out to be.

It's pretty clear that Elon wants to launch robotaxi soon. So I expect the desire to launch soon will lead to the question, "How do we actually get started?"

I'm just doing the thought experiment where Tesla decides to launch. How would that look? First, pick a city where FSD already works well. Second, make sure FSD works well enough to actually launch. If overfitting gets you there without appreciable harm to the ultimate goal of universal FSD, then Tesla is likely to take that route.

 

[diplomat33]

I agree with you, but Elon has been known to change his mind. Elon and others thought universal FSD would be a lot easier than it has turned out to be.

It's pretty clear that Elon wants to launch robotaxi soon. So I expect the desire to launch soon will lead to the question, "How do we actually get started?"

I'm just doing the thought experiment where Tesla decides to launch. How would that look? First, pick a city where FSD already works well. Second, make sure FSD works well enough to actually launch. If overfitting gets you there without appreciable harm to the ultimate goal of universal FSD, then Tesla is likely to take that route.

Sure, a robotaxi launch might look like that. But as far as I can tell, Elon is still 100% committed to the idea of universal FSD and he keeps believing that it will happen very soon. Elon wants Tesla to "solve L5" first and then launch robotaxis everywhere all at once. Elon is convinced that L5 is right around the corner. So he thinks that they won't need to "pick a city and and overfit data to that city to launch driveress" because he thinks FSD will be driverless everywhere soon. And he is sticking to it.

 

[aronth5]

Another Omar V12 drive in LA but this time with his father who also uses FSD. His father was very positive and I think it's becoming clear that V12 will finally be a significant upgrade. About time! As much as people complain about Omar and I've been one of them he has done a pretty good drive showing V12 including the AI Driver video and this video.

 

[Usain]

Sure, a robotaxi launch might look like that. But as far as I can tell, Elon is still 100% committed to the idea of universal FSD and he keeps believing that it will happen very soon. Elon wants Tesla to "solve L5" first and then launch robotaxis everywhere all at once. Elon is convinced that L5 is right around the corner. So he thinks that they won't need to "pick a city and and overfit data to that city to launch driveress" because he thinks FSD will be driverless everywhere soon. And he is sticking to it.

If that's true then I'm wrong.

I don't think it's feasible to launch everywhere at once. For logistical reasons Tesla will need to pick a city or two to get started. If universal FSD is good enough to launch soon then Tesla stock goes through the roof.

But I expect the desire to start the robotaxi service soon (1 to 3 years) will win out and it will happen before universal FSD is good enough. I actually do hope I'm wrong though.

 

[diplomat33]

If that's true then I'm wrong.

I don't think it's feasible to launch everywhere at once. For logistical reasons Tesla will need to pick a city or two to get started. If universal FSD is good enough to launch soon then Tesla stock goes through the roof.

But I expect the desire to start the robotaxi service soon (1 to 3 years) will win out and it will happen before universal FSD is good enough. I actually do hope I'm wrong though.

I agree with you with launching robotaxis everywhere at once is not feasible, certainly not right now. I just think Elon will stick to his universal FSD notions even if he is wrong. Heck, Elon said that FSD was a solved problem back in 2015. Here we are 9 years later and he is still convinced that L5 is right around the corner. If he was going to give up the notion of L5 and do localized robotaxis, it would have happened by now. And if Elon keeps believing that universal FSD is going to happen "at the end of the year" then he will be able to justify delaying localized robotaxis.

But yeah, the question will be at one point does the desire to actually launch a legit robotaxi service trump Elon's over optimism that L5 is close to being solved? Or does Tesla never launch a robotaxi service because Elon just sticks to the idea of universal FSD?

 

[lzolman]

Why shoot for L5 out of the gate? That would be like NASA planning to launch a manned mission to Mars before testing any of the rocket/ship/lander components first. L5 is not particularly important except maybe for a robotaxi, and even then there can be rules for just pulling over when necessary. IMO, L4 would be "just as good" in terms of advancing the FSD ecosystem, as it would be so much better than what we currently have.

 

[sleepydoc]

Given that is difficult for a human, I think it is impossible for FSD, unless they include a Super-sensitive Pothole Avoidance Maneuvers (SPAM) mode.

This mode would swerve all over the road all the time, avoiding anything that looks out of place.

The problem is false positives. Distinguishing depth vs simple pavement markings is really difficult. Potholes can vary significantly in how they look.

They could map them of course, but that is also going to be sad for some people. Would need a rapidly responding system not based on Tesla driving data. Waze has this but is of course hit or miss.

Elon said it’s coming in v69. Going out to testers in 2 weeks.

 

[sleepydoc]

Unless it's written that the driver is responsible. To take liability for pothole damage is not gonna happen.

But it is - down on page 132 of the terms and conditions.

 

[sleepydoc]

I wonder how you have the setting that allows you to save some time by allowing the car to make an automatic change. Make the time savings unrealistic or turn that setting off.

I’m not sure if Griz is talking about the same issue but on one route I take FSD will deviate from the course and try to take a left turn into a residential neighborhood virtually every time. There is no viable path to my destination through this neighborhood and it it will attempt to do so even when there’s no traffic. It’s simply defective lane planning/selection

 

[sleepydoc]

I left out "should".




What it sees "should" always take priority over the map.

If it is relying too heavily on the map, that is a crutch that needs to be discarded.

If / when you get V12, it will be interesting to see if the behaviour persists,

I posted this before but the DOT put up some temporary stop signs on a detour route last summer. FSD clearly saw the stop signs but frequently failed to stop.
More recently there’s a road/intersection at which FSD will come to a complete stop for no reason, treating it like a stop sign. (Full stop, ‘creeping ahead for visibility,’ proceed) There is no stop sign here and never has been.

It’s pretty clear to me that FSD prioritizes map data over visual data. (Except where speed limits are concerned)

 

[Daniel in SD]

Detailed extensively in 10.69 thread.

Same metric as before.

No interventions by the driver or other drivers. (Covers creeping at the wrong time or too far, cutting it too close, not accelerating up to speed and forcing traffic to go around or slow down, etc.). Any honking from other drivers is disqualifying.

Missing large easy opportunities (more than five-second gaps).

So basically just normal driving. No mistakes, no weirdness, no interventions.

Using the median is weirdness and very silly, but that’s allowed.

Anyway no change from prior rules in 10.69 thread. That covers minimum number of attempts too.

I can almost taste the beer. We might even see Chuck's neighborhood is the perfect location to start the robotaxi service.

https://twitter.com/x/status/1754879057756315837

 

[powertoold]

Incredible post, incredible insight

https://twitter.com/x/status/1695562811190210890

 

[AlanSubie4Life]

I don't think there is any way to solve that

Umm...I should hope there is, otherwise Robotaxis may have a rough go of it in the next couple years.

Someone points out that this is difficult to train for...and then you say it's not solvable. Hmm.

Robotaxi is already a reality with Waymo. They have driverless vehicles that pick up passengers and take them to their destination.

Not what I meant in this context. There's an entire thread about this, anyway.

and I think it's becoming clear that V12 will finally be a significant upgrade

How do you know that V11 similarly configured would not be just as assertive and smooth?

I'm not saying it would be - just pointing out we're not comparing apples to apples (released version, vs. alpha version in limited release to "vetted" (and I use the term loosely) users).

I can almost taste the beer.

Surprised you're not concerned about the apparent lack of a simulator.

 

[JB47394]

Another Omar V12 drive in LA

That was a good drive. From sunset to darkness, and mostly in light rain.

• False positive collision warning when passing cars stopped in an adjacent lane
• Passing through an automatic security gate
• Setting speed to 38 on a four lane divided road with a posted speed limit of 25 mph.
• Turning around in a cul-de-sac.
• Pulling into a parking spot in front of Target at the end of a drive

 

[AlanSubie4Life]

Pulling into a parking spot in front of Target at the end of a drive

Notice it did not. (This is a persistent problem with Autopark of course, in the reverse direction.)

Difficult to watch (misses a 12-second gap):

 

[aronth5]

Notice it did not. (This is a persistent problem with Autopark of course, in the reverse direction.)

Difficult to watch (misses a 12-second gap):

Did I misunderstand your post? But @ 30:03 FSD did park in a customer pick up spot in front of Targets. Omar then moved the car since they weren't picking up anything.
FSD also pulled over when they got to a friends house @17:40 which is a nice improvement.

 

[sleepydoc]

This for one example, but there are many others

DIRECTIONAL EMERGENCY SIRENS LTD

Fire & Rescue Sirens Ltd

www.directionalsirens.com

Minneapolis has ‘shot spotter’ microphones in some areas that are able to localize gunshots. If they can do that a car should be able to localize sirens.

 

[JB47394]

Yes FSD parked in a customer pick up spot in front of Targets.

I think he's objecting to the fact that the car didn't pull completely into the spot. It was about 80% of the way in. At this point, I don't care. It identified a spot and pulled in. As a first effort with neural networks, I thought it worth pointing out.

 

[Usain]

Umm...I should hope there is, otherwise Robotaxis may have a rough go of it in the next couple years.

Someone points out that this is difficult to train for...and then you say it's not solvable. Hmm.

So a pedestrian is suddenly thrown in front of your car and you think avoiding the collision is a solvable problem?

I guess you could solve it with a crystal ball.

The thing I said you could solve for is what happens after the collision. The Cruise vehicle made the situation worse by dragging the poor soul as it tried to pull over. That one can be solved with simulations.

 

[AlanSubie4Life]

you think avoiding the collision is a solvable problem?

No. The original question was addressing what happened after the unavoidable collision (that is what got Cruise shut down) and how to train for that:

How are you going to use end-to-end training to solve the incident that got Cruise shut down?

So back to that question: how does one train for this with end-to-end? Your answer was simulations - hopefully that will work! (I think we kind of talked past each other here.)