FSD v12.x (end to end AI)

[Joboo7777]

I can't wait for the day when the highway stack goes to v12.

The v11 highway stack is usually pretty good, but it does a terrible job of merging into a lane w bumper to bumper traffic. It will slow down, speed up, turn on its blinkers, turn off its blinkers, and repeat a combination of these behaviors before finally getting into the new lane.

It's extremely unnatural and unhuman like and if I was the car behind the tesla, I would think the person driving is an idiot.

Interesting you say this. I find merging has been quite good these days. Everyday on my way home from work I have this specific use case. Bumber to bumper traffic on the highway and as I enter using the on ramp it has done a pretty good job at finding a slot to merge. The traffic I am merging in to is almost at a dead stop and is barely crawling so maybe that has something to do with it?

I have a few other merge use cases on different roads that have also been quite good. Weird how the experience can be so different across cars.

 

[KArnold]

Or, your map was updated. How do you know it wasn't? If you live in a place where the maps are constantly updated and accurate, you wouldn't know.

Doesn't really matter. A bug by any other name smells just as sweet.

 

[MP3Mike]

Or, your map was updated. How do you know it wasn't? If you live in a place where the maps are constantly updated and accurate, you wouldn't know.

Because my car hasn't downloaded updated map data? (Tesla only updates the map data once or twice a year.)

 

[zoomer0056]

Or, your map was updated. How do you know it wasn't?

Look at the software page to determine map data version.

 

[JB47394]

It does in some cases. An intersection I go through just had a turn lane added, and FSD saw and visualized it as soon as it opened up.

The traffic visualization is V11 code, and routing is provided by a central service. So could V11 be actively visualizing the lanes, and V12 is maneuvering by the latest central data? I assume that the local map data is for the map visualization. It doesn't need updating very often.

By this theory, DrGriz's intersection in Idaho hasn't been updated by the central service, while MP3Mike's in Oregon was.

 

[OldManCan]

Talking about visualization... what happened to the trash container visualization? Somehow I notice they are not shown anymore when lined up on the side of the road. They used to show before but I think post 12.3.4 or .5 maybe they were dropped.

 

[MP3Mike]

Talking about visualization... what happened to the trash container visualization? Somehow I notice they are not shown anymore when lined up on the side of the road. They used to show before but I think post 12.3.4 or .5 maybe they were dropped.

They went away with the V12.x updates. (It appears that the visualizations are a stripped down version of what V11.x used.)

 

[David Wang]

Look at the software page to determine map data version.

I have the latest map data NA-2024.8-14924, but it didn't fix any of the incorrect lane information in my neighborhood. Here's hoping that 12.4 adds the ability for fsd to read (improve?) the lane markings in real time.

 

[David Wang]

Interesting you say this. I find merging has been quite good these days. Everyday on my way home from work I have this specific use case. Bumber to bumper traffic on the highway and as I enter using the on ramp it has done a pretty good job at finding a slot to merge. The traffic I am merging in to is almost at a dead stop and is barely crawling so maybe that has something to do with it?

I have a few other merge use cases on different roads that have also been quite good. Weird how the experience can be so different across cars.

I think it depends. If the lane that you are merging "from" is moving at a similar speed to the lane that you are merging "into", the merging algorithm is pretty good.

The issue that i see is when the lane that you are merging "from" is moving at a much faster speed than the lane you are merging "into", then the lane merging algorithm does all these weird unnatural things that I pointed out in my original post.

 

[OldManCan]

They went away with the V12.x updates. (It appears that the visualizations are a stripped down version of what V11.x used.)

it still visualizes traffic cones on the side of the road so I think it sees the objects such as trash cans but not shows it on the screen. As long as it sees them and avoids them I'm ok with it.

 

[stopcrazypp]

Or, your map was updated. How do you know it wasn't? If you live in a place where the maps are constantly updated and accurate, you wouldn't know.


What you are proposing, that FSD reads signs and lanes some places and not in others, makes no sense to me. How could that even be? Mind you, I agree that it used to. It's broken now.

Turn arrows on the lane itself is one of the few things FSD visualizes. It doesn't do that for road signs other than speed limits and stop signs.

Whether it acts on that information is another matter.

For your particular example are you saying that even when the road is completely clear (such that turn arrows is visible) the car doesn't visualize it?

What I did observe just driving is the lane arrow marking will disappear if it is partially obscured by another car. That seems to indicate it is read in real time, not something that is based solely on a map. If also doesn't show up until it is in visual recognizable range.

 

[Mardak]

When it is perfectly trained, what does it do when it has two exactly equally correct scenarios? That is what I am curious about.

Depends on what you mean by equally correct. If it's taking either of 2 turn lanes that are empty with no need for upcoming maneuver, e.g., subsequent turn, it could learn what humans tend to do in picking whichever lane is "easy" (could depend on the road layout), so there is still some implicit bias that results in a split of say 60%/40%.

Even with close to 50%/50% in selecting a lane, additional training could also learn that humans tend to stay in the lane once entering, so a true 50%/50% could be basically random in picking the left or right, but once the steering wheel starts biasing to a particular lane, the split could shift more towards 60%/40% avoiding the indecision.

More complicated would be either lane is equally correct in one aspect but a lane has more vehicles waiting in it, and some people do indeed start in one lane and decide a bit later to switch into the shorter lane. Perhaps initially entering the longer lane the preference was 60%/40% then shifts towards 50%/50% as it gets closer to the long line and flips to 40%/60% deciding to switch.

For each of these, additional training can make these decisions more sharp / opinionated with a higher drop-off to the next best action, e.g., 70%/30% is a 40% difference instead of 60%/40%'s already 20% difference. This can make the networks more confident but also has the potential drawback of overconfidence in not realizing something needs to change until a really strong signal is detected that lets it change its mind.

 

[MP3Mike]

it still visualizes traffic cones on the side of the road so I think it sees the objects such as trash cans but not shows it on the screen. As long as it sees them and avoids them I'm ok with it.

Only when you are on a controlled access highway and it is using the V11 stack. When you are on surface streets using the V12 stack it doesn't show traffic cones anywhere.

I suppose if there was a trashcan on the side of a highway it would show there as well when the V11 stack was in use.

 

[JulienW]

Not a big deal and actually the way it should be but I found it mildly interesting. Auto park would not recognize my parking neighbors open space since she placed a (my) cone in it. I even tried several times backup up/forward and pressing the screen but it stayed blank.

Kinda feel bad and need to wipe that cone off since it is covered in dust.

 

[Mardak]

Is there a reason why they can't poll the entire fleet for video training data, even those with just basic AP?

Tesla most likely does poll the entire fleet potentially even those in Europe with FSD 12.3.6 included in the software but not exposed to the driver to activate. It depends very much on the type of data Tesla needs to collect as some might want active usage of FSD Supervised to find disengagements and others can be collected in shadow mode without Autopilot even engaged.

The context we were discussing FSD mileage is how Tesla has been reporting cumulative total miles driven FSD Beta/Supervised in the quarterly reports. Typically these updates only included up to the last month of the report, but Tesla specially for Q1 2024 included data through April 21st to highlight the sudden spike.

The slope of the blue line around January and February this year is slightly less than the slope before it (~65M/mo vs ~75M/mo) while the slope around March slightly increased (~85M/mo) with the red line including V12 presumably from increased usage on city streets with better performance of V12 compared to V11 from the existing fleet. But the big increase mid April (~300M/mo) seems to primarily come from Tesla giving FSD free trials, so as the 30-day trial ends in May for some (while others previously stuck on 2024.8.x are newly getting the trial with 2024.14.x), the slope should decrease to less than the red spike but it should still be steeper than March.

 

[FSDtester#1]

Isn't there a rumor that Tesla already has a Robotaxi specific version of FSD?

Yes, it was me passing said rumors

 

[JulienW]

Yes, it was me passing said rumors!

Wait, I heard a rumor that you only pass facts and occasional gas.

 

[JulienW]

Look at the software page to determine map data version.

We do get some Delta Map updates in-between found by Greentheonly.

Tesla Hacker Reveals How Maps Are Augmented with Fleet Data Between Updates

Tesla hacker Green reveals the augmentation of Tesla's maps with fleet data between updates, enhancing Autopilot and Full Self-Driving experiences for safer and

www.notateslaapp.com

 

[Mardak]

because the UI doesn't show a full lane plan for the whole route, it's not possible to know directly if there is one

Yeah, the navigation / map data can include lane information and is shown for upcoming turns, but there's a lot of lane information that isn't shown that 11.x definitely relied heavily on to avoid upcoming turn lanes and incorrectly got out of lanes because of incomplete map data making 11.x think it was in the wrong lane.

Has anybody found a better way to get at this map data that might be dynamically updated with live navigation? I've definitely noticed different behaviors when it uses offline navigation presumably based on outdated or inaccurate map data previously downloaded. One possibility is to keep updating navigation to make the next turn (instead of continuing straight) so that the UI might show the number of lanes and the turn lanes. But this is pretty inconvenient as the UI only shows the next turn and sometimes doesn't even show that information (e.g., expanding directions then collapsing seems to sometimes dismiss the lanes UI).

 

[FSDtester#1]

Wait, I heard a rumor that you only pass facts and occasional gas.

Definitely a rumor. Not the gas part
I am the king of the Dutch Oven...