FSD v12.x (end to end AI)

[stopcrazypp]

Nice sunny Sunday morning. The intersection was just painted after the winter sand and salt scrubbed the paint. No one else in the intersection. Couldn't be more perfect. The car drove into the middle (straight only) lane (green light) and ignored the left turn only lane (green arrow) and attempted the left turn I had set navigation for. This is the same spot that would have been the left turn lane a year ago. It followed an out of date map.

Another example. Same intersection, nav set to go straight this time. It pulls into the right turn only lane (also with an arrow) and attempts to go straight.

It acts as as though there are only two lanes (which it did a year ago) instead of three.

I didn't give it too much thought when the lanes weren't clearly striped, but now they are.

I have 2024.8 map software. Date doesn't make it accurate.

But does the arrow show up in the visualization of the road for the correct lane (s)? That was my main question.

Again, whether the car can read and recognize the arrow is separate from whether it acts on it. It may weigh any map data more heavily, but that doesn't mean it can't read the turn arrow marking.

An example of this is the speed limit signs (it has been tested before that it can read fake speed limit signs that do not exist on a map), but whether it acts on that sign is a completely different manner.

 

[enemji]

But does the arrow show up in the visualization of the road for the correct lane (s)? That was my main question.

Again, whether the car can read and recognize the arrow is separate from whether it acts on it. It may weigh any map data more heavily, but that doesn't mean it can't read the turn arrow marking.

An example of this is the speed limit signs (it has been tested before that it can read fake speed limit signs that do not exist on a map), but whether it acts on that sign is a completely different manner.

FSDs will always pick the rightmost turn lane on the left turns to avoid curb rash / jumping the median for most times when getting in the lane on the other end

 

[swedge]

I noticed that black tesla got updated to 2024.14.x. Isn't he one of the original testers?

In the video points out that this is a main branch version, and that it includes FSD 12.3.6. This version was released starting on 4/6/24. That 12.3.6 version of FSD was first released in 2024.3.25 on 4/28/24. So, if this car did not already have 12.3.6, it is not a tester.

Here are some of my observations about the differences between the tester and main branches.

Not all on 2024.3.25 are "testers", it is a mix of testers and cars which first opted for FSD after 5/23/23 when version 2023.12.10 with 11.3.6 became available to anyone who signed up for FSD after that date. I think the everyone on FSD prior to that were testers who had received 11.3.6 starting 4/8/23.

As a general rule, a car is not updated to a lower release or a to a lower FSD number. Main branch cars tend to receive multiple updates with the same FSD version, but testers generally do not. (An exception was 2023.44.30.x w/ FSD 11.4.9 with a flurry of updates related to a recall.) Usually, testers' updates have year.week release numbers lower than what main branch cars already have.

It appears that tester may have all received FSD before 5/23/23 when 2023/12/10 with FSD 11.3.6 included. After that, all releases included the FSD code, and was available to anyone who bought, subscribed or free-trialed FSD.

I've been taking advantage of TeslaFi's ability to display the update history of each car in their fleet, trying to understand the tester branch, which I am on. I posted a comparison between my updates and a typical main branch car a few days ago here.

Like much about Tesla, it is a bit of a black box.

 

[joelkfla]

Turn arrows on the lane itself is one of the few things FSD visualizes. It doesn't do that for road signs other than speed limits and stop signs.

Whether it acts on that information is another matter.

For your particular example are you saying that even when the road is completely clear (such that turn arrows is visible) the car doesn't visualize it?

What I did observe just driving is the lane arrow marking will disappear if it is partially obscured by another car. That seems to indicate it is read in real time, not something that is based solely on a map. If also doesn't show up until it is in visual recognizable range.

Clearly, speed limit signs are being digested. Else it wouldn't speed up to 50 mph after passing a Route 50 marker.

 

[phxazcraig]

In the video points out that this is a main branch version, and that it includes FSD 12.3.6. This version was released starting on 4/6/24. That 12.3.6 version of FSD was first released in 2024.3.25 on 4/28/24. So, if this car did not already have 12.3.6, it is not a tester.

Here are some of my observations about the differences between the tester and main branches.

Not all on 2024.3.25 are "testers", it is a mix of testers and cars which first opted for FSD after 5/23/23 when version 2023.12.10 with 11.3.6 became available to anyone who signed up for FSD after that date. I think the everyone on FSD prior to that were testers who had received 11.3.6 starting 4/8/23.

As a general rule, a car is not updated to a lower release or a to a lower FSD number. Main branch cars tend to receive multiple updates with the same FSD version, but testers generally do not. (An exception was 2023.44.30.x w/ FSD 11.4.9 with a flurry of updates related to a recall.) Usually, testers' updates have year.week release numbers lower than what main branch cars already have.

It appears that tester may have all received FSD before 5/23/23 when 2023/12/10 with FSD 11.3.6 included. After that, all releases included the FSD code, and was available to anyone who bought, subscribed or free-trialed FSD.

I've been taking advantage of TeslaFi's ability to display the update history of each car in their fleet, trying to understand the tester branch, which I am on. I posted a comparison between my updates and a typical main branch car a few days ago here.

Like much about Tesla, it is a bit of a black box.

I've been trying to figure out the statistics too. I have 2024.3.25 and FSD 12.3.6. I had subscribed to it for 2 months before 12.3 came out, which may explain why I got it to start with.

In looking at the stats (notateslaapp, software updates, pick a version, statistics) for 2024.14.8. and the overall stats, it seems clear that the bulk of the latest updates are getting people that never had 12.3 a change at it, plus the free month trial. Almost all of the movement is 2024.8 to 2024.14. Meanwhile the 2024.3 sits unchanged, week after week.

I noticed that the 2024.14 rollout really picked up speed, then crashed with bug reports cropping up. Now it seems to be proceeding again, cautiously. Looking at the daily stats for 2014.14 you see waves of updates, every other day on odd-numbered days. I wonder which vehicles get picked for those days?

 

[stopcrazypp]

Clearly, speed limit signs are being digested. Else it wouldn't speed up to 50 mph after passing a Route 50 marker.

But the point is it doesn't always strictly follow the speed sign, especially if you have the automatic speed setting. The same thing may apply to the turn arrows on the ground. It may read the turn arrow, but that doesn't mean it has to follow it (it can choose to favor map data).

 

[sleepydoc]

There are hundreds (exaggeration, obviously) of posts of people disengaging FSD and TACC still being on and scaring them or FSD not activating and TACC activating instead.

I never used TACC and found it annoying that steering wheel disengagement kept TACC on.

To which I say, "if you're not going to pay attention, don't drive a effin' car!"

Every time I turn on FSD I listen for the double chime and glance down at the screen to verify that it's engaged. Every time I turn on TACC I listen for the chime, check the set speed on the screen and hold my foot over the accelerator until I've verified that it's engaged. Every time I use cruise control on our other car I do the same.

When I'm using FSD I'm watching the road with my hands next to the wheel so I can take over if it does something stupid. If it disengages it gives an audio signal and the screen changes and oh, I happen to notice as the car starts to drift out of its lane and I take over immediately.

Now what they've done is make it so you can't use TACC at all but you can still disengage FSD just as easily. One can argue that it's even more dangerous since it will suddenly stop in the flow of traffic and since speed control is so horrible with 12.3 you just assume it's messing up anyway.

Sorry, I have no sympathy for those who don't pay attention and can't figure out what the car is doing.

 

[spev18]

In the video points out that this is a main branch version, and that it includes FSD 12.3.6. This version was released starting on 4/6/24. That 12.3.6 version of FSD was first released in 2024.3.25 on 4/28/24. So, if this car did not already have 12.3.6, it is not a tester.

Here are some of my observations about the differences between the tester and main branches.

Not all on 2024.3.25 are "testers", it is a mix of testers and cars which first opted for FSD after 5/23/23 when version 2023.12.10 with 11.3.6 became available to anyone who signed up for FSD after that date. I think the everyone on FSD prior to that were testers who had received 11.3.6 starting 4/8/23.

As a general rule, a car is not updated to a lower release or a to a lower FSD number. Main branch cars tend to receive multiple updates with the same FSD version, but testers generally do not. (An exception was 2023.44.30.x w/ FSD 11.4.9 with a flurry of updates related to a recall.) Usually, testers' updates have year.week release numbers lower than what main branch cars already have.

It appears that tester may have all received FSD before 5/23/23 when 2023/12/10 with FSD 11.3.6 included. After that, all releases included the FSD code, and was available to anyone who bought, subscribed or free-trialed FSD.

I've been taking advantage of TeslaFi's ability to display the update history of each car in their fleet, trying to understand the tester branch, which I am on. I posted a comparison between my updates and a typical main branch car a few days ago here.

Like much about Tesla, it is a bit of a black box.

In case you missed it, Elon said on May 20th that software branches and FSD are going to be synced up after the 12.4 rollout. If true I'm hoping this means that beginning with FSD 12.5, we should receive the latest FSD and vehicle software in the same package. No more new FSD releases on an old branch which would eliminate being held back like the 2024.3.25 folks currently. That will be a welcome change.

https://twitter.com/x/status/1792568940943565246

 

[uscbucsfan]

To which I say, "if you're not going to pay attention, don't drive a effin' car!"

Every time I turn on FSD I listen for the double chime and glance down at the screen to verify that it's engaged. Every time I turn on TACC I listen for the chime, check the set speed on the screen and hold my foot over the accelerator until I've verified that it's engaged. Every time I use cruise control on our other car I do the same.

When I'm using FSD I'm watching the road with my hands next to the wheel so I can take over if it does something stupid. If it disengages it gives an audio signal and the screen changes and oh, I happen to notice as the car starts to drift out of its lane and I take over immediately.

Now what they've done is make it so you can't use TACC at all but you can still disengage FSD just as easily. One can argue that it's even more dangerous since it will suddenly stop in the flow of traffic and since speed control is so horrible with 12.3 you just assume it's messing up anyway.

Sorry, I have no sympathy for those who don't pay attention and can't figure out what the car is doing.

There was enough mode confusion for the NHTSA and Tesla to make a change.

The Refresh MS/X have always had the option for single pull/disable TACC. I thought that was a great compromise, but it's been decided that wasn't good enough.

I personally find TACC useless, but as always welcome options.

 

[David Wang]

Speaking of lane selection, as people have noticed and commented here, fsd v12.3.6 behaves differently based on its surroundings for the same route.

There is an intersection in my area where v12.3.6 will go straight in the straight lane when there are no cars in front of me.

If there are cars in front of me in the straight lane, v12.3.6 will go into the left turning lane and then try to go straight.

The problem with this behavior, besides the fact that its trying to go straight in a left lane, is that there are 2 traffic lights, one for turning left and one for going straight. If the turning left light is green, the car won't move because its looking at the straight traffic light which is still red.

And then obviously i have to take over.

Is this an issue with map data or incorrect AI training data or both?

 

[sleepydoc]

There was enough mode confusion for the NHTSA and Tesla to make a change.

The Refresh MS/X have always had the option for single pull/disable TACC. I thought that was a great compromise, but it's been decided that wasn't good enough.

I personally find TACC useless, but as always welcome options.

To be honest, with 12.3 I have far less need for TACC. Ironically, 12.3 seems to do everything better except manage speed. With past versions I would routinely use it when I didn't feel like dealing with FSD. I also use TACC/cruise control routinely for city driving as I've found it to be the best way for me to avoid speeding (especially in my Tesla.)

Even with its improvements many people still have a lot of occasions in which FSD does not do well, and many people prefer to drive rather than using FSD. This change makes it difficult to use both, essentially forcing people to use one or the other. Yes, you can do things like set up multiple profiles but if someone can't figure out whether they're in FSD vs TACC do we have any hope of them having the wherewithal to manage multiple driver profiles?

 

[PianoAl]

It seems to me most of the times it doesn't use a signal [for merging] is when there are no nearby vehicles.

In the three merges I've done since reading this, that's been the case:

No cars, no signal
Car in right lane, signal
Car in left lane, no signal.

But when you think about it, what would be the point of this? Easier to just always signal. But perhaps this is due to the NN training.

 

[MP3Mike]

But when you think about it, what would be the point of this? Easier to just always signal. But perhaps this is due to the NN training.

Why waste the energy flashing the lights if it isn't necessary?

Yes, easier to always signal if you are writing procedural code. But with this being end-to-end NN based, that means that it is likely that in the training videos human drivers only signaled when they thought it was necessary, not when they are supposed to. I think this will likely be an issue with various things, and likely recalls in the future, where they will have to better curate their training videos to only include clips from good/professional drivers that always follow the rules of the road. (It is currently too much like a normal/bad driver in some situations.)

 

[David Wang]

Why waste the energy flashing the lights if it isn't necessary.

Yes, easier to always signal if you are writing procedural code. But with this being end-to-end NN based, that means that it is likely that in the training videos human drivers only signaled when they thought it was necessary, not when they are supposed to. I think this will likely be an issue with various things, and likely recalls in the future, where they will have to better curate their training videos to only include clips from good/professional drivers that always follow the rules of the road. (It is currently too much like a normal/bad driver in some situations.)

i'm wondering how much of the training data is from real traffic and how much is from using a simulator?

Can't tesla model this behavior by injecting data from a simulator?

 

[MP3Mike]

Can't tesla model this behavior by injecting data from a simulator?

Sure, they probably could. But that would likely be a lot of work as well. You would have to simulate all of the different highway merge topologies, with all the various traffic conditions. Essentially program to replicate all of the real-world video clips they are using, but always following the laws. (Curating the videos clips might be easier and cheaper.)

They would, also, have to find, and remove, all of the current training clips showing bad behavior.

 

[OldManCan]

Why waste the energy flashing the lights if it isn't necessary?

This following is from the code book for CA. It doesn't talk about whether or not there is another car in the lane you're merging into etc. You must signal or face the potential ticket.

Motorists must give a signal at least 100 feet before they turn or change lanes. A driver who violates Vehicle Code 22108 must pay a fine of $238.00. A motorist who does not signal also receives one point on his DMV driving record.

 

[sleepydoc]

This following is from the code book for CA. It doesn't talk about whether or not there is another car in the lane you're merging into etc. You must signal or face the potential ticket.

If you turn on your blinker but no one’s around to see it, does it actually blink?

 

[swedge]

In case you missed it, Elon said on May 20th that software branches and FSD are going to be synced up after the 12.4 rollout. If true I'm hoping this means that beginning with FSD 12.5, we should receive the latest FSD and vehicle software in the same package. No more new FSD releases on an old branch which would eliminate being held back like the 2024.3.25 folks currently. That will be a welcome change.

https://twitter.com/x/status/1792568940943565246

I did see that, but it is not clear what any of it meant. The X posting starts with a complaint about waiting for updates, which always happens. But it states that many are stuck on FSD 11.1, which is not true. TeslaFi shows no one still on FSD 11.1. Probably, the poster was referring to the UI version, which everyone has had for a long time.

Elon's response does address an issue which slows down their process of releasing updates, the proliferation of hardware variants. Each revision of FSD or the vehicle code requires modifications and testing for each variant. In cases where both FSD and the vehicle software are changed, as is happening with 2024.14.x, which is brining the main branch up to V12.3.6 FSD from V11.4.9 FSD.

The issue I brought up is what I call the "tester" branch, most of whom are still on 2024.3.25, and I am fining none who have been updated to 2023.14.x. Among this group are folks who requested FSD as far back as 2021, and many had to pass the Safety Score test to be allowed in. One article suggests there were around 100,000, and those (like myself) who had purchased rather than subscribed have not found a way to leave the tester branch.

Since the FSD beta testing started, these folks have been comparing notes on these TMC threads. Now that V12 is becoming available to all owners, folks new to FSD are also coming to this thread which has extensive commentary on the 7 prior versions of FSD 12 waiting back to Feb, all of which are obsolete. Further, the testers are expecting and commenting on FSD 12.4 which may drop any day now, but which the main branch owners may not get for some time. We'll see.

All that said, I agree that it would be nice if all the FSD updates were released with the current main branch vehicle software. But Telsa may well keep the tester group separate so they can rapidly cycle through iterations to clean them up before rolling out to the entire fleet, as they have been doing since 2021.

 

[Dewg]

This following is from the code book for CA. It doesn't talk about whether or not there is another car in the lane you're merging into etc. You must signal or face the potential ticket.

The argument here is that a merge is neither changing lanes nor turning.

 

[JulienW]

The argument here is that a merge is neither changing lanes nor turning.

Yes there is ONLY ONE direction the car can go. So a signal to say you are going the ONLY direction you can go is redundant.