FSD v12.x (end to end AI)

[rlsd]

v12.2.1 community gate recognition is still not very good yet.
Last night when entering the right side gate, FSD attempted to make left turn on the opening space for people to make U turn while the gate was opening. On the way to go out of the community, FSD attempted to make left turn to go through the opened left side gate after another car coming to the community.
Basically it doesn't know gate concept. It just knows blocking objects and openings.

 

[Mardak]

So, V12, if it is following the rules would need to confirm that the opposite turning car has turned into its lane or not immediately made a lane change

Yes, what 12.x should do is obvious, I was pointing out that an average selection of lane changes will likely not have a lot of "check opposite/oncoming" vehicles and more of "check behind/side" examples. Even for your intersection, maybe it's above average that people will attempt a simultaneous turn, but it requires 1) enough traffic that people are impatient enough to attempt and 2) people feel comfortable enough in doing the maneuver. As others have pointed out, this behavior is probably not common for a reason, so end-to-end training probably does not naturally have a lot of examples to learn from, but the foundation of 12.x neural networks should allow for learning the correct behavior.

 

[Jeff N]

A few random notes about V2.2.1 that I’m not sure I’ve seen mentioned here yet:

1. It no longer visualizes traffic cones or trash cans (except on the freeway where the old software stack is running).

2. It occasionally does this weird Creepy Wheel thing where every few seconds it turns/nudges the steering wheel a little further to the right when stopped at a traffic light waiting to make a left turn. After 30-40 seconds it will have turned the wheel at quite an angle — I wonder what drivers in the lane to the right of me think is going on as they see my front wheels angling over towards their lane. Then, when the light turns green and we are ready to go, the wheel will smoothly return to the forward direction before the car begins moving forward.

3. Yesterday just after I got in the car I saw a pop-up notice on the center screen telling me that the next time that the car is turned off it would install some system parameter updates of some kind. I’ve never seen a notice like that before. Could these be tweaks to my FSD stack? I dunno.

 

[MP3Mike]

2. It occasionally does this weird Creepy Wheel thing where every few seconds it turns/nudges the steering wheel a little further to the right when stopped at a traffic light waiting to make a left turn. After 30-40 seconds it will have turned the wheel at quite an angle — I wonder what drivers in the lane to the right of me think is going on as they see my front wheels angling over towards their lane. Then, when the light turns green and we are ready to go, the wheel will smoothly return to the forward direction before the car begins moving forward.

I wonder if that is some kind of nag defeat device detection routine.

 

[Mardak]

Hallucination in this context is like swerving around a non-existent car or obstacle, seeing an non-existent situation and responding to it

"Hallucination" has been adopted to describe Large Language Model mispredictions probably to convey the sense that a response output can sound reasonable but not grounded in reality of what the LLM knows or was prompted for. Hopefully we don't end up with people debating what it actually means in the context of end-to-end or whether "phantom braking/swerving" is more appropriate, but people can casually use it to describe any wrong behavior of 12.x's neural networks.

Given that it's end-to-end, the there could be internal/hidden perception issues "detecting" phantom vehicles or misclassifying a vehicle as double-parked, or these could also be control issues of incorrectly deciding it's an appropriate time to go around a vehicle or switch lanes. Similarly, the inconsistent speed on empty roads could be from overfit perception of noise especially at nighttime that control incorrectly decides it needs to be more cautious and slow down, or more generally it's hallucinating away from what people would expect how speed should be maintained.

 

[Mardak]

i did try out Auto Speed Offset for a few days, but for the routes i drive, it's just too slow… I find that even using the old setpoint offset limit, the car isn't zooming up to the setpoint like I'm used to v11.x doing, so this might be somewhat intended v12 behavior that i just have to get accustomed to.

I would think Automatic Set Speed Offset generally allows 12.x to drive at whatever speed it thinks is appropriate (except for what seems like a hardcoded heuristic of no more than 50% of detected speed limit -- this matches the NHTSA recall: "Added a visual glow behind the speed limit icon on the user-interface to alert the driver when the vehicle’s set speed exceeds the detected speed limit by more than 50%"). So your setting an even higher setpoint offset higher than what AUTO would want probably doesn't do much as you've experienced it casually going at a slower speed, and only setting a lower offset changes behavior. I suppose one test is to bump up the offset to the maximum allowed and does it behave any differently from AUTO?

I believe from the original V12 livestream, Elon Musk's vehicle showed the maximum 85mph as the set speed, but that might be concerning for some to see, so they replaced it with AUTO.

 

[rlsd]

A few random notes about V2.2.1 that I’m not sure I’ve seen mentioned here yet:

1. It no longer visualizes traffic cones or trash cans (except on the freeway where the old software stack is running).

2. It occasionally does this weird Creepy Wheel thing where every few seconds it turns/nudges the steering wheel a little further to the right when stopped at a traffic light waiting to make a left turn. After 30-40 seconds it will have turned the wheel at quite an angle — I wonder what drivers in the lane to the right of me think is going on as they see my front wheels angling over towards their lane. Then, when the light turns green and we are ready to go, the wheel will smoothly return to the forward direction before the car begins moving forward.

3. Yesterday just after I got in the car I saw a pop-up notice on the center screen telling me that the next time that the car is turned off it would install some system parameter updates of some kind. I’ve never seen a notice like that before. Could these be tweaks to my FSD stack? I dunno.

1. I saw the same thing.
2. When there are 2 lanes for left turn, FSD sometimes swing back and forth about 1 or 2 feet to the other lane. There were no other cars around.

 

[lzolman]

My driving with it and seeing videos, the approach is sound

People will always be nitpicking stuff about it though, just as I can nitpick any Uber driver

Wow, not the reply I was expecting.
I'm not sure how driving with it and seeing videos reveals any kind of "approach".
I happen to agree that the approach is sound, but my opinion is based on the end-to-end AI approach being taken. I believe that's a sound approach. I have yet to form an opinion on how well they've executed on it.
I can wait, however. I don't want to get it on my car if it's going to make me cry (from exasperation).

 

[AlanSubie4Life]

Specifically checking oncoming traffic for a lane change?

Guess I don’t follow. The car came from the outside turn lane so there was going to be a potential conflict regardless of which target lane FSD chose.

If car maintained lane, would be in outside lane on turn. This would conflict with Tesla’s/FSD’s path.

If oncoming car cut lane to inside turn lane, then final FSD position and car’s final position would conflict.

These seem like common situations which one has to routinely evaluate when dealing with multi-lane turning traffic (which is super common, though usually protected, but not in this case because it was an off-ramp with no continuing road - but in any case one has to time your turn to properly account for what turning traffic is there).

I guess this guy needs a wider angle camera? Annoying that we can’t see the oncoming vehicle except at a distance in the video. It really hides behind the other turning traffic. B-pillar camera (I assume) and the driver could see it though.

Anyway, seems like v12 definitely made the wrong move (certainly should have been a disengagement!!!). While of course the oncoming turning traffic was in the wrong. Hopefully FSD v12 can work out these subtleties! Wonder when they will incorporate reading street signs or whether they expect that to magically come out of the training (if they were doing true end-to-end they would have to)?

 

[fcastrillo86]

I would think Automatic Set Speed Offset generally allows 12.x to drive at whatever speed it thinks is appropriate (except for what seems like a hardcoded heuristic of no more than 50% of detected speed limit -- this matches the NHTSA recall: "Added a visual glow behind the speed limit icon on the user-interface to alert the driver when the vehicle’s set speed exceeds the detected speed limit by more than 50%"). So your setting an even higher setpoint offset higher than what AUTO would want probably doesn't do much as you've experienced it casually going at a slower speed, and only setting a lower offset changes behavior. I suppose one test is to bump up the offset to the maximum allowed and does it behave any differently from AUTO?

I believe from the original V12 livestream, Elon Musk's vehicle showed the maximum 85mph as the set speed, but that might be concerning for some to see, so they replaced it with AUTO.

i haven't tried setting larger offset percentages for city streets driving yet to try and address the granny driving. What I have tried is instead using the scroll wheel to bump up my speed from the 12% offset, (i.e. 40mph posted speed limit > 13% overage = 45mph set speed, then scroll wheel override up to 60mph) with no noticeable effect on the car's behavior. Maybe this is the new normal acceleration profile for v12?

Auto Set Speed basically acts as you understand it, but i wish it would bias towards a default speed offset without cars around. For me, it got a little annoying to be going with the flow of surrounding traffic (i.e. 45mph in a 30 zone), having the nearby traffic eventually leave me (either from them speeding, or separated by traffic lights, etc.), then my car decides to drive at or under speed limit while other cars aren't around to pace, but then another batch of cars catches up (often a combination of being passed in adjacent lanes and some riding my bumper) and only then does the car decide to up the gently accelerate to keep pace again. I wasn't really liking that large of a speed fluctuation, but i wonder if this is the approach it's going to need if the end goal is still driverless/unsupervised?

I will play with the setting some more when i get more seat time. It's my wife's car so I don't get nearly as much time with it as I would like

 

[TK211X]

We’re legitimately due for a new V12 version to pop. It should be pretty imminent based on the Tesla Fi trends.

 

[powertoold]

Day 3 12.2.1:

1) Not so good today
2) Disengaged twice in 6 miles, one was more preferential, another one was in the parking lot, there was a PED crossing with me at stop sign turning left, car was patient for a long time, waiting for 3-4 other peds to cross, but the last PED to cross, seems like car wanted to go (very similar of that edge case video of the PED crossing while he was unprotected lefting)
3) otherwise similar to yesterday

Btw, I'm at a disengagement every 10 miles or so, not counting parking

 

[Daniel in SD]

https://twitter.com/x/status/1762611196455448617

So smart. Maybe it’s trying to cause collisions to lower the bar for deployment?

 

[JulienW]

So smart. Maybe it’s trying to cause collisions to lower the bar for deployment?

Why does he think that is good? That is the most obtuse and accident causing maneuver you can do. If your lane is clear you keep going. NEVER stop in the street to let people turn in front of you with traffic moving. I absolutely HATE when people do things like this. I'm like go the hell on and get out of my way and I will still turn at the about the same time. You are saving me nothing by doing this and endangering lots of people.

 

[aronth5]

Interesting U-turn video
From driver- "v12 makes a wrong turn and then makes a U turn to get back on the right direction"

https://twitter.com/x/status/1762130723048505695

 

[powertoold]

Why does he think that is good? That is the most obtuse and accident causing maneuver you can do. If your lane is clear you keep going. NEVER stop in the street to let people turn in front of you with traffic moving. I absolutely HATE when people do things like this. I'm like go the hell on and get out of my way and I will still turn at the about the same time. You are saving me nothing by doing this and endangering lots of people.

Yea I hate it when people stop for a PED waiting to cross in the middle of a multilane road, it's actually dangerous for the PED in that situation because you're basically pressuring the PED to cross and at the same time obstructing cars in other lanes from seeing the ped crossing lol

That situation in the video isn't ideal, but it's not horrible imo, it does seem like fsd understands the situation and is considering options and chose the more conservative one to stay put

 

[JulienW]

That situation in the video isn't ideal, but it's not horrible imo, it does seem like fsd understands the situation and is considering options and chose the more conservative one to stay put

There was NO reason for it to stop. the lead car was moving and never stopped. Even had it stopped (which it didn't) there was still room for another car before the "box" (what we have and call in ATL).

 

[Daniel in SD]

Yea I hate it when people stop for a PED waiting to cross in the middle of a multilane road, it's actually dangerous for the PED in that situation because you're basically pressuring the PED to cross and at the same time obstructing cars in other lanes from seeing the ped crossing lol

That situation in the video isn't ideal, but it's not horrible imo, it does seem like fsd understands the situation and is considering options and chose the more conservative one to stay put

Don’t be obtuse. The difference is the pedestrian has the right of way. Also the pedestrian has a better view of cars coming.

The situation in the video makes no sense since the car didn’t have to stop at all. You only stop if you can’t make it through the keep clear area.

 

[powertoold]

There was NO reason for it to stop. the lead car was moving and never stopped. Even had it stopped (which it didn't) there was still room for another car before the "box" (what we have and call in ATL).

Yea it was an unnecessary and poor decision, I'm just saying that it does seem like it made a decision and persisted that decision, so at least it wasn't jumping back and forth between decisions

 

[AlanSubie4Life]

https://twitter.com/x/status/1762611196455448617

So smart. Maybe it’s trying to cause collisions to lower the bar for deployment?

I do like how Edge Case 411 is maximizing his opportunity to monetize this. And the more “edge cases” he shows the longer his exclusivity will last!

I don’t think I would pay for that ride though. I think someone would have to pay me. I only trust myself with FSD Beta.