FSD v12.x (end to end AI)

[jwalkingjon2]

If speed < TargetSpeed (
Set Acceleration = ON;
)
if (NextCarDistance < FollowDistance
|| (VRU == TRUE)
|| (TargetSpeed > 500ftTargetSpeed)
|| (PassengerScreaming == TRUE)){
Set Acceleration = OFF;
}
//*shrug* seems pretty easy to me!

Needs more if statements

IF car <bumpersticker = "You're following too close -or "*insert other dumb bumpersticker*" {

Set acceleration = Max
Set breaking = Off
Play murica f ya theme song after rear end = true
}

 

[enemji]

Joking aside, 12.3 would generally drive 5-10 MPH under the speed limit.

Nope. Mine drive 7+ mph over

 

[jabloomf1230]

Then it's never going to work.
There isn't enough storage nor compute in the universe to code the proper response to every possible combinations of inputs.
The only reason any approach could be feasible is that things can be generalized. Rather than deal directly with every combination of pixels that can be a car, the NN outputs show how "carish" an object is. Then the next layer(s) takes those approximations and acts on them.

For hand coding to work, it needs accurate and complete source data (categorization). If those layers are working, then there are still too many variable combinations to hand code and more approximation is needed.

NN training is slower and 'fuzzier' than saying "if x>0.567" , but what if the code needed to be "if x>0.571"? Apply that across all values and it's beyond human ability in a useful amount of time, but a big block of training compute can tweak all the parameters simultaneously while checking for regression.

You've left out the fact that Tesla also uses simulations to train FSD.

 

[Todd Burch]

The UX isn't great with 12.x Automatic Set Speed Offset, but it seems like you've turned that on effectively telling FSD it's allowed to go 50% above the speed limit. If you don't want it to go faster than 35 in 35, set the percentage offset to 0% instead. The guardrail C++ code you describe already exists in 12.x in clamping the speed end-to-end is allowed to go.

Are you saying my set speed offset of +5 MPH really means “go no more than 50% above the speed limit?”

More proof it needs work. I don’t want to go the speed limit, or 7 below, or 15 over in close-to-reckless driving territory (20 over here in Virginia).

I know it ultimately needs to be much more complex than that, and it’s not a trivial problem, but giving free trials to all Tesla drivers when the car will frequently go 40%+ over the known speed limit is not a good look… such things trigger the attention of our good friends at NHTSA. Or if you watch Munro’s videos, “nichschta”.

 

[lzolman]

Get ready for more changes. The development pace continues to ramp.

Ashok Elluswamy works for Tesla. Autopilot / AI

https://twitter.com/x/status/1774968328475340938

For a start, how about just negotiating intersections correctly?

 

[bedoig]

I remember way back in pre v9 days, FSD could handle light to moderate traffic on the interstate pretty well.... and kind of do straight roads on backstreets. Then with v10...

I don't think there was a public release of anything before 10.something, was there? Not disagreeing overall, I'm also enjoying the journey. $2k (after EAP) doesn't look bad now that it feels like things are moving. This is what I was always hoping for really - some fun along the way getting to experience it develop. Granted my car is like 5.5 years old now, various people have gotten screwed at earlier and later times, etc. etc. etc.

 

[Daniel in SD]

The B pillar is where the camera resides. If you measure the distance from the front of a Tesla to the camera, you will see it is the SAME distance from the front of the vehicle to the driver’s eye on many other vehicles. And they manage just fine.

Or do you have data showing otherwise to back up your claim?

Again, I'm not talking about the b pillar camera. I'm talking about looking forward out the windshield. The cameras are mounted in the center of the windshield, whereas the driver can move their head to see around cars in front of them. This occurs at an intersection while waiting for a left turn with a car in the opposing left turn lane and while passing on a two lane road.

 

[JHCCAZ]

I'd rather get my dk slammed in a car door

what about a cyberfrunk?

That one has remote control.
So I'm guessing that if he could set it for continuous open/close mode, he'd probably sign up for that.

 

[FSDtester#1]

That one has remote control.
So I'm guessing that if he could set it for continuous open/close mode, he'd probably sign up for that.

Probably would be the most action he's had in a while!!

 

[JHCCAZ]

Probably would be the most action he's had in a while!!

I wouldn't know, fortunately.
But maybe one can program it along with one of those musical Tesla light shows.
" PDub why have you been spending so much time in the garage lately?"

 

[JHCCAZ]

Again, I'm not talking about the b pillar camera. I'm talking about looking forward out the windshield. The cameras are mounted in the center of the windshield, whereas the driver can move their head to see around cars in front of them. This occurs at an intersection while waiting for a left turn with a car in the opposing left turn lane and while passing on a two lane road.

I strongly agree with this and I was writing about it when we were discussing the early HW4 rumors:

This newly posted article from notateslaapp discusses dual repeater cameras as part of hw4 and the Highland/Juniper redesign.

Exclusive: Tesla's Model 3, Project Highland Exposed: What’s Changing

I'll also quote this relevant comment about the repeaters and add some thoughts:

I wouldn't be so sure that the second camera in the repeater will be only side facing. I don't know for sure, but it makes complete sense to me that the second camera will be able to look forward along the fender of the car.

This would take care of what I think is one of the most dangerous present blind spots: the issue of seeing oncoming traffic when you are trying to turn left with another left turning car in the median turn bay facing you. This is already a problem for a human driver, but at least he can lean towards the driver window. The centralized windshield cameras cannot.

Peripheral forward l-looking cameras on the mirrors or in the headlight area would have been even better for seeing past such obstructions, but the repeater location is still a big advantage over the present central cameras. However, with the repeater location it's especially important that FSD should pose the car relatively straight when awaiting a left turn, so as not to have the cars on front fender obstruct the oncoming lane vew. (one of the advantages of the headlight location is that the oncoming lane view is maximized for almost any left-turning pose angle of the ego car).

If the rear repeater camera sees straight back along the fender, and about 110° meaning a bit forward, then a roughly symmetrical front repeater camera could take in the view from straight forward along the fender back to an overlap region with the rear repeater camera. I would think that both of these would also see the ground for parking purposes.

This begs the question of what the pillar camera is then needed for. I think that even if the front and rear repeater cameras overlap, the pillar would still contribute its undistorted non-overlapped side view, also biased more for the upper vertical region and not the curb region. These pillar cameras are presently slightly mysterious because we don't get to see them in Sentry or Dashcam views, but apparently they're being kept in HW4, l so presumably they continue to be fairly important in FSD.

And this other commentary on the need for looking past oncoming traffic:

The reason I don't really like that top-center windshield location is that it's nearly useless for a very common use case: seeing the high-speed oncoming lanes, past the obstruction of a left turning car in the oncoming turn lane or turn bay.

This is already somewhat of a challenge for a human who can lean towards the driver side window. The center windshield location, even though it's high up, is markedly inferior. The front left corner location is the best for this, the A-pillar or driver side mirror location next best, and a forward-facing repeater camera (that can at least see along the fender) is the third best.

 

[jebinc]

Which car do you have? That sometimes help to try to make sense of it, but Tesla never communicates anything to their FSD customers because they honestly don’t give a damn about what we know or don’t know so there’s rarely a 100% definite answer.

Nah, it’s another case of Tesla provided BOHICA!

 

[EVNow]

Test? 5 minutes.

LOL. What is this - "Hello World" program ?!

 

[EVNow]

Get ready for more changes. The development pace continues to ramp.

Ashok Elluswamy works for Tesla. Autopilot / AI

https://twitter.com/x/status/1774968328475340938

All that would actually mean .... wait for it .... Feature Complete !!!

 

[Ramphex]

I retract my statement about v12 EVNow having less nags. Going to wait for the next revision and reassess.

 

[AlanSubie4Life]

1) Same issue with speed (maybe it goes 1-2mph faster? I have no idea - probably not, or just chance). But a) doesn't keep up with traffic b) may travel below speed limit for no reason c) won't travel set target speed (ignores it - it's treated as a cap) when ASSO mode is disabled (or when it is enabled and the hidden limit is in effect of course) - this item c is of course a key factor which means it's still broken - it's really all that matters.

2) Still stops abruptly, then too slowly. Stops many feet behind stop line as usual. Exactly the same.

3) Still too close to curbs on right turns in particular (though it'll hit left turns too from what we've seen). Seemed to take exactly the same line on the same corners.

1) In ASSO mode, maybe it goes faster? (It's so hard to tell, because this is basically a random walk anyway.) It certainly does not keep up with traffic that is going 5-10mph over the limit as is the custom (we have a 45mph unenforceable (it's literally unenforceable due to California Speed Trap law) limit and it is legal to go 55mph, or whatever you want as long as it is not reckless, which is what everyone does since that's the reasonable speed for the road - it just stuck to 45mph in the fast lane as a car cruised past in the slow lane at 50mph (I had to turn left soon so didn't change lanes)).

******
2) The key thing is that with ASSO mode off: It will NOT go at your desired set speed (unless it decides it wants to, meaning that set speed is equal to or lower than its desired speed). That's really all anyone cares about, of course.

If no one complains that manual mode does not respect their set speed, I will concede I was wrong.

This has aged very well! Plenty of complaints, as I expected. I do not have to concede I was wrong! Whew - I was starting to think I had overstepped last night. (I'll gladly concede when I am wrong, of course.)

Fair enough. To be clear, "better" or "much better" doesn't mean that it will never go slower than user-preferred speed or entered cap speed or posted speed limit. It means it will do things like that less often, hence better than the acknowledged behavior of 12.3.

Yes, it does seem that it goes very slightly faster. Tonight, I saw 47mph in a 45mph (manual target was 52mph). And I think I saw 52mph in a 50mph (manual target 57mph). I think before I might have seen 43mph-44mph and 48mph-49mph on these same roads? But I'd have to review my footage (which I'm not going to).

My initial sense that maybe it was a couple mph faster seemed to be correct, even though I doubted it, and I wholeheartedly concede that is an improvement, which is why people say it is "better" - but it is far from resolved.

Of course, on freeways, the nice familiar strong behavior comes back with v11 - it just goes the speed you ask for! Not looking forward to v12 on freeways until they fix this problem.

To me this seems like the same FSD as 12.3 though. I don't think 12.3.3 is fundamentally retrained. It seems likely that they somehow just tweaked some parameters to achieve this ASSO change, rather than doing retraining. The behavior is just so similar in all the areas where I have problems - I cannot detect any obvious differences which you'd expect to see if it were actually retrained. It just drives exactly the same way in exactly the same places on the street.

For the unprotected lefts there definitely seems to be a change in a "caution" parameter. It just sits in the middle of traffic lanes now most of the time on my unprotected left. Creeps, then commits at 3-5mph into traffic lanes, then just stops (1-2mph). No relevant traffic present. This did not happen on 12.3. However, in every other way it approaches this turn the same way (down to the exact same incorrect lateral position on the unmarked street, blocking right-turning traffic), so it seems like 12.3 and 12.3.3 are the same underlying neural net. 3/4 times I have tried this turn, this is what it has done. I've seen others report the same today. I had no such failures on 12.3.

So I conclude that 12.3 and 12.3.3 are actually the same, with parameter adjustments.

Does manual mode honor the percent offset that we can control? (I haven't tried manual mode at all in V12+.) I'll have to give it a shot and see on my next trip.

I think at this point everyone knows how it all works:

1) In ASSO mode, speed offset is +50% (it doesn't seem to matter if you're in chill, moderate, assertive, @arnolddeleon - I checked this tonight as best I could). So in a 50mph it's set to 75mph, etc. Presumably it maxes at 85mph but I haven't checked. This is adjustable (invisibly), and it acts as a cap on speed.

2) In manual mode, with % offset of your choosing, it seems to work as before, with one major difference: it won't (in general) go at your requested set speed (this is the big problem that we hear many complaints about, for which there is no workaround, and it is a clear regression from v11). It is adjustable on the fly as normal, and it will honor it as a cap, of course.

3) When switching on the fly from ASSO mode to manual mode, the ASSO limit will be preserved (that's how you determine what ASSO limit is set to empirically). If you disengage and reengage the manual limit will come into force as normal.

Are you saying my set speed offset of +5 MPH really means “go no more than 50% above the speed limit?”

No. If you have a set speed offset (which can only be a % now) that will be honored as a cap. In ASSO mode that % is set to 50%. In manual mode it's whatever you set.

but giving free trials to all Tesla drivers when the car will frequently go 40%+ over the known speed limit is not a good look…

ASSO mode has a terrible interface, but since this is a cap, it's quite unlikely to go 40%+ over the limit.

In my particular area, I've only seen it going about 5% over the limit, max. And it's often so slow to get to that point that on average it is under the limit.

It certainly could go higher than that. It's probably calibrated to limits on California roads, not Oregon roads, which I know from experience are much lower limits than California roads, all else being equal. That may be why we see higher excursions in Oregon. It should take the limit into account of course but it does not seem to (as usual you can use manual mode to just apply a cap, though).

But overall I don't think this is a major source of concern for releasing to all Tesla drivers. I think it's fairly well calibrated to the average driver, and some will think it accelerates too fast, but that's not related to speeding.

 

[arnolddeleon]

1) In ASSO mode, speed offset is +50% (it doesn't seem to matter if you're in chill, moderate, assertive, @arnolddeleon - I checked this tonight as best I could). So in a 50mph it's set to 75mph, etc. Presumably it maxes at 85mph but I haven't checked. This is adjustable (invisibly), and it acts as a cap on speed.

I'm not sure if you are making a distinction between "surface streets" and "freeway mode". With Auto Speed Set Enabled and while driving on the freeways (w/ V11 for controls) the Max Speed is definitely affected by chill/moderate/assertive mode. The setting only affects the Max speed at speed limit changes or upon engagement. So, even though freeway driving control is V11, this part of V12 does "leak" into it.

I don't know how (if) affects the hidden Max speed on "surface streets". I don't recall making any claims about that.

 

[lascavarian]

I think it's fairly well calibrated to the average driver, and some will think it accelerates too fast, but that's not related to speeding.

I would posit that acceleration is an introduced problem. The acceleration I am encountering is novel to recent FSD IMO.

Where is this coming from? It is not from millions of miles of actual data. This seems emergent or residual from something. It is almost like an artifact from using synthetic data and accelerated training has crept into the machine.

 

[AlanSubie4Life]

I'm not sure if you are making a distinction between "surface streets" and "freeway mode". With Auto Speed Set Enabled and while driving on the freeways (w/ V11 for controls) the Max Speed is definitely affected by chill/moderate/assertive mode. The setting only affects the Max speed at speed limit changes or upon engagement. So, even though freeway driving control is V11, this part of V12 does "leak" into it.

I don't know how (if) affects the hidden Max speed on "surface streets". I don't recall making any claims about that.

Ah, ok, got it. Good to know. Only one freeway and I was in manual mode.

What are the percentages for each mode?

 

[kpanda17]

MYP 12.3.3 multiple drives yesterday, overall feel very good, smooth, I can see using SFSD over time becoming regular use, addictive
Still edge cases rare but exist, we're not at robo level as expected, V12 is a reboot, but def on the path to robo
Summon did not work again, kept saying get closer, feel no need to keep testing summon until the release of the new SmartSummon in the next 60-90 days
Auto park very good and consistent
Im a believer in SFSD V12