FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[KLam]

6 Hours, Zero takeovers.

Completely useless* because its by a shill* according to #JunkTeam.

Its not fake. But if you care to watch at 6:04:3 and 6:08 you will find he didn't flag two of the most unsafe moves of FSDb.

1. swing left across lane to do a right turn
2. cut onto left lane to do a left turn

It almost like someone who do not drive coded the moves

 

[Ramphex]

Installing newest version of perfection

 

[EVNow]

How do I best use it to get it to select turning lanes appropriately? In one of my videos it tried jumping into a wrong turning lane, in the other it failed to get into the turning lane, stopped in the middle of intersection, and then went straight clickity clacking the turn signals in different directions.

As I’ve been saying that is a bad regression in 11.4.2.

If you select “minimize lane changes” it seems to get better. Today on a drive FSD insisted on moving to the left lane on a two lane road with equal traffic. After canceling twice, I switched on minimize lane change and it didn’t try again.

 

[kabin]

I'm sure it felt close, but I'm not sure what you're talking about it getting confused while creeping. You can tell from the position of the hood and the position of the bus relative to the center lane-line that the FSD Beta visualization is accurate, and there's at least a foot of space between the two vehicles at the closest:

View attachment 946624

It might be near impossible to tell from wide angle videos but the rendering on the display shows a separation of maybe half the ego's width and it's no where close to that. Admittedly the UI isn't high precision. It's possible FSD thought it had that much room to spare but it looks too close from my vantage point and the driver sounded anxious about clearing the bus.

I've had two instances where 11.4.2 kept creeping closer and closer to stationary lead vehicles. It was as if the planner and/or creep logic had lost sanity. I've never experienced this before 11.4.2 and I'm seeing it in uploaded videos too. It seems to be a new issue.

Adding another data point here. I had to look at it again and as the ego passes the bus it's right front wheel appears inside the white line's of the lane of the bus up to the point where the front wheels are almost even with the bus rear wheel. That's nuts!

 

[Ramphex]

As I’ve been saying that is a bad regression in 11.4.2.

If you select “minimize lane changes” it seems to get better. Today on a drive FSD insisted on moving to the left lane on a two lane road with equal traffic. After canceling twice, I switched on minimize lane change and it didn’t try again.

They aren’t random lane changes though, they’re turns I actually need to make . But I can try those same spots with that option on to see if it does anything.

 

[EVNow]

Its not fake. But if you care to watch at 6:04:3 and 6:08 you will find he didn't flag two of the most unsafe moves of FSDb.

1. swing left across lane to do a right turn
2. cut onto left lane to do a left turn

It almost like someone who do not drive coded the moves

The moves are so unsafe that 90% drivers do it. Basically if the other lane is empty most drivers don’t care about cutting into the lane. Just like when they turn, they don’t mind going over the center line a bit. There is nothing to “flag” unless it was unsafe because there was a car in that place at that moment FSD was going to hit.

Sometimes those are needed because there may not be enough room. But it looks like by giving those moves “low cost”, cost optimized path selection is literally “cutting corners”.

 

[KLam]

The moves are so unsafe that 90% drivers do it. Basically if the other lane is empty most drivers don’t care about cutting into the lane. Just like when they turn, they don’t mind going over the center line a bit. There is nothing to “flag” unless it was unsafe because there was a car in that place at that moment FSD was going to hit.

Sometimes those are needed because there may not be enough room. But it looks like by giving those moves “low cost”, cost optimized path selection is literally “cutting corners”.

I think it has to be coded to not cross lanes unless its absolutely necessary. Actually the 2nd illustrated problem of cutting into the left lane and not following a solid guide turn lane blocked a following car on its left.

Testers should flag all these violations and stop driving with FSDb to alert Tesla to fix all of these fundamentals!

 

[3rror]

I updated to 2023.12.11 last night. I was concerned that Teslafi did not tie it to FSDb, but updated completed, and I did NOT lose FSDb.

I mentioned that on Twitter also to those groups, but they haven’t updated their sites to show that 2023.12.11 does in fact include FSDb, it’s just the older version 11.3.6. I wish it were the newer flavor.

 

[LowlyOilBurner]

As I’ve been saying that is a bad regression in 11.4.2.

If you select “minimize lane changes” it seems to get better. Today on a drive FSD insisted on moving to the left lane on a two lane road with equal traffic. After canceling twice, I switched on minimize lane change and it didn’t try again.

Trying this tomorrow, I have just the spot where the same behavior you described has been happening since 11.4.2. Definitely a regression

 

[JHCCAZ]

If you select “minimize lane changes” it seems to get better. Today on a drive FSD insisted on moving to the left lane on a two lane road with equal traffic. After canceling twice, I switched on minimize lane change and it didn’t try again.

I have one egregious and repeatable example of its insistence on getting out of the left lane about 500 yaeds ahead of a planned double left turn. This is the one that solved itself in May and then came back about a week ago. It will try at least twice, sometimes three times if I don't disengage first.

On 11.4.2, selecting Minimal Lane Changes, and/or Chill, does not suppress this error. It's pretty clearly a map/nav problem, and I hold out hope that the same fix I got during 11.3.6 will come through again.

I just got 11.4.3, but I'm pretty sure that alone won't fix this problem spot. Tomorrow afternoon will tell.

 

[EVNow]

I think it has to be coded to not cross lanes unless its absolutely necessary. Actually the 2nd illustrated problem of cutting into the left lane and not following a solid guide turn lane blocked a following car on its left.

Testers should flag all these violations and stop driving with FSDb to alert Tesla to fix all of these fundamentals!

Doesn’t work like that. Basically they use some kind of cost optimizer to find out the best path since you can’t actually hard code all these.

Yes, it would be best if it doesn’t do things like this and drives “perfectly” - but we are not there yet.

Worst part to me is - earlier when turning to a branching turn lane, FSD would use the dotted line are to turn properly. Now it just changes lane like it’s a normal lane change.

 

[JHCCAZ]

11.4.2 started biasing hard to the left at the stop sign where I have to turn left out of my residential road, onto another unstriped neighborhood feeder road. First try on 11.4.3 is doing exactly the same thing. It's not a big problem unless someone happens to be coming along to turn onto my street; in that case they understandably get a little frosted. If I let it happen, the car is hogging the middle of the road and posed at a left angle while it's leisurely mulling its next move at the stop sign.

The feeder road comes up the hill, and there's a poorly placed cactus that sonewhat obstruct the already shallow view of cars coming up to this intersection. So possibly it's reacting to that, and the angled pose is an attempt to get the forward wide camera involved. However I'm not really sure that's a good explanation.

Yet another example iMO where a front corner camera would provide a superior view.

The basic driving is getting really good now, but I think map confusion (quite solvable) and camera viewpoint geometry (a fundamental problem) are ongoing issues.

 

[fasteddie7]

Yet another example iMO where a front corner camera would provide a superior view.

there have been several revisions of the hardware since 2.0. Any ideas why front corner cameras haven’t been introduced. It seem like such a no brainer for superior performance to a person in the drivers seat. I’ve been trying to figure out how far and at what angle the new HW4 cameras can “see” and if it meets or exceeds me leaning forward to look left and right, but I still think corner cameras at the front and rear would really be measurably better than a person could achieve alone. They must be seeing something with the current configuration that I can’t yet grasp.

 

[Mardak]

For those with 11.4.3, does the software screen have an entry for "Full Self-Driving (Beta) Software" version separate from the vehicle software "display" version 11.1? Here's an example from FSD 2023.12.10 to 2023.12.11 ?

Previously, there would only be one line for "FSD Beta 11.4.2 (2023.7.10 a165af53442e)." I wonder if this means we'll keep getting "earlier" FSD Beta releases (still after pre-Safety-Score group) ahead of Tesla deploying a "stable" version to main vehicle software branch?

 

[jebinc]

there have been several revisions of the hardware since 2.0. Any ideas why front corner cameras haven’t been introduced. It seem like such a no brainer for superior performance to a person in the drivers seat. I’ve been trying to figure out how far and at what angle the new HW4 cameras can “see” and if it meets or exceeds me leaning forward to look left and right, but I still think corner cameras at the front and rear would really be measurably better than a person could achieve alone. They must be seeing something with the current configuration that I can’t yet grasp.

That would be a good question to tweet to the Great Charlatan.

 

[Ramphex]

For those with 11.4.3, does the software screen have an entry for "Full Self-Driving (Beta) Software" version separate from the vehicle software "display" version 11.1? Here's an example from FSD 2023.12.10 to 2023.12.11 ?

Previously, there would only be one line for "FSD Beta 11.4.2 (2023.7.10 a165af53442e)." I wonder if this means we'll keep getting "earlier" FSD Beta releases (still after pre-Safety-Score group) ahead of Tesla deploying a "stable" version to main vehicle software branch?

I always wondered why they don’t just bake it into the same main firmware with a flag activation per VIN to get the Junk features

 

[JHCCAZ]

For those with 11.4.3, does the software screen have an entry for "Full Self-Driving (Beta) Software" version separate from the vehicle software "display" version 11.1? Here's an example from FSD 2023.12.10 to 2023.12.11 ?

Previously, there would only be one line for "FSD Beta 11.4.2 (2023.7.10 a165af53442e)." I wonder if this means we'll keep getting "earlier" FSD Beta releases (still after pre-Safety-Score group) ahead of Tesla deploying a "stable" version to main vehicle software branch?

I just snapped a photo of my software screen after getting 2023.7.15 today. Interestingly, it doesn't say that at all on that screen; it only lists the FSD version and the map version.

The main software version number is shown in the app but not in this software screen. Possibly because the only people who get this particular firmware are FSD beta testers. I'm pretty sure it was never offered to mainstream users.

 

[archae86]

For those with 11.4.3, does the software screen have an entry for "Full Self-Driving (Beta) Software" version separate from the vehicle software "display" version 11.1? Here's an example from FSD 2023.12.10 to 2023.12.11 ?



Previously, there would only be one line for "FSD Beta 11.4.2 (2023.7.10 a165af53442e)." I wonder if this means we'll keep getting "earlier" FSD Beta releases (still after pre-Safety-Score group) ahead of Tesla deploying a "stable" version to main vehicle software branch?

No. The first line under "Software" reads:
FSD Beta 11.4.3 (2023.7.15 ....

then it goes on the Navigation Data.

I'll leave any inferences to others.

 

[JHCCAZ]

there have been several revisions of the hardware since 2.0. Any ideas why front corner cameras haven’t been introduced. It seem like such a no brainer for superior performance to a person in the drivers seat. I’ve been trying to figure out how far and at what angle the new HW4 cameras can “see” and if it meets or exceeds me leaning forward to look left and right, but I still think corner cameras at the front and rear would really be measurably better than a person could achieve alone. They must be seeing something with the current configuration that I can’t yet grasp.

I said repeatedly that the camera placements need work, OTOH I'm always ready to hear a good technical counter argument- I just haven't.

There was a YouTube interview a year or so back, with a guy who had been an employee (though I think not a principal design engineer) at the time the model 3 platform was designed. He said something to the effect that they talked about it, but "the car can see okay". I can try to find this if you're interested.

I've also had people, once or twice, say that the car has 360° coverage so what's the problem? The answer to this is obvious I think: you can have 360° view that can be subject to blockages in common situations, versus a 360° view that is less vulnerable to the most common blockages.

There was a rumor, yet unfulfilled, that HW4 would bring an additional forward-angled repeater camera. Though it's not a corner placement. It would still shift the viewpoint farther forward than the driver's head, instead of farther back as with the B pillar. Just as importantly I think, it would provide a far superior view of the oncoming high-speed lane when you're facing another left-turning car in front of you. So I was pretty happy to hear about that but so far it hasn't shown up.

 

[sleepydoc]

there have been several revisions of the hardware since 2.0. Any ideas why front corner cameras haven’t been introduced. It seem like such a no brainer for superior performance to a person in the drivers seat. I’ve been trying to figure out how far and at what angle the new HW4 cameras can “see” and if it meets or exceeds me leaning forward to look left and right, but I still think corner cameras at the front and rear would really be measurably better than a person could achieve alone. They must be seeing something with the current configuration that I can’t yet grasp.

there would be 2 possible places to put it - the A pillar and somewhere in the front quarter panel/near or in the headlights. The quarter panel/headlight placement would be useless for half the year in any of the winter states because of road spray. Also likely close to useless in the rain. The A pillar location might work but there's not much lateral real estate there and it needs to cover a lot of the side view. It would also be more susceptible to road spray, bug splats, etc if it's facing too far forward.