FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[mgs333]

Some random thoughts. 11.3.6

• I really would like the disengaging by steering wheel to be smoother, this is quite unsettling to passengers
• I think it would be useful to have a 'confidence meter' so you can tell when the car is starting to be less confident so I can be more alert and ready to take over. Most of the time, it's not hard to tell when the car will get nervous/confused but sometimes this happens for no apparent reason at a place where it's done fine many times before
• My car regularly cuts off cars when doing lane changes on the freeway, sometimes they change lanes to get away, sometimes they just stay riding my rear bumper
• Today, the car was terrible on a drive it has done great at before, my wife was scared after I disengaged a few times. (missing a freeway exit, almost driving into a curb on a 35mph curve, suddenly darting out into the next lane as we're stopping for red lights, etc etc.)

 

[sleepydoc]

Not everywhere, and not under all conditions, and not without a variety of extra rules and possibly training or equipments to be able to be aware of audible signals.

Besides, Teslas can already hear, so that's not an issue. The only question is how they will react to what they hear.

They don’t have the ability to localize audio to know where the sound is coming from

 

[rlsd]

You do.

Sorry. I meant voice command. I used "Save that" before but it's not a documented feature. I prefer "Save video".

 

[D Good]

Some random thoughts. 11.3.6

• I really would like the disengaging by steering wheel to be smoother, this is quite unsettling to passengers
• I think it would be useful to have a 'confidence meter' so you can tell when the car is starting to be less confident so I can be more alert and ready to take over. Most of the time, it's not hard to tell when the car will get nervous/confused but sometimes this happens for no apparent reason at a place where it's done fine many times before
• My car regularly cuts off cars when doing lane changes on the freeway, sometimes they change lanes to get away, sometimes they just stay riding my rear bumper
• Today, the car was terrible on a drive it has done great at before, my wife was scared after I disengaged a few times. (missing a freeway exit, almost driving into a curb on a 35mph curve, suddenly darting out into the next lane as we're stopping for red lights, etc etc.)

Are there still people on .3.6?

 

[mgs333]

Are there still people on .3.6?

Yes those of us who recently joined beta are on 2023.12.10 with 11.3.6

 

[powertoold]

11.4.2 is such a big step up from 11.3.6. I'm consistently doing no disengagement and sometimes no intervention drives from home to work and back every day in varying traffic conditions. 11.3.6 was randomly too jerky or unnecessarily slowing at too many points.

I also use it without disengagements to get food, from parking lot to parking lot.

 

[JulienW]

11.4.2 is such a big step up from 11.3.6. I'm consistently doing no disengagement and sometimes no intervention drives from home to work every day in varying traffic conditions. 11.3.6 was randomly too jerky or unnecessarily slowing at too many points.

If Tesla updates the new Beta people on 23.12.10 to a 11.4.x version Tesla sure as hell better update us long timers to 23.12.x or > also.

 

[EVNow]

11.4.2 is such a big step up from 11.3.6. I'm consistently doing no disengagement and sometimes no intervention drives from home to work and back every day in varying traffic conditions. 11.3.6 was randomly too jerky or unnecessarily slowing at too many points.

I also use it without disengagements to get food, from parking lot to parking lot.

Are you not having any lane change issues ? I think that's the main regression (or even first time bug).

 

[EVNow]

suddenly darting out into the next lane as we're stopping for red lights,

Yes - I've seen this multiple times. I've to be extra vigilant to cancel those. Close to stopping for the traffic light FSD will change to the wrong lane - esp. if there are multiple turn lanes.

 

[Supcom]

Not everywhere, and not under all conditions, and not without a variety of extra rules and possibly training or equipments to be able to be aware of audible signals.

Besides, Teslas can already hear, so that's not an issue. The only question is how they will react to what they hear.

Do you have any examples of a state that has special requirements for deaf people to drive a car? In Texas, there are none, and no hearing tests are given in order to get a license. Some states may require things like additional mirrors to provide a wider field of view for seeing emergency vehicles. Teslas obviously have that covered with the multiple cameras.

 

[powertoold]

Are you not having any lane change issues ? I think that's the main regression (or even first time bug).

Do you mean lane selection or the actual lane change maneuvers? If you're talking about maneuvers, no, it's wayyy better and more responsive for me on 11.4.2. With 11.3.6, sometimes I would activate a manual change, and it would never do it. Sometimes with 11.3.6, I would cancel a lane change, and it'd still do it!

Lane selection is better on 11.4.2 for me, but it still does some odd both and forth changes. I don't mind it as much since the lane change maneuvers are done well.

 

[EVNow]

Do you mean lane selection or the actual lane change maneuvers? If you're talking about maneuvers, no, it's wayyy better and more responsive for me on 11.4.2. With 11.3.6, sometimes I would activate a manual change, and it would never do it. Sometimes with 11.3.6, I would cancel a lane change, and it'd still do it!

Lane selection is better on 11.4.2 for me, but it still does some odd both and forth changes. I don't mind it as much since the lane change maneuvers are done well.

I mean the latter.

Lane change itself is excellent and probably better than I can do. I'd argue it's probably better than average humans.

Lane selection and the decision to change lanes is definitely worse for me (and to a lot of folks here as we see). Sometimes it really insane - like I posted a while back - leading to missed exits etc.

 

[Dewg]

If Tesla updates the new Beta people on 23.12.10 to a 11.4.x version Tesla sure as hell better update us long timers to 23.12.x or > also.

I was one of the last updated to 11.3.6, but I just got 11.4.2 on Friday, so it's coming.

 

[powertoold]

I mean the latter.

Lane change itself is excellent and probably better than I can do. I'd argue it's probably better than average humans.

Lane selection and the decision to change lanes is definitely worse for me (and to a lot of folks here as we see). Sometimes it really insane - like I posted a while back - leading to missed exits etc.

Oh weird, lane change decision and selection are better for me, especially on highway. Not sure what the difference is. I use chill 90% of the time.

As for the decision, 11.4.2 tends to err on the side of slowing down behind the adjacent car for lane changes.

 

[willow_hiller]

Lane selection and the decision to change lanes is definitely worse for me (and to a lot of folks here as we see). Sometimes it really insane - like I posted a while back - leading to missed exits etc.

It's somewhat of an issue that most of us got 11.4.2 and map version NA-2022.44-14515 at the same time. Makes it hard, if not impossible, to distinguish issues with the firmware and issues with the map.

I have noticed 11.4.2 trying to take incorrect exit lanes, but I cannot tell if this is because of the new map or not.

 

[EVNow]

Tesla's warning might be the most appropriate:

Warning: This Vehicle May do the Wrong Thing at the Worst Time.

 

[willow_hiller]

View attachment 944022

Jokes about bumper stickers aside, I would worry about how this might effect your legal liability in a collision. Similar to how saying "I'm sorry" after an accident can be admissible in court as evidence of wrongdoing.

 

[EVNow]

Jokes about bumper stickers aside, I would worry about how this might effect your legal liability in a collision. Similar to how saying "I'm sorry" after an accident can be admissible in court as evidence of wrongdoing.

I hope dash cam videos + my video recording will help assign the blame properly.

 

[EVNow]

As for the decision, 11.4.2 tends to err on the side of slowing down behind the adjacent car for lane changes.

I'm particularly talking about this
- The car is coming close to an exit, the lanes are slowing down
- Now FSD needs to change to the right lane for exit
- Instead, FSD changes to the left lane !
- After 100 ft FSD changes back to the original lane
- Then it tries to change right to the exit lane - but too late

Or
- The car is coming close to an intersection
- Now FSD needs to go straight
- Instead, FSD changes to the right, turn lane !
- After 100 ft FSD tries changing back to the original lane
- But too late, so turns right ....

It's somewhat of an issue that most of us got 11.4.2 and map version NA-2022.44-14515 at the same time. Makes it hard, if not impossible, to distinguish issues with the firmware and issues with the map.

I've driven on these routes umpteen times without such issues. I doubt its a mapping issue (I've to check whether I got a map update at the same time as 11.4.2).

Seems to me - FSD long term planning is disconnected from short term planning to overtake vehicles or avoiding traffic jams. So, even though the car imminently needs to turn/change in one direction - it does the opposite.

 

[powertoold]

I'm particularly talking about this
- The car is coming close to an exit, the lanes are slowing down
- Now FSD needs to change to the right lane for exit
- Instead, FSD changes to the left lane !
- After 100 ft FSD changes back to the original lane
- Then it tries to change right to the exit lane - but too late

Yea, that's odd. I've never had that happen on the highway with 11.4.2. The only reason it'd change to the left is either speed based or "changing lanes to stay out of rightmost lane."

I've found a "bug" or consistent issue with 11.4.2 today. It doesn't deal well with people turning left while waiting for an unprotected right. There was a bunch of cars turning left in front of me, and 4.2 just decided it was impatient and wanted to squeeze in, lol. I found this impatient behavior somewhat consistent on other unprotected rights.