FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[kabin]

Funny, I had 11.3.2 for less than a week before I got 11.3.3. My experience on 11.3.2 was totally different than it is on 11.3.3. 11.3.2 was good.

Whor mars was recently saying something similar.

 

[Mardak]

Could this be the same for 2023.6.8 and 2023.6.9? .8 appears USS and .9 is Vision.

Tesla seems to be pushing out a not-FSD-Beta software 2023.6.11 that merged those those two versions: with and without USS. Those updating are coming from both those versions as well as those who were on recent 2023.2.100/.200 factory software versions for new 3/Ys.

This potentially is in preparation for a 2023.6.15 / FSD Beta 11.3.6 merging 11.3.4/11.3.5 to the latest production branch allowing FSD Beta additions to recent deliveries and FSD Capability acquisitions already on 2023.x software, which is probably at least 90% of the fleet not already on FSD Beta.

 

[TK211X]

If I could compare my first impressions of FSDb it would be like if your uncle who is slightly visually impaired is driving you around while invested and talking to you not 100% thinking about what he’s doing.

 

[JB47394]

Also seeing a huge drop off in ability to actually engage fsd now - even when lane lines are great it often says “temporarily unavailable”. Anyone else seeing that a ton now?

Folks have been talking about how the icon just doesn't show up (Chuck Cook associated it with being too close to a lane edge). No notification, but no icon either. I've encountered it as well and I relate it to narrow lanes. I have a drive that takes me on a two lane road that is in perfect condition, with perfect lane lines - but FSD won't be available there until the road widens a bit. With something so fundamental as detecting lane width being wonky, I can imagine lots of functionality misbehaving in odd ways.

 

[Ramphex]

I’m starting to think the FSDb team consists of one dude who just codes 40 hours a week. Elon keeps him around to keep something rolling out here and there, to create a perception that there’s progress, so more and more people drop $$ on FSD. Then the firmware team bakes the small changes into a stale firmware version and pushes it out.

After-all Geo Hotz did something similar… so one man team isn’t too unrealistic.

When was the last time FSDb had something significant change, that wasn’t just tweaking or editing the current code?

 

[DarkForest]

Anyone else noticed degraded USS functionality?

I'm on 2022.45.13 (11.3.4), and I noticed the USS lines and behavior are significantly different now. I have a tiny garage and need to get within 10 inches to the wall to fit. This was easy on v10, but now the USS lines are drawn at a different distance than before and less consistently. It also starts saying STOP much earlier.

I've had to start putting the car in reverse to compare lines in order to park where I need to.

 

[DarkForest]

But… isn’t it also illegal? Some people also see it as a courtesy to pull up FIRST to the stop sign, wait for me to stop, then wave ME across first like they’re doing me a favor. No. Just obey the rules of the road and go first since you got there first. Grrr.

Maybe...

Yes, totally agree. I value being predictable over courtesy like you described.

I value safety above all. If I'm the one turning, the person behind me can see my blinker, but the line of cars behind them can't. People who are checked out while driving these long boring county roads may not stop in time if I hold everyone up to make my turn. Getting onto the shoulder is safer in those situations. Granted, the ideal solution is that everyone always pay attention while driving, but since reality is often disappointing, defensive driving is a thing.

If I had my way, I'd want it to do this shoulder turn on high speed roads and maybe not on lower speed roads.

 

[Gilliland]

I find that FSDb's creeping behavior is genuinely frightening both for the occupants and for oncoming traffic. I wish it would just stop and not move at all when there is obvious oncoming traffic. It should only be creeping forward when there is no threat of an immediate collision.

When I'm driving, I do creep forward even when there is oncoming traffic so I can see what's coming behind them. I get that. But FSDb doesn't creep forward at a snail's pace like a normal driver, Instead it moves with short bursts of motion, leading me and the oncoming traffic to think that it might just decide to go without warning. The blue creep line doesn't give me much comfort, especially since it often appears to be far too close to or even beyond the path of oncoming traffic. And I'm usually too busy watching the traffic with my foot poised over the brake pedal to look for the creep line anyway.

Human drivers are able to creep forward without appearing to threaten oncoming traffic. FSDb has no such subtlety.

 

[Ramphex]

Maybe...

Yes, totally agree. I value being predictable over courtesy like you described.

I value safety above all. If I'm the one turning, the person behind me can see my blinker, but the line of cars behind them can't. People who are checked out while driving these long boring county roads may not stop in time if I hold everyone up to make my turn. Getting onto the shoulder is safer in those situations. Granted, the ideal solution is that everyone always pay attention while driving, but since reality is often disappointing, defensive driving is a thing.

If I had my way, I'd want it to do this shoulder turn on high speed roads and maybe not on lower speed roads.

Personal opinion, I don’t like the shoulder turn idea at all. The shoulder is usually not wide enough for another car to fit side by side causing it to stick out over the yellow line which could become problematic for anyone coming from the left and making a right turn while towing a trailer, requiring the full use of the lane to make the turn. Just my opinion though.

 

[AlanSubie4Life]

But FSDb doesn't creep forward at a snail's pace like a normal driver, Instead it moves with short bursts of motion, leading me and the oncoming traffic to think that it might just decide to go without warning.

Old issue oft discussed.

Simple body language. Very amenable to algorithmic solution. Imagine how difficult it will be when they start using neural nets.

 

[Hayseed_MS]

Personal opinion, I don’t like the shoulder turn idea at all. The shoulder is usually not wide enough for another car to fit side by side causing it to stick out over the yellow line which could become problematic for anyone coming from the left and making a right turn while towing a trailer, requiring the full use of the lane to make the turn. Just my opinion though.

Agreed. Most places do not allow turning from a shoulder - you must be in a designated lane to turn. If you pass any cars when turning from the shoulder can be considered illegally passing on the right.

 

[AlanSubie4Life]

Agreed. Most places do not allow turning from a shoulder -

Except for California (improved shoulders aka bike lane of course). Where plurality of Teslas exist.

It is perfectly fine and in fact encouraged within 200 feet of an intersection. Need to keep traffic moving.

 

[Hayseed_MS]

Except for California (improved shoulders aka bike lane of course). Where plurality of Teslas exist.

It is perfectly fine and in fact encouraged within 200 feet of an intersection. Need to keep traffic moving.

I guess if CA does it - it must be best...

Here, we just make it a turn lane if we want to encourage that type of behavior. Also, we call improved shoulders turn lanes. Shoulders are not part of the road.

 

[aronth5]

This is incorrect. It can see just fine as long as it is far enough forward. This is for the pose.

Watch the planner.

It’s not going to weirdly use the repeater cameras by turning right first.

It would make no sense on a right turn anyway because it would be hurting the field of view of both relevant cameras to turn left first. The problem, to the extent any exists on right turns, is not the left-hand extent of the FOV of the pillar cam and repeater cam.

Can watch the planner also for understanding that right-turn behavior (which is bizarre).

You could be right since nobody really knows but don't forget the front camera is used for turns too. The question still remains why does the car turn this way?

 

[AlanSubie4Life]

The question still remains why does the car turn this way?

It’s for the median pose. Probably to not hit curb (exaggerated) on right turns. Just check the planner and I think should help clarify.

Anyway it is dumb and unnecessary. No one drives this way unless they are pulling a trailer.

Definitely a problem. If they could disconnect the steering wheel from the planner that would improve things but still not ideal and it would be a surprise when the car proceeded.

 

[AlanSubie4Life]

Also, we call improved shoulders turn lanes. Shoulders are not part of the road.

Yeah. No idea what the car does with shoulders then. Does anyone know? I have only seen the car using “turn lanes” in this case. Also from people’s pictures only turn lanes have been used (improved shoulders).

No place to test it around here since everything is improved.

I would think unimproved shoulders would normally not be drivable space but probably unreliable.

 

[Supcom]

Agreed. Most places do not allow turning from a shoulder - you must be in a designated lane to turn. If you pass any cars when turning from the shoulder can be considered illegally passing on the right.

An example of such prohibition would be useful here. Many times people assert a particular driving maneuver is illegal, when, in fact, it is actually legal.

In Texas, using the shoulder is acceptable for making turns, as well as other uses:

Sec. 545.058. DRIVING ON IMPROVED SHOULDER. (a) An operator may drive on an improved shoulder to the right of the main traveled portion of a roadway if that operation is necessary and may be done safely, but only:
(1) to stop, stand, or park;
(2) to accelerate before entering the main traveled lane of traffic;
(3) to decelerate before making a right turn;
(4) to pass another vehicle that is slowing or stopped on the main traveled portion of the highway, disabled, or preparing to make a left turn;
(5) to allow another vehicle traveling faster to pass;
(6) as permitted or required by an official traffic-control device; or
(7) to avoid a collision.

Clearly, in Texas, pulling into the shoulder when making a right turn is not only courteous, it is perfectly legal.

 

[Hayseed_MS]

An example of such prohibition would be useful here. Many times people assert a particular driving maneuver is illegal, when, in fact, it is actually legal.

In Texas, using the shoulder is acceptable for making turns, as well as other uses:

Sec. 545.058. DRIVING ON IMPROVED SHOULDER. (a) An operator may drive on an improved shoulder to the right of the main traveled portion of a roadway if that operation is necessary and may be done safely, but only:
(1) to stop, stand, or park;
(2) to accelerate before entering the main traveled lane of traffic;
(3) to decelerate before making a right turn;
(4) to pass another vehicle that is slowing or stopped on the main traveled portion of the highway, disabled, or preparing to make a left turn;
(5) to allow another vehicle traveling faster to pass;
(6) as permitted or required by an official traffic-control device; or
(7) to avoid a collision.

Clearly, in Texas, pulling into the shoulder when making a right turn is not only courteous, it is perfectly legal.

I agree with you. I think I am just getting hung up on my semantics of improved vs unimproved. You can drive on an improved shoulder unless posted otherwise.

 

[Ramphex]

Yeah. No idea what the car does with shoulders then. Does anyone know? I have only seen the car using “turn lanes” in this case. Also from people’s pictures only turn lanes have been used (improved shoulders).

No place to test it around here since everything is improved.

I would think unimproved shoulders would normally not be drivable space but probably unreliable.

Could you post a pic of a shoulder, improved shoulder, and a turning lane. Just so everyone can be on the same page with what we’re talking about. Also keep in mind that there are 49 other states aside from California.

 

[AlanSubie4Life]

Could you post a pic of a shoulder, improved shoulder, and a turning lane. Just so everyone can be on the same page with what we’re talking about.

Too lazy. Google image search works though.

Improved - paved, same level. On the far side of the solid or dashed white lane line.

Unimproved - soft shoulder unpaved or at least much different surface than rest of road. Not at same level also not improved even if paved.

Turn lane - marked with road markings as a lane and a turning arrow somewhere.

There are probably myriad permutations which do not confirm to the above. Use your noggin.