FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Ramphex]

I cant help but notice that beta testing this software is sort of like accumulating flight hours. I am sure you see a lot of parallels lately with this sort of thing.

Flight hours are definitely more enjoyable

 

[FSDtester#1]

11.3.5 The driver monitoring and hands requirements are very stringent. I had a passenger waving their phone around trying to take a picture of the moon and autopilot got pissed off so I look down to see what it’s upset about and it gave me a insta strike even though my hand was on the wheel and I was looking at the road.

Though, I hear it resets after two weeks if it strikes out but I hope the full reset isn’t actually logged against you in the future. With how sensitive it is there is no doubt in my mind that I'm going to eventually strike out.

Keep in mind I’m a commercial pilot and have experience with automation management daily but they’re going a bit overkill. I can be manually driving and change the song I’m listening to no issue but in FSDb it starts to get upset.

Has anyone here heard an update on the supposed 10k miles restriction? Elon tweeted that those with 10k autopilot miles can get the DM/hand req. removed but I can’t find anything recent on it.

Commercial pilot? Hmmm.. rumor has it they have a fully working autopilot, that can take off, land, fly, etc. If it can be solved for planes, cars should be a cakewalk.

 

[valem]

Is everyone able to send voice notes? Even legacy MS and MX?
I have not seen the option once, whether I just disengage or take over or press the brake.

Am I only one that doesn't get the option? I'm on 11.3.4

Given the insane amount of replies I got... looks like I am the only one that doesn't get voice note prompts after disengagement.

 

[KLam]

FSD used to perform a California Stop at clear intersections. NHTSA shut that down. Then FSD did a 1 mph pause. NHTSA again said no (this latest recall). So now the vehicle comes to a full stop and waits even at clear intersections.

It's kind of funny NHTSA have such requirements for a drive assist requirement. Take Cruise controls on most cars, they drive straight/or not by itself (kind of automated). Did NHTSA require that Cruise control has to stop on Stop signs? Did NHTSA require that Cruise control monitor driver attentions?

 

[sandman1330]

Commercial pilot? Hmmm.. rumor has it they have a fully working autopilot, that can take off, land, fly, etc. If it can be solved for planes, cars should be a cakewalk.

Erm.... It's not the same thing.

What you are referring to is a CAT III ILS approach. This relies entirely on an external signal that the aircraft follows right down to the runway, and in the short strokes a radar altimeter that gives precise height above ground. This would be far more analogous to the use of some kind of external cueing and LIDAR for self driving, not the tech currently employed by FSD. And, it is heavily reliant on all other traffic being kept away from the aircraft conducting autoland (both to obviously avoid collision, and to avoid other aircraft blocking the vital ILS signal guiding the aircraft down). Any kind of FSD will never be able to do the same.

Further, aircraft automation is far more like level 2, maybe gusting level 3. One of the most common things you'll hear on a flight deck these days is "what's george doing now?" (George is the nickname for the autopilot). While largely hands off, it's constantly monitored by the flight crew and with the one (sort of) exception of CAT III, is entirely reliant on the crew inputting the altitudes, direction, speeds, etc. It's more like Tesla autopilot than it is FSD.

 

[Dewg]

It's kind of funny NHTSA have such requirements for a drive assist requirement. Take Cruise controls on most cars, they drive straight/or not by itself (kind of automated). Did NHTSA require that Cruise control has to stop on Stop signs? Did NHTSA require that Cruise control monitor driver attentions?

I think cruise control is L1, so they don't regulate it.

 

[TK211X]

According to the trend data we should see a wave of some sort of FSDb Ver. today or tonight.

 

[aronth5]

FSD used to perform a California Stop at clear intersections. NHTSA shut that down. Then FSD did a 1 mph pause. NHTSA again said no (this latest recall). So now the vehicle comes to a full stop and waits even at clear intersections.

I just wish Tesla didn't initially allow for California rolls anywhere from 4mph to 7mph. Lots of videos showed those speeds. That caused the attention of a lot of people including NHTSA. I suspect if Tesla did the iniital rolls at say 1-2 mph we'd probably be here with 1-2mph rolling stops. Tesla most likely created this problem themselves.

 

[Spinkick]

According to the trend data we should see a wave of some sort of FSDb Ver. today or tonight.

Its funny how people pay $$$ and all sit around waiting for the Wizard of Oz, oracle, whatever, to push out its next miracle.

 

[TK211X]

Its funny how people pay $$$ and all sit around waiting for the Wizard of Oz, oracle, whatever, to push out its next miracle.

It’s like waiting on an Amazon package. I support waiting for FW releases rather than smoking crack under a bridge or becoming a hostage in Mexico any day.

 

[EVNow]

FSD now runs closer to the curb on the right side of the unmarked road. But it still wants to leave some space on the right when turning from an unmarked road. This transition is very jerky - so the car is going smoothly towards the right hand side of the unmarked road, then at the stop sign it suddenly jerks to the left. Then creeps forward to get a better view - very jerkily - before making the turn.

Surprise .... yesterday's drive was much better. No jerky lunge to the left to leave space on the right .... even stop & creep were ok.

Almost as if, it either got some extra map data to help with exact stop line location or .... just random behavior. A repeat of better driving today (and a few more days) will confirm either way.

 

[EVNow]

Its funny how people pay $$$ and all sit around waiting for the Wizard of Oz, oracle, whatever, to push out its next miracle.

Any worse than people lining up around the block for a shiny new white phone that is 0.01% better than the one they already have ?

 

[Joboo7777]

Two more good drives around Dallas today, mostly on Bush Turnpike. About 20 miles each way. The route out had zero interventions and drove pretty well. I had one minor intervention on the way back where FSDb slowed down on the turnpike due to misapplying traffic lights that are for an intersection on the frontage road that is above the grade of the turnpike. The turnpike curves such that the traffic lights are seen directly above the road briefly. FSDb has been slowing for these lights for months, especially at night when they are more visible. Sadly, V11 has the same behavior, though a brief application of accelerator solves the problem.

Other than that issue, I continue to be impressed by how well V11 drives on limited access roads in my area. City streets still needs more polish to smooth out performance at uncontrolled intersections. I think the car is now stopping too long at stop signs, causing other drivers to become confused and proceeding out of turn. It's a fine line between too little and too long of a stop.

I live up in McKknney and the biggest problem I have is how FSDb handles the big 6 lane city streets that we have in DFW. If I can just get the car to stop missing turns because it’s not getting in to the correct turning lanes with adequate time to make the turn that would be great. And if there is any level of traffic it NEVER makes the turns. I usually miss the turn and it re routes or I just take over.

Another similar issue I have is on highway where major highways intersect. For example I was getting on Sam Rayburn heading toward McKinney from Legacy Drive and immediately getting on there is the Dallas North Tollway on-ramps(north & Soith) within a mile. The car simply does not act fast enough to get out of the left lanes that head to DNT. I’ve now tried this a few times and everytime it gets on DNT north bound and re routes me an additonal 10 mins. Not the end of the world but is alittle inconvenient.

Edit:
I think part of the problem here in DFW area is the constant road expansion projects going on. I think some of the issues we experience down here in DFW is related to inaccurate or old map data. Roads are constantly expanding as the population grows and I don’t see an end in sight to this. At least not within the next few years.

I just ran in to an issue where the road was recently expanded with a right turning lane. The car simply does not recognize this turning lane. I’m guessing that’s because of old map data?

 

[KLam]

I think cruise control is L1, so they don't regulate it.

So, if a car manufacturer claimed a system with stop sign detection can be recalled when it failed to do so at some instances?

 

[JB47394]

I saw an FSDb improvement today. I have a road with a series of right turn lanes that spring up. FSDb has always changed lanes into at least one of them. Today I got a clean drive down that road. Here's hoping it's not a one-off success.

I've also mentioned that I've tried using turn signals to direct the car when it is on FSDb, but doesn't have a destination. This is for turn lanes that are added to the side of a through road. The car will make the change into the turn lane and even plan for the turn, but it maintains whatever speed you were carrying until very late in the turn lane. I was doing 50, and I don't recommend it. Perhaps at 30 it will have time to slow for the turn in a more natural way.

Is anyone else getting spring-induced FSDb fatigue? It's a beautiful day and I really don't feel much like letting FSDb screw it up. So apart from the specific tests above, I drove the car.

 

[EVNow]

I've also mentioned that I've tried using turn signals to direct the car when it is on FSDb, but doesn't have a destination. This is for turn lanes that are added to the side of a through road. The car will make the change into the turn lane and even plan for the turn, but it maintains whatever speed you were carrying until very late in the turn lane. I was doing 50, and I don't recommend it. Perhaps at 30 it will have time to slow for the turn in a more natural way.

That is not a supported scenario for FSDb. So, won't expect it to work properly.

 

[Dewg]

So, if a car manufacturer claimed a system with stop sign detection can be recalled when it failed to do so at some instances?

I'm not sure I understand the question.

Here are some explanations for L1 and L2:

Level 1 Driving Automation – Driver Assistance
At Level 1, the lowest rung of automation, a vehicle has at least one driver support system that provides steering assistance OR braking and acceleration assistance. The driver remains responsible for driving the vehicle and must be prepared to take control at any time and for any reason.

Adaptive cruise control is an example of a Level 1 driver assistance technology. It maintains a safe following distance between your vehicle and traffic ahead without any intervention by the driver. A steering assistance feature, such as lane-centering assistance or lane-following assistance, would also qualify as Level 1 autonomy.

However, a vehicle with both of these features working together qualifies as Level 2 driving automation <this is essentially Tesla Autopilot>

Level 2 Driving Automation – Partial Driving Automation
Level 2 driving automation applies to vehicles with advanced driving assistance systems (ADAS) that can take over steering, acceleration, and braking in specific scenarios. But, even though Level 2 driver support can control these primary driving tasks, the driver must remain alert and is required to actively supervise the technology at all times.

An example of Level 2 driving automation is Highway Driving Assist, installed in Genesis, Hyundai, and Kia vehicles. It requires the driver to have her hands on the steering wheel but actively steers, accelerates, and brakes the vehicle when traveling on highways. BlueCruise is a new hands-free partial driving automation technology from Ford. It is more sophisticated than Highway Driving Assist, allowing the driver to take her hands off of the steering wheel on specific, approved highways in the U.S. and Canada.

Both of these examples of Level 2 driving automation require the driver to remain alert, engaged, and ready to take control at any time. For the record, and according to what the automaker told the state of California, Tesla’s new Full Self Driving Capability technology is a Level 2 system, and it will remain so when Autosteer for city streets arrives as an over-the-air software update.

 

[Ramphex]

Erm.... It's not the same thing.

What you are referring to is a CAT III ILS approach. This relies entirely on an external signal that the aircraft follows right down to the runway, and in the short strokes a radar altimeter that gives precise height above ground. This would be far more analogous to the use of some kind of external cueing and LIDAR for self driving, not the tech currently employed by FSD. And, it is heavily reliant on all other traffic being kept away from the aircraft conducting autoland (both to obviously avoid collision, and to avoid other aircraft blocking the vital ILS signal guiding the aircraft down). Any kind of FSD will never be able to do the same.

Further, aircraft automation is far more like level 2, maybe gusting level 3. One of the most common things you'll hear on a flight deck these days is "what's george doing now?" (George is the nickname for the autopilot). While largely hands off, it's constantly monitored by the flight crew and with the one (sort of) exception of CAT III, is entirely reliant on the crew inputting the altitudes, direction, speeds, etc. It's more like Tesla autopilot than it is FSD.

Imagine the AP needed you to jerk off the yoke while staring out of the window nonstop

What, in your opinion, do you think is more impressive, Tesla’s FSDb or Cirrus’ Safe Return?

 

[KLam]

I'm not sure I understand the question.

Here are some explanations for L1 and L2:

Level 1 Driving Automation – Driver Assistance
At Level 1, the lowest rung of automation, a vehicle has at least one driver support system that provides steering assistance OR braking and acceleration assistance. The driver remains responsible for driving the vehicle and must be prepared to take control at any time and for any reason.

Adaptive cruise control is an example of a Level 1 driver assistance technology. It maintains a safe following distance between your vehicle and traffic ahead without any intervention by the driver. A steering assistance feature, such as lane-centering assistance or lane-following assistance, would also qualify as Level 1 autonomy.

However, a vehicle with both of these features working together qualifies as Level 2 driving automation <this is essentially Tesla Autopilot>

Level 2 Driving Automation – Partial Driving Automation
Level 2 driving automation applies to vehicles with advanced driving assistance systems (ADAS) that can take over steering, acceleration, and braking in specific scenarios. But, even though Level 2 driver support can control these primary driving tasks, the driver must remain alert and is required to actively supervise the technology at all times.

An example of Level 2 driving automation is Highway Driving Assist, installed in Genesis, Hyundai, and Kia vehicles. It requires the driver to have her hands on the steering wheel but actively steers, accelerates, and brakes the vehicle when traveling on highways. BlueCruise is a new hands-free partial driving automation technology from Ford. It is more sophisticated than Highway Driving Assist, allowing the driver to take her hands off of the steering wheel on specific, approved highways in the U.S. and Canada.

Both of these examples of Level 2 driving automation require the driver to remain alert, engaged, and ready to take control at any time. For the record, and according to what the automaker told the state of California, Tesla’s new Full Self Driving Capability technology is a Level 2 system, and it will remain so when Autosteer for city streets arrives as an over-the-air software update.

Let's rephrase it this way.
If today Tesla said they have introduced a driver assist feature called Stop sign detect and Auto stop. This feature could be subject to recall if it failed sometime to detect Stop sign and stop? If that's the case, why would car manufacturers introduce more advanced driver assist features.

 

[eli_]

Simple solution is Tesla open sources their code ala Open Pilot, and if you want to roll stops like any sane person then you can go and rebuild it with ROLL_STOPS=1 and NHTSA can't do anything about it.