FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Yelobird]

I doubt most v11.x data will be of much value to v12 given its need for high quality data. I assume high quality includes being courteous, safe, following traffic laws, no speeding, no rolling stops, no abrupt accel/decel, smooth steering. Elon said only 5% of drivers come to a full stop. High quality data collection might be a unicorn. It would be ironic if TSLA needed to pay FSD owners to drive appropriately in specific areas.

Believe he actually corrected that to say less then .5% of drivers full stop. Also noted they had to custom program that goofy rule to accommodate the complaint department.

 

[PACEMD]

More FSDj; it’s like a merry-go-round, except there isn’t any music….

More like Mary, Mary, Quite Contrary........

 

[PACEMD]

My fun today started when FSD/Nav etc decided that instead of being in Austin, TX that I was actually in Killeen, TX (65 miles away)
FSD(wtf) then decided to make turn decisions based on that ridiculous assumption and ignored all camera input when deciding turns - obviously I disabled it.

Was it out late last night? Sounds a bit hungover.........

 

[Tronguy]

I doubt most v11.x data will be of much value to v12 given its need for high quality data. I assume high quality includes being courteous, safe, following traffic laws, no speeding, no rolling stops, no abrupt accel/decel, smooth steering. Elon said only 5% of drivers come to a full stop. High quality data collection might be a unicorn. It would be ironic if TSLA needed to pay FSD owners to drive appropriately in specific areas.

Actually... Not sure I agree with this. Your assumption is, I think, that the Actual Incident, as recorded on the car's cameras and such, would be directly put into the training set as an example of What Not To Do.

But that doesn't have to be the case. At some point humans may (or may not, this is guessing at Tesla's internal processes) be in the loop. Some gonzo does a search for a pareto chart sorted by Evil and Cause; another gonzo gets assigned one to investigate. Investigation occurs, correlation (as in, human correlation) occurs, fault case gets synthesized. Maybe from video evidence, or maybe just something gonzo #2 puts together.

And the above is the back side, looking for an error. The front side is what-shoulda-happened. Which gonzo #2 may have at hand from other drivers or may have to synthesize from scratch.

Something along the lines of, "It's a dirty job, but somebody's got to do it." More power to Tesla if they can figure a way to automate, or partially automate the process.

 

[PACEMD]

Oh this is good!

This needs to replace @Ramphex ‘s FSDj.

All hail FSD(wtf)!

FSDwtf..........I like it...........

 

[archae86]

More power to Tesla if they can figure a way to automate, or partially automate the process.

Partial, at least. When Elon and the big-hit FSDb development guy were talking about the paucity of code governing actual car operation in the V12 alpha, they were careful to say there was lots and lots of code devoted to the difficult task of selecting training material. It may not be a highly unified system, as I think at least four languages got reeled off in one sentence as being part of what was used in the effort. (think I remember C++ and Python).

I suppose the logical next step is training AI to select AI training material. They did not hint that was going on just yet, or even under development.

 

[JulienW]

Instead of FSDj I'm more thinking 20.7.xBS.

 

[PACEMD]

Oh this is good!

This needs to replace @Ramphex ‘s FSDj.

All hail FSD(wtf)!

And where is Ramphex by the way? No fun trashing FSDb without him...........I mean Jebinc is fun and all but we really depend on the team.......

 

[PACEMD]

So the transfer of FSD from our 2019 M3P- to the new 2023 M3P went well. At pickup in Seattle yesterday, when I set up the iPhone key in the actual car, FSD was already present. Not only was it there but software version 2023.20.200 was installed. Not the latest FSD software but it is FSDb, a beta version, 11.3.6. I had worried I would be back in line for beta, but not the case. Others have said the same. Also at pickup/setup for the 2023, FSDb disappeared from the 2019 model. A few hours later the 2019 was updated to 2023.26.9, the latest non-FSD software.

 

[Tronguy]

So the transfer of FSD from our 2019 M3P- to the new 2023 M3p went well. At pickup in Seattle yesterday, when I set up the iPhone key in the actual car, FSD was already present. Not only was it there but software version 2023.20.200 was installed. Not the latest FSD software but it is FSDb, a beta version, 11.3.6. I had worried I would be back in line for beta, but not the case. Others have said the same. Also at pickup/setup for the 2023, FSDb disappeared from the 2019 model. A few hours later the 2019 was updated to 2023.26.9, the latest non-FSd software.

Um. Stupid question: New, 2023 M3P. HW3 or HW4?

I was thinking about doing the 2018 M3 RWD LR to the modern equivalent; but the various Federal and State rebates wouldn't work, and $20k+ seemed a bit steep for a car that's working fine anyway.

 

[JulienW]

Um. Stupid question: New, 2023 M3P. HW3 or HW4?....

HW3. HW4 is not coming until Highland (maybe Oct.).

 

[PACEMD]

Um. Stupid question: New, 2023 M3P. HW3 or HW4?

I was thinking about doing the 2018 M3 RWD LR to the modern equivalent; but the various Federal and State rebates wouldn't work, and $20k+ seemed a bit steep for a car that's working fine anyway.

It is HW3 so there's that, but the tax rebate applies, and the 15k saved with the FSD transfer helps my attitude. I also got $5k off by selecting an on-lot vehicle. See my post above. All in all, I saved too much money to pass on the update. I may be sorry when the refresh model comes out but so be it. With the tax rebate, I saved $27k total. I'm by nature too cheap to walk on by. I was raised by grandparents who were young adults during the Depression.

 

[JulienW]

It is HW3 so there's that, but the tax rebate applies, and the 15k saved with the FSD transfer helps my attitude. I also got $5k off by selecting an on-lot vehicle. See my post above. All in all, I saved too much money to pass on the update. I may be sorry when the refresh model comes out but so be it. With the tax rebate, I saved $27k total. I'm by nature too cheap to walk on by. I was raised by grandparents who were young adults during the Depression.

Yea I can see it is sorta a catch-22 situation. Losing out on the $15k "amnesty" (as if you were one of the 19 arrested here last week) is a hard "free option" to just pass up. That is some serious green stamps.

 

[tivoboy]

Um. Stupid question: New, 2023 M3P. HW3 or HW4?

I was thinking about doing the 2018 M3 RWD LR to the modern equivalent; but the various Federal and State rebates wouldn't work, and $20k+ seemed a bit steep for a car that's working fine anyway.

No M3 with HW4,, S and X only, and I think some Y but won’t be M3 I think at all or for now at least and that apparently includes highland from some reports, but we’ll have to see.

 

[kpanda17]

Hope they don’t create two v12 code trains like they did before but maybe unavoidable
Anyone say v12.3 FSDb and v12.4 same but for testers ?

 

[PACEMD]

Yea I can see it is sorta a catch-22 situation. Losing out on the $15k "amnesty" (as if you were one of the 19 arrested here last week) is a hard "free option" to just pass up. That is some serious green stamps.

And my Cybertruck reservation had FSD at $7k. Think they will honor that? We probably still have some green stamp books in junk boxes with my grandmother's stuff. Will Tesla or Amazon take green stamps? My grandmother, young during the depression, never threw out anything. When she died we found plastic bags, about a foot in diameter, filled with those twisty wire things bread enclosures were/are sealed with. Now called twist ties, but we had no name for such things. Or the internet for that matter. Never know when you might need many multiples of thousands of those twisty things.

 

[PACEMD]

Yea I can see it is sorta a catch-22 situation. Losing out on the $15k "amnesty" (as if you were one of the 19 arrested here last week) is a hard "free option" to just pass up. That is some serious green stamps.

I'm guessing that's why the 2-month limited-time FSD transfer offer showed up. To sell existing cars to people who otherwise would wait for the Highland version........and I fell for it..........

 

[Acisplat]

https://twitter.com/x/status/1695479308729598241

For us early and safety score testers, are we going to get a rebase of 11.4.7 on the 2023.26.x branch next week or are we stuck with 2023.7.x indefinitely?

 

[amadeusneo]

For us early and safety score testers, are we going to get a rebase of 11.4.7 on the 2023.26.x branch next week or are we stuck with 2023.7.x indefinitely?

This is the million dollar question. Just wish we had concrete posts about this somewhere on Tesla.com and not whenever Elon feels like tweeting (posting? Xing?) it out.

 

[Gwgan]

Had given up on using 11.4.4 except as a smart cruise control on well marked roads but Elon posted that .7 was mostly AI so I test drove 11.4.7 for 36 miles in just over an hour over a usual course

Tldr: I’ll be giving up on it as well.

Serious Interventions 6

Multiple disengagements for driving in the middle of the road







The Good:



Stopped at a stop sign and took a left at a T intersection, which is greater than 90° and partly obscured by bushes without creeping forward.



Sometimes did a better job of staying to the right in an unmarked road then it had but NOT doing so was the cause of many interventions.



Took a right hand turn at a stop sign with favorable traffic conditions.



Nice job smoothly slowing down for car turning to the right out of the highway ( speed limit 50 driving 58 )



Nice job slowing for traffic turning left off of the highway (speed limit 45 traveling 52 )



The Bad:



Drove down my curved hilly gravel driveway with speed set to 10mph but the car kept stopping or driving at 3mph. Came to the top of the T intersection and car a hard time deciding which way to go but eventually went in the correct direction and did get to the bottom of the driveway and turn onto the road.



Slowed down abruptly and unnecessarily from 35mph for a pedestrian well off to the side of the road.



A little slow to get started in general which is not about driving fast but about quickly getting out of the way—across an intersection or just to not force following traffic to slow for no reason.



Intervention: On a stretch of road with poor right side lines car randomly put on the right turn signal and moved over to the right



Intervention: After taking a left at a stop sign car drove into a parking lot on the right instead of staying on the road



Did not have the headlights on despite having the wipers going and that is against state law



Signaled too late for the traffic behind to prepare for a fairly abrupt right hand turn off of the highway (speed limit 45 )



Slowed unnecessarily for a car crossing across the road at a good distance



Intervention: Taking an oblique left from highway speeds at 58 miles an hour the car failed to signal in time, and then braked very abruptly in a short amount of distance, and then crawled across the opposing lane before entering the target road. FSDb has yet to get this turn right



Several disengagements on roads with no center line as the car was not staying enough to the right while there was oncoming traffic. One time was coming out of a right hand turn onto the unmarked road and met opposing traffic as was in the middle of the road.



Intervention: car said “stopping for a stop sign” and proceeded to do so but it wasn’t my stop sign. It belong to the road entering from my right.



Once out of four similar situations car moved to the right before turning left. This is dangerous around here as people behind a left turning vehicle are apt to pass on the right, partially driving in the breakdown lane.



Accelerated uncomfortably fast just before an anticipated left hand turn where it was going to have to stop, and then braked abruptly



Intervention: Stopped at the wrong place at a stop sign on an oblique intersection. This was an intervention because once it started going again, I didn’t know if it was going to stop for the crossing traffic.



After traffic diversion for construction it did not go back to the normal speed limit but thought limit was still 25 mph



Intervention: Turned left off the highway onto an oblique road way too fast the. swerved too far the right then too far to the left. Did not hit anything and did recover but disengaged to leave a note.



Intervention: Car stopped to take a left-hand turn off the highway at a place where traffic will usually just pass on the right. It did not pull to the right. However, it began to turn left across the opposing lane and in front of oncoming traffic.