FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[JB47394]

If it is this difficult to program factual input someone please let us know how do you program intuition? Wait for AI?

Yes, but not general artificial intelligence, if that's your meaning. A neural nets is a tool that is perfectly capable of pulling off intuition. A neural net develops an "intuition" about "cars" just by being shown lots of them versus lots of things that are not cars. That type of formation can be applied to any input, including driving scenarios. It's a matter of figuring out how to quantify the scenarios as well as the desired outputs such that a neural net can do the learning, compute the weights, form that "intuition" about driving scenarios and come up with useful instructions for what to do when faced with them.

Having enough scenarios to train such a network is a bit of a challenge unless you're a company like Tesla.

 

[kabin]

11.3.6 is having too many near misses. I'm starting to see online videos and complaints as well as experiencing them for myself. Today 11.3.6 approached a stop sign with the usual brake early and crawl but didn't respond appropriately to a Mercedes delivery van with right-of-way making a left turn directly in front of me. FSDb kept crawling into the van's path. The van driver assumed I was being aggressive and was clearly upset. FSDb assumed the creep limit was safe but it wasn't as the van drove through the creep limit.

 

[Reddy Kilowatt]

I had an impressive failure today on 11.3.6. It stopped at a stop sign to make a right turn. This is the road that I take from my house daily to enter the main highway (55 mph). The car proclaimed "creeping for visibility", then luckily out the corner of my eye, I saw a car moving at the 55 mph to my left. Unfortunately, FSDb decided to make the turn anyway, which would have caused the other car to slam on the brakes. I disengaged before my car could make the turn.

Now all that isn't too exciting, except that the cross traffic vehicle was a county sheriff.

I have had that happen a couple times with 13.6. I have also had it try and turn right into the side of a car just passing me. I have had this on earlier versions.

 

[Therealm80]

I wonder how much shadows play a role in what it sees. The car was on the side street. There was a landscaper trailer parked off to the side. It started to go around and then slammed on the brakes, but there was a big shadow going across the road. It was really weird.
On another note, it’s been doing roundabouts very very well but yes, things have definitely regressed on city roads.

 

[AlanSubie4Life]

A neural net develops an "intuition" about "cars" just by being shown lots of them versus lots of things that are not cars.

I know next to nothing about neural nets but I do know that what they are doing has little to do with our concept of intuition.

The pattern matching and categorizing behavior of neural nets (my sense for what they are doing, though maybe I am wrong in my understanding) is substantially more fragile than anything we might be running in our much more powerful minds.

Just take a look at the primitive behavior of ChatGPT as an example of how lacking in intuition or thought these constructs have. (The poetry is amazing though! )

They are extremely powerful in many applications of course. Just very very limited and fragile.

 

[ewoodrick]

General point: autonomous driving (FSD or otherwise) has long way to go, even if the antiquated “old school” airplane tech doesn’t get incorporated.

Specific point. Autonomous driving is easy.

Autonomous driving with stupid humans still needs some work.

Reminds me of a video of a 4-way intersection with autonomous cars driving from MIT. The autonomous cars barely had to slow down for the intersection. They interleaved perfectly. Compare this to humans who may only get 6 cars through per minute.

 

[PACEMD]

My car maps stop for Horse crossings..
I also have the option to navigate to the nearest Kmart or RadioShack

Dang, you got the new maps. Mine came over on the Mayflower........navigates to Boston Harbor for tea........

 

[PACEMD]

No, seriously, new maps already. I did get 11.3.6 yesterday but new maps? Noooooooooo..........

 

[PACEMD]

NOT stopping for a would make for a VERY bad day.

We joke but here in Montana, some roads are blocked with cows, horses, and at times bison.........

 

[PACEMD]

My mother and my significant other hate FSD and always ask me to turn it off because of how jerky and sloppy it is.

So THATS why my wife and mother-in-law call me a sloppy jerk. Good to know.........

 

[PACEMD]

Okay. My maid hates it when I use FSDb while driving.

Not that there's anything wrong with that.........

 

[Supcom]

I had an opportunity to test 11.3.6 on some high speed two-lane roads. While, in general, the car behaved well, there are two items of interest. When turning onto one of these roads, where the speed limit was 65 mph, the car took way too long to accelerate up to my set speed of 70 mph. It was better than NOA ever was, but still far too slow. Fortunately, there was no approaching traffic from the rear, or I would have felt obligated to give it some go pedal.

The second issue was that this version still attempts to slow for flashing yellow light strung above the highway. I guess they don't have these in California.

 

[AlanSubie4Life]

The second issue was that this version still attempts to slow for flashing yellow light strung above the highway. I guess they don't have these in California.

They do, though maybe not exactly the same case…for example around blind corners with traffic lights they flash with timing appropriate to warn you if the light is going to be red when you get there (does not flash otherwise).

The car sometimes slows down for them. The main problem is that you notice it slowing down (it should slow down in such a way that you don’t notice). And then it speeds up again and slams on the regen around the corner when it finally starts slowing for the red light, too late.

 

[jebinc]

Not that there's anything wrong with that.........

Well, it could be quite painful - when FSD “jerks the wheel” or slams the brakes. Abrasions even possible.

@Ramphex

 

[etorres]

Add me to the 11.3.6 is worse camp. I was generally happy with 11.3.3 (on a relative scale it was better than the 10.69.x with a number of nice improvements).

So many phantom slow downs, odd lane changes/blinkers and jerky turns today. Really frustrating after feeling like things were on a better trajectory with 11.3.3. I was starting to feel fairly at ease on city trips with .3 but all that went away with .6. Hoping today was just a bad drive but fearing not.

 

[Supcom]

They do, though maybe not exactly the same case…for example around blind corners with traffic lights they flash with timing appropriate to warn you if the light is going to be red when you get there (does not flash otherwise).

The car sometimes slows down for them. The main problem is that you notice it slowing down (it should slow down in such a way that you don’t notice). And then it speeds up again and slams on the regen around the corner when it finally starts slowing for the red light, too late.

View attachment 927801

The ones I have trouble with are strung above the traffic lane on a 65 mph road to alert you to an uncontrolled intersection. There are four lights arranged in a rectangle with opposite corners lit and unlit together. The car will slow, and possibly stop, for them, though I never allow to progress that far!

 

[Krash]

Oh, ok. That is pretty big of you. He probably didn't realize what could happen by just downvoting and not posting.

You mean that you get harassed by the schoolyard bullies? Some of the posts got snipped and got some warnings.

If one doesn’t have anything to say, a downvote is actually my preferred method of disagreement.

 

[j0shm1lls]

11.3.7 when? 11.3.6 has been a big steaming pile of regression for me.

 

[powertoold]

3.6 is more robotic for me. When following a car, it feels like I'm being towed by the lead car.

Maybe I haven't been paying much attention to 3.4, but 3.6's parking lot perception seems to be a lot more detailed, predicting far away parking lot geometry obscured by rows of parked cars.

 

[cbdoc]

Interestingly, I know of no means to report a problem trying to engage FSDb. I disengaged on a road where I have problems with reengaging just so I could report that I was testing a problem with reengaging. I couldn't reengage, and I hope the car's data somehow communicated that.

I know that I cannot reengage based on the width of the roadway on that road, but I also got the whack-a-mole game on a four lane 45 mph roadway with a center median. There's tons of room, but the car didn't want to engage. I'll have to keep an eye on when the icon appears and see if I can correlate it to the conditions around me.

Press voice button and say: report bug [bug].