FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[kabin]

TeslaBull is driving a HW3 car and testing 11.4.

Yep. HW3 is the motherload source of data. Going forward HW4 cars will hopefully have improved camera location and image resolution which will force a transition.

 

[kabin]

As I keep writing - there is a large learning curve. Those who don't get over the curve won't find FSD useable.

Those who do can even use it for Uber and passengers have to look for various clues to see if the driver is driving or FSD.

It would be fun to hop in the back seat and give an honest critique but it's all luck of the draw in the Uber world.

The koolaid is heavy with that one. He has issues but tries to gloss over them. It's all about seeing what's happening versus listening to his interpretation.

 

[powertoold]

Going forward HW4 cars will hopefully have improved camera location and image resolution which will force a transition.

Just all speculation at this point. It's possible that all the extra compute for HW4 will be used to process 4x the resolution at a higher frame rate, but all the NNs will be outputting the same kinds of predictions.

 

[powertoold]

mention the dumb stuff like changing lanes in the middle of intersections or entering turn lanes at the last second, dawdling at stop signs when no traffic is anywhere in sight, ...

Changing lanes in intersections isn't illegal in most states, and if it's done safety, it's not a big deal.

Entering turn lanes at last second is stupid and annoying for sure, but it'll get fixed. And you can always manually get into the correct lane by using the stalk.

Dawdling at stop signs is mostly the result of the NHTSA requirements. FSD was a lot more natural when it was rolling stop signs at 4-5mph.

There's a lot of polishing to do with FSD beta, but there's no point in polishing when there are still several safety issues.

 

[EVNow]

It would be fun to hop in the back seat and give an honest critique but it's all luck of the draw in the Uber world.

The koolaid is heavy with that one. He has issues but tries to gloss over them. It's all about seeing what's happening versus listening to his interpretation.

No need for Koolaid - I go with my own experience. Obviously I don't post videos so I post others that show similar experience.

Yes - the whole idea is to not nit-pick when FSD drives. Watch out for safety and let it drive. If you nit-pick, you can not really use FSD. That is the learning curve part.

Here is an example from last couple of days. When taking a left turn (> 90 degrees) in one of the neighborhood roads, 10.69 used to make a wide turn. In 11.x the turns are sharp - so the car seemed headed either to the curb or to a car waiting in the perpendicular road. So I intervened a couple of times. One day I let it go a little further, FSD adjusted in the middle of the drive and made a safe turn. Ever since its been making safe turns, sometimes adjusting in the middle of the turn. If I keep intervening because the turn is not smooth, FSD won't be useable on that road ...

I can give quite a few examples like that.

ps : TL&DR : Wide ranging experiences people report on FSD is atleast partly because of the "safety drivers".

 

[powertoold]

Look at 3.6 fumble around this pedestrian lol. I guess it's still better than Cruise

 

[kabin]

Any reason you didn't answer the question regarding employees testing V11.4 on HW3 or didn't you think about that?

@kabin- "I wouldn't be surprised if 11.3.x was HW3's last hurrah with 11.4 and beyond destined for HW4"

Keeping it real - it's all talk from my behind but I think both can be true. Said another way, V11.3 won't have much significance for HW4 vehicles. Plus the abundance and familiarity of HW3 yada yada yada for V11.4 testing. There should be a significant transition to HW4. Now please allow me some flexibility to overstate just a little - This Is The Way!

 

[EVNow]

Look at 3.6 fumble around this pedestrian lol. I guess it's still better than Cruise

Initial part was ok. But once it decided to go, it should just go. Thats what pedestrians expect ... they may start walking expecting the car to pass before they cross.

FSD gives pretty bad clues to pedestrians, first it stops - by the time pedestrian moves, sometimes it starts moving as well - and the pedestrian stops. FSD also stops. This goes on. Worst part is - FSD goes all the way till the pedestrian but stops right infront of them, blocking their way. Or may be a few feet infront.

Sometimes the interaction goes smoothly. I've got a thank you wave from pedestrians a number of times

 

[kabin]

No need for Koolaid - I go with my own experience. Obviously I don't post videos so I post others that show similar experience.

Yes - the whole idea is to not nit-pick when FSD drives. Watch out for safety and let it drive. If you nit-pick, you can not really use FSD. That is the learning curve part.

Here is an example from last couple of days. When taking a left turn (> 90 degrees) in one of the neighborhood roads, 10.69 used to make a wide turn. In 11.x the turns are sharp - so the car seemed headed either to the curb or to a car waiting in the perpendicular road. So I intervened a couple of times. One day I let it go a little further, FSD adjusted in the middle of the drive and made a safe turn. Ever since its been making safe turns, sometimes adjusting in the middle of the turn. If I keep intervening because the turn is not smooth, FSD won't be useable on that road ...

I can give quite a few examples like that.

ps : TL&DR : Wide ranging experiences people report on FSD is atleast partly because of the "safety drivers".

For sure the current FSD state of affairs isn't for everyone however that impacts mainstream acceptability so they eventually need to prioritize these changes for the masses. And that time has come and gone with FSD being available to anyone with sufficient funds.

I never want to feel comfortable with FSD's short choppy uncomfortable steering, accel, decel, moving closer and closer to oncoming traffic, while deciding if it can jam in between car flow. Those higher bandwidth psychotic decisions should happen in the software not at the brake/throttle/steering control.

 

[JHCCAZ]

Yep. HW3 is the motherload source of data....

Motherlode

 

[powertoold]

FSD gives pretty bad clues to pedestrians, first it stops - by the time pedestrian moves, sometimes it starts moving as well - and the pedestrian stops. FSD also stops. This goes on. Worst part is - FSD goes all the way till the pedestrian but stops right infront of them, blocking their way. Or may be a few feet infront.

11.4 has improved VRU interactions, so hope it's better

I'm excited about 11.4, been watching the videos 5-10x lol.

Elon said that Tesla has been closing the loop on interventions. It seems they've established a good workflow for their data engine.

From the notes:

- Improved the decision to assert or yield for pedestrians at more crosswalks by evaluating multiple possible futures in the joint space of ego's actions and the pedestrian's response.

- Improved ego's behavior near VRUs by measuring their probability of intersecting ego's path, based on their kinematic data, and preemptively decelerating when the estimated risk is high.

 

[AlanSubie4Life]

FSD percent improvements always sound spectacular but the status quo remains for me.

60-80% is pretty poor. Would mean another 3-4 major releases to get to something sort of remotely useful, and another 10 or more or so before it’s nearly autonomous.

 

[Enright]

I can see where Beta was thinking about it, and it's good to see the addition of logic to handle Keep Clear sections. Next step is to work on anticipation around it. If the light ahead is green and traffic is flowing ahead, no need to stop.

These "second order" situations are the ones that fascinate me when it comes to the challenge of AI and driving - the ones where the right decision depends on not one, but two or three variables, each with their own set of rules. It reminds me of the "Three Body Problem" - where you can easily calculate the future position of two objects that are gravitationally locked, but it's computationally impossible to do the same for three objects due to the sheer number of permutations.

E.g. "Don't enter a keep clear zone unless there is room to cross it" vs. "Don't enter a keep clear zone unless there is room to cross it unless the light is green, but hasn't been green for a long time [indicating the intersection is block and/or traffic jam] and the car in front is either moving, or is likely to move shortly because the light turned green a short time ago and the green light is not stale and likely to turn red anytime soon, and you are not in a turn lane where the car in front might not be able to make the turn due to traffic / backup on the perpendicular road and the lane in front doesn't end or merge... and on and on..."

It's amazing how well our brains solve this kind of problem many times per minute while driving, and how good neural nets are getting at doing the same.

 

[powertoold]

60-80% is pretty poor. Would mean another 3-4 major releases to get to something sort of remotely useful, and another 10 or more or so before it’s nearly autonomous.

11.4 seems to be much better at slowing / stopping, based on the videos.

I can't wait for the day V11 is consistently smooth at going and stopping on highway. That's my main annoyance with V11 on highway with intermittent stop and go traffic.

 

[EVNow]

60-80% is pretty poor.

That depends on what we are talking about - and even how the improvement is calculated.

Equally important, is degradation in some other areas. In 11.x turns are smoother and slower when there is no need to speed up quickly, but the sharper turns sometimes require adjustments on the way.

 

[AlanSubie4Life]

11.4 seems to be much better at slowing / stopping, based on the videos.

Lol, I will believe it when I get it and experience it. Or, when a video is posted which calculates jerk and acceleration and plots them at all times during the video.

I agree it is very poor at this, though I have not experienced stop-and-go traffic, since we do not seem to have that in San Diego (at my chosen times of course).

That depends on what we are talking about - and even how the improvement is calculated.

It was disengagement rate that was quoted. Presumably per-mile disengagement rate.

A (claimed) 5x improvement (80% reduction) for a major release is a little sad, given the starting point.

 

[Ramphex]

My non-FSD Model Y got a 2023.12.1.1 update, while FSDj Model S is still on 2022.45.15…. Just saying….

 

[loquitur]

Does "kinematic data" in 11.4 imply something beyond modeling pedestrians
as mere cuboids? This is inferred from a Musk tweet in answer to Omar, who
was complaining about such (for 11.3.6) here:

https://twitter.com/x/status/1644908343155363841

Stuff like this, I guess (174 pages from a Ph.D. thesis) about pedestrian intention
kinematics: https://www.researchgate.net/public...n_on-board_camera_to_predict_their_intentions

 

[mgs333]

Get a Tesla if you want to learn about AI trying to kill you, says Apple cofounder Steve Wozniak

Apple cofounder Steve Wozniak had some harsh words for Tesla’s self-driving effort; he said in an interview that people who...

electrek.co

 

[black knight 3]

Get a Tesla if you want to learn about AI trying to kill you, says Apple cofounder Steve Wozniak

Apple cofounder Steve Wozniak had some harsh words for Tesla’s self-driving effort; he said in an interview that people who...

electrek.co

what does this have to do with the topic of the thread "FSD Beta V11.x"? It's not appropriate. find another forum if you want to entertain conversations like this,.