Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register
This site may earn commission on affiliate links.
The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.


Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.


Here is the AI Day explanation by in 4 parts.


screenshot-teslamotorsclub.com-2022.01.26-21_30_17.png


Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

 
Last edited:
I’ve gotten so tired of writing big performance reviews of the latest beta release that I just can’t do another one. So I’m going to summarize…

11.4.7 is so much of a regression in so many areas, the only word to describe it is ”junk.”

The last positive thought I can muster is that since v12 is supposed to be new from the ground up, they’ve given up on the 11 branch and all available brain power is now focused on 12. I mean, honestly… that’s the only reason they could release a version that is worse than what we had back in January, right?

/sigh
 
I suspect these all pipe to /dev/null now that they are focused on getting v12 out.
There's a lot riding on this release. They are going to have to get wipers, autopark and summon, essentially all of the features that were removed due to sensors being removed, down perfectly, in addition to a FSD system that makes way fewer mistakes, or it’s going to forever be labeled as party tricks, and $12k+ is a lot to ask for party tricks.
 
Yesterday morning driving home using FSD. 11.4.7 HW4 2023 MSP pulls into the left-most lane of a two lane left hand turn. I am the first vehicle at the light with a red arrow. Car stops as it should, and I think “nice”, relax a bit, and then it begins to accelerate through the left hand turn WITH A RED ARROW STILL SHOWING. I hit the brakes before it made it through the crosswalk, but I no longer trust FSD 11.4.7 with red lights, stop signs, unprotected left turns, protected left turns… he!!, I don’t trust it with anything other than driving on the freeway and rural roads with no turns. Any other time, I am literally hovering my foot over the brake and I have my hands both firmly on the wheel.

From where I sit, 11.4.7 is the worst FSD release in the 5 months since I have owned my Tesla.

I truly feel sorry for those of you who paid for FSD and live in urban areas unless you drive frequently on freeways. I would not trust my Tesla any further than I could throw it in an urban setting.

Joe
 
  • Like
Reactions: Phlier
The funniest thing about all of this is that you know Tesla knows FSDj is j with a... uh... capital J. They can continue to move the goalposts for years and years, by which point they've banked the money, and most customers will never have received the product despite having moved on from their automobile. I think there's a word for this in the English dictionary... Rhymes with clam, or bam, or wham. Take your pic.

And the YouTube shills? They should be flat-out ashamed of themselves.
 
I actually liked my 11.4.7 ride today, but who knows what tomorrow will bring?

11.4.x behaviors may be changing as they replace more code (or old NN's} with new NN's in order to test their data set. Not really trying to make 11 better, just prepping the data set for v12.

Or 11.4.x may be getting worse so that 12.x.x will look like a big improvement.

In either of those cases they're not wasting time on 11. If they are trying to improve 11.4.x, it's just been a game of whack-a-mole.
 
  • Informative
Reactions: pilotSteve
Yesterday morning driving home using FSD. 11.4.7 HW4 2023 MSP pulls into the left-most lane of a two lane left hand turn. I am the first vehicle at the light with a red arrow. Car stops as it should, and I think “nice”, relax a bit, and then it begins to accelerate through the left hand turn WITH A RED ARROW STILL SHOWING. I hit the brakes before it made it through the crosswalk, but I no longer trust FSD 11.4.7 with red lights, stop signs, unprotected left turns, protected left turns… he!!, I don’t trust it with anything other than driving on the freeway and rural roads with no turns. Any other time, I am literally hovering my foot over the brake and I have my hands both firmly on the wheel.

From where I sit, 11.4.7 is the worst FSD release in the 5 months since I have owned my Tesla.

I truly feel sorry for those of you who paid for FSD and live in urban areas unless you drive frequently on freeways. I would not trust my Tesla any further than I could throw it in an urban setting.

Joe
I can’t seem to figure out what actually changes with updates, as it seems all of the changes in behavior happen between updates. I had a drive on 11.4.7 this morning that would have qualified as v12 out of beta. Perfect turns, stopped and actually DID NOT turn on a no turn on red (the first time ever) I posted about it first thing this morning. I was so excited I grabbed my camera and went for a drive a few hours later to make a video, and it was like a different car, it actually stopped in the middle of a left turn with no one around right in the middle of the road, I kept thinking if someone was behind me they would have rear ended me. i don’t understand how it can go from flawless to anything but just like that.
 
Last edited:
I can’t seem to figure out what actually changes with updates, as it seems all of the changes in behavior happen between updates. I had a drive on 11.4.7 this morning that would have qualified as v12 out of beta. Perfect turns, stopped and actually DID NOT turn on a no turn on red (the first time ever) I posted about it first thing this morning. I was so excited I grabbed my camera and went for a drive a few hours later to make a video, and it was like a different car, it actually stopped in the middle of a left turn with no one around right in the middle of the road, I kept thinking if someone was behind me they would have rear ended me. i don’t understand how it can go from flawless to anything but just like that.
FWIW, I have a couple no turn on red intersections on my commute and 11.4.x has always handled them correctly. Is the sign on your drive at an angle or otherwise obfuscated?
 
FWIW, I have a couple no turn on red intersections on my commute and 11.4.x has always handled them correctly. Is the sign on your drive at an angle or otherwise obfuscated?
No. They seem pretty standard to me. every sign looks a little different, but they all say the same thing. One is illuminated because the rules change based on the time of day, I never expect that one, but the others are just regular no turn on red. It also misses stop except right turn.IMG_0212.jpegIMG_0211.jpegIMG_0213.jpeg
 
Last edited:
  • Helpful
Reactions: pilotSteve
With adding all the "rules" in code we are just running up against computational power limits.

Probably. Not clear that the redundancy is going to magically come back with the v12 approach though. Not that it matters much probably.

natural learning which is more like the way we learn and drive.

It's not clear to me that the NN method is more like the way we learn and drive than the heuristic-based, programmatic approach. Remember the structure of the NNs does not resemble our brain at all (NN is just a term of art, not meant to suggest some sort of analog to human brains), nor is it as complex.

It's hard to say what our brains do exactly, but the process that the FSD NN is going through to produce control signals doesn't seem that similar to what I do when I'm driving. I still don't see how they're going to create understanding with this new approach.

It's very common for FSD now to not understand what is happening, and I don't see that changing in v12.

While the demo is far from perfect the behaviors of the car is FAR different than the jerky robotic way our car's drive

It was really hard to tell from the video. Early on the car rocketed up to stopped traffic and slammed on the brakes. Elon said it was smooth but since the camera work was so poor it's really hard to tell. Roundabouts seemed better.

Stops seemed better, but remember his car was not set to stop at stops. And most likely all the "sliders" or whatever they use in the v12 approach (and presumably in Elon's car as a general rule) to limit car behaviors were really loose. When they have to start obeying traffic laws and reduce the certainty, I think some of the slow and jerky behaviors will probably come back.

At this point, we all know that without a high quality video and extremely careful analysis to determine jerk & acceleration, it's hard to say how good the experience is. I briefly looked to see if anything could be determined but I gave up.

I see a possible glimmer of hope that a path to L3 may exist.

I'm not too optimistic about even L3 at this point.

I guess we'll find out in 6-12 months or whatever. I am curious what form v12 will ultimately take - will it really be end to end (nets on nets with accessible intermediate outputs, etc.), or will they still leave some rules in place? I have a hard time understanding how they can do this reliably without rules as was claimed in the video. I'll be surprised if that works. But I guess I just don't understand, which is fine.
 
No. They seem pretty standard to me. every sign looks a little different, but they all say the same thing. One is illuminated because the rules change based on the time of day, I never expect that one, but the others are just regular no turn on red. It also misses stop except right turn.View attachment 971107View attachment 971108View attachment 971110
Yeah, that does look pretty clear. My signs are all on a post on the right side of the road at eye level. Seems doubtful, though, that placement would matter.
 
Yea, I am still seeing it get confused by stop sign placement on adjacent roads. I thought that map data might help it, but a lot of the time when there is a minor road intersecting a more major road at say 45 degrees the stop sign is clearly visible on the more major road in such a way that it is totally understandable why it abruptly starts stopping for it.
I also had it stop when I passed a horse farm where the owner put up their own "stop" sign that said "Whoa!" and had a horse on it!
 
  • Funny
Reactions: pilotSteve
Yeah, the center lane cannot turn on red, but the rightmost lane can. this sign is one of tens of thousands on the roads that FSDb is going to need to be able to understand if they hope to pull off autonomous driving.
Exactly. And one reason why hard-coded rules will fail. A lot.

I'm not saying that they'll actually be successful with their "no rules" approach, but it does at least appear that it's the correct rabbit to chase at this point; there's just no way they'll be able to successfully pull off all of these types of intersections with rules, *especially* with rules that change according to time of day.

One huge stumbling block I see to this new approach is data curation. How in the world are they going to be able to automatically label a clip as "this driver did it right, let's use them for the training data set" vs one that didn't do it right? They are talking about recruiting test drivers at this point. Hmmmm.
 
Last edited: