Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register
This site may earn commission on affiliate links.
The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.


Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.


Here is the AI Day explanation by in 4 parts.


screenshot-teslamotorsclub.com-2022.01.26-21_30_17.png


Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

 
Last edited:
A little hyperbole there sir? Down the team rah rah rabbit hole you go. All I said was Jun 22 should be interesting.
How is it hyperbole? As others have pointed out, O'Dowd not only has a financial interest in Tesla failing, but has staked his name on it and has a history of deception.

I don't really trust Musk, either, but despite his long history of overpromising and underdelivering and wildly overoptimistic timeframe promises he falls short of the type of deceit that O'Dowd has committed.
 
Driving today I noticed some other changes that occured with a visible update. I see that it is now stopping at stop signs much further forward so it doesn't need to creep as much (awesome!), and it has also been making spit second alternate route decisions that don't do anything, such as suddenly changing route by making a turn only to make a turn to get right back on the road at the same spot. (Not awesome)
 
How is it hyperbole? As others have pointed out, O'Dowd not only has a financial interest in Tesla failing, but has staked his name on it and has a history of deception.

I don't really trust Musk, either, but despite his long history of overpromising and underdelivering and wildly overoptimistic timeframe promises he falls short of the type of deceit that O'Dowd has committed.
No one disputes that. I don't understand your confusion.
 
Absolutely sounds like a thing that could be a map data update, specifically on stop lines.
So I had something similar happen with on ramps, and I suspected the same thing, so I recorded the version number each day, there was no change in version number between yesterday and today, or does map data update but keep the same version number? I do see around 21 gb uploaded last night and 1.5gb downloaded on the car though. IMG_0137.jpeg
 
  • Informative
Reactions: FSDtester#1
So I had something similar happen with on ramps, and I suspected the same thing, so I recorded the version number each day, there was no change in version number between yesterday and today, or does map data update but keep the same version number? I do see around 21 gb uploaded last night and 1.5gb downloaded on the car though. View attachment 948048
Y'know... There's a contingent around these forums who swear up and down that a firmware download is a firmware download and that, once downloaded, the behavior of the car under FSD-b should be absolutely deterministic. Same exact environment, same exact results.

Then one throws in map updates. And real time map updates. And that makes the determinism get a leettllee bitty kind of hard to come by.

And then there's maniacs like Yours Truly who was, by gum, writing self-modifying code back in the 1980's. Admittedly, it was BASICA on a PC, but BASICA was an interpreter, not a compiler, and it was perfectly allowable and acceptable to let the program itself write a bunch of lines to a file, import those lines to the running program, then execute a GOTO to the lines in question. For that matter, I've had the wonder and glory of working with compiled Microsoft FORTRAN files and, while stepping through the code looking for bugs, watched good 'ol Microsoft overwrite assembly language instructions with new instructions from time to time. (This was back in the day when math coprocessors were few and far between; so the standard compiler would put in calls to multipliers and dividers that did their thing without the coprocessor by default. If there was a coprocessor present, it would check for that - then overwrite the calls with actual coprocessor assembly language instructions. And then use that for all the rest of the looping. Fun.)

And, fundamentally, neural net systems are all about feedback. And some of those feedback loops can certainly modify weights in the NN's, if allowed to.

And we've had reports: One day, everything is $JUNK. The next day, it works better. The following day, it's all in left field again, or half-way there. I mean, I've read the reports around here that, after a week, FSD-b whatever-it-is is working better than it did the first day.

And then there's the car-to-car variations. As an example: Recently, in the forums, there's been all these comments about how bad the windshield wipers are. Which is flat out strange, since my experience is that they got seriously better a couple of years ago and have stayed that way.

Finally, here you are, keeping track of the uploads and downloads on your Beta Program FSD-b enabled car. And 1.5 GB of download is a heck of a lot of data. Forget self-modifying code: Some system at Tesla could be modifying the heck out of some NN to improve its performance without having to do a full download.

I think that, collectively, we should re-think this, "The car's software is written in granite and doesn't change from release to release" idea.

My idea, based upon nothing but handwaving: Something like a NN state machine has internal variables with ranges. I suspect that, at a minimum, Tesla is dumping changed limits on those variables on random punters, then evaluating how the car(s) are operating after the changes. As to what Tesla could be changing? The sky's the limit.
 
So I had something similar happen with on ramps, and I suspected the same thing, so I recorded the version number each day, there was no change in version number between yesterday and today, or does map data update but keep the same version number? I do see around 21 gb uploaded last night and 1.5gb downloaded on the car though. View attachment 948048

You get real time map data sent to the car each time you put in a route- that doesn't change the general version of the map.


Maps may record the location of the stop signs but something as granular as the position of the stop line relative to the sign has to be done by the car itself. If it can’t there’s no hope for FSD

And yet, it's not just done by the car itself. It's in the map data. So is location of crosswalks as well as stop signs. Which is updated each time you put a destination in. Like I already told you and you disagreed with for no good reason.

I agree for L5 the car SHOULD be able to just see and understand all that stuff with no maps at all. But that's not the reality of how the system works today. Possibly it never will be- at least not on this hardware.


Y'know... There's a contingent around these forums who swear up and down that a firmware download is a firmware download and that, once downloaded, the behavior of the car under FSD-b should be absolutely deterministic. Same exact environment, same exact results.

Then one throws in map updates. And real time map updates. And that makes the determinism get a leettllee bitty kind of hard to come by.

That, plus the environment is never exactly the same- sun at different angles, different moving and stationary objects, etc.

Remember when the guy posted the video of FSD failing a sort-of-u-turnish situation a couple times, then suddenly it worked a couple times, and people here were JUST SURE that THE CAR IS LEARNING.

Until someone pointed out the failures the car was going 15 mph faster than the successes (set speed was higher)



And, fundamentally, neural net systems are all about feedback. And some of those feedback loops can certainly modify weights in the NN's, if allowed to.

Except we know anything not in CRCed firmware, which can only be updated as a whole, does not survive a reboot-- and we know the driving computer crashes and reboots a lot more often than is ever gonna get you to L5.


And 1.5 GB of download is a heck of a lot of data. Forget self-modifying code: Some system at Tesla could be modifying the heck out of some NN to improve its performance without having to do a full download.

Except, again, it can't- without updating the whole thing.


Again, the one guy who can actually see the code has explained this.

Green said:
hey must update entire firmware at least on the autopilot. Can't change a single file - breaks dm-verity. Can't overlay and have it survive a reboot (no dev overlay hooks in prod firmwares)

Then someone suggested what you did- maybe they're just pushing NN weight changes- and he replies

Green said:
where would they be stored and what would load them after restart?
As is the weights are part of the firmware rootfs, measured with dm-verity. Sure you can replace entire rootfs - but that's firmware update of the autopilot unit.
 
  • Like
Reactions: FSDtester#1 and GSP
It would be really nice if the car stopped launching me into the exit lane for an offramp every single time and overcorrect back, making me look drunk

But... but...

 
  • Like
Reactions: Therealm80
You get real time map data sent to the car each time you put in a route- that doesn't change the general version of the map.




And yet, it's not just done by the car itself. It's in the map data. So is location of crosswalks as well as stop signs. Which is updated each time you put a destination in. Like I already told you and you disagreed with for no good reason.

I agree for L5 the car SHOULD be able to just see and understand all that stuff with no maps at all. But that's not the reality of how the system works today. Possibly it never will be- at least not on this hardware.




That, plus the environment is never exactly the same- sun at different angles, different moving and stationary objects, etc.

Remember when the guy posted the video of FSD failing a sort-of-u-turnish situation a couple times, then suddenly it worked a couple times, and people here were JUST SURE that THE CAR IS LEARNING.

Until someone pointed out the failures the car was going 15 mph faster than the successes (set speed was higher)





Except we know anything not in CRCed firmware, which can only be updated as a whole, does not survive a reboot-- and we know the driving computer crashes and reboots a lot more often than is ever gonna get you to L5.




Except, again, it can't- without updating the whole thing.


Again, the one guy who can actually see the code has explained this.



Then someone suggested what you did- maybe they're just pushing NN weight changes- and he replies
No arguments about what you said. But.. "works bad one week, works tons better the next" sounds awful suspicious.

My opinion: Never underestimate the inventiveness of a bunch of CS types. There may be sneak paths: The main code base may be check summed within an inch of its life, but perhaps there's some FLASH variable storage with all sorts of flags and trackers that controls what the main code base might be up to. Possible on a major rev said FLASH area gets initialized; but then, over time, it gets... modified.

There's no way to know, of course. I'm probably wrong. But the operative word here is, "probably".
 
But... but...

I’ll believe it when I see it. FSD has come far regardless, though.
 
No arguments about what you said. But.. "works bad one week, works tons better the next" sounds awful suspicious.


FWIW the pretty recently discovered "each drive gets real time updated map data" - especially HIGHLY DETAILED map data, down to exact locations of crosswalks, # and type of lane, etc removes a LOT of that suspicion to my mind.

In fact it even makes it make MORE sense the degree of "improvement" varies by location too, since the amount of newer/better map data will vary by location for any given drive.

Also consider the degree of behavior change you can manage with JUST map data.

For example, say you want to, I don't know, have every FSD car stop 2 feet closer to the intersection at stop signs... Just send a global map update moving all mapped stop lines forward 2 feet.

No need to touch NNs, driving code, or anything else checksummed.
 
Sure, it's solved I'd guess around 10% of the city driving. The other 90% is the edge cases of which we encounter a veritable plethora of times every drive, and where FSDj tries to kill you.


By actual amount of driving time FSDb on for like 90% of it is more the reality I've seen in the cities around here anyway (and 99.99% highways)

But the amount of progress toward that last 10% city has appeared pretty minimal for a good while now.
 
  • Like
Reactions: GSP