Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register
This site may earn commission on affiliate links.
The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.


Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.


Here is the AI Day explanation by in 4 parts.


screenshot-teslamotorsclub.com-2022.01.26-21_30_17.png


Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

 
Last edited:
Bold or obtuse prediction:
  • Now we know all HW4 cars are getting FSD Beta.
  • V12 ACTUALLY appears to be a complete rewrite/change of programing philosophy/fundamentally different
[crazy a$$ speculation]We could see 11.4.x near the end of the V11 development line and within a few weeks see all cars moved to about the same software and eliminate the Early Access program (at least for now). Then other than bug fixes we would all be on about the same software with about the same FSD Beta version. Tesla may even remove the FSD Beta version number altogether from releases. We could see a little stagnation of any new FSD Beta features or improvements while most resources are detected to the "new way of programing" V12. So we my see a period of around 4 to 12 months with little changes and a lot of anxious waiting.

Formost I was skeptical of Elon's V12 proclamations but it does look like V12 is NOTHING like V11 so maybe this time his hyperbole is near reality.

If (big if) V12 is truly the complete change in the way FSD is developed and it pans out to be successful we could see a significant improvement and be well worth the short hiatus in FSD Beta updates.

Of course Tesla will probably prove me a fool by releasing 23.12.30/11.5.1 next week.😱
 
If (big if) V12 is truly the complete change in the way FSD is developed and it pans out to be successful we could see a significant improvement and be well worth the short hiatus in FSD Beta updates.
EM says Tesla will have to curate the training data to influence (improve) the resulting NN.

This is fine but the unknown is the magnitude required to produce a particular durable result. No metrics.

Additionally IMO this is an attempt to tilt reality toward a particular result by negating the premise that billions of miles of training data is the secret sauce to deliver autonomous driving perfection.

It seems to me that this invalidates the secret sauce premise. How are we to know that the desired result would appear in the NN on its own with just bit more actual driving data?

How are we to know that falsifying (or curating) the training data does not result in the appearance of undesirable NN behavior in some unexpected area?

These questions cannot be answered because the emergent behavior that we call AI is not understood.

V12 is a step into a dark place filled with untested wishful presumptions that a designed reality delivered by good intentions is likely to be a good idea? Welcome to our new world overlords.

Then there is the liability of creating an intensional synthetic reality for training. What is the defense to the consequence?

There is a lot to ponder over the next 6 months. YMMV
 
Did the temp language change bump and it did not force an upgrade but on Teslafi I don’t see anything newer than what I have, 26.10

Btw, it did force a check
Eng to Brit eng and back eng
How do you do this temp language bump thing? Just change the language setting somewhere? This is the type of thing I was looking for to see if I could get up to 26.10 or 26.11 from 26.7 before end of this week when we leave on a long road trip.
 
I really need a block diagram of this stuff. Is there a big bank of computers, (CPUs and GPUs) in let's say Palo Alto, that form a huge network of neural net architecture? The images and videos from Tesla vehicles are dumped in that network and out comes the control signals to turn right, turn left, slow down, speed up, stop, start, etc?

2,000 miles away, I'm driving my Tesla which has a few CPUs and GPUs. but not a neural network or even huge banks of memory large enough to store the whole library of each of those visuals and their corresponding control commands? I can understand the car having the capacity to store and run a program with 300,000 lines of code, (as evidenced by taking 30 minutes to download even on a fast network).

I'm okay with how the neural net systems analyzes the camera data and decide what the correct control signals are for that data. But what's the connection with my car 2,000 miles away where I need the correct control signals for what ITS cameras are seeing?

My car doesn't "learn" and store solutions for specific situations. How does having a giant library of control commands in Palo Alto, do my car any good in BFE. My car doesn't have neural nets making decisions about what to do? It needs real time adjustments on each of the car's controls, according to what it is seeing right then.

Is the concept then that the neural net system in P.A. is clever enough that it will analyze all those videos and images, and for each, develop the correct control signals. Then is clever enough to write a program so wonderful that it can write a program with only 100 lines of code, or even 300,000 lines of code, that will be able to make the same decisions without training? Then you just update the car to the better program and everything is wonderful? And somehow the car will be able to make good decisions about every situation that was presented to the training computer, without even thinking about it? The present HW1 - HW4 does not provide for "learning" does it?
 
I'll take a swing at this; hopefully I won't make things more confusing. I'm going to start with what I understand Tesla to be doing with v12 as I actually think that's easier to get your head around.

Super (over) simplified concept of what's going on: Machine learning engineers put together what's called a model, it's constructed of (lots of) nodes and connections between the nodes (synapses) but it is untrained. Each node does some relatively simple calculation, part of the calculation comes from the input information it gets and part of the calculation is what are called weights (or parameters, etc) - information stored in each node. For the purposes of this over simplification, think of it as conceptually the Ml engineers have wired the cameras to the steering steering wheel, brakes, go pedal, etc. The car still has no idea how to drive, but you if you point the camera at something, control changes (random) are going to pop out the other end of this "program".

Next, you have to tune all those weights to get the behavior you want. Modern models can have millions or even billions of parameters involved. Again, over simplifying, but you show an image to a camera, out pops some control changes. You compare that to what you want (part of the training data) and then tweak the weights of the nodes along the way to get it closer to what you wanted to happen. Repeat. A lot. Like a lot a lot. This is what is happening in Palo Alto. It takes massive amounts of data (images, desired control changes, etc) and massive amounts of processing (to continually readjust the weights).

When that's done, meaning it's behaving within your tolerances against separate test data, those *weights* are captured along with the model. *THAT* is what is sent to your car in the next firmware - the model, plus the latest and greatest set of weights. (Again, I'm over simplifying a lot here, but for understanding purposes, this is conceptually close enough.)

What your car does is called inference - it takes images from the camera as you drive around and locally, on the car, runs them through the model + weights to get control outputs. Inference is dramatically less computationally intensive than training. There's no reaching back to Palo Alto or anything like that. Palo Alto is busy working on a better set of weights with more / improved data and maybe a tweaked model architecture, etc.; ultimately it's moved on.

What I'm describing is what v12 appears to be, vs versions before that didn't use neural nets all the way through. They used them for pieces of the problem and that information was fed into manually written control code (that's the 300k lines of C++ folks are referring to). Think of those versions as using neural nets to pick out lane lines and cars and stop signs, etc then some hand written code takes that information to then say "ok - I'm not quite centered in the lane, but I want to be, so steer left X degrees", etc. I'm totally making that example up, but, again, conceptually FSD < 12 is closer to that.

Again, I'm being loose with my terminology and am dramatically simplifying, but conceptually this is what's going on.
 
My Tesla 2003 M3LR(11.4.4)just blew through a stop sign! Yikes. I did not expect that at all. It’s the first time I’ve driven this particular route so maybe it’s GPS data related? It’s a pretty established road in a large development so I was very suprised. Hard to keep up with this thread since it moves very quickly so my apologies if this behavior has been discussed. But I’ll ask anyway..;) has anyone experienced this?

I’m going to drive the same route later to see if it happens again.
 
  • Informative
Reactions: sleepydoc
My Tesla 2003 M3LR(11.4.4)just blew through a stop sign! Yikes. I did not expect that at all. It’s the first time I’ve driven this particular route so maybe it’s GPS data related? It’s a pretty established road in a large development so I was very suprised. Hard to keep up with this thread since it moves very quickly so my apologies if this behavior has been discussed. But I’ll ask anyway..;) has anyone experienced this?

I’m going to drive the same route later to see if it happens again.
That's why we all need to be paying attention when using this, anything can happen and at the worst time.
 
My Tesla 2003 M3LR(11.4.4)just blew through a stop sign! Yikes. I did not expect that at all. It’s the first time I’ve driven this particular route so maybe it’s GPS data related? It’s a pretty established road in a large development so I was very suprised. Hard to keep up with this thread since it moves very quickly so my apologies if this behavior has been discussed. But I’ll ask anyway..;) has anyone experienced this?

I’m going to drive the same route later to see if it happens again.
Mine also did this and waiting for the photo ticket
Think when FSD sees red and green lights back to back, as in my case 50 feet in between, it defaults to green and goes through
 
I wonder it's necessary to re-calibrate cameras after installing 11.4.7.
I've had my car for over 4 years and I have recalibrated the cameras exactly .........0 times. Unless there is a hardware change or a specific problem it is unnecessary, but no harm other that having to wait on calibration if you want to.
 
Since 11.4.7 is less than a near to nothing update from 11.4.4 I'm going to not download/instal and see if there is ANY chance I could get pushed 23.26.11. I know there is little chance but nothing to lose trying this since 23.7.30 is NO real update over 23.7.20.
 
  • Like
Reactions: Pdubs
Since 11.4.7 is less than a near to nothing update from 11.4.4 I'm going to not download/instal and see if there is ANY chance I could get pushed 23.26.11. I know there is little chance but nothing to lose trying this since 23.7.30 is NO real update.
It feels like a pretty large leap, tbh. There are still major issues, but it drives a lot different. Elon said there's a lot of AI replacement in 11.4.7.

You won't be able to go to 2023.26.11 unless you can unsubscribe.
 
  • Like
Reactions: sleepydoc and Pdubs