It doesn't matter what version you have. So far as has been disclosed, Tesla has only released 12.x to a limited number of employee vehicles. By any measure of past performance, it will take months to go from first employee rollout to regular users.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
FSD v12.x (end to end AI)
- Thread starter Buckminster
- Start date
-
- Tags
- #fixphantombraking autopilot chill is not chill elon is an ass fix fsd extreme late deceleration fsd does not commit to go fsd does not maintain proper speed fsd hard and late braking fsd has decision wobble fsd jerky driving fsd late deceleration fsd sometimes may not detect train crossings fsd stops too early too quickly fsd stops too far behind line fsd tailgating in city streets
Agreed. It seems like many are getting expectations Way to high for this release based on a few employees starting to test it. I wouldn’t be surprised if it’s 4 months plus till anyone gets it and that’s what my Assumed expectations are.
KArnold
Active Member
Even if it were wide release today I would not install until the other guinea pigs (yous guys) give it a good shakedown for two or three weeks if not longer. History repeats and it will be filled with bugs.
Be careful what you wish for. Been there, done that.
Be careful what you wish for. Been there, done that.
Probability of getting V12
1. Last week of December 2023 (.05)
2. Washington Birthday WE (.10)
3. Memorial Day WE (.25)
4. Independence Day WE (.40)
5. Labor Day WE (.60)
6. Thanksgiving WE (.75)
7. Christmas 2024 WE (.95)
8. After Christmas 2024 (.00) - FSD project fold. Elon moves to Mars. Customers won't get refunded.
1. Last week of December 2023 (.05)
2. Washington Birthday WE (.10)
3. Memorial Day WE (.25)
4. Independence Day WE (.40)
5. Labor Day WE (.60)
6. Thanksgiving WE (.75)
7. Christmas 2024 WE (.95)
8. After Christmas 2024 (.00) - FSD project fold. Elon moves to Mars. Customers won't get refunded.
jebinc
Well-Known Member
Yeah, people have tried looking at small/individual parts of neural networks at various levels/depth to try to understand what each might represent, and often times especially in larger models, there can be multiple partial concepts, e.g., both dog head and cat tail but also say mailbox in "one." As you point out, additional neural networks can help in the probably tedious task of finding the collection of network parts that correspond to each of the actual human-level single concept of a cat vs mailbox.
These secondary neural networks can be built right next to the larger network as additional fine-tuned heads such as those with outputs to be used for generating FSD visualizations of the road, objects, etc. even for a world model that was not explicitly trained on human-interpretable labels for perception.
willow_hiller
Well-Known Member
I was also thinking more about this. Because Tesla is so familiar with their own neural network architecture, they may be able to design a visual area of the monolithic model, and a planning/control area, and sample the intermediate weights straight from the visual area without needing a second network.
This might be theoretically possible because neural networks are ultimately optimization problems; so if you have a section of the system architecture that has a structure that is best-suited to learning visual recognition, and a section of the architecture that is best-suited to planning/control, the model will naturally tend to build the visual learning into the first area of weights. Because we know that perceiving the world and recognizing objects is prerequisite to planning a course around them, the network will necessarily contain that information; Tesla just needs to know where to look for it.
Unless of course the network learns a way to navigate the world in a way that doesn't involve some form of human-recognizable perception.
Does anyone understand Musk’s comments during the v12 demo in August that HW4 cars would not get it until HW3 cars were dialed in, probably about six months later? It puzzles me since our new in August Model Y has received the newest v11 FSD Beta, the same day as our 12/2016 Model S. I would think our Model Y would also be behind in receiving the current v11 FSD Beta if data from HW4 cars is insufficient.
(And yes, I do realize anything Musk says about timelines is suspect.)
(And yes, I do realize anything Musk says about timelines is suspect.)
As you probably guessed, there is no chance in the near term FSD picks up on the many queues that are so obvious to a human driver. But given enough time, almost anything is possible with tech advances, the right team, and leadership.
How will this overtime work? Need more OTA UG or the code is there and merely needs to have access to more and more data?
If the later is true, it’s a paradigm shift where code function gets better over time just with more data
willow_hiller
Well-Known Member
Probably just two reasons:
1. There are enough changes to the camera angles/filters that v12 for HW4 will require fine-tuning of the base model. There is a lot more training data from HW3 for them to pull from in the first round of training, and HW4 will have to be trained on top of that after-the-fact.
2. The uncompressed HW4 data has a much higher bit-rate, so even if they had a comparable volume/quality of training data from HW4, training V12 from scratch on HW4 would take a lot longer or require a lot more compute.
stopcrazypp
Well-Known Member
Other than cropping to simulate one of the missing cams (HW4 only has 2 cams vs 3 in front) and for front dashcam purposes, I imagine currently Tesla is doing 2x2 binning currently to emulate HW3, which would not require much extra compute if at all (depending on how much they are doing sensor side).
As per other thread, it appears they are doing binning on the repeater cams anyways:
There will be NO HW4 upgrade for HW3 owners
And you know snapping in the HW4 cameras will fix this? I am expecting as they have a wider FOV.
teslamotorsclub.com
We started seeing HW4 vehicles getting the same FSD Beta version as HW3 vehicles as early as May with 11.4.x, so presumably Tesla figured out some way to share the model between hardware potentially as you suggest with cropping and binning. This inclusion of HW4 was a 2 month delay after HW3 got any 11.x back in March, but most of HW4 fleet had to wait until late August for wide release of 11.4.4 -- 3 months after initial HW4 release and basically 6 months after HW3.
Perhaps we'll initially see a similar shared model between HW3 and HW4 for end-to-end with the compatibility hacks/solutions, and maybe the model will even be tainted/trained with HW4-simulated-as-HW3 inputs. Unclear if end-to-end will be more sensitive to artifacts introduced by supporting both sets of cameras than 11.x.
So unclear if Elon Musk's "HW4 software will lag HW3 by at least another six months" is referring to even getting the compatibility working with end-to-end or potentially the longer-term goal of dedicated HW4 model trained with full HW4 cameras making use of HW4 compute.
Some HW4 cars received and used Beta in March. Most are on 11.4.4, but those cars were upgraded with everyone else to 11.4.7.
On the topic of additional outputs, as for vidualization:
I have to say that I'm not seeing why this is any sort of a fundamentally separate problem.
The FSD network must be trained to perform all required actions for the driving task, and these include at least low-level communication outputs to give helpful information to other drivers. The obvious and simplest example is activating turn signals, not only for the correct direction but at the most useful point in the decision to planning to control queue. Judicious use of the horn is another (and actually rather difficult to get right if your goal is communication rather than emotion). A more subtle one, but I claim also very important, is to accelerate, decelerate and pose the vehicle in a way that gives helpful cues to surrounding actors. (By the way, the current FSD 11.x needs work on all of the above.)
Providing the visualizations (or the precursor database from which they can readily be drawn) is not fundamentally a different requirement in my view. Yes it's more complex graphically, but probably simpler in terms of setting and judging the correctness of the output, i.e. the training goal function. It's just being treated as an "extra" problem in many discussions, because it happens to be something new in the set of tasks that we traditionally associate with automobile design and operation. It actually doesn't have to be something bolted on to an otherwise opaque FSD robocar, it simply has to be a target of the development all the way through.
In fact, I suspect that a winning NN FSD architecture, descending from all the prior work, already contains the precursor database because that's essentially what the car needs for its planning. Even if this hadn't been crafted by prior heuristic code development (which it has), it's very likely that something a lot like lane maps, traffic controls, various road users and occupancy network would eventually have come out of a completely blank NN slate. It's already been pointed out here that LLMs appear to gravitate towards an internal data representation of world geography, that essentially mimics a human-style map of the globe - even if this was not explicitly encouraged in the initial model setup.
Then that internal NN traffic map, though really made up of NN weights and not intrinsically comprehensible to human engineers, is the best and most efficient source to feed the goal-driven production of a very tractable (and recordable) database of the local traffic environment. Probably a lot like the data structures that video game programmers have been using for decades, so from there the path is clear to get to the on-screen visualization. In theory, they could train directly for the final graphic image as a goal, but I think that's probably unnecessary and even counterproductive.
I have to say that I'm not seeing why this is any sort of a fundamentally separate problem.
The FSD network must be trained to perform all required actions for the driving task, and these include at least low-level communication outputs to give helpful information to other drivers. The obvious and simplest example is activating turn signals, not only for the correct direction but at the most useful point in the decision to planning to control queue. Judicious use of the horn is another (and actually rather difficult to get right if your goal is communication rather than emotion). A more subtle one, but I claim also very important, is to accelerate, decelerate and pose the vehicle in a way that gives helpful cues to surrounding actors. (By the way, the current FSD 11.x needs work on all of the above.)
Providing the visualizations (or the precursor database from which they can readily be drawn) is not fundamentally a different requirement in my view. Yes it's more complex graphically, but probably simpler in terms of setting and judging the correctness of the output, i.e. the training goal function. It's just being treated as an "extra" problem in many discussions, because it happens to be something new in the set of tasks that we traditionally associate with automobile design and operation. It actually doesn't have to be something bolted on to an otherwise opaque FSD robocar, it simply has to be a target of the development all the way through.
In fact, I suspect that a winning NN FSD architecture, descending from all the prior work, already contains the precursor database because that's essentially what the car needs for its planning. Even if this hadn't been crafted by prior heuristic code development (which it has), it's very likely that something a lot like lane maps, traffic controls, various road users and occupancy network would eventually have come out of a completely blank NN slate. It's already been pointed out here that LLMs appear to gravitate towards an internal data representation of world geography, that essentially mimics a human-style map of the globe - even if this was not explicitly encouraged in the initial model setup.
Then that internal NN traffic map, though really made up of NN weights and not intrinsically comprehensible to human engineers, is the best and most efficient source to feed the goal-driven production of a very tractable (and recordable) database of the local traffic environment. Probably a lot like the data structures that video game programmers have been using for decades, so from there the path is clear to get to the on-screen visualization. In theory, they could train directly for the final graphic image as a goal, but I think that's probably unnecessary and even counterproductive.
Wow I can't believe there is a conspiracy that V12 isn't really end to end!
I work in machine learning / deep learning, and having following all of Tesla's comments on V12, there isn't nor ever was a question in my mind they were talking about going from pixels into to steering commands out. Tesla did not invent that comment, the idea has probed for 10 years.
Whether the end to end is explicity one network, or multiple networks chained together does not matter as much as long as it allows backprop to flow through gradients from one part to the next.
The discussion about ways to extract intermediate steps is interesting - I would just add in my mind there are some pretty obvious things they can do. For one, they easilly set up secondary outputs from whichever layer in the network that also get supervised feedback. What would the labels be? V11's output!
See having a parallel model that aleady produces visiualizations can be used as the labeler for training V12 to output visualizations. This is quite straightforward.
I work in machine learning / deep learning, and having following all of Tesla's comments on V12, there isn't nor ever was a question in my mind they were talking about going from pixels into to steering commands out. Tesla did not invent that comment, the idea has probed for 10 years.
Whether the end to end is explicity one network, or multiple networks chained together does not matter as much as long as it allows backprop to flow through gradients from one part to the next.
The discussion about ways to extract intermediate steps is interesting - I would just add in my mind there are some pretty obvious things they can do. For one, they easilly set up secondary outputs from whichever layer in the network that also get supervised feedback. What would the labels be? V11's output!
See having a parallel model that aleady produces visiualizations can be used as the labeler for training V12 to output visualizations. This is quite straightforward.
Doesn't back propagation have the effect of turning any involved layers into a monolith? I'm seeing this as a case of a 'perception' network being nothing more than a starting point for the overall system's training. That is, after its initial training, nothing is going to require that the perception network deliver stop signs and lane lines. Those signals will be tweaked and tuned by the back propagation until the ultimate output - control - is optimized.
