-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
Elon: FSD Beta tweets
- Thread starter Terminator857
- Start date
willow_hiller
Well-Known Member
Any good educated guesses about what exactly Elon means when he says they're "polishing" or "smooth[ing]" out a point release?
Do we think Tesla is fine-tuning the end-to-end model with clips collected from employees and early testers?
Or maybe there's some sort of filter or smoothing applied to the inputs or outputs or the model that they're tweaking per point release?
Why don't you tell us since you see as much info as EVERYONE else in the world sees (sans Tesla engineers).
Reading between the lines when he says a few it sounds like they lost control of v12.4 and hope to stumble upon a few lucky breaks.
willow_hiller
Well-Known Member
AlanSubie4Life
Efficiency Obsessed Member
It is meant to obfuscate, so it is
.Since 12.3.x seemed to not have had any major retraining, using the same fundamental driving model throughout (as evidenced by IDENTICAL driving behavior in each point release except in the prompt-engineered locations - literally it takes EXACTLY the same line in a few test scenarios every time on every 12.3.x - completely different than 11.4.9), I assume the same prompt-engineering occurs for 12.4.x.
It’s just the polish on the
. 
!It certainly would be interesting to know more about how they do this, but I have not seen any educated discussion about how it is done on this site. Of course, that is not too surprising, because even on the much more scrutable mechanical side of things, it’s tough to get good information here much of the time. It’s usually just someone spouting off the latest myth or rumor and that is what we get. It’s not like NASIOC was back in the day!
Last edited:
Ben W
Chess Grandmaster (Supervised)
Presumably, "lost control" = serious regressions that they don't yet understand the cause of, or how to fix, and "lucky breaks" = figuring it out in a solvable way in a reasonable amount of time.
At some point, they will likely reach the compute ceiling of HW3 (given their model architecture), and after that the more information they try to stuff into the network for new tasks, the more it will degrade the old tasks. The progress of e.g. ChatGPT (from 2 to 3 to 4) has shown that increasing model size and complexity yields amazing gains, but this requires increasingly capable hardware to support: each successive GPT model has been about an order of magnitude bigger than the last. On fixed hardware, progress is much tougher, and HW3 is over five years old, practically an eternity in computer years. And HW4 is only about 50% faster than HW3. (3 cores vs 2, but otherwise similar.) So it's possible they're already bumping up against this ceiling. "AI5" will presumably be a much bigger leap, but none of the current fleet will benefit from it, unless they somehow make an upgrade path.
Last edited:
At the Shareholder Meeting, when asked about FSD improvement he said, "I don't know...maybe 2-3x?"
As if anyone needed definitive proof not to take anything Elon says literally.
AlanSubie4Life
Efficiency Obsessed Member
Yeah I think we all know with certainty now there will be no huge improvements in FSD with current hardware. We are where we are; just incremental improvements like v12 from v11 going forward. But improvements of course are MUCH more difficult to get now. So improvements will slow down.
Even 5x improvement in mpi would be wonderful (though for me it is way easier for them to achieve that than it is for some users).
willow_hiller
Well-Known Member
Are you going to cite a timestamp?
Based on the times in the recording of the event:
At 1:27:35 he says:
"12.4 is actually like a whole different version than 12.3 and 12.5 is a whole different version than 12.4, so you'll see really giant improvements I think sometimes factor of ten improvements between successive versions."
At 2:06:50 he says:
"Between each one there's anywhere from a minimum of doubling of improvement to, in some cases, I think a five to ten fold improvement."
And anyway, these are likely estimates based on how the Autopilot team has observed the loss function scaling as data and compute scale. They're just theoretical estimates of performance and may not fully bear out in the real world.
AlanSubie4Life
Efficiency Obsessed Member
No they are not. Elon said a factor of 10 improvement sometimes. That is real world.
And he set a bar of minimum of doubling regardless of conditions for each release.
Since it is reasonable to think he is talking about general expectations of improvements, any reasonable reading of this says that Elon expects real world improvements of 5-10x in most cases, in short order, over the next few months.
It wouldn’t make sense to talk about theoretical improvements that won’t happen or are on timescales of many many months. He’s talking directly to users about what they are going to see in short order.
willow_hiller
Well-Known Member
Where do you think Elon is getting these estimates before the model is even fully trained? It's based on the scaling laws, as discussed by Ashok at the Q1 2024 earnings call.
Here that is quoted again in case you haven't read it:
I think it's likely that Elon is presuming, and perhaps incorrectly so, that loss scaling is directly proportional to performance scaling.
AlanSubie4Life
Efficiency Obsessed Member
I don’t know, nor do I care, nor do I understand how it is relevant.
Elon is speaking directly to owners/users (and investors) about what they should expect in short order. This means that once they have 12.5 in ~fall they will be getting an absolute minimum of 4x reduction in real world intervention rate vs 12.3.x, and in many cases 25x-100x reduction.
It’s a direct statement by a CEO with access to the most information in the world about the exact expected improvement in real world performance (this is what will matter for the company’s extremely expensive strategic objectives - not the theoretical improvement!!! - so you can be sure they have a very good idea of this). Companies are very good about being able to accurately project 4-6 months out, in general, particularly on their technology roadmap.
So that is what Elon is (presumably) talking about. Saying anything about theoretical performance would be negligent and misleading to investors. Obviously saying he was just “presuming” does not give him an out - he is the CEO so he does not get to do that, especially in this context of speaking directly to investors.
What on earth makes you think 12.5 is not fully trained? Ashok says it only takes a few weeks, and it has been about six months since 12.3 was trained. They’ve seen the real-world scaling laws in real life implementation (or at least simulated life which is just as good in this context).
It seems clear to me that Elon is clearly stating that he expects 25x improvement in 12.5 vs. 12.3 in general, with absolute worst case of 4x. This puts Tesla in a dominant position in the next couple quarters, since there are no competitors in a position like this. This is the hockey stick. This is real world. This is not based on assumptions or Elon’s hopes.
Last edited:
willow_hiller
Well-Known Member
If you don't understand how these things work, and you refuse to try and learn, then you only have yourself to blame when your misunderstanding doesn't align with reality.
AlanSubie4Life
Efficiency Obsessed Member
I don’t see how we could possibly know where Elon is coming up with his estimates, so I am not going to expend energy on that futile effort.
I am not really sure what you are referring to. I have been pretty clear on my expectations. (To quantify, I will roughly estimate 2x improvement in intervention rate by end of the year.) Do you think there will be bigger improvements?
But we’re talking about what Elon said - not why. I know better than to believe what he says, of course. But he’s not going to be wrong (we of course know he will be wrong) simply because he was innocently thinking about theoretical improvements. Now thinking THAT would be incredibly naive.
That is just not how things work. Remember they are putting billions of dollars into this, instead of building cheaper and better cars to compete in the EV market. They are planning to not sell cars! (Elon is on record saying they will not sell cars once they have Robotaxis.)
Last edited:
2 hr, 6 min, 50 sec:
"Between each one (version), there's, I don't know, a doubling of improvement to, in some cases, I think a 5 to 10 fold improvement.
I think we're close to eliminating almost all static object collisions."
Yikes. So he's admitting they know they've had a problem with cars on FSD hitting static objects. E.g. medians, parked emergency vehicles.
willow_hiller
Well-Known Member
You just cited the same timestamp as me, and it does not match your claimed quote. Are you admitting you made it up?
What a curious way to rephrase that statement. It went from "We're reducing collisions" to "We're admitting a known problem". This is why companies use meaningless corporate-speak when they say anything in public.
Similar threads
