FSD Beta v11.x

Knightshade · May 24, 2022

DrChaos said:
So, do you think that after the 'merge' and legacy Autopilot is completely off, there will be more compute available to improve the frame rates for FSDb? What level? Do you have any specific technical knowledge here?

I do not believe HW3 is sufficient to run L3 or better in a single node (I think it might be good enough to run L3 across both nodes or even L4 highway, but don't believe Tesla would do that without any node redundancy-- and I don't think L4 in cities can be done with out at minimum 2-4 more cameras but that's another discussion)

I think it's POSSIBLE they can get L2 to run in a single node after they unify and heavily optimize the code--- but it's also quite possible they can't-- and honestly it doesn't make a ton of sense to do so given we know HW4 exists.

I have some technical knowledge, but don't have deep Tesla specific technical knowledge beyond what some folks like Green have found and reported.

Ultimately what I've said generally is when someone tells you "X is enough compute for redundant generalized self driving" they are lying-- because nobody knows how much is required for that until they actually get it working- and nobody has.

From what we DO know it seems incredibly incredibly, unlikely HW3 is it-- given how far the system still is, and how it's already near the max of both nodes running cross-node code.

It's possible HW4 is enough to do it in a single node....or maybe across both... but again nobody knows either way, and can't until it's actually working- so it's possible that's not enough either.

DrChaos said:
That might improve performance or safety all on its own.

Sure- sticking at L2 I'd expect freeing up currently used compute would give them headroom to improve things... but I don't think they have nearly enough freeable to get you past L2 on this HW since they're not going to ever do L3 or higher without redundant compute nodes- a system were a node crash crashes the car would be insane.

DrChaos said:
For better autonomous performance I would want: stereoscopic 4k to 8k cameras---much more data all on its own. Current main front camera is monocular and 1280x960 fixed orientation---humans have better vision than this for sure.

Plus multiple 77 GHz imaging radar sets. Agree on no lidar, but new radar is excellent. Elon was making up bullshit excuses about inability to do sensor fusion between vision and radar. You can train machine learning models for it, and use better sensors.

I mean, I want a pony too... (might be a bad example in current climate

)

FWIW we know higher res cameras are already coming from what Elon has previously said about cybertruck.... the question will be if it's JUST that and faster driving computer (in which case Tesla can likely retrofit all existing cars with them).... or if it's MORE cameras in additional locations.

Personally I think they need a pair of forward-mounted side facing cameras to solve city driving-- the B pillars simply can't see around corners with obstructions well enough.... you can PROBABLY technically get to L3 with current cams simply requiring the driver to take over at tough corners but that experience would suck.

It wouldn't hurt to put a similar pair on the back either for rear cross traffic detection.

But adding cameras in new places would be vastly harder to retrofit on existing fleet.

As to radar- I agree modern (much more expensive) radar is very good.... but I don't think you need it to drive better than a human. Humans don't have radar either, but the car can at least see in all directions at once-- so if you solve the AI part you get a better-than-human driver without radar.

COULD it be better still WITH radar? Probably. But you still need to solve vision anyway- so that's their focus.

drtimhill · May 24, 2022

DrChaos said:
So, do you think that after the 'merge' and legacy Autopilot is completely off, there will be more compute available to improve the frame rates for FSDb? What level? Do you have any specific technical knowledge here?

Yes, and that is probably more than anything else what I driving the merge .. Tesla needs the horsepower for the FSD stack, and want to jettison as much legacy stuff as possible to free up resources. It also makes sense logistically .. having two stacks to train/maintain/debug is a huge waste of resources (though of course a lot of that will migrate to the single stack in-house, but you still get efficiency from single toolchain and reduced testing surface area etc).

drtimhill · May 24, 2022

Knightshade said:
Ultimately what I've said generally is when someone tells you "X is enough compute for redundant generalized self driving" they are lying-- because nobody knows how much is required for that until they actually get it working- and nobody has.

From what we DO know it seems incredibly incredibly, unlikely HW3 is it-- given how far the system still is, and how it's already near the max of both nodes running cross-node code.

It's possible HW4 is enough to do it in a single node....or maybe across both... but again nobody knows either way, and can't until it's actually working- so it's possible that's not enough either.

My suspicion is there are still some people who think HW3 can achieve their FSD goals (maybe even Elon), and hence the push to discard legacy asap. If not, Tesla have a nasty problem .. either they end up with "FSD lite" on HW3 and "FSD Full" on HW4 (and a lot of very angry HW3 owners) or they end up doing yet another round of free (or at-cost) HW3 to HW4 upgrades (assuming that is even possible).

Knightshade · May 24, 2022

drtimhill said:
My suspicion is there are still some people who think HW3 can achieve their FSD goals (maybe even Elon), and hence the push to discard legacy asap.

Let's just say I admire their optimism

Given even fully using both nodes they still don't have L3 working anywhere, I find it unlikely in the extreme they'll be able to BOTH:
Get the system to L3 on HW3 at all
and
Then squeeze 100% of that code back into a single node on the same HW so they can have failover redundancy

The absolute best case scenario would be they get a system they believe is safe for >L2 except for the fact it requires both nodes of compute to do it. In that case they finally know how much total compute they need, and perhaps HW4 is "it"

(this ignores the whole do they need at least 2 more side/facing forward cams question for city driving of course)

drtimhill said:
If not, Tesla have a nasty problem .. either they end up with "FSD lite" on HW3 and "FSD Full" on HW4 (and a lot of very angry HW3 owners) or they end up doing yet another round of free (or at-cost) HW3 to HW4 upgrades (assuming that is even possible).

The upgrade cost, assuming it's possible, would be trivial to Tesla at this point..... the bigger issues would be:

If they never get to "this is actually enough" compute with HW3, and thus when they move to HW4 they still won't know if even THAT is sufficient.

And, again, the whole "enough" cameras or not question-- because adding them on existing cars is a ton harder than upgrade in place existing ones.

EVNow · Jun 2, 2022

https://twitter.com/x/status/1532195571519430656

FSDtester#1 · Jun 2, 2022

EVNow said:
https://twitter.com/x/status/1532195571519430656

Well, that explains it perfectly to me....

EVNow · Jun 2, 2022

Ryan27 said:
Well, that explains it perfectly to me....

You can search online for more resources.

Why Transformers are Slowly Replacing CNNs in Computer Vision?

Before getting into Transformers, let’s understand why researchers were interested in building something like Transformers inspite of…

becominghuman.ai

Raurele · Jun 8, 2022

Knightshade said:
I do not believe HW3 is sufficient to run L3 or better in a single node (I think it might be good enough to run L3 across both nodes or even L4 highway, but don't believe Tesla would do that without any node redundancy-- and I don't think L4 in cities can be done with out at minimum 2-4 more cameras but that's another discussion)

I think it's POSSIBLE they can get L2 to run in a single node after they unify and heavily optimize the code--- but it's also quite possible they can't-- and honestly it doesn't make a ton of sense to do so given we know HW4 exists.

I have some technical knowledge, but don't have deep Tesla specific technical knowledge beyond what some folks like Green have found and reported.

Ultimately what I've said generally is when someone tells you "X is enough compute for redundant generalized self driving" they are lying-- because nobody knows how much is required for that until they actually get it working- and nobody has.

From what we DO know it seems incredibly incredibly, unlikely HW3 is it-- given how far the system still is, and how it's already near the max of both nodes running cross-node code.

It's possible HW4 is enough to do it in a single node....or maybe across both... but again nobody knows either way, and can't until it's actually working- so it's possible that's not enough either.

Sure- sticking at L2 I'd expect freeing up currently used compute would give them headroom to improve things... but I don't think they have nearly enough freeable to get you past L2 on this HW since they're not going to ever do L3 or higher without redundant compute nodes- a system were a node crash crashes the car would be insane.

Green himself said that V11 is currently running in the background. So NoA stack, Beta stack, and the combined single stack are currently running at the same time… once V11 drops, wouldn’t that free up a HUGE amount of what is needed?

Knightshade · Jun 8, 2022

Raurele said:
Green himself said that V11 is currently running in the background.

Green was somewhat vague beyond it being in shadow mode (which is only part of the full stack by definition) and "The if-then statements are exploding and the complexity is being amassed."

He mentions in the same thread he'd seen nothing to suggest actual full self driving (the L5 kind) was coming anytime soon (which fitting everything into a single node would be required for)

Again the issue is nobody knows how much compute you need for this- until someone does it's all guesswork where you can only consider what's likely or not.

Code that exploded out of single node over 2 years ago now fitting back into 1- on hardware we already know is being replaced with something more powerful within the next year- seems unlikely.

Mardak · Jun 22, 2022

Some interesting information about FSD Beta progress from this video "Elon Musk Unfiltered: Interview Part 3 (Bonus Material)"

I believe Beta 9 introduced Tesla Vision and removed radar:

Once we got rid of radar — which by the way was initially strongly opposed by the Autopilot team — and now nobody wants radar back… I had to lay down the law and say “the radar is coming out…” Once radar was removed, it became clear that actually our neural nets were much worse than we think they are. They were being helped by radar too much.

There was a mention of Beta 10.11's "bag of points" to neural network vector space and that improvement isn't actually slowing:

The reason why [FSD Beta 10.]12.1 didn’t seem that great is because we refactored so many of the neural nets, so that from the outside it looks like 2 steps forwards 2 steps back and it feels like the improvement is flat.

Unclear if this unifying will happen before Beta 11 / single stack:

We still don’t have a unified vector space where all the neural nets pour their outputs into a single agreed upon vector space for both fixed and moving objects… Currently they’re fairly separate — the moving objects network and static objects network, if they disagree on the position of a car, then you could say “this car is in a non-drivable space position because of a disagreement between neural nets.” And if they’re not running at the same frame rate then you get a time error as well.

Maybe as a suggestion for those with FSD Beta and what types of video snapshots to send back to make progress towards the next "major" Beta release:

Our goal right now is to get to 0 interventions in point-to-point drives in cities. So the polish is less important than solving for 0 interventions.

dramsey · Jun 22, 2022

Well, God forbid they should include data from a separate sensor that “helps too much”!

It does seem odd that he complains that radar was producing “more signal than noise” and seconds later says it was “helping tout much”. But Elon’s never been the best speaker.

Mardak · Jun 22, 2022

dramsey said:
he complains that radar was producing “more signal than noise”

The noise probably is part of the reason why he made the call to remove it from real-time driving decisions. But the signal is good to generate training data for the neural networks. Most likely FSD Beta vehicles with radar still have it active for data collection triggers as well as to send back with snapshots for the auto-labeler to process offline where it can better determine with radar signals from future and past if it's actually signal or noise.

In some sense, all previous versions of Autopilot / FSD Beta architectures were crutches like radar but are still useful in progressing to the next architecture. Also like how radar can provide useful training data, previous single-camera predictions that have high confidence were used to train 4D multi-camera neural networks. A similar thing will probably happen leveraging the separate static vs moving object predictions to train the unified one that hopefully allows the legacy networks to be removed allowing compute necessary for Beta 11 / single stack.

drtimhill · Jun 23, 2022

dramsey said:
Well, God forbid they should include data from a separate sensor that “helps too much”!

It does seem odd that he complains that radar was producing “more signal than noise” and seconds later says it was “helping tout much”. But Elon’s never been the best speaker.

When you learn to ride a bike you start with training wheels to stop you falling over .. but later you need to discard them as they ultimately are a hinderance to riding. That's I think basically what he means.

dramsey · Jun 23, 2022

drtimhill said:
When you learn to ride a bike you start with training wheels to stop you falling over .. but later you need to discard them as they ultimately are a hinderance to riding. That's I think basically what he means.

That's a reasonable interpretation; although of course with Elon it's almost always "interpretation", isn't it? Still, makes sense.

Tesla was doing some amazing things with radar. Remember how proud Elon was when they figured out a way to analyze the signal so that they could get data on the car ahead of the car in front of you?

Elon Musk explains Tesla Autopilot's new capacity to see ahead of the car in front of you

One of the most impressive new features that Tesla announced today with the Autopilot’s new radar processing capacity is the...

electrek.co

bizi · Jun 23, 2022

Is there anyone in the United States that purchased a new Tesla in 2022 and the FSD beta? If yes:

When did you buy?
What state are you in?
What model do you have?
How long did you have to wait before getting the FSD beta?

Knightshade · Jun 23, 2022

The beta is not available for purchase.

It is by invite, at no charge- but the group of folks eligible to be invited is limited to those with both FSD and who opted in to be considered for an invite and with a good enough safety score (and likely other location based criteria not fully disclosed)

bizi · Jun 24, 2022

Knightshade said:
The beta is not available for purchase.

It is by invite, at no charge- but the group of folks eligible to be invited is limited to those with both FSD and who opted in to be considered for an invite and with a good enough safety score (and likely other location based criteria not fully disclosed)

Yes I understand that. I asked a poor question. Is there anyone who purchased in 2022 who received a Beta invite?

JulienW · Jun 24, 2022

bizi said:
Yes I understand that. I asked a poor question. Is there anyone who purchased in 2022 who received a Beta invite?

Beta is available for all Tesla cars with HW3 that have paid for or subscribe to FSD. So there are cars from about 2017 onward. Why would you think 2022 cars are excluded and why ask? Also I have acquaintance who has a Model S Long Range that is (I'm reasonably sure) a 2022 he got Beta a couple of weeks ago.

growltyger · Jun 24, 2022

bizi said:
Yes I understand that. I asked a poor question. Is there anyone who purchased in 2022 who received a Beta invite?

Yes. I purchased a 2022 Tesla and have also received an invitation to beta test FSD Beta.

KnifeAfterDark · Jun 25, 2022

What does the invite to FSD Beta look like? Is it an email, notification, or simply an update?

FSD Beta v11.x

Well-Known Member

Active Member

Active Member

Well-Known Member

Well-Known Member

Craves Electrolytes

Well-Known Member

Member

Well-Known Member

Active Member

Member

Active Member

Active Member

Member

Member

Well-Known Member

Member

Well-Known Member

Member

Member

Similar threads