Tesla autopilot HW3

[strangecosmos]

Those curious about why Tesla would do a custom ASIC (application-specific integrates circuit) instead of buying an Nvidia GPUs should check out these tweets from James Wang. James is an analyst at ARK Invest, and he formerly worked at Nvidia.

“Why it makes financial and strategic sense for Tesla to build its own AI chip:” James Wang on Twitter

Tesla’s speed in deploying new computing hardware vs. other automakers: James Wang on Twitter

Your hardware doesn’t have to be better if you can get into your cars faster. As Amnon Shashua (CEO of Mobileye) one said, Tesla is faster than other automakers at deploying new hardware because it is its own Tier 1 supplier for many parts: Mobileye CEO: Tesla is 'willing to take more risks' in its self-driving efforts

 

[strangecosmos]

Explanation of the automotive supply chain: https://link.medium.com/z1M49lQQoU

Tier 1 suppliers: “Companies that supply parts or systems directly to OEMs [original equipment manufacturers, e.g. Tesla] are called Tier 1 suppliers. Some of these brands are recognizable, like Bosch or Continental. Some of them are less so.”

Tier 2 suppliers: “Many firms supply parts that wind up in cars, even though these firms themselves do not sell directly to OEMs. These firms are called Tier 2 suppliers.”

Tier 3 suppliers: “In the automotive industry, the term Tier 3 refers to suppliers of raw, or close-to-raw, materials like metal or plastic.”

 

[banned-66611]

The problem with doing an ASIC is that it's a major investment in something that they don't know will actually work.

Typically you try to avoid committing to silicon before you have a working system based on generic parts like GPUs or FPGAs. If they build an ASIC and start fitting it to cars there is a very good chance that they will have to replace it eventually, which means a costly re-design, costly upgrades, yet another hardware platform to support for decades...

I'd expect them to demonstrate FSD works with off-the-shelf parts first, then move to an ASIC for production. What did Musk say, 2022?

 

[mongo]

The problem with doing an ASIC is that it's a major investment in something that they don't know will actually work.

Typically you try to avoid committing to silicon before you have a working system based on generic parts like GPUs or FPGAs. If they build an ASIC and start fitting it to cars there is a very good chance that they will have to replace it eventually, which means a costly re-design, costly upgrades, yet another hardware platform to support for decades...

I'd expect them to demonstrate FSD works with off-the-shelf parts first, then move to an ASIC for production. What did Musk say, 2022?

Which is why they (likely) already developed a FSD NN on server hardware, then ran it with off the shelf HW in cars, then distilled the custom HW need to run it (plus margin), then used FPGA or such to validate that approach, then produced a small run of chips along with modules for S/X and 3, tested those, then ordered production quantities...

Final FSD NN can still be in process of refinement. Worst case, they double (or any X factor) the chips on the board to match final NN size. The system needs full redundancy, so (if upsizing were required) they couod initially use 100% of the processing power in EAP+ mode with human backup and no redundancy. If the chip is optimized for their type of NN, that is an efficient stop gap to a 'perfect' sized chip.

 

[banned-66611]

Which is why they (likely) already developed a FSD NN on server hardware, then ran it with off the shelf HW in cars, then distilled the custom HW need to run it (plus margin), then used FPGA or such to validate that approach, then produced a small run of chips along with modules for S/X and 3, tested those, then ordered production quantities...

Final FSD NN can still be in process of refinement. Worst case, they double (or any X factor) the chips on the board to match final NN size. The system needs full redundancy, so (if upsizing were required) they couod initially use 100% of the processing power in EAP+ mode with human backup and no redundancy. If the chip is optimized for their type of NN, that is an efficient stop gap to a 'perfect' sized chip.

Interesting theory but they have not been able to demonstrate FSD in a car yet. If it was that close they could do the promised cross country demo with the development hardware. They are gambling on having identified the right solution without actually proving it in the real world yet.

 

[boonedocks]

Interesting theory but they have not been able to demonstrate FSD in a car yet. If it was that close they could do the promised cross country demo with the development hardware. They are gambling on having identified the right solution without actually proving it in the real world yet.

Whaaaaat. You mean that video I saw in December 2016 of a model X driving an entire route by itself and dropping off the driver and parking itself....the one that had me ordering an. S100D EAP HW 2.0.....the one that had me PRE-PAY for FSD was not a demonstration of FSD demo by Tesla. What?!?!? LOL LOL LOL (all that was true though)

 

[mongo]

Interesting theory but they have not been able to demonstrate FSD in a car yet. If it was that close they could do the promised cross country demo with the development hardware. They are gambling on having identified the right solution without actually proving it in the real world yet.

Not revealing progress is not the same as not making progress, or not being able.You are trying to prove something didn't happen because you have no data of it happening. However, your knowledge of it not happening has zero bearing on it having happened or not. (null hypothesis)

Did Roadster 2020 not exist at all before the reveal? No, it did exist, and no one, publicly, knew about it.

 

[fmonera]

Whaaaaat. You mean that video I saw in December 2016 of a model X driving an entire route by itself and dropping off the driver and parking itself....the one that had me ordering an. S100D EAP HW 2.0.....the one that had me PRE-PAY for FSD was not a demonstration of FSD demo by Tesla. What?!?!? LOL LOL LOL (all that was true though)

I bought the FSD package too (not feeling cheated though, I already knew where I was getting myself into).

This forum is filled with a lot of posts with DEMOS from Mobileye/Nvidia/... trying to compare future technology with what Tesla has now in actual cars you can buy. That 2016 Tesla demo is great to be compared with all that Mobileye/... demos that we are bombarded here. In fact, in "The fifth element" film (1997), I saw a city full of flying taxis !!!! This can be compared in a fair way with Tesla/Mobileye too.

Sorry to all Tesla haters and competitors (not you @bonedocks ), but for some technology to be seriously compared to what Tesla has TODAY in production, the technology has to be in production in an open environment:

• Not geofenced
• With enough confidence on the technology that they release to a wide part of the world (having to limit the released features to avoid killing a lot of people on unexpected conditions)
• With enough time in the market (to be able to statistically compare the number of accidents caused by the unexpected environment)

Claiming L2+ on a very limited list of specific geofended roads on a specific country as long as you stay on the same lane does not qualify, sorry. Even I can make my car full L5 as long as it is stopped inside my garage, I know I will not kill anyone and the environment is controlled.

For competitors (and haters) there is indeed a fair way to use their DEMOS to compare to Tesla, which is to compare "Future Technology" and "Wishfull thinking" of both Tesla and Mobileye/NVidia/.... For that, in Tesla side it has two contenders:

• The 2016 Demo
• Elon promises and "disclosures" about what they have in development (including hovering above the road with the "Tesla Roadster SpaceX Package", Summon from the other side of the country -in a few months-, FSD already on EAP, and so on).

I don't think comparing future technology or wishes from the CEOs would be fair for competitors. Elon is unbeatable on promising incredible "future technology". Please, compare Apples to Apples and Oranges to Oranges.

 

[dgatwood]

I'd expect them to demonstrate FSD works with off-the-shelf parts first, then move to an ASIC for production. What did Musk say, 2022?

Do you mean Tesla? AFAIK, they're already shipping a custom ASIC since January 9.

Not revealing progress is not the same as not making progress, or not being able.You are trying to prove something didn't happen because you have no data of it happening. However, your knowledge of it not happening has zero bearing on it having happened or not. (null hypothesis)

This. Tesla sends back large quantities of video data at disengagements that they can use for training models offline. They have presumably been doing a huge amount of training, and they know precisely how much compute power is needed to handle the resulting models.

Sure, they can't say for certain that their training data is sufficient or that the net(s) is (are) deep enough. The first of those problems is trivially solvable, but requires lots of human intervention to tag images and stuff, and might still need to be solved before they consider it safe enough to do a full self-driving test. Only the second one would require a major hardware change.

More importantly, the second one would require a major hardware change to hardware that probably isn't practical to build right now. But even if that does eventually prove necessary, deploying HW3 will still bring them much closer in the real world, and will give them a much smaller set of disengagements to work with, which will reduce the amount of human effort needed to determine what does and does not need to be tagged, bringing it to a more manageable level, which will make the next steps easier.

 

[Bladerskb]

Not revealing progress is not the same as not making progress, or not being able.You are trying to prove something didn't happen because you have no data of it happening. However, your knowledge of it not happening has zero bearing on it having happened or not. (null hypothesis)

Did Roadster 2020 not exist at all before the reveal? No, it did exist, and no one, publicly, knew about it.

Elon himself said he will never withhold an AP Update. We already know that the public release is at most 6 months behind the internal development. There's no secret development that is years ahead of the public release. The C2C demo didn't happen because they couldn't do it point blank. We know that they tried it because the maps showed up in the engineering/dev cars.

Sorry to all Tesla haters and competitors (not you @bonedocks ), but for some technology to be seriously compared to what Tesla has TODAY in production, the technology has to be in production in an open environment:

• Not geofenced
• With enough confidence on the technology that they release to a wide part of the world (having to limit the released features to avoid killing a lot of people on unexpected conditions)
• With enough time in the market (to be able to statistically compare the number of accidents caused by the unexpected environment)

Claiming L2+ on a very limited list of specific geofended roads on a specific country as long as you stay on the same lane does not qualify, sorry. Even I can make my car full L5 as long as it is stopped inside my garage, I know I will not kill anyone and the environment is controlled.

I'm sorry but what are we really comparing, adaptive cruise control and lane keeping system that also suggests lane change? Really?
If you want to do a real comparison, compare the complete subsystem individually. Things like hardware, perception software and driving policy.

 

[mongo]

Elon himself said he will never withhold an AP Update.

Not sure, what this has to do with anything I wrote. I assume you mean a vetted/ tested update, otherwise this statement makes no sense:

We already know that the public release is at most 6 months behind the internal development.

There's no secret development that is years ahead of the public release.

Well.. you just said the development is up to 6 months ahead of public release...

The C2C demo didn't happen because they couldn't do it point blank.

They couldn't do it in a 'real win' way. Pre-planed route: maybe. Arbitrary route: no.

We know that they tried it because the maps showed up in the engineering/dev cars.

And Elon said so...

 

[dgatwood]

Elon himself said he will never withhold an AP Update. We already know that the public release is at most 6 months behind the internal development. There's no secret development that is years ahead of the public release.

The new AP neural net requires much faster hardware than what is available in any current vehicles. They aren't withholding it for the fun of it. They're withholding it because hardware that can handle the necessary computation doesn't exist yet except in cars shipped within the last month or so. Tesla almost certainly won't want to beta test it by rolling it out on every vehicle with HW3, which means they can't realistically start deploying it until there are enough HW3 cars out there for 1% of HW3 users to be a meaningful sample size.

It goes without saying that a deeper NN will yield more reliable results. Whether that qualifies as "years ahead" depends largely on how much time they have spent training the larger models so far. That said, given that the main SoC in HW3 is reported to be from way back in 2015, my guess is that they have been working on this for at least three years. So yes, I expect that it will literally be years ahead of what we have now.

 

[S4WRXTTCS]

The new AP neural net requires much faster hardware than what is available in any current vehicles. They aren't withholding it for the fun of it. They're withholding it because hardware that can handle the necessary computation doesn't exist yet except in cars shipped within the last month or so. Tesla almost certainly won't want to beta test it by rolling it out on every vehicle with HW3, which means they can't realistically start deploying it until there are enough HW3 cars out there for 1% of HW3 users to be a meaningful sample size.

It goes without saying that a deeper NN will yield more reliable results. Whether that qualifies as "years ahead" depends largely on how much time they have spent training the larger models so far. That said, given that the main SoC in HW3 is reported to be from way back in 2015, my guess is that they have been working on this for at least three years. So yes, I expect that it will literally be years ahead of what we have now.

Is there any proof that Tesla has been shipping HW3 since Jan 9th?

I did hear about that rumor, but I haven't seen anyone post any actual evidence.

In any case I agree with you in that this is one time where the internal development is going to be certainly well ahead of production since it's brand new hardware. Whether it's shipping now or not.

Of course it won't fix things like weather related camera issues, or navigation related issues due to map issues. Or a million other things, but I think most of us have accepted that it's the slow march.

There is something in HW3 for everyone.

HW2/HW2.5 EAP owners will get to be mad that EAP on HW2+ won't be as good as EAP on HW3 (will likely be around 6 months after HW3 ships out on new vehicles in North America).
FSD owners will finally have something for their $3K even if it's not FSD
Blader has at least 2 years left of joy in comparing a shipping Tesla with Tech demos, and watching all the Tesla fans being mad that some new feature didn't meet expectations. I'm sure I'll be one of them.
Lawyers will be kept fed by people suing Tesla for what they didn't get, and felt promised.

Exciting times despite the gradual tick of actual progress.

 

[S4WRXTTCS]

The problem with doing an ASIC is that it's a major investment in something that they don't know will actually work.

Typically you try to avoid committing to silicon before you have a working system based on generic parts like GPUs or FPGAs. If they build an ASIC and start fitting it to cars there is a very good chance that they will have to replace it eventually, which means a costly re-design, costly upgrades, yet another hardware platform to support for decades...

I'd expect them to demonstrate FSD works with off-the-shelf parts first, then move to an ASIC for production. What did Musk say, 2022?

You avoid committing to silicon before the hardware works. But, the hardware is just a component of FSD and most of the work is software work.

So you simply estimate what you'll need, and you test out sample versions of it on large FPGA's to prove the hardware part out. You can do this well before the final software is done.

I also imagine they planned on doing iterations of versions of the ASIC to continue to scale up as they added more, and more functionality to the ADAS computer.

There are too many unknowns to wait around before locking down the chip.

I also question whether FSD was ever truly the goal in how we see FSD. I think we tend to see FSD from what Elon Musk says from a marketing standpoint. Elon has clearly demonstrated that he's disconnected from the development team in that he didn't even know Sentry Mode required HW 2.5, and was developed using the stuff they developed for Dashcam. He also tried to say NoA was proof of FSD on the freeway or at least came across that way.

Instead I think the Engineers went for a much more step by step approach. Where they knew everything got stacked on top of each other. If one component doesn't work the whole thing falls down.

HW3 in it's entirety is a dramatic shift from what was in HW. Different SOC, different neural net accelerator, etc. It's as if they wanted a clear departure without changing any of the sensor suite from HW2.5.

From a development perspective I doubt the Engineers really even think in terms of EAP versus FSD. Instead it's a singular effort towards eventual FSD with whatever that might look like.

What was committed to silicon was likely what they were comfortable achieving at this point in time despite whatever Elon Musk might say. It's a little hard to be serious about FSD when the rear camera gets dirty so easily. Or things like not having down facing cameras or rear facing radar.

 

[strangecosmos]

Not revealing progress is not the same as not making progress, or not being able.You are trying to prove something didn't happen because you have no data of it happening. However, your knowledge of it not happening has zero bearing on it having happened or not. (null hypothesis)

Did Roadster 2020 not exist at all before the reveal? No, it did exist, and no one, publicly, knew about it.

It’s worth asking: what would a cross-country demo actually prove?

Suppose you have an autonomous car that works 5% of the time. Do 20 attempts and publish a video of the one that worked. Presto! You have a great demo video!

Suppose you have an autonomous car that works 95% of the time. You can do a live demo for the press, or a livestream, or whatever, and have pretty good odds of it going off without a hitch. Voila! An amazing live demo!

For the demo to actually mean anything, there would have to be some evidence of something more than the usual tricks. Say Tesla livestreamed 100 cars all taking different cross-country routes, and all 100 made it across perfectly. That would be impressive.

It still wouldn’t prove that the system is 99.999%+ reliable, but it would be unlikely for all 100 cars to succeed if the system is only 95% reliable.

 

[S4WRXTTCS]

It’s worth asking: what would a cross-country demo actually prove?

Suppose you have an autonomous car that works 5% of the time. Do 20 attempts and publish a video of the one that worked. Presto! You have a great demo video!

Suppose you have an autonomous car that works 95% of the time. You can do a live demo for the press, or a livestream, or whatever, and have pretty good odds of it going off without a hitch. Voila! An amazing live demo!

For the demo to actually mean anything, there would have to be some evidence of something more than the usual tricks. Say Tesla livestreamed 100 cars all taking different cross-country routes, and all 100 made it across perfectly. That would be impressive.

It still wouldn’t prove that the system is 99.999%+ reliable, but it would be unlikely for all 100 cars to succeed if the system is only 95% reliable.

One does challenges because it's emotionally filling. I think most of us can agree that automobiles are a bit silly, but they fulfill an emotional element.

Nothing fulfills that emotional element like the open road.

To demonstrate autonomous cross country driving would be monumental in both marketing terms, and in pushing for acceptance of autonomous driving. Now not a fake spliced together one of hundreds of attempt, but a proper attempt.

They likely have enough data to do a thing within a simulation so it would be just a matter of getting to the point where they felt comfortable attempting such a thing IRL. They'd also likely test out each leg of the journey without publishing the result of all of those attempts.

 

[strangecosmos]

To demonstrate autonomous cross country driving would be monumental in both marketing terms, and in pushing for acceptance of autonomous driving.

But why? Only because the general public is not yet savvy enough to distrust demo videos of autonomous driving?

Anthony Levandowski did a cross-country autonomous drive and it didn’t seem to make much of a splash:

Controversial ex-Uber engineer claims to have completed a coast-to-coast self-driving trip

I agree that demo videos are exciting as a taste of what might be possible in the future. But the reason the topic of the cross-country demo came up in this thread is as evidence about technical progress. Mongo’s argument was that the absence of a cross-country demo isn’t evidence of the absence of progress. My argument is that a cross-country demo isn’t evidence of progress. Not really.

There are ways Tesla could make such a demo evidence of progress. For example, it could do it with the production version of Navigate on Autopilot. But then even better evidence would be customers trying out the new version of NoA themselves, and so the demo would be a bit superfluous.

As I said above, Tesla could livestream 100 cars making the trip along different routes to demonstrate a higher level of robustness than the typical demo video. But that would be quite an event to organize!

 

[verygreen]

Is there any proof that Tesla has been shipping HW3 since Jan 9th?

The closest to that is the model3 wiring diagram effective from Jan 9th that lists ECU3.0 where ECU2.5 was in prior releases.

Nobody did any teardowns yet to see what's inside those cars of course.

 

[kengchang]

The closest to that is the model3 wiring diagram effective from Jan 9th that lists ECU3.0 where ECU2.5 was in prior releases.

Nobody did any teardowns yet to see what's inside those cars of course.

And Europe Model 3 requires separate approval process to turn on AutoPilot

 

[emmz0r]

Tesla almost certainly won't want to beta test it by rolling it out on every vehicle with HW3, which means they can't realistically start deploying it until there are enough HW3 cars out there for 1% of HW3 users to be a meaningful sample size.

Well then. They can just retrofit mine already, I paid for it Then the % of HW3 cars will increase.