Tesla autopilot HW3

[lunitiks]

But Tesla has to do it.

 

[strangecosmos]

It seems dubious that Tesla would try to run FSD on HW2 since this would require the AI team to spend its time — one of Tesla’s most scarce, precious resources — designing two versions of the vision neural network: one for HW2 and one for HW3.

Obviously, Navigate on Autopilot runs on HW2 and maybe some other FSD features will as well. But in the long term, I can’t see Tesla trying to run everything on both HW3 and HW2.

If the price of FSD exceeds the cost of the upgrade, then Tesla has a financial incentive to offer upgrades.

There is also potentially a data incentive. Upgrading HW2 cars is a way to get HW3 cars on the road. HW3’s NN/NNs will likely be able to make better predictions (i.e. judgments or determinations). That predictions data could be useful. Instead of uploading the raw camera data, Tesla could just upload the predictions data. If the predictions are accurate, that saves getting a human to manually label the raw camera data. This wouldn’t be useful for supervised learning of perception, obviously — where’s the source of ground truth? — but it could, in theory, be useful for weakly supervised learning of perception or for behavioural cloning (a.k.a. imitation learning).

 

[lunitiks]

Obviously, Navigate on Autopilot runs on HW2 and maybe some other FSD features will as well. But in the long term, I can’t see Tesla trying to run everything on both HW3 and HW2.

Valid point. What do you think, however, of the fact that HW2.0 and 2.5 sport different camera- and radar sensors? Mustn't there be different NNs, or at least, different capabilities? I know too little about the consequences of having one fleet with RCCC and one with RCCB. And I'm just now trying to work out the difference between the radars

 

[strangecosmos]

Valid point. What do you think, however, of the fact that HW2.0 and 2.5 sport different camera- and radar sensors? Mustn't there be different NNs, or at least, different capabilities? I know too little about the consequences of having one fleet with RCCC and one with RCCB. And I'm just now trying to work out the difference between the radars

Did we find out whether the difference between the colours the cameras can perceive is the firmware or the hardware?

On the ARK Invest podcast, Elon said that HW2 can’t run the NN they want to run with input from all 8 cameras at full resolution and full framerate. They use suboptimal tricks like cropping images to reduce the computational load. The work that would need to be done by the AI team is all those tricks to make the NN run on HW2. That seems labour intensive.

I don’t know about the differences between the sensors. I am actually curious whether Model 3, Model X, and Model S can share the same training datasets and the same trained neural networks because the spatial configuration of their cameras is different. jimmy_d says the new NN is camera agnostic, although I’m still not entirely sure what that means.

 

[lunitiks]

Did we find out whether the difference between the colours the cameras can perceive is the firmware or the hardware?

Different hardware. (Different part numbers, different color filters.)

I think it's been proven by @verygreen and @DamianXVI in their videos. (Well, that doesn't prove different hardware, but. Still, different part numbers.)

 

[mongo]

It seems dubious that Tesla would try to run FSD on HW2 since this would require the AI team to spend its time — one of Tesla’s most scarce, precious resources — designing two versions of the vision neural network: one for HW2 and one for HW3.

Based on my understanding of Karpathy's SW 2.0 talk:
I don't think it would take a lot of developer time. The NN training would be done with a 2.x version network using the same data sets as AP3.0. The time investment would be reducing features and camera data rates to find a maximal solution.

(or if they SW2.0 the SW2.0, a bigger NN does the work of fitting the FSD NN into the HW2.x capabilities)

 

[jimmy_d]

That's because his job -- if I'm not mistaken -- is to obfuscate, cherry-pick and otherwise dissemble in order to advance a particular position.

This is not meant to be a personal attack. This is a statement of fact about @strangecosmos's day job over at Seeking Alpha, where he writes essays espousing particular viewpoints (in other words, opinion reporting rather than news reporting -- there's a very good reason that reputable news outlets separate these two things). It also does not mean that the particular position(s) he advances in his opinion reporting are incorrect or harmful.

Though, as a matter of my own opinion, those positions are both incorrect and harmful.

I think this probably qualifies as a personal attack.

 

[strangecosmos]

(or if they SW2.0 the SW2.0, a bigger NN does the work of fitting the FSD NN into the HW2.x capabilities)

If they could do that, why release HW3 at all?

The time investment would be reducing features and camera data rates to find a maximal solution.

Elon did get Karpathy to spend time working on a NN solution to rain-sensing windshield wipers, so... Maybe ¯\_(ツ)_/¯ It could save a lot of money.

Ultimately though I think there comes a time when HW2 hits its limits, even with all the tricks, and needs to be replaced with HW3 for safety reasons.

 

[jimmy_d]

There’s so much noise and confusion in here about SAEs levels it’s unbearable.

J3016 is a taxonomy. It deals with the question: To which degree is the human expected to perform the driving tasks? And it lets us assign a Level from 0-5. "0" meaning the human is absolutely necessary in all respects, "5" meaning the human is not expected to do anything, anytime, anywhere.

J3016, chapter 8.1:


As for mr. Lex Friedman, he said that J3016 is “the most widely accepted taxonomization of autonomy”. And that it’s useful for describing the system, for legal discussion and for policy making. Which, of course, is right.

But then Friedman goes on to say that he thinks/feels J3016 is not useful for “design and engineering” because he thinks/feels there are/should be only two “real” categories – one where the system “is not fully responsible”, and one where the system “is fully responsible”.

Which is where he messes up.

First of all, responsibility is a legal matter - so it’s exactly what Friedman began saying J3016 is useful for describing.

Secondly, Friedman doesn’t seem to appreciate the fact that J3016 draws an unambiguous line between L0-L2 and L3-L5, where you are doing the driving in L0-L2, while you are not doing the driving in L3-L5. Because your car manufacturer has designed the system such that you're not supposed to be driving.

Which is why your car manufacturer must take legal responsibility for the car's actions in L3-L5. The manufacturer can’t tell you that you’re not supposed to be monitoring the car’s surroundings, and at the same time say you’re liable because you failed to monitor the car’s surroundings.

So J3016 is as much about manufacturer intent than anything else. Chapter 8.2:



So when Waymo publicly writes that their system is Level 4, it's because it's their design intent. It's what Waymo expects of it, and takes responsibility for.

Tesla on the other hand publicly makes it clear you must «keep your hands on the wheel at all times», that the car is not autonomous, and that YOU are responsible. So L2 is Tesla’s design intent with NoA, Auto Lane Change and Summon. They’re not intended to be, which is why they’re not, L3 features.


Get it? Good. Now wake me up when Tesla demonstrates anything L3.

So the disconnect is that we’re comparing an unfinished L4 system to a consumer released L2 system? If manufacturers final product design intent in the discriminatory factor then you’ll get weird results when you apply it to the current operating features of an incomplete test vehicle.

Does that mean that the Apple is an L2 system and the orange *will be* an L4 system?

 

[lunitiks]

My orange is certainly L5

 

[mongo]

If they could do that, why release HW3 at all?



Elon did get Karpathy to spend time working on a NN solution to rain-sensing windshield wipers, so... Maybe ¯\_(ツ)_/¯ It could save a lot of money.

Ultimately though I think there comes a time when HW2 hits its limits, even with all the tricks, and needs to be replaced with HW3 for safety reasons.

The NN on NN would only be able to find a maximal feature fit (which option can you have, how big a camera image, how high a frame rate) for the HW2.0, it would not be able to fit more in than is computationally possible.

The rain sense via NN is necessary because only the cameras can tell if they are occluded (bird droppings/ insects).

Yeah, HW3 is bigger, better, faster, more.

 

[jimmy_d]

My orange is certainly L5

Oh man I seriously LOL’d on that. Good one.

 

[Bladerskb]

There is also potentially a data incentive. Upgrading HW2 cars is a way to get HW3 cars on the road. HW3’s NN/NNs will likely be able to make better predictions (i.e. judgments or determinations). That predictions data could be useful. Instead of uploading the raw camera data, Tesla could just upload the predictions data. If the predictions are accurate, that saves getting a human to manually label the raw camera data. This wouldn’t be useful for supervised learning of perception, obviously — where’s the source of ground truth? — but it could, in theory, be useful for weakly supervised learning of perception or for behavioural cloning (a.k.a. imitation learning).

If we follow your thesis to the letter, then they dont need to wait for HW3 to collect data, train and deploy the "exponential" RL. In the case of NOA, traffic lights and stop signs dont exist on limited divided highways where NOA functions in and RL models are not computationally intensive and can certainly run on HW2.X.

NOA is currently unusable according to most people, I guess we will see in 5 days on your RL thesis.

 

[diplomat33]

Here are 4 strong reasons why I think Tesla will give AP2 owners a free AP3 upgrade:

1) Tesla had the fore thought of designing the AP computer to be easily upgraded. Musk said the AP2 computer can be swapped out and the AP3 computer swapped in. This suggests that Tesla planned in advance to upgrade computers over time in order for older cars to remain compatible with FSD. So Tesla planned for the possibility that FSD might be delayed past the life span of AP2 but wanted those cars to still be compatible for FSD. Why do that and then pass?

2) Tesla has made a promise to AP2 owners that they will get the AP3 chip if is required for FSD so that AP2 owners won't miss out on what they paid for. To renege on that promise now and use some loophole that AP2 is good enough, would be a huge breach of trust and open Tesla up to lawsuits. Even if the lawsuits don't scare Tesla, losing the trust of their initial and most loyal customers who helped them get to where they are now, would be terrible, maybe even fatal for Tesla. Plus, Tesla has always been good in the past about at least throwing a bone to older customers as a thank you for their loyalty. Didn't Tesla give reservation priority to Model S owners for the Model 3 launch explicitly as a thank you to them for their loyalty? To give their loyal customers a big FU now would seem out of character honestly.

3) Upgrading AP2 cars with the AP3 chip would allow Tesla to increase the number of cars that can use FSD properly and thus increase the data collection they get to further improve FSD. Plus, it would reduce the disparity in hardware and thus make software updates easier. It would be a lot easier for Tesla if they just have to write FSD software for AP3 than if they have to worry about compatibility across AP2, AP2.5 and AP3 platforms. So it is certainly to their advantage to give AP2 cars the AP3 upgrade.

4) FSD is a fundamental piece of Tesla's Master Plan. Tesla has made it clear that FSD and eventually the Tesla Network is a key part of their entire vision of sustainable transportation. Tesla wants YOU to have FSD! Giving AP2 owners the AP3 upgrade would fit perfectly in that plan. To deny AP2 cars the upgrade or cheat their way out of it would be completely inconsistent with their master plan.

 

[emmz0r]

Unfortunately this has not stopped Tesla from not following through on promises before. Many AP1 promises remain undelivered (including the navigation-based exit-taking for 8.1 in December 2016) as does the Model S free lighted vanity mirrors promise for all cars. Of course P85D never got its HP and P90DL never reached its performance numbers but went through several versions for new buyers leaving old buyers in the dust...

They seem to promise enough to keep current sales going but over time those promises may not be kept.

Indeed, but this is different:
FSD is a defined package now where you actually need the hardware to function. They can't make an AP2 FSD package, and a different AP2.5+ FSD package. They have to come under one. So that's why I see no reason not to retrofit those cars who ordered FSD. How many can there be anyway? It's not a mundane task to do either. There's simply no excuses.

 

[electronblue]

I genuinely hope Tesla does retrofit HW3 on AP2/2.5 cars. Despite my cynicism, I still hold out hope — I think the risks are there but this is by no means a closed chapter yet luckily.

 

[emmz0r]

I genuinely hope Tesla does retrofit HW3 on AP2/2.5 cars. Despite my cynicism, I still hold out hope — I think the risks are there but this is by no means a closed chapter yet luckily.

Either that, or they have to refund the full price of HW2/2.5 cars that bought FSD.
The separate product name was FSD. It is a distinct product. On top of this you have a huge amount of Model 3s that have 2.5, and have bought FSD.

The only thing they can do is to drag it out forever, but how long can they legally do that? Didn't they get a class action for lacking feature parity with AP1?

 

[electronblue]

BTW what’s the latest: Is Model 3 HW3 retrofit really that much harder than Model S/X? I remember reading speculation that the sandwich design and water cooling could make a potential replacement that much harder on Model 3? Is it really so? I mean they must have designed the part to be serviceable anyway...

@lunitiks @verygreen

 

[verygreen]

It's doable, but harder. Teh sandwich design pretty much ensure they would need to replace the whole thing, not just the autopilot unit. That adds a quite a bit of reprogramming and data copying all around. The 30 minutes replacement time? not even in realm of possibilities on the model3. Quoted replacement time on S/X for the AP unit is 54 minutes...

 

[fbitz777]

The only thing they can do is to drag it out forever, but how long can they legally do that? Didn't they get a class action for lacking feature parity with AP1?

Yes I got a refund on my EAP for about 150$
Note that back in 2016 you had AP as a $2500 option but then they introduced AP2.0 and "forced" people to EAP (5k) ...

If I follow the thread even EAP owners should be upgraded to HW3...but since now EAP is no longer offered it will be hard to prove that.