Tesla autopilot HW3

[Cirrus MS100D]

My concerns about the NN development is the fact that my lowly HW2 car has Smart Summon. As has been discussed in other threads, it’s possible that because the speeds are so slow, THAT is the reason why it works, but my hope was the “latest” HW3-focused networks would be SO huge and complicated that HW2 would falter, thus indicating really progress and a real need for additional hardware.

Don’t get me wrong, I’m thankful for Smart Summon and prepaid for FSD in mid 2017, am hopeful they’re doing such a great job with efficiency they don’t (yet) NEED the additional hardware resources, but I’m also simultaneously concerned.

 

[thewishmaster]

My concerns about the NN development is the fact that my lowly HW2 car has Smart Summon. As has been discussed in other threads, it’s possible that because the speeds are so slow, THAT is the reason why it works, but my hope was the “latest” HW3-focused networks would be SO huge and complicated that HW2 would falter, thus indicating really progress and a real need for additional hardware.

Don’t get me wrong, I’m thankful for Smart Summon and prepaid for FSD in mid 2017, am hopeful they’re doing such a great job with efficiency they don’t (yet) NEED the additional hardware resources, but I’m also simultaneously concerned.

I am assuming something like smart summon which is designed to work on all enhanced autopilot cars (e.g. HW2 and HW2.5) will never become something so feature-rich that you *need* HW3+ to run it (after all it'd be hard to advertise Smart Summon with a bunch of features that don't actually work on older cars, though it's not unprecedented e.g. sentry mode or all the MCU2+ entertainment stuff). I would bet it will get rebranded for HW3+ cars (Smart Summon 2?) to have new features and take advantage of the newer hardware.

 

[EVNow]

I am assuming something like smart summon which is designed to work on all enhanced autopilot cars (e.g. HW2 and HW2.5) will never become something so feature-rich that you *need* HW3+ to run it (after all it'd be hard to advertise Smart Summon with a bunch of features that don't actually work on older cars, though it's not unprecedented e.g. sentry mode or all the MCU2+ entertainment stuff). I would bet it will get rebranded for HW3+ cars (Smart Summon 2?) to have new features and take advantage of the newer hardware.

If smart summon diverges and becomes better for HW3 - they won't talk about it specifically. Would be just part of the City NOA.

BTW, once they start targeted HW3 development (or more accurately production release, since development is going on already) - they can port to HW2/2.5 by reducing the number of layers. It may not perform as well - but all the procedural code could work. When HW3 starts having new tasks that are not ported to HW2 - the features will be completely missing in old cars.

 

[thewishmaster]

If smart summon diverges and becomes better for HW3 - they won't talk about it specifically. Would be just part of the City NOA.

BTW, once they start targeted HW3 development (or more accurately production release, since development is going on already) - they can port to HW2/2.5 by reducing the number of layers. It may not perform as well - but all the procedural code could work. When HW3 starts having new tasks that are not ported to HW2 - the features will be completely missing in old cars.

Yeah, probably. Basically it's too soon to worry about things mentioned in the post I responded to. Either way, where's my HW3 upgrade...

 

[bdub85]

Have you never heard of a code fork? Parallel development?

If you have 10x the computing power you can't dev the new code on the old hardware. You start building that software using more capable hardware.

Yes the same code on old and new hardware won't change things. But once you have the new hardware you are eligible to run the new code base when it is released.

If they are bothering to do field upgrades you can bet they have code for the new hardware well along in the dev process.

They 100% have to have a development branch for dedicated HW3 only. I'm sure they've been working on it for quite some time if they are pushing out the actual hardware upgrade to customers like you mentioned.

 

[Saghost]

They 100% have to have a development branch for dedicated HW3 only. I'm sure they've been working on it for quite some time if they are pushing out the actual hardware upgrade to customers like you mentioned.

V10 came to HW3 first, and although there are only a couple datapoints, it seems like the HW3 cars may have gotten a much larger download than others.

That might mean they are already working to different NNs, or maybe they’re driving with the same ones but using new full size ones on the other processor for validation.

 

[bdub85]

V10 came to HW3 first, and although there are only a couple datapoints, it seems like the HW3 cars may have gotten a much larger download than others.

That might mean they are already working to different NNs, or maybe they’re driving with the same ones but using new full size ones on the other processor for validation.

Or maybe the HW3 code is baked into the version of v10 which goes out to the fleet and only HW3 enabled cars boot up the HW3 software? It will be hard to know without being able to look under the covers.

From a practical ease of deployment/maintainability standpoint, the easiest solution will be only push out the HW3 software once every car is updated to HW3. That is not the most ideal solution obviously, as they will want to get the code out as soon as it's ready to ship. My guess is they will have to support at least 2 versions until HW3 is fully deployed to the fleet.

 

[Pale_Rider]

Yeah, was hoping @jimmy_d or @verygreen or others with the necessary insight could give us an idea on what NN HW3 is using now...

 

[Cirrus MS100D]

@verygreen can obviously speak for himself, but he had stated a few times to be of the mindset that there is no second/hidden/mysterious/magical FSD/HW3 NN code branch lurking anywhere and what we see is as good as what Tesla currently has. I had wondered about the "monster" NN that @jimmy_d described early this year (full 360, camera "agnostic", etc.) and, again, it appears to have been an idea that Tesla toyed around with, possibly with HW3 in mind, but there appears to be no sign of this in the wild. Perhaps they are codeveloping it in a new branch perhaps they abandoned it.

 

[chillaban]

@verygreen can obviously speak for himself, but he had stated a few times to be of the mindset that there is no second/hidden/mysterious/magical FSD/HW3 NN code branch lurking anywhere and what we see is as good as what Tesla currently has. I had wondered about the "monster" NN that @jimmy_d described early this year (full 360, camera "agnostic", etc.) and, again, it appears to have been an idea that Tesla toyed around with, possibly with HW3 in mind, but there appears to be no sign of this in the wild. Perhaps they are codeveloping it in a new branch perhaps they abandoned it.

I dunno, it depends on what one means by the magical hidden NN. For example, the v10 NN is definitely improved and apart from early access, that basically appeared out of nowhere and all of a sudden our cars classify pickup trucks, stop and start more responsively in traffic, and cars don’t dance around much anymore.

I totally agree that it’s very unlikely that Tesla has some secret NN that lets you nap on the way to work, but at the same time it’s very unlikely that they don’t have a branch for what they are doing a few months vs a year into the future.

 

[emmz0r]

verygreen etc can only see what's deployed . So The "dumbed down" NN for HW3 is what he will see. That will say nothing about what the actual real database is, and what NNs can be generated from it.

 

[Fiver]

I dunno, it depends on what one means by the magical hidden NN. For example, the v10 NN is definitely improved and apart from early access, that basically appeared out of nowhere and all of a sudden our cars classify pickup trucks, stop and start more responsively in traffic, and cars don’t dance around much anymore.

For all we know the ability to distinguish between vehicle types on the main display was always there in data, but the MCU wasn't coded to properly process them and then display them as separate vehicle types on screen. Over the last year it started with generic cars, they added motorcycles, and now we get more granularity. The same goers for the dancing cars. That was an artifact of the raw data stream being represented that way visually. They've now programmed the MCU to present that same information in a more consistent, less jittery way. Didn't change the data, just changed how it's shown to the end user. Tesla still gets the raw data stream for their work going on behind the scenes.

What we see on the MCU is just an interpretation of the data fed from the AP computer. Two separate computers, two purposes, running two different sets of code. (MCU and AP)

 

[Saghost]

The same goers for the dancing cars. That was an artifact of the raw data stream being represented that way visually. They've now programmed the MCU to present that same information in a more consistent, less jittery way. Didn't change the data, just changed how it's shown to the end user.

This seems like an unwarranted assumption. I think it's much more likely that they've made improvements to the NN recognition of car orientation and therefore the cars stopped dancing in the data and the display.

 

[ChrML]

For all we know the ability to distinguish between vehicle types on the main display was always there in data, but the MCU wasn't coded to properly process them and then display them as separate vehicle types on screen. Over the last year it started with generic cars, they added motorcycles, and now we get more granularity. The same goers for the dancing cars. That was an artifact of the raw data stream being represented that way visually. They've now programmed the MCU to present that same information in a more consistent, less jittery way. Didn't change the data, just changed how it's shown to the end user. Tesla still gets the raw data stream for their work going on behind the scenes.

What we see on the MCU is just an interpretation of the data fed from the AP computer. Two separate computers, two purposes, running two different sets of code. (MCU and AP)

Still, if the AP computer bases its driving decisions thinking that the environment is full of phantom dancing cars, then it's not very reassuring. Could even be the root cause of all the phantom braking if the inaccuracies are in the vision system.

 

[Fiver]

Still, if the AP computer bases its driving decisions thinking that the environment is full of phantom dancing cars, then it's not very reassuring. Could even be the root cause of all the phantom braking if the inaccuracies are in the vision system.

Yes, perhaps the data stream going from the AP to MCU is different in V10, but regardless, the MCU doesn't do any hard lifting like the AP computer does. All the MCU does is take whatever it's fed and present that visually to us. So they either finally told the MCU how to properly interpret the data coming in and stopped the dancing, or changed the way the AP computer feeds the MCU, or a little of both.

Notice the jittery cars only happens at very low speeds or complete stop? Once forward motion happens they stop dancing pretty damn quick. Up until enhanced summon started being a thing perhaps they were well aware of this "issue" but figured it didn't matter to fix the problem, if you'd even call it that, since they were working on AP at speed. Now they have shifted more work towards FSD, and FSD will essentially be a faster enhanced summon. Currently enhanced summon is basically low speed/often stopping full self driving.
If the "jittery data" was really a problem, training the NN using enhanced summons seems like a good way to teach the NN what's actually going on around it. I personally believe the dancing cars was less about bad data, and more about bad presentation to the end user of data. Regardless, it should have been worked on (at least visually) sooner, as it did present an interesting show on screen at stoplights.

 

[rjdoc74]

My 4/18 made MX P100D was upgraded to HW3 yesterday. Drove all day today. I see no difference in regular driving or AP behavior. Other than a new annoying “beep” proximity sound which I don’t seem to be able to turn off even in Joe mode.

 

[chillaban]

For all we know the ability to distinguish between vehicle types on the main display was always there in data, but the MCU wasn't coded to properly process them and then display them as separate vehicle types on screen. Over the last year it started with generic cars, they added motorcycles, and now we get more granularity. The same goers for the dancing cars. That was an artifact of the raw data stream being represented that way visually. They've now programmed the MCU to present that same information in a more consistent, less jittery way. Didn't change the data, just changed how it's shown to the end user. Tesla still gets the raw data stream for their work going on behind the scenes.

What we see on the MCU is just an interpretation of the data fed from the AP computer. Two separate computers, two purposes, running two different sets of code. (MCU and AP)

I am not arguing whether the v10 improvements are NN, control algorithm, or MCU interface based, more that they do have the notion of dropping in changes out of the blue that represents a longer period of development. The counter argument to the secret branch theory tends to be “because Tesla frequently leaks development features in public firmware releases, they are likely just giving us monthly drops of their latest and greatest and turning features on and off”

All I’m saying is that I think the truth is a mix of both. We aren’t going to wake up one day and find out FSD on HW3 is a solved problem. But that doesn’t mean the weekly firmware updates we see are what Tesla is 100% working on. They do seem to have branched efforts that are ongoing and result in major visible progress drops once or twice a year.

 

[bdub85]

I am not arguing whether the v10 improvements are NN, control algorithm, or MCU interface based, more that they do have the notion of dropping in changes out of the blue that represents a longer period of development. The counter argument to the secret branch theory tends to be “because Tesla frequently leaks development features in public firmware releases, they are likely just giving us monthly drops of their latest and greatest and turning features on and off”

All I’m saying is that I think the truth is a mix of both. We aren’t going to wake up one day and find out FSD on HW3 is a solved problem. But that doesn’t mean the weekly firmware updates we see are what Tesla is 100% working on. They do seem to have branched efforts that are ongoing and result in major visible progress drops once or twice a year.

It’s a byproduct of agile development IMO. They are completing and constantly deploying small finished pieces of the larger fsd puzzle.

 

[Fiver]

It’s a byproduct of agile development IMO. They are completing and constantly deploying small finished pieces of the larger fsd puzzle.

All I’m saying is that I think the truth is a mix of both. We aren’t going to wake up one day and find out FSD on HW3 is a solved problem. But that doesn’t mean the weekly firmware updates we see are what Tesla is 100% working on. They do seem to have branched efforts that are ongoing and result in major visible progress drops once or twice a year.

Completely agree. I think most of the improvements we see come in bits and pieces via the point releases we see about every month. It's a given that there probably are several branches being worked on behind the scenes. Coordinating those branches and synchronizing what works in those, prior to wide release is the challenge.

 

[diplomat33]

Update from Musk on HW3 upgrades: