Forking the codebase (and expectations)

[Teefal]

The new split between EAP and FSD has generated much conversation surrounding the relative merits of individual features, but I suspect there's something bigger going on here. First read about FSD ...

"HW3 includes a custom Tesla-designed system on a chip. Tesla claimed that the new system would process 2,300 frames per second (fps), which is a 21x improvement in image processing compared to HW2.5, which is capable of 110 fps. The firm described it as a "neural network accelerator". Each chip is capable of 36 trillion operations per second, and there are two chips for redundancy. Tesla claims HW3 has 2.5× improved performance over HW2.5 with 1.25× higher power and 0.8× lower cost. HW3 features twelve ARM Cortex-A72 CPUs operating at 2.6 GHz, two Neural Network Accelerators operating at 2 GHz and a Mali GPU operating at 1 GHz."

Now they said only FSD features need HW3. But they just changed the definition of FSD, moving more into the EAP column. So people are saying "I don't need street driving" but they are missing the underlying truth here. Those specs are NOT just for street driving.

People with HW3 will have a better experience than those with HW2.5, guaranteed. The only reason it's not obvious now is that they aren't fully using HW3 yet.

I'm a software dev, and if I'm Tesla I'm thinking:

1. Many people won't care about optimum AP
2. We can't upgrade all these cars anyway, we don't have the resources
3. We HAVE to fork the codebase and code to the new specs

So what do they do? They make an EAP bucket and an FSD bucket. Maximize revenue and get people to pay for the version that does as much as HW2.5 can deliver. Get the people who won't know the difference between EAP Autosteer and FSD Autosteer happy where they're at.

Then focus all new dev on HW3. Put maintenance devs on EAP ... the B team, the port team. Because no top flight dev wants to work with the old hardware, especially with an increase like that. They've probably been phasing it in (or crippling it) to ramp up expectations.

So a year from now? Callouts in the manual saying FSD cars will see this and EAP will see that. Or even a general warning that EAP experience will be different than FSD.

They are two entirely different computing/gpu platforms.

EAP will start to suck more.

 

[jjrandorin]

Interesting post, but they didnt "change the definition of FSD, moving some features to EAP". Those features were always in EAP. The feature set of EAP is what it was previously (source, my car I purchased with EAP in 2018, and multiple other people who bought EAP back then).

The fork in features is what is shown now as the features of the two. I doubt EAP as it stands will get anything more than maintenance... there wont be any more features for it, most likely.

 

[Twiglett]

There are already more than two versions of AP hardware out there currently.
Don't forget Tesla have already been through this a few times and the initial release of AP hasn't been getting worse over time or losing features as HW3 has been developed.
EAP and AP aren't going to be getting worse because of it.
Additionally, this upcoming rewrite is the second rewrite of AP - with the first rewrite also not causing feature loss in earlier versions.
The main difference now is that new features won't get added.

 

[Teefal]

Elon said it himself ... 4D, labeling/training nets on moving video instead of snapshots. One of my projects last year was annotating live professional sports video using ML. HW3 is a completely different beast than HW2.5. If they seem the same now it's because they're doing a lowest common denominator approach.

It's another pair of eyes. Photos vs films.

 

[CYBER_Y]

It’s very simple each feature is a part of a package that is it’s own chunk of code that is included at will. So that means they can include or exclude any features at any time. No need for a code fork. It’s as simple as setting a flag in the database.

 

[Knightshade]

There are already more than two versions of AP hardware out there currently.

Yeah there's 3.

HW2, HW2.5, and HW3 (though HW2 is only on older S/X cars)

And I mean HW1 exists but has been functionally dead since like 2016 as Tesla lacks access to core code owned by mobileye.

Don't forget Tesla have already been through this a few times and the initial release of AP hasn't been getting worse over time or losing features as HW3 has been developed.
EAP and AP aren't going to be getting worse because of it.
Additionally, this upcoming rewrite is the second rewrite of AP - with the first rewrite also not causing feature loss in earlier versions.
The main difference now is that new features won't get added.

Well, the main difference is it's going to be incapable of running on anything older than HW3.

All versions prior to the coming-soon re-write process each camera individually as its own thing... so the pre-HW-3 inability to handle all cameras at full frame rate could be worked around.

The new re-write requires the ability to handle all cameras at full frame rate at the same time. Something HW2 and HW2.5 is not able to do.

So HW2.x will have to remain on the old codebase because it fundamentally can not do the basic visualization method the new codebase relies on.

It’s very simple each feature is a part of a package that is it’s own chunk of code that is included at will. So that means they can include or exclude any features at any time. No need for a code fork. It’s as simple as setting a flag in the database.

Nope- see above.

This isn't just "turn off the red light feature"

This is the fundamental core aspects of all the AP/EAP/FSD related features can not run on older HW

 

[S4WRXTTCS]

There are already more than two versions of AP hardware out there currently.
Don't forget Tesla have already been through this a few times and the initial release of AP hasn't been getting worse over time or losing features as HW3 has been developed.
EAP and AP aren't going to be getting worse because of it.
Additionally, this upcoming rewrite is the second rewrite of AP - with the first rewrite also not causing feature loss in earlier versions.
The main difference now is that new features won't get added.

That's tough to say.

For example take my experience with the HW2.5 to HW3 upgrade.

Before the surgery I observed bugs in the NoA behavior that other people reported. In fact these were show stopping bug where people reviewing it didn't even bother giving it a grade because it was just broken. Things like not getting out the passing lane for example. Bugs that weren't in previous versions.

After the surgery I noticed that some of the bugs were gone.

It likely was that way because they were running two separate pieces of code, and resource constraints meant that Tesla didn't have a chance to go back to the old one to fix it. Maybe they did since then or maybe its been growing dust. I simply don't know.

There was also other bugs that were remained between the two. Things inherent in the way the features were implemented like re-centering when the lane widens during a merge point in the right lane. Or auto-lane change aborting because it thinks the semi-truck that is in the right lane is in the middle lane for a brief second, but its not.

How does Tesla fix the remaining bugs on HW2/HW2.5 vehicles with EAP?

With the new 4D labeling it's likely going to fix the auto-lane changes to be way more consistent, but how do you fix it on HW2/HW2.5 vehicles that aren't compatible with the new network?

If I recently paid $4K for EAP I'd be pretty livid if the auto-lane change worked way better on a HW3 vehicle where the owner also only paid $4K for EAP.

So Tesla could choose to make every EAP vehicle have the old net, but then things aren't as safe.

 

[Mardak]

At least early last year on an ARK Invest podcast On the Road to Full Autonomy With Elon Musk - ARK Invest

Tasha Keeney: Is the amount of compute processing that you need for full autonomy fixed or do you have to upgrade this every two years?

Elon Musk: I do think we can achieve full autonomy with the NVIDIA hardware, but it's a much harder software problem. You really have to budget your compute and do all sorts of tricks to manage how you use your compute. With hardware that's 2000% better, you don't have to do that constant budgeting, so the software problem is much easier. I think with the current hardware and a lot of effort, we could get to full self-driving maybe 50% to 100% safer than a person, but with hardware 3, I think it's probably like 1000% safer than a person.


So theoretically the codebase doesn't need to be entirely forked and instead have some budgeting adjustments if safety can still be achieved even with heuristics like sampling. E.g., if the rewrite requires data from all 8 cameras at full resolution, is reducing to 10fps with 100ms gaps between output safe enough (in addition to neural network processing time)? Then again, if the software actually requires video without dropped frames, sampling wouldn't be useful.

Also, this is the same podcast where Elon Musk was certain "without a question mark" that by end of 2019, they would have "feature complete" full self-driving that is capable of picking up from parking lot and driving to destination without an intervention (still requiring supervision). So the above guess might be totally wrong now 1.5 years later.

Practically, I agree that engineering resources will focus on improving Autopilot with FSD computer. So similar to how Autopilot is relatively unchanged for pre-HW2 vehicles, pre-HW3 vehicles will probably not get too much more. I will note Teefal mentioned a split between EAP and FSD software, but I believe the split will be along the hardware, as I think the rewrite will require HW3 and will improve basic Autopilot and EAP functionality including highway driving, lane changes, detecting merges, etc.

 

[Knightshade]

Musk himself has been pretty clear that you should not believe him regarding how hard something is if he hasn't actually already done it.

He's been consistently, badly, wrong on how hard FSD is, and what's actually needed to do it for years and years now.

So I'd take his suggestion they could ever have done FSD on pre-HW3 compute with a mars sized grain of salt.

 

[Mardak]

If I recently paid $4K for EAP I'd be pretty livid if the auto-lane change worked way better on a HW3 vehicle where the owner also only paid $4K for EAP.

Most people with the EAP upgrade option already have HW3. Roughly 400k vehicles were sold with HW2/.5, and a portion of those have been retrofitted with HW3. Since 2019Q2, Tesla has been selling roughly 100k vehicles with HW3 per quarter, so including 2020Q3, there should be more than 600k vehicles with HW3 today.

I agree that it would be unfortunate for someone with HW2/.5 paying the same dollar amount for EAP and getting less capable software than someone with HW3. But again to engineering resourcing constraints, practically even if one includes pre-HW2 vehicles, Tesla's overall fleet is majority HW3.

 

[S4WRXTTCS]

Most people with the EAP upgrade option already have HW3. Roughly 400k vehicles were sold with HW2/.5, and a portion of those have been retrofitted with HW3. Since 2019Q2, Tesla has been selling roughly 100k vehicles with HW3 per quarter, so including 2020Q3, there should be more than 600k vehicles with HW3 today.

I agree that it would be unfortunate for someone with HW2/.5 paying the same dollar amount for EAP and getting less capable software than someone with HW3. But again to engineering resourcing constraints, practically even if one includes pre-HW2 vehicles, Tesla's overall fleet is majority HW3.

Keep in mind there are multiple ways that a person could end up with EAP on a HW2+ vehicle.

The primary path was EAP was an option from early 2016 (roughly) all the way till around 2019 before they pulled the EAP option. All of these cars had HW2 or HW2.5. If a customer purchased a car then without FSD they be on HW2+.

The second path is having a HW2.5 vehicle and upgrading to EAP this month. Now sure there are less of these than HW3 vehicles, but keep in mind that Tesla was continuing to put HW2.5 into vehicles well after HW3 cars were shipping. In fact they pulled a rather crazy stunt where if someone didn't order FSD they would end up with HW2.5 instead of HW3. I'm not sure even when this stopped. During the initial period with HW3 they had supply issues.

 

[helvio]

I found this link and think it's an interesting read:
Full Self-Driving Computer Installations

 

[Mardak]

It'll be interesting to see how much basic Autopilot improves with the rewrite to make use of HW3/FSD computer. Basic Autopilot stays "within its lane" while FSD software would support stopping and turning for intersections.

Driving on roads without lane lines and avoiding parked vehicles definitely would need to be handled for FSD, but those things seem basic enough to support generally. Notably, it would seem to be more work to run the current/old behavior on non-FSD HW3 vehicles than to have all HW3 vehicles running the new code.

 

[Mardak]

I suppose this kinda answers the question:
https://twitter.com/elonmusk/status/1315448376163524608

Assuming "old architecture" refers to the software that can run on HW2+ while "FSD build" requires HW3+. I guess we'll find out soon how the new architecture is additionally split on software purchases Basic Autopilot vs Enhanced Autopilot vs FSD software.

 

[Knightshade]

It wasn't ever really a question to folks who understood software development

It's the reason I got FSD in the first place- I knew anybody without HW3 would be on a dead codebase long term.

 

[Teefal]

Yep. Good to see some acknowledgement from Tesla.