Let's talk Autopilot on HW 2.5 vs HW 3.0

[CarlK]

I guess the logic here is with the new computer, the calculation is faster and thus, it will reduce the chance of collision.

But I agree with you that if software makes a decision to go ahead and crash into a Semi Trailer then it doesn't matter whether that decision is slow or fast, the collision will occur until the software will start to issue the correct instructions by braking instead of speeding.

Instruction part is straightforward the key is the neural net needs to recognize an object as either 100% a hazard or 100% not a hazard. When the software failed to do that it most likely was because neural net had not been trained (enough) for that particular situation. The new neural net (Karpathy said was developed using Software 2.0) is likely not the same and most importantly it will get a lot more training utilizing the faster HW3.

From one of the comments made from people who took the 4/23 test ride it looks the entire HW/SW system had been put in the field and learned how to do those very impressive thing for only ~3 months.

The assumption I'm making is the capability of the SW is being limited by the capability of the HW.

So with more capable HW means that the SW can be significantly improved.

One possible example of this is the visualization that's shown on the FSD demo video from autonomy day. I'd love to have that Visualization (that I believe was possible due to the Neural Network running on HW3) on my car.

Definitely. Both software itself and the deep learning capability due to faster processor had be significantly improved. I don't think what old AP used to do has any relevance in the new FSD although it's hard for people not to make the connection.

 

[turnem]

I still don't believe that they won't need further changes to the non-CPU hardware to achieve their goals. I feel like they will ultimately add more (or different types of) sensors before fully realizing the goal of fully autonomous driving.

I think they are fairly close to human supervised self driving in the city and that they'll accomplish this with current hardware (including HW3 and existing sensors and cameras). But full level 5 is way off and will likely involve new technology somewhere along the way.

 

[S4WRXTTCS]

I guess the logic here is with the new computer, the calculation is faster and thus, it will reduce the chance of collision.

But I agree with you that if software makes a decision to go ahead and crash into a Semi Trailer then it doesn't matter whether that decision is slow or fast, the collision will occur until the software will start to issue the correct instructions by braking instead of speeding.

It's really about speed, and accuracy of detections.

The new hardware allows the speed of the detection events to significantly improve, and therefor lowering latency. It also allows the accuracy of the Neural Network detections to significantly improve by allowing much larger and sophisticated neural networks.

There is so much more power that they can implement processing intensive algorithms for stereo processing. This is important when it comes to detecting large objects that the Neural network hasn't be explicitly trained on.

Now I don't know how aggressive Tesla will be in taking advantage of the new hardware for even basic things like AP or safety related things. But, the capability is there to make significant improvements.

 

[Tam]

...Neural Network detections...

Will the high speed of future 5G play any roles in Tesla Autopilot / FSD / Neural Network detection?

 

[CarlK]

Wow this is one of the most vicious Elon’s burns ever:

“Wife certain that husband goes to grave before FSD. Requests visit to grave site in the far future when FSD is a reality”.

Go everywhere, sleep in our cars FSD I don’t expect to be a reality until every car on the road can communicate with one another. That will take decades.

Getting from A to B without touching your wheel in geofenced, well mapped routes - possible in a decade.

HW3 computing over HW2 computing is substantial.

Inside of 5 years, I can see HW4 being 10 times more powerful than HW3. In ten years HW5 could be 10 times more powerful than HW4.

Much like with chess, driving can be solved through computational brute force.

I hope you make it 10 years and your wife is wrong for once.

I once thought it was hard but I stared to understand how it's done after hearing Karpathy's presentation. You don't need car to car communication. The vision NN can learn to sense slight movement of pedestrians and other cars and use that to predict their intentions. That's exactly how human drivers do under those situations. That seems to be able to solve the biggest problems such as four way stop or pedestrian crossing, what used to be thought of as show stoppers. With the ability to acquire a lot of machine learning data I do feel Tesla could get FSD out sooner than most have thought possible.

Will the high speed of future 5G play any part in Tesla Autopilot / Neural Network detection?

Probably not with Tesla's approach. The neural net resides in each car. Most decisions are made within the car.

 

[animorph]

If Tesla decides to use an AP2.x compatible NN for AP on HW3 then there might not be much difference in performance between hardware platforms. There could still be some processing speed improvements if Tesla allows the AP NN to run full speed in HW3. That might buy a little response time improvement. I think this is what we're seeing now.

It seems reasonable that Tesla could use the FSD software for both AP and FSD in cars that have the FSD option. This would be HW3 only, but would provide a much smarter AP, or even just TACC, in cases where you didn't want to use FSD. In the best case, this would also apply to AP-only cars with HW3. That would mean a much better-performing AP in HW3, with all the smarts people wish AP had now. It would also mean that Tesla would probably drastically reduce development of AP/EAP on old hardware since it would have no commonality with any of the software on HW3. If not soon, I think this will happen in a few years.

On the other hand, FSD software might provide an AP that is too good. Tesla might want to reserve that performance for FSD buyers. In that case the non-FSD software remains independent of FSD, even in HW3, and compatible with all AP2+ hardware. AP2.x would get the latest AP-only developments since there is still commonality with AP on HW3. Given Tesla's focus on FSD, I don't think this scenario is as likely. Using FSD software should allow better "shadow mode" data gathering, even if driving with AP only or manually. It would be awkward to run AP-only software and the FSD NN at the same time, though possible. And Tesla would have to support the AP-only software, in which they will have little interest.

As far as the hardware history, this forum has known that AP2.x was running on hardware that was generally suitable only for EAP. The processing power was not close to what was assumed to be needed for FSD. But until Tesla had their chip in production, a real FSD computer would have been too expensive, especially when including them in all their cars. So Tesla used a cost-effective, and upgradable, EAP computer and planned (my assumption) to replace them later for FSD as needed.

I expect FSD will start out very much like AP2.0, requiring lots of supervision, with lots of people not willing to put up with it. We'll be the trainers, every time we disengage FSD for doing something stupid. Tesla will fill in the corner cases we uncover during the next few years. Initially I think AP-only will remain pretty much as is for now, but in the longer term (one or two years?) it will become a subset of FSD on HW3+ cars.

 

[S4WRXTTCS]

If Tesla decides to use an AP2.x compatible NN for AP on HW3 then there might not be much difference in performance between hardware platforms. .

This can't be dismissed outright because that is a very real possibility as that's what currently exists, but I don't see any evidence from Tesla's track record on safety related items that they'd do this.

A good example of this is the 4 cameras versus 8 cameras. Originally EAP was supposed to only use 4 of the 8 cameras available, and FSD was going to use all 8. But, when Tesla eventually activated cameras for side monitoring they ended up activating all the cameras even for people who didn't have EAP or EAP/FSD.

At every juncture Tesla has opted to utilize the HW there to improve safety even for people who didn't buy the AP package.

 

[insaneoctane]

IMO, HW2.x will become inferior over time. While the AP2.x will continue to perform AP and EAP, probably using the ~ same NN as HW3, it will be doing it with fewer cameras and cropped video. The resultant of this is not zero difference. I suspect, over time, even during just AP, that HW3 will be able identify fringe or dangerous cases more often, better, and/or sooner. I can not quantify this, but it's non-zero

 

[groovidad]

Wow this is one of the most vicious Elon’s burns ever:

“Wife certain that husband goes to grave before FSD. Requests visit to grave site in the far future when FSD is a reality”.

Go everywhere, sleep in our cars FSD I don’t expect to be a reality until every car on the road can communicate with one another. That will take decades.

Getting from A to B without touching your wheel in geofenced, well mapped routes - possible in a decade.

HW3 computing over HW2 computing is substantial.

Inside of 5 years, I can see HW4 being 10 times more powerful than HW3. In ten years HW5 could be 10 times more powerful than HW4.

Much like with chess, driving can be solved through computational brute force.

I hope you make it 10 years and your wife is wrong for once.

“Go everywhere, sleep in our cars FSD I don’t expect to be a reality until every car on the road can communicate with one another. That will take decades.”

EXACTLY!

 

[CarlK]

There is no question FSD and original AP2 software were different. AP2 was already there when Karpathy joint the company. He said he rewrote the neural net with software 2.0. What we don't know is whether AP/EAP had been updated to the new neural net as a subset of the FSD. A good guess is it had. We just don't know when that happened. Perhaps when NoA was released?

 

[ammulder]

I don’t feel like FSD is possible without tracking objects over time — whereas right now I believe it analyzes each frame in isolation. Which probably accounts for some of the dancing, and makes it harder to recognize the broad side of an 18-wheeler, and probably contributes to the phantom braking, among other things.

Anyway, I’m not sure whether 2.5 and 3 sound differ in the ability to do that — you would think only a modest amount of memory would be needed, but I’m not sure what all those AP boards/packages include.

I think it’s another layer on top of the basic NN though. The NN processes a frame, and then something else inputs the frame data and the last known object data and outputs a more refined ‘view of the world.’ Perhaps a second-layer NN.

 

[vjason]

It doesn't matter what =< HW 2.5 can do, vehicles with that hardware who bought EAP but didn't/don't buy FSD are most likely feature complete with regard to driving automation. I say this because it keeps the code base simple as they will focus on 3.0 when it comes to new features along those lines. Based on happenings so far I can't see them allocating resources to back port major new features from HW3 to earlier versions, as there is zero immediate revenue benefit to doing so (yes, it is all about revenue).

In my opinion the writing was on the wall when they killed EAP and offered the brief FSD sale, which is why I ponied up my $2K even knowing that any significant form of FSD is still (probably) a ways off. The way I see it is that I got probably as good a FSD deal as I'm going to get, and if I decide to go another direction (or trade up) it will make my car desirable over other cars who face a $6K FSD upgrade cost. Best case scenario is that Elon was right and FSD arrives early, which I'm fine with and will hope for (for the sake of Tesla).

 

[Giru]

Just got my new May 19 build date M3 last week. Ordered with FSD, came with 2.5. When I found this out at pickup, they said the upgrade will not be this year, and pointed out that there would be a long line ahead of me.

Yes I will get an the upgrade I already paid for, but when?

Elon did say those that order the car with FSD, will get the upgraded computer free. However I did not hear him say those that order after delivery would not also have to pay parts and labor for for the upgrade, on top the FSD FEE.

How you do know if you have hw2.5 or 3.0 in the model 3? I just got mine 2 days and dunno where to check that

 

[willow_hiller]

How you do know if you have hw2.5 or 3.0 in the model 3? I just got mine 2 days and dunno where to check that

This post covers it pretty well Definite Guide: Find out if you have HW3.0 or HW2.5 You just need a set of trim tools to pop off the cover and take a look at the part number.

Also, I've thought of a logical progression that convinced me HW3 will improve base AP.

1. HW3 is required for city NOA. Why else would Tesla be bothering with retrofits?
2. City NOA will involve more nuanced AP behavior, like not suddenly braking for a vehicle turning ahead of you.
3. The above nuanced behavior would greatly improved regular AP functionality.

Therefore, HW3 will improve AP function, even for those of us without FSD. Any fault in my assumptions?

 

[garbia]

I am ordering mine tomorrow, will it come with HW3?

 

[preilly44]

I am ordering mine tomorrow, will it come with HW3?

Yes, it will have AP3 (FSD computer).