Rumor: HW4 can support up to 13 cameras

[Terminator857]

4x pixels and 60% more cameras to process. What was the estimate for HW4 compute over HW3 again? Better be, to use one of Elon's favorite phrases, an order of magnitude more powerful considering HW3 is supposedly close to maxed out...

Mobileye SuperVision™ | The Bridge from ADAS to Consumer AVs

Mobileye SuperVision™ is our most advanced driver-assist system, providing eyes-on/hands-off automated operation of standard driving functions.

www.mobileye.com

If 4 cameras are for parking like mobileye's implementation then that means 13 - 4 = 9 cameras or one additional camera for driving? Or is the indoor cabin camera counted in the 13 count? Then no additional FSD cameras for HW4, just 4 additional parking cameras.

I'll start a new rumor: 2023 vehicles will come with parking cameras or 360 park view.

 

[EVNow]

4x pixels and 60% more cameras to process. What was the estimate for HW4 compute over HW3 again? Better be, to use one of Elon's favorite phrases, an order of magnitude more powerful considering HW3 is supposedly close to maxed out...

Lots of things to consider.
- Only need to merge 13 cameras to get the 360 view. After that its the same NN.
- Currently HW3 runs 2 separate stacks. With a single stack compute needed is significantly less.
- The NN & procedural code is still very raw & immature. Lots of scope for optimization.

 

[thewishmaster]

Lots of things to consider.
- Only need to merge 13 cameras to get the 360 view. After that its the same NN.
- Currently HW3 runs 2 separate stacks. With a single stack compute needed is significantly less.
- The NN & procedural code is still very raw & immature. Lots of scope for optimization.

True re: merging 13 cameras. But there's still some image transformation happening before the data is fed into the NNs. I'm sure it's far from the largest task the compute has to perform, but there will still be impact from that.

Multiple stacks - well, yes, theoretically true but remember they are not running at the same time. Merging will only save them from having to run vastly different NNs and logic in various modes of operation. They'll still be switching things on/off depending on whether the car is on city roads, on the highway, etc. - e.g. you don't need speed bump detection when you're traveling at high speeds, but the hardware will still be maxed out as far as they can take it I'm sure.

NN / procedural code immature - yeah, I suppose we can say that since we don't have first-hand knowledge of its current state. But barring some breakthroughs it's unlikely they'll be able to optimize the NNs a ton, as they're still adding functionality and it's clear the accuracy isn't quite there yet for safe operation. Reducing NN complexity would only reduce accuracy which can't be the goal until some level of accuracy is achieved.

 

[Phlier]

Yep this was addressed at AI day. Elon expects HW3 will provide FSD 200-300% safer than a human. HW4 may be 1000% safer.
I doubt they'll ever upgrade any HW3 cars even if they don't achieve those safety targets but who knows.

I interpreted what he was saying exactly as you did; if HW 3 can fulfill the advertised functionality suite of FSD, then there wouldn't be any upgrading to HW 4. HW 3 might not perform as well as 4, but as long as it meets the minimum requirements, that's what we'll be stuck with.

But at this point, I don't think anyone really knows is HW 3 is going to meet that minimum bar yet.

Ah well, if the CyberTruck really does end up shipping with HW 4, at least one of my cars will have it.

 

[mongo]

They would need to integrate the new camera views into their 3D bird's eye view. So it would probably require another "rewrite" of their software.

Yeah, the first level image stitcher would need updated. This implies more overlaping fields of view such that the cameras can see behind smaller occlusions.
Seems like the training data on object recognition would carry over, possibly with up sampling. Driving logic might not be impacted.

 

[SageBrush]

Why would the supply chain know about # of cameras per car ?

Perhaps this is inference based on presumed Tesla car production in the future

 

[mongo]

Why would the supply chain know about # of cameras per car ?

Perhaps this is inference based on presumed Tesla car production in the future

Different part numbers for different positions.
For quoting, they usually have quantity per vehicle. If they supply other parts, they can back calculate it.

 

[EVNow]

True re: merging 13 cameras. But there's still some image transformation happening before the data is fed into the NNs. I'm sure it's far from the largest task the compute has to perform, but there will still be impact from that.

Yes - all the transformation needed for stitching will be impacted.

Multiple stacks - well, yes, theoretically true but remember they are not running at the same time. Merging will only save them from having to run vastly different NNs and logic in various modes of operation. They'll still be switching things on/off depending on whether the car is on city roads, on the highway, etc. - e.g. you don't need speed bump detection when you're traveling at high speeds, but the hardware will still be maxed out as far as they can take it I'm sure.

True - what they save significantly is on firmware and dev cost. May be not all that much on compute.

NN / procedural code immature - yeah, I suppose we can say that since we don't have first-hand knowledge of its current state. But barring some breakthroughs it's unlikely they'll be able to optimize the NNs a ton, as they're still adding functionality and it's clear the accuracy isn't quite there yet for safe operation. Reducing NN complexity would only reduce accuracy which can't be the goal until some level of accuracy is achieved.

We "know" the software is immature - and thus will have a lot of inefficiency. First they have to get it "working" properly - then they will optimize.

w.r.t. NN - I don't know whether better training can result in more optimized NN.

 

[Knightshade]

Things we know for a fact right now:


HW4 is coming (Was mentioned as far back as autonomy day when HW3 was announced- and AI day reinforced this saying likely coming next year on cybertruck)

Next gen cameras are coming (mentioned same time as the CT HW4 thing).

Tesla has said in the past anyone who bought (not rented) FSD gets any needed HW upgrade for free (and they have kept this promise one at least 2 different occasions now, with the 2->2.5 camera upgrade, and the 2/2.5 computer->HW3 upgrade).

HW3, currently, is heavily needing to borrow compute from node B, meaning redundancy is not possible on HW3 with current code.



Things we do NOT know:

The number of type and placement of next gen cameras (though forward/low and side/front facing to handle the creep situations seems the single most obvious need).

If HW4 is "enough" compute for L4/L5 either. Tesla almost certainly doesn't know this yet either.

If they can somehow magically BOTH add vastly more capability to the code AND vastly shrink the compute it needs to fit back in a single HW3 node.... this seems SUPER unlikely to me, meaning HW4 will likely be required for anything about L2 driving.

How easily a retrofit on existing fleet would be... (the computer is probably easy as HW3 was... swapping existing cams MIGHT be easy... adding new ones is likely to be significantly harder.

If the post 3/19 FSD buyers will get free retrofits (which has been exhaustively debated 783,000 times roughly already- but I suspect they will if the retrofit lift isn't ridiculous)

 

[Terminator857]

I feel fairly confident that the thirteen cameras are the regular 8 FSD cameras + 4 parking assist cameras + internal cabin camera. The only FSD camera change is 2x resolution. Which is cool, which means there is an easy upgrade from FSD HW3.

 

[stopcrazypp]

4x pixels and 60% more cameras to process. What was the estimate for HW4 compute over HW3 again? Better be, to use one of Elon's favorite phrases, an order of magnitude more powerful considering HW3 is supposedly close to maxed out...

They can use 2x2 binning (which in many sensors have built in) and cropping for extra resolution to reduce processing requirements, although obviously it's better if the processing hardware is improved to handle the higher res.

 

[thewishmaster]

They can use 2x2 binning (which in many sensors have built in) and cropping for extra resolution to reduce processing requirements, although obviously it's better if the processing hardware is improved to handle the higher res.

Yeah I’d bet they would do binning for a while - if they switch to higher res cameras they’d need to retrain everything with new higher-res data before they can take advantage of the extra resolution

 

[stopcrazypp]

I feel fairly confident that the thirteen cameras are the regular 8 FSD cameras + 4 parking assist cameras + internal cabin camera. The only FSD camera change is 2x resolution. Which is cool, which means there is an easy upgrade from FSD HW3.

That seems to be the exact same layout as the XPeng P5 (which has 13 cameras).
小鹏P5申报图曝光 配置无疑是最大亮点 - 全网搜

Seems like a waste for 4 whole cameras to go to parking alone (would hope they would do some coverage in regular driving especially for the areas that other discussions point out may need better coverage), but it does seem like a layout many in the industry is adopting.

 

[EVNow]

That seems to be the exact same layout as the XPeng P5 (which has 13 cameras).
小鹏P5申报图曝光 配置无疑是最大亮点 - 全网搜

Seems like a waste for 4 whole cameras to go to parking alone (would hope they would do some coverage in regular driving especially for the areas that other discussions point out may need better coverage), but it does seem like a layout many in the industry is adopting.

For all we know - the rumor started because someone heard an "EV" company had ordered 13 cameras per car.

 

[mongo]

Yeah I’d bet they would do binning for a while - if they switch to higher res cameras they’d need to retrain everything with new higher-res data before they can take advantage of the extra resolution

Would they? Doesn't the system already clip and scale regions of interest to identify objects? If so, high res cameras just allow seeing details futher away.

 

[stopcrazypp]

Would they? Doesn't the system already clip and scale regions of interest to identify objects? If so, high res cameras just allow seeing details futher away.

That definitely works for objects (and stuff like sign recognition), but for things like drivable path labeling (which would basically involve the entire scene), I imagine binning would be used extensively if the goal is to reduce processor demand. This lets you offload a lot downsampling tasks to the sensor chip itself (or if it doesn't support it, many image processor chips support such a thing).

 

[stopcrazypp]

For all we know - the rumor started because someone heard an "EV" company had ordered 13 cameras per car.

Might as well post it here also. Per other discussion, the 8+4 layout being discussed (ignoring 1 camera for cabin camera) is exactly what Nvidia's Drive Hyperion is using. From the unveiling video, a ton of Chinese manufacturers are using Nvidia's solution. That definitely increases the chance of a supplier mix-up on this.
Autonomous Car Progress
NVIDIA DRIVE Hyperion Developer Kit

 

[Terminator857]

I feel fairly confident that the thirteen cameras are the regular 8 FSD cameras + 4 parking assist cameras + internal cabin camera. The only FSD camera change is 2x resolution. Which is cool, which means there is an easy upgrade from FSD HW3.

I believe cybertruck has already been spotted with additional parking cameras.

Tesla Cybertruck is equipped with a camera under front bumper

The latest outing of the Tesla Cybertruck revealed that the electric pickup truck is equipped with a camera under the...

electrek.co

 

[Knightshade]

Yeah I’d bet they would do binning for a while - if they switch to higher res cameras they’d need to retrain everything with new higher-res data before they can take advantage of the extra resolution

If only they were building the worlds most advanced NN training supercomputer or something....

 

[thewishmaster]

If only they were building the worlds most advanced NN training supercomputer or something....

Mostly thinking of the data itself, not the training - outside of simulation it will take a bit of time to collect a similar variety of data they have collected already using cams with different specs