Future Autopilot Hardware Rumors and Speculation

[ord3r]

With Elon Musk commenting that HW4 could be introduced with the cybertruck, and rumors that this chip will use TSMC and go into production Q4 2021, seems like a decent time to partake in the time-honored forum tradition of baseless speculation and rumor mongering..

Elon Musk said on the earnings call new hardware is all about increasing the factor of safety, and that FSD remains achievable at greater than human safety levels on old hardware. He said this referring to the HW4 computer, but does anyone think Tesla will also revise the cameras used? Or even add more cameras? Or even other sensor hardware? Curious to hear if anyone has seen any hints of upcoming hardware in the public firmwares, or even just speculation on how the AP system may evolve in the next few years.

 

[Tam]

With Elon Musk commenting that HW4 could be introduced with the cybertruck, and rumors that this chip will use TSMC...

It's Samsung not TSMC

Tesla Cybertruck’s HW4.0 computer will be built by Samsung: report

Recent reports have indicated that Tesla’s Hardware 4 computer, which would make its debut in the Cybertruck, would be produced by Samsung Electronics Co. The world-leading chipmaker is reportedly certain to beat out its rival, the larger Taiwan Semiconductor Manufacturing Co. (TSMC), for the...

www.teslarati.com

 

[J1mbo]

I wonder if the chip will be an evolution of the FSD3 chip, or something more closely related to a dojo chip (single dojo chip is probably too hungry at 400W TDP).

 

[Tam]

...revise the cameras used?...

I doubt it. The last camera revision was the color filtering but not anything else such as more processing power, sensitivity, pixels...

Or even add more cameras?...

Right now the configuration fits well for ADAS that a driver is still responsible for driving so I don't think there's an incentive to add cameras when Tesla has a free human driver to the job.

Or even other sensor hardware?...

Not for a foreseeable future but eventually, when LIDAR becomes a norm in other cars to reliably avoid collisions and if Tesla Pure Vision still won't reliably avoid collisions then maybe Tesla will be pressured to add more sensor hardware.

...how the AP system may evolve in the next few years...

Depends on who you ask. Elon Musk predicted that it would go driverless in 2020 with no human drivers in Robotaxis.

 

[Bugeater]

Saw this at Whole Foods in Los Altos, CA today. The car had Tesla plates, so I asked the occupants if they were from Tesla. They said yes. I asked if the stuff on top was some new FSD hardware. The sort of hesitated. Then they said that I would like it! I asked if they can't really talk about it and they smiled and said yes.
Has anyone else seen anything like this before?

 

[pilotSteve]

LIDAR for baseline perception, plus other possible new production sensors (cameras) being evaluated or benchmarked?

 

[S4WRXTTCS]

They were probably just testing the 54th revision of the repeater camera.

 

[cobra]

That looks like a lidar setup. There was mention of Tesla using one to help test the cameras against, so it might be that. It's somewhere here on the forum. Could you see if the front or back had anything besides that side sensor mounted (same type of sensor?)

 

[Tam]

...revise the cameras...

There's a rumor for a higher resolution of 5.4 instead of 1.2 megapixels.

Tesla's FSD hardware 4.0 to use cameras with LED flicker mitigation

At Tesla's AI Day event last summer, Elon answered some questions during the Q&A portion of the event that revealed Tesla's plans for FSD hardware. Elon inf

www.notateslaapp.com

 

[serendipitous]

My wild-ass prediction is higher res cameras, HW4 to handle the additional pixels, and new cameras integrated into the headlight housings. Retrofits offered to people who bought, but not subscribe, to FSD.

 

[LowlyOilBurner]

Someone on here posted pics of a new design of the tri-focal cameras, or at least the housing they live in?

 

[EVNow]

There's a rumor for a higher resolution of 5.4 instead of 1.2 megapixels.

Tesla's FSD hardware 4.0 to use cameras with LED flicker mitigation

At Tesla's AI Day event last summer, Elon answered some questions during the Q&A portion of the event that revealed Tesla's plans for FSD hardware. Elon inf

www.notateslaapp.com

Makes sense - hires sensors are pretty common now.

ps : But that comparison of field of view is BS. FOV is more to do with optics than sensor resolution.

 

[JulienW]

One thing (or 2 things) I have wondered since the cameras are just cheeper consumer grade are they capable of true 10 bit depth? The difference from 8 bit to 10 bit is a 256x3 to 1026x3. Are they really capable of detecting and resolving that much dynamic range?

Also photon counting vs image processing usually requires more specialized/commercial quality CCDs. Aren't consumer CCD's generally a little slow for this job?

 

[EVNow]

One thing (or 2 things) I have wondered since the cameras are just cheeper consumer grade are they capable of true 10 bit depth? The difference from 8 bit to 10 bit is a 256x3 to 1026x3. Are they really capable of detecting and resolving that much dynamic range?

Also photon counting vs image processing usually requires more specialized/commercial quality CCDs. Aren't consumer CCD's generally a little slow for this job?

These are apparently Sony auto grade - Sony's IMX490 automotive sensor. My guess is they wouldn’t directly lie about tech specs like no name Chinese imports on ebay.

IMX490 - Sony Semiconductor Solutions Corporation | CMOS Image Sensor

www.gophotonics.com

120db of dynamic range - 10 bits @ 40 fps and 12 bits @ 30 fps.

 

[S4WRXTTCS]

One thing (or 2 things) I have wondered since the cameras are just cheeper consumer grade are they capable of true 10 bit depth? The difference from 8 bit to 10 bit is a 256x3 to 1026x3. Are they really capable of detecting and resolving that much dynamic range?

Also photon counting vs image processing usually requires more specialized/commercial quality CCDs. Aren't consumer CCD's generally a little slow for this job?

I don't see them as cheap consumer grade sensors as they are specifically designed for automotive use where you need a lot of dynamic range.

My understanding is HW3 uses the On-Semi AR0136AT, but I can't find the datasheet for that. It's similar to the AR0132AT though.

https://www.mouser.com/datasheet/2/308/AR0132AT-D-888230.pdf

Assuming they are using the HDR mode it actually combines three exposures into one 20bit value that is than compressed to 12bit (page 19).

So answer your question they are capable of detecting and resolving that much dynamic range. I don't know how Tesla is using them though.

As to photon counting I'm a bit confused by the entire thing. Normally with photon counting its for very extremely low light situations where you want something sensitive, but with low noise. These can be so precise that your counting single photons (SPAD detectors).

Now you can certainly say you're doing photon counting with a regular image sensor, but I think its just fancy talk for dealing with raw images.

 

[drtimhill]

I wonder if the chip will be an evolution of the FSD3 chip, or something more closely related to a dojo chip (single dojo chip is probably too hungry at 400W TDP).

Highly unlikely .. Dojo is optimized for totally different workload than the car, quire apart from its power profile.

 

[drtimhill]

Or even add more cameras?

There has been some speculation about adding side-facing cameras approx at the headlights so the car can see left/right at an intersection without creeping so much (or at all), but I've not seen anything to substantiate this.

 

[S4WRXTTCS]

This is what I wish Tesla would use for Frontal SPAD Depth Sensor.

Sony to Release a Stacked SPAD Depth Sensorfor Automotive LiDAR Applications, an Industry First*1Contributing to the Safety and Security of Future Mobility with Enhanced Detection and｜News Releases｜Sony Semiconductor Solutions Group

Sony Semiconductor Solutions Group develops device business which includes Micro display, LSIs, and Semiconductor Laser, in focusing on Image Sensor.

www.sony-semicon.co.jp

 

[EVNow]

This is what I wish Tesla would use for Frontal SPAD Depth Sensor.

Sony to Release a Stacked SPAD Depth Sensorfor Automotive LiDAR Applications, an Industry First*1Contributing to the Safety and Security of Future Mobility with Enhanced Detection and｜News Releases｜Sony Semiconductor Solutions Group

Sony Semiconductor Solutions Group develops device business which includes Micro display, LSIs, and Semiconductor Laser, in focusing on Image Sensor.

www.sony-semicon.co.jp

Is a 597 x 168 pixels lidar useful ?

Fairly sure someday in the future there will be a lot more & better sensors in cars. Just not in cars we have today.

 

[drtimhill]

As to photon counting I'm a bit confused by the entire thing. Normally with photon counting its for very extremely low light situations where you want something sensitive, but with low noise. These can be so precise that your counting single photons (SPAD detectors).

Now you can certainly say you're doing photon counting with a regular image sensor, but I think its just fancy talk for dealing with raw images.

Yes, it pretty much is. They are not "counting photons" in the literal sense, they are just getting the raw data from the CCD, which, in a certain general sense, can be said to represent a measure of the overall quantity of photons that were received by the sensor. The idea is sound, that the raw data contains information that the NN can extract/use that is not available in the processed version the cameras spit out for human eye consumption.