AP2.0 Cameras: Capabilities and Limitations?

[tescroft]

For me, this rain sensing using video story just makes no sense at all. COTS rain sensors are good, cheap and easy to integrate. There is no strategic advantage doing this inhouse. It seems some bad decisions have been made at Tesla in this case.

 

[mrkisskiss]

Yeah, seems like someone at Tesla was "all in" on vision doing absolutely everything...

Having said that, it does make sense that Autopilot learns when it can't see properly... for sure, this is true in the rain - but also perhaps if there's something else on the windshield; dirt, grime, squashed bugs. If it can work out when it *can't* see, then it could do things like activate a windscreen washer spray and wipe to see if it improves... Not super important whilst we're in the car, but being able to clean itself would be advantageous when it's more autonomous

 

[daktari]

What are the possibilities for Tesla doing a recall with a retrofit rain sensor? Is the technical architecture in the camera bay ready for another sensor to signal the wipers? You need a new housing of course. Bigger recalls have been done by other brands.

 

[J1mbo]

What are the possibilities for Tesla doing a recall with a retrofit rain sensor? Is the technical architecture in the camera bay ready for another sensor to signal the wipers? You need a new housing of course. Bigger recalls have been done by other brands.

Zero.

 

[daktari]

Zero.

Zero technical possibilities ? Why?

 

[verygreen]

Yes, its YUV422, but with channels stored as separate planes.

View attachment 245239 View attachment 245240 View attachment 245241

So what would be a good tool to use then? Thanks!

 

[J1mbo]

What are the possibilities for Tesla doing a recall with a retrofit rain sensor?

You didn't say technical.

Technically, a sensor needs to be connected somehow to a controller.

But then how does AP control the wipers? It still had to learn to know when it is raining.

If it has to learn this anyway, why do you need a sensor and a controller?

 

[DamianXVI]

So what would be a good tool to use then? Thanks!

Well, I've used a proprietary vision library from my work and a couple lines of C++, so I can't really recommend any publicly available tool. Sorry.

I could add a function to the tool I previously posted, when I have some free time.

 

[daktari]

But then how does AP control the wipers? It still had to learn to know when it is raining.

If it has to learn this anyway, why do you need a sensor and a controller?

No, the wipers need to be connected to a sensor, closed system. FSD on= always auto wipers activated. (or other way around, not able to initiates FSD without auto wipers activated).
Keeping camera clean is then handled. If not clean enough, FSD need to let driver take over (lvl 3) or stop (lvl 4).

 

[stopcrazypp]

No, the wipers need to be connected to a sensor, closed system. FSD on= always auto wipers activated. (or other way around, not able to initiates FSD without auto wipers activated).
Keeping camera clean is then handled. If not clean enough, FSD need to let driver take over (lvl 3) or stop (lvl 4).

If the CAN bus connection allows, the AP processing hardware can also read in the raw or processed status of the sensors (similar to how it handles the radar data). I didn't follow the discussion on this in general, so the current wiring may not make this possible though.

I think the theory about Tesla thinking they have to design the vision detection in general anyways (for dirt, grime, bugs, etc for all cameras, not just the front ones) makes more sense.

 

[J1mbo]

No, the wipers need to be connected to a sensor, closed system. FSD on= always auto wipers activated. (or other way around, not able to initiates FSD without auto wipers activated).
Keeping camera clean is then handled. If not clean enough, FSD need to let driver take over (lvl 3) or stop (lvl 4).

The bold part would need a custom controller, to allow FSD to override the physical stalk setting, which I would guess is a bit more expensive than the off-the-shelf unit Tesla used from the original MS until AP2. The other way around wouldn't work for situations where there is no driver.

The fact is, for whatever reason, Tesla have decided to move to a different architecture for auto wipers. This would have happened after a lot of discussion in the various engineering teams and a clear alternative being agreed on.

My guess is that auto-wipers has been a victim of the agile methodology Tesla use for development. Auto lights, high beam and driving dynamics have taken priority.

 

[bjornb]

On AP1 cars the rain/light sensor is connected on a LIN bus to the BCM (Body Control Module). AP2 has a Light/Humidity/Temp sensor instead on that LIN bus. If Tesla really want to I think it would be possible to retrofit the rain sensor by creating a new AP2 camera housing and necessary components, reusing the existing LIN bus, and appriopriate software changes in the BCM etc (but I dont believe it will happen).

The bold part would need a custom controller, to allow FSD to override the physical stalk setting, which I would guess is a bit more expensive than the off-the-shelf unit Tesla used from the original MS until AP2. The other way around wouldn't work for situations where there is no driver.

I disagree.
As far as I can see the wipers are controlled by software in the BCM wich responds to certain inputs - wiper stalk position and rain data (dedicated rain sensor on AP1 cars, and hopefully CAN data over the chassis CAN bus on AP2 cars in the future). I believe Tesla can disregard the wiper stalk input today if they want to, they actually do that when you select wiper service position from the controls menu on the MCU..

 

[scottf200]

For me, this rain sensing using video story just makes no sense at all. COTS rain sensors are good, cheap and easy to integrate. There is no strategic advantage doing this inhouse. It seems some bad decisions have been made at Tesla in this case.

If they are thinking about all cameras and the necessity of them for as much FSD as possible then @stopcrazypp point about them needing this software tech for all their cameras makes logical and strategic sense.

I think the theory about Tesla thinking they have to design the vision detection in general anyways (for dirt, grime, bugs, etc for all cameras, not just the front ones) makes more sense.

 

[lunitiks]

Anyone with AP2.0 / 2.5 care to teardown their tri-cam block and post some good pics of its insides?
@kdday ? @verygreen ? @bjornb ? @AnxietyRanger ? Anybody ?
I'm particularly interested in any text incscribed on the cameras (manufacturer? resolution/focal length/etc?) and of course good pics of the infamous HVAC-sensor.

(Long-shot I know, but hey what can you do.)

Anyway, here's the recipe:

First and foremost: Disconnect 12V. (Official walkthrough for S and X. Kdday's TLDR-version here.) Not sure if it's sufficient to turn car off.

Step one (1): Remove the rear view mirror. You'll have to remove the small cover like shown in this video. Then I thing you have to turn the mirror clockwise until it pulls free. (At least on AP1.)

Drugi korak (2): Remove the mirror cover. Simply pull it down to release the clips.

Schritt drei (3): Release the screws that secure the mirror bracket carrier to the glare shield.

четвертый шаг (4): Disconnect the camera harness connectors, and then remove the camera.

Passo cinque (5): Release the fasteners that secure the glare shield.

Let's anti-degrade the Community!

 

[lunitiks]

 

[KyleDay]

Anyone with AP2.0 / 2.5 care to teardown their tri-cam block and post some good pics of its insides?
@kdday ? @verygreen ? @bjornb ? @AnxietyRanger ? Anybody ?
I'm particularly interested in any text incscribed on the cameras (manufacturer? resolution/focal length/etc?) and of course good pics of the infamous HVAC-sensor.

(Long-shot I know, but hey what can you do.)

Anyway, here's the recipe:

First and foremost: Disconnect 12V. (Official walkthrough for S and X. Kdday's TLDR-version here.) Not sure if it's sufficient to turn car off.

Step one (1): Remove the rear view mirror. You'll have to remove the small cover like shown in this video. Then I thing you have to turn the mirror clockwise until it pulls free. (At least on AP1.)

Drugi korak (2): Remove the mirror cover. Simply pull it down to release the clips.

Schritt drei (3): Release the screws that secure the mirror bracket carrier to the glare shield.

четвертый шаг (4): Disconnect the camera harness connectors, and then remove the camera.

Passo cinque (5): Release the fasteners that secure the glare shield.

Let's anti-degrade the Community!

I am replacing my windshield on my AP 2.0 car (got a cracked from a rock) in the next two weeks or so (windshield on backorder apparently). I'm not sure if while replacing the windshield this would be a good time to disassemble the housing, but might be an opportunity.

 

[verygreen]

Interesting development, people report that FSD/EAP upgrades are now missing for already purchased cars that did not prepay for the options (I assume only HW2 is affected since hw 2.5 are not really in anybody's hands yet). Just a glitch?

AP2 upgrade option gone from MyTesla. • r/teslamotors

 

[KyleDay]

It's a glitch. The chance to upgrade battery from 60 > 75 is also gone

 

[DamianXVI]

As I said earlier - below is a new version of image conversion tool with a function for converting images from the backup camera. It's accessible with "-back" switch (with no additional options, this is a straightforward conversion to RGB BMP).

I was also experimenting a bit with this tool since. Some functions are rewritten for SSE for faster computation (useful when converting replay videos). I also wanted to add the ability to export a full precision image to some advanced external image editor, for better HDR processing and enhancements, so I added an option to save to TIFF files ("-tiff" switch). This saves a 32bit floating point TIFF with full linear range (without histogram stretching and gamma correction). As far as I understand, it should be possible to load this file with Photoshop or similar application without any precision loss, but I didn't properly tested that.

 

[bjornb]

Interesting development, people report that FSD/EAP upgrades are now missing for already purchased cars that did not prepay for the options (I assume only HW2 is affected since hw 2.5 are not really in anybody's hands yet). Just a glitch?

AP2 upgrade option gone from MyTesla. • r/teslamotors

FSD upgrade is still available in my account for my AP2 car with EAP.