How can Tesla do rain sensing on AP2?

[thegruf]

Been bugging me this one, and now relevant to me having switched to an AP2 ModelS

Tesla don't seem to have any quick answers either, though the images of the Model3 don't seem to show a rain sensor either, so they must have a concept of how to do it at least.

The biggest issue with the cameras is that they are deliberately designed to be out of focus at the range of the windshield glass, else every bit of dirt (and rain!) would ruin AP performance.

So if the image is out of focus close up - how are Tesla going to "see" the rain without a separate sensor?

Use the cameras
to "see" the raindrops at distance - is this possible to check for appropriate noise between successive images?
to look for wet pavement - inaccurate
to watch other cars wipers? - only works if there is other traffic
(and when all cars are Teslas would be interesting, first one to manually use the wipers sets the whole fleet going!)
check tor spray using the rear camera - not impossible but likely inaccurate

Use internet based rainfall radar? - intersting but not accurate enough

radar - nope?

somebody explain how this can be done please

 

[Matias]

Raindrops on lens cause characteristic flare on picture. I think they train neural net to detect this.

 

[WarpedOne]

I see a rain sensor ...

 

[J1mbo]

I see a rain sensor ...

That's a light sensor, not a rain sensor...

 

[lunitiks]

That's a light sensor, not a rain sensor...

According to the parts manual, it's a "Light/Temperature/Humidity"-sensor. One could argue that rain is pretty humid... But it's still a strange word to use for something that's supposed to detect rain (why call it "humidity"). The same sensor is also called "HVAC-sensor" in Teslas service manual. Anyway, the sensor in question is attatched to a custom Tesla PCB (so no off-the-shelf stuff like AP1-cars have, which is manufactured by HELLA).

So either they ran into problems with that LTH-sensor, or its taking longer than expected to train the neural nets to flawlessly detect rain drops with the cameras...

 

[J1mbo]

According to the parts manual, it's a "Light/Temperature/Humidity"-sensor. One could argue that rain is pretty humid... But it's still a strange word to use for something that's supposed to detect rain (why call it "humidity"). The same sensor is also called "HVAC-sensor" in Teslas service manual. Anyway, the sensor in question is attatched to a custom Tesla PCB (so no off-the-shelf stuff like AP1-cars have, which is manufactured by HELLA).

So either they ran into problems with that LTH-sensor, or its taking longer than expected to train the neural nets to flawlessly detect rain drops with the cameras...

IIRC the temperature & humidity measurement is for the sensor side of the glass, to control the demisting wires in front of the cameras.

 

[nowtleft]

They simply dont work until there is so much rain on the windscreen you have to either flick wipe or switch wipers on full. Why mess with rockets if you cant sort a rain sensor out or buy one from Ford that works?

 

[J1mbo]

They simply dont work until there is so much rain on the windscreen you have to either flick wipe or switch wipers on full. Why mess with rockets if you cant sort a rain sensor out or buy one from Ford that works?

Maybe because the auto sensors from "Ford" don't work so well in Full Self Driving cars when the driver has the "auto" bit switched off?

 

[Haxster]

Raindrops on lens cause characteristic flare on picture. I think they train neural net to detect this.

View attachment 237076

Unfortunately, other things could produce the same flair patterns.

Here's a scenario:
You park under a tree on a hot day. Clear tree sap drops on the window over the camera. It drys hard. When you get in the car, it thinks there's a rain drop or two on the windshield. The wipers start and, because the sap is hard, they continue running.

Maybe they should turn on the washer, but they don't. If they did, the car could run out of fluid before the tree sap was cleared. Life can get complicated.

Note, this could also be an issue for conventional IR rain sensors.

 

[nowtleft]

Maybe because the auto sensors from "Ford" don't work so well in Full Self Driving cars when the driver has the "auto" bit switched off?

I cant see why they wouldn't - thye are triggered by rain on the windscreen not how the cars autopilot is set up. Just remember my Mondeo's auto rain sensing wipers being very good (car was crap!)

 

[lunitiks]

Unfortunately, other things could produce the same flair patterns.

Here's a scenario:
You park under a tree on a hot day. Clear tree sap drops on the window over the camera. It drys hard. When you get in the car, it thinks there's a rain drop or two on the windshield. The wipers start and, because the sap is hard, they continue running.

Maybe they should turn on the washer, but they don't. If they did, the car could run out of fluid before the tree sap was cleared. Life can get complicated.

Note, this could also be an issue for conventional IR rain sensors.

... and for the cameras?

Unavoidable issue.com?

 

[Tam]

...Temperature...-sensor

I hope that is not the one that measures outside temperature as displayed on the dashboard because the windshield can get very hot!

 

[_jal_]

This question has come up in other forums and the received wisdom is that they are trying to do something as outlined in this paper.

Vision-based rain sensing with an in-vehicle camera - IEEE Xplore Document

 

[whitex]

That's a light sensor, not a rain sensor...

Ok, so what you're saying is Tesla with it's 8 cameras cannot tell whether it's dark outside, but they will be able to detect rain? You'd think it should be easier to tell day vs. night over rain vs. no rain.

 

[lunitiks]

Good point @whitex. Kind of a paradox

 

[Farzinfar]

What baffles me is why Tesla removes the infrared rain sensor that these days can be found on any cheap car in favour of something not proven or tested yet. With all those sensors and cameras all over the car, yet it cannot see the rain coming! Why didn't they do a test before removing the normal cheap sensor!

An infrared sensor will not cost more than 5 USD and it would have been wiser for Tesla to keep the sensors. Or perhaps we are just too spoiled! I had rain sensors in all my cars since its introduction in 1996! and it is very annoying to see such an expensive car doesn't have a rain sensor!

 

[lunitiks]

The term "rain sensor" is kind of confusing in itself in this context, because Tesla is attempting to sense the rain visually (through cameras and algorithms), instead of through IR emitters, receivers and a microchips dedicated for that job only. Some may argue AP2 has a rain sensor, because it has a system for rain sensing - just not ready yet - others may argue that AP2 has no rain sensor and tries to mitigate this by other means.

Does AP2 have a rain sensor? I don't know how to answer that. All I know AP2 doesn't do auto rain-sensing wiper functionality today

 

[Farzinfar]

@lunitiks: thanks for your reply. Well I have AP2 car and I know for fact when i set the wiper control on first (dotted) setting, it moves the blades in a few second intervals regardless of rain and it doesn’t have rain sensor yet. But this is not the issue.

The issue is that Tesla as you described have removed the simple infrared sensor in hope of using the current cameras instead to detect the rain.

My argument is when they have not yet developed such sensitivity to sense the rain (even though they will do in the future with an on air update), they should not remove the old one and keep the infrared until they properly develop sensor using cameras.

The same argument is for AP1 and AP2 which I see from other posts in the forum there have been some serious backward moves from AP1 to AP2. What is expected from such company is to keep the good work and not to go backward even if on air update can rectify things later they should not remove functionality from the car. I hope I was clear. Thanks again.

 

[lunitiks]

It's clear as day @Farzinfar. You're suggesting that Tesla - as long as they hadn't solved the cameras-only rain sensing capability - should've kept the HELLA hardware in AP2-cars. Even if it would mean that the HELLA unit would be rendered completely useless ("redundant") the second the great wide-angle-camera-rain-sensing-Neural-Network-action-man stuff was beamed to the fleet OTA.

Well, your opinion will be disagreed to death by a lot of people. They'll say Tesla never promised you auto rain-sensing wipers in the first place -- because it's true; they would be legally correct; Tesla's webpage doesn't sport that feature at all. And then they'll say have patience. And then they'll say the goal justifies the means. Or some such.

So good luck. I've been here long enough to know that this particular discussion will be a too exhausting to even bother to argue.

In conclusion: Wait. Or sell. Your choice

 

[Farzinfar]

@lunitiks: I read content of your message and it was well noted. First please note that I thanked you twice (at the beginning and at the end) for taking time to answer me. Secondly this is a forum to exchange ideas and no place for hostility and bad manner!

As for your first paragraph: yes that is what exactly I meant. An infrared unit only costs 5 USD and perhaps even less for mass purchase so a 5 dollar unit being redundant and useless kept on some models is not a big deal.

I also mentioned about AP1 vs. AP2 and that is also several steps backward move but let us just concentrate on rain sensor.

Your second paragraph: I totally disagree with you for obvious reason that I will explain shortly. Before that I would like to say that I don't think people would disagree with me to death since this is not really an important issue to die for. Now without going to contract law (by the way I am a lawyer by training so the chances of beating me on this is very slim), I refer you to the manual. As you have been around long enough, I am sure you have already read the manual several times. This is a refresher: Please refer to page 53 of the manual (Facelift model) which it clearly says:

*Model S has a rain sensor located on the

inside of the windshield at the base of the

interior mirror. When wipers are set to Auto,

the frequency at which they wipe depends on

how much water the sensor detects. When

wipers are set to the 2nd level, the sensor is

more sensitive."

This is a clear indication that this function is not an option and should have been provided by Tesla. You might argue they forgot to modify the manual when they introduced AP2 and face lift model. Well, many car manufacturers use the wording "Where applicable" or "if available" for those functionalities that are available as option. Moreover a car has so many bits and pieces and functions that are not listed in the contract of sale so it doesn't mean they are not included on delivery. Have you ever seen in a sale contract of a car that they list doors, windows, door handle etc...? Does it mean that a manufacturer can give you a car without doors because it was not mentioned in the sale contract?

Finally on your last paragraph: well as I said above, this is a place to exchange ideas and I am sure there are positive results coming out of these discussions. Participating to one is a personal choice. The same way you suggested "wait or sell", I would suggest don't or do: read or participate or contribute. Choice is yours.

And one final comment, I don't think Elon himself is as defencive as you are about Tesla. So my friend take it easy and take criticism, suggestion or ideas with open mind. It can be constructive. Thank you. The same way I took your advice and I will wait. cheers

Best wishes