Tesla replacing ultrasonic sensors with Tesla Vision

[kabin]

There was an odd fast wiper speed with light rain - I'm not sure witnessed it with this release.

One thing I do notice is repeated wiper action when the windshield might not be perfectly clear. As an example I tried to clean the front windshield the other day and although it looked clear to me the wipers kept turning on during a later drive as if it was raining. Had it been dirt or debris related I would've expected the wiper & washers to come on. It just repeatedly applied the dry wipers. Maybe that function needs a little param limit adjustment and/or logic tweak?

 

[sleepydoc]

There was an odd fast wiper speed with light rain - I'm not sure witnessed it with this release.

One thing I do notice is repeated wiper action when the windshield might not be perfectly clear. As an example I tried to clean the front windshield the other day and although it looked clear to me the wipers kept turning on during a later drive as if it was raining. Had it been dirt or debris related I would've expected the wiper & washers to come on. It just repeatedly applied the dry wipers. Maybe that function needs a little param limit adjustment and/or logic tweak?

One issue/problem is that the cameras are on an area covered by the end of the passenger side wiper at the end of its travel - essentially the place you’re most likely to get smearing. Normally it doesn’t matter for the driver because you don’t really care if there’s a smudge behind the mirror but it is an issue for the FSD cameras. When I clean the windshield I usually end up spraying more wiper fluid than I normally would to make sure that area gets cleaned well.

 

[Boza]

As mentioned, the $5 chip doesn't solve it either, it has the same problem that it only observes a small section of the windshield (although usually they are positioned a bit lower than where a camera would be). A true solution that would work in all situations would be a camera observing the entire windshield, but beyond that, it's all heuristics (usually assumption wetness of windshield is uniform, which many times it's not, examples pointed out elsewhere).

There is a difference between 100%, 99% and 70%. No one claims that the $5 solution works in 100%. The cost and the fact that it performs at least as well as the vision in wider spectrum of use cases makes it better, overall.

 

[Boza]

One issue/problem is that the cameras are on an area covered by the end of the passenger side wiper at the end of its travel - essentially the place you’re most likely to get smearing. Normally it doesn’t matter for the driver because you don’t really care if there’s a smudge behind the mirror but it is an issue for the FSD cameras. When I clean the windshield I usually end up spraying more wiper fluid than I normally would to make sure that area gets cleaned well.

Frankly, I think it has something to do with the noise from the sensor. No matter what kind of parameter adjustments they introduce the level of noise is so high that the signal becomes random.
That level of noise could be due to high sensitivity to non-related factors: variability in the glass material, camera itself, etc; or environmental factors like dust, lights, etc.
Bottom line is that vision is a much more expensive solution that works in a narrower set of circumstances.

 

[whitex]

I think it makes more sense if you consider that there are two drivers: the human is one, and arguably would be better served by the tried and true IR sensor that everyone here wants. But the other is the computer, looking through the front camera assembly. The optics and POV of the camera, right up against the upper windshield, mean that rain and dirt have a different effect on the computer's vision then on the human's vision.

So, even if the industry standard solution were adopted (even if free), it would still be just as necessary to develop the AI response network to keep the computer vision adequately clear and functional.

There is obviously a fair amount of commonality between the computer's windshield-clearing needs and the human's (which is why you and many others think it's "pretty OK"), but not full commonality (hence the frequent complaints).

Remember also that it's not a matter of "I'm not even using Autopilot, so don't bother me". Tesla cannot choose to disable the camera clearing needed for AEB or other accident-avoidance features, so logically it's a moot point whether AP/FSD is engaged.

Considering all that, I conclude that the camera-driven AI auto-wiper functionality is not at all superfluous or misguided, but in fact required - unavoidable. The next and separate question is, should Tesla have kept the IR sensor anyway, just for the bio-driver?

Maybe, but the reasoning above would dictate that those who think the wipers are too aggressive would still be annoyed - it's what the computer vision needs, in situations where a few sprinkled droplets right in front of the camera are occluding its vision.

The only aspect that would become better with a backup IR sensor, would be situations where the computer is satisfied but the human wants More Faster Wipers (aka "run the MF Wipers" ). These are the situations that require the driver to periodically goose the wipers with a manual button-press. Not the end of the world, but I do think a little algorithm could be written (I think I could do it but then it wouldn't be an NN) to pick up on "N wiper requests over the prior M seconds" and to adjust accordingly.

Ok, so why does only the front camera behind the windshield need wiping? Obviously Tesla believes the car can reach robotaxi level autonomy (they have been selling the "FSD capable hardware" for years) with cameras, most of which do not have any mechanical cleaning. Or are you a super-believer of Elon and still believe his Autopilot 1 marketing that he will reach full self driving with just stereo cameras behind the windshield because humans only have 2 eyes behind the windshield?

Until Tesla has implemented, tested, and approved for use fully autonomous driving, they need to make cars for human use. As long as humans are required to drive, they need clear windshields, whether or not the computer needs them or not. If we take your argument that computer has different needs and those take priority over the needs of the human, and take that to the extreme, that would mean Tesla could just remove the steering wheel and pedals, substitute those with a phone app, since the computer doesn't need them. Obviously that would not fly. Same goes for parking sensors, or any other driver aid, until Elon can prove that Tesla's fully self drive by insuring against any damage the car causes when sent to drive on its own anywhere, including coast to coast summon Elon said the FSD cars will be able to do years ago.

 

[Battpower]

why does only the front camera behind the windshield need wiping?

.... while, back at the ranch, both b-pillars are either blinded or fogged up and the trunk cam covered in cr*p with no sign of concern or remedy?

However, that video posted a way back in this thread iirc showed cars able to see enough to attempt self driving with pretty well all cams obscured (taped up) . The main forward facing cam seems to take the biggest role in providing data to base forward motion on, so may be that is sufficient to justify wiping its window!

 

[JHCCAZ]

Ok, so why does only the front camera behind the windshield need wiping? Obviously Tesla believes the car can reach robotaxi level autonomy (they have been selling the "FSD capable hardware" for years) with cameras, most of which do not have any mechanical cleaning. Or are you a super-believer of Elon and still believe his Autopilot 1 marketing that he will reach full self driving with just stereo cameras behind the windshield because humans only have 2 eyes behind the windshield?

Until Tesla has implemented, tested, and approved for use fully autonomous driving, they need to make cars for human use. As long as humans are required to drive, they need clear windshields, whether or not the computer needs them or not. If we take your argument that computer has different needs and those take priority over the needs of the human, and take that to the extreme, that would mean Tesla could just remove the steering wheel and pedals, substitute those with a phone app, since the computer doesn't need them. Obviously that would not fly. Same goes for parking sensors, or any other driver aid, until Elon can prove that Tesla's fully self drive by insuring against any damage the car causes when sent to drive on its own anywhere, including coast to coast summon Elon said the FSD cars will be able to do years ago.

I did not say that only the front camera behind the windshield needs wiping.

What I actually said, was that Tesla has no choice but to provide cleaning for the windshield cameras, and that the sensing and decision-making for said cleaning must be driven from the camera view. This is product design reasoning and it is sound.

I did not say that other functionality shouldn't be provided, but I did discuss a few of the pros and cons. I think my message indicated that I appreciate the concerns a human drivers, of which I am one. This is also product design reasoning.

Everything else you wrote, extrapolating emotional and tribal associations, inferring robotaxi-only philosophy, inferring sycophantic devotion to Elon etc., is a distortion of my message. I'm sorry if you're annoyed with people who don't just reinforce the general whining about auto-wipers. I was responding to a message that specifically questioned why ML engineering resources should be applied to auto-wipers, and I think I provided a logical and defensible explanation.

There aren't just two camps for every question, and it's a false argument to assign an entire opposite-camp belief system to someone who doesn't simply echo your favored complaints. I have innumerable disagreements and concerns with the present state of my Tesla and its automation, but I'm not hopelessly disgruntled. I also greatly admire much of the engineering I see in Tesla products and in SpaceX et. al., yet I'm not a hopeless Elon fanboy. When I see a problem, and/or other people expressing frustration about a problem, I tend to think about it as a product designer which I am. Most solutions and design choices have benefits, costs and problems. I think about all those, and the new sets of benefits, costs and problems that come with proposed solutions.

Again, I did not say that only the cameras matter, nor that the popular demand for an industry standard IR sensor was wrong or should be ignored - but I did say that it would not end the complaints, and that it would not obviate the need for the camera-driven auto wiper software, both of which are true.

 

[Battpower]

it would not end the complaints

The slightly polarizing issue I see is between those who see the existence of a reasonable justification / explanation as representing some form of mitigation of responsibility (Tesla's) and those who focus more on feeling their car(s) are Tesla's science experiment and are not happy with that.

You are probably correct that there will always be some dissatisfaction. In this case, it isn't obvious to many why established, low cost solutions need to be reinvented apparently at the owner's inconvenience.

 

[JHCCAZ]

The slightly polarizing issue I see is between those who see the existence of a reasonable justification / explanation as representing some form of mitigation of responsibility (Tesla's) and those who focus more on feeling their car(s) are Tesla's science experiment and are not happy with that.

You are probably correct that there will always be some dissatisfaction. In this case, it isn't obvious to many why established, low cost solutions need to be reinvented apparently at the owner's inconvenience.

I think I agree with your take. And yes, I agree it isn't obvious why the established IR sensor solution is (even if helpful and cheap) still insufficient. I thought it was interesting point, so my explanation was an attempt to clarify the non-obvious reason.

 

[SO16]

I did not say that only the front camera behind the windshield needs wiping.

What I actually said, was that Tesla has no choice but to provide cleaning for the windshield cameras, and that the sensing and decision-making for said cleaning must be driven from the camera view. This is product design reasoning and it is sound.

I did not say that other functionality shouldn't be provided, but I did discuss a few of the pros and cons. I think my message indicated that I appreciate the concerns a human drivers, of which I am one. This is also product design reasoning.

Everything else you wrote, extrapolating emotional and tribal associations, inferring robotaxi-only philosophy, inferring sycophantic devotion to Elon etc., is a distortion of my message. I'm sorry if you're annoyed with people who don't just reinforce the general whining about auto-wipers. I was responding to a message that specifically questioned why ML engineering resources should be applied to auto-wipers, and I think I provided a logical and defensible explanation.

There aren't just two camps for every question, and it's a false argument to assign an entire opposite-camp belief system to someone who doesn't simply echo your favored complaints. I have innumerable disagreements and concerns with the present state of my Tesla and its automation, but I'm not hopelessly disgruntled. I also greatly admire much of the engineering I see in Tesla products and in SpaceX et. al., yet I'm not a hopeless Elon fanboy. When I see a problem, and/or other people expressing frustration about a problem, I tend to think about it as a product designer which I am. Most solutions and design choices have benefits, costs and problems. I think about all those, and the new sets of benefits, costs and problems that come with proposed solutions.

Again, I did not say that only the cameras matter, nor that the popular demand for an industry standard IR sensor was wrong or should be ignored - but I did say that it would not end the complaints, and that it would not obviate the need for the camera-driven auto wiper software, both of which are true.

Don’t take whitex’s comments personally. He has been soured for years. And I literally mean YEARS!!! And yet he’s STILL on here. Why? Because he has said he wants to turn away people from Tesla since he has been “wronged”. He used to convince everyone at one point to buy a Tesla and now he’s doing the opposite. Some sort of weird crusade I guess.

Anyway, it does seem that Tesla removes items before the replacements are ready. I think some of it is forced due to parts supply (ultrasonics and chips?) and due to licensing (MobileEye dropping AP1).

Other times, simply a premature self inflicted change like the wiper sensor.

I wish they would wait until the replacement is close to on par with the original before removing. (At least auto wipers are working better than they used to. Still not excellent though.)

 

[LiamPope]

I'll never buy the idea that all the auto wiper problems are simply because it is operating for the cameras and not for the human driver, with the implication it's doing a fantastic job and it's just our expectations that are wrong! Makes no sense! First, this is not an autonomous car - it's a car with ADAS. The human is in full control. Second, I'm inclined to go with the simpler explanation that it just does a lousy job (ymmv). I mean you could say the same for a rain sensor system anyway - it's also operating on and for the patch of glass top centre of the windscreen, but somehow does a better job for the human than Tesla Vision. It's not a matter of location or who/what they are serving - that's stretching too hard to excuse Tesla - it's simply the wrong tool for the job.

Also, to the idea that there MUST be a camera based rain detection system - nonsense! There's really no reason or benefit that the cameras themselves must decide whether or not it is raining. Sure in Tesla's Vision dogma it fits, but in reality the car as an autonomous system can take input from different types of sensor. Simple fact is the system would be better informed it is raining by a rain sensor than its own 'eyes' which can't even focus on the screen they need to determine the status of and a NN. Just as I am better informed of the time or temperature, for example, by looking at a clock or thermometer than relying on my eyes or other less suited sensors and a NN to figure it out. A rain sensor can be literally an inch away from the screen the cameras need to see out of - arguments the conditions can be different are rubbish.

My own 'cameras' keep themselves clean by blinking. Not because they see any obstructions and figure it out with the meat NN, but because they directly sense contact on the protective outer screen and operate the wipers. Yes - dedicated dumb sensors hard wired to the wipers - no Vision needed!

 

[Battpower]

it's simply the wrong tool for the job.

I am in this camp - based on my own experience that suggests the camera solution 'doesn't work' to the extent that it might 'do something' but what it does is neither good enough nor as good as other established alternative designs.

the idea that there MUST be a camera based rain detection system

I haven't really picked up a common theme among owners suggesting there 'must be camera based solution' . Most just want a working solution that's at least as good as what's commonly available. Tesla on the other hand must be at liberty to use whatever solution they want, but it must satisfy owners expectations as well as Tesla's objectives.

Wishing that you have a great VO system does not make it that way! Stating that it will work or could work doesn't make it work! This is science experiment talk.

But, finding the boundaries of what's reasonable, desirable and deliverable is not always straightforward. Given that owners should not be co-opted into their science experiment, would it change anything if, for argument's sake, owners were told that they have to keep the windscreen perfectly clean and free from waxes and other products / coatings in order for the wiper system to work correctly?

I have noticed some jet wash / car washes put treatments on your car due to residue in pipes etc, even if you don't intentionally select it. Although I haven't linked this to wiper bahaviour (and it doesn't seem to cause an obvious issue with my non-Tesla car) I have seen smeary patches on the windscreen that appear to be due to car wash chemicals.

 

[LiamPope]

I haven't really picked up a common theme among owners suggesting there 'must be camera based solution' .

I didnt specifically quote anyone, but just scroll up. And I've heard it before too - that for some reason the self driving system cameras must determine for themselves if it is raining. No other way. They can't just ask an adjacent rain sensor. Makes no real sense and other manufacturers don't seem to think so.

 

[jabloomf1230]

Can the auto wipers either be coded better than they presently are or is it hopeless? This discussion is analogous to the Lidar-Tesla Vision arguments. It always seems better to use a source-sensor pair, that is, as long as one ignores the added complexity and cost.

My thought is that comparing front camera frames can be made to work as well as an IR source-sensor pair. Tesla just hasn't put the time and effort into doing so.

 

[Mr. Michael]

When your only tool is tesla vision, then every problem looks like something that needs nothing else. It seems that currently tesla is on a vision quest, and we are along for the ride. A sometimes very scary ride.

 

[Battpower]

for some reason the self driving system cameras must determine for themselves if it is raining.

Hmm.

In the context of Tesla and VO this makes perfect sense.

In the real world it's either raining or not and either the human driver or other device is going to / needs to wipe.... or not.

I suppose that if you place a higher level of trust in one sensor vs another....

you'd still probably be better off with a tested IR sensor's view of if it's raining or not. Until VO works at least as well as IR, then I doubt any one will care which one controls the wipers.

 

[Mr. Michael]

Imagine the failures of the vision system in training on elon’s Full Self Wiping Toilet. Maybe don’t…

 

[Battpower]

That would probably be really cr*p!

 

[sleepydoc]

I haven't really picked up a common theme among owners suggesting there 'must be camera based solution' . Most just want a working solution that's at least as good as what's commonly available. Tesla on the other hand must be at liberty to use whatever solution they want, but it must satisfy owners expectations as well as Tesla's objectives.

Yes - I generally describe myself as a pragmatist. I’m far more interested in results than in how they are achieved. It’s also important to note that no sensor system is perfect. The best system I’ve used was on my Audi A4 and I’d describe it as ’pretty good.’

Part of the problem with Teslas approach is that you forced to have the wipers on ‘auto’ if you’re using FSD, AP, or even just TACC. In my Audi if it wasn’t working well I could just turn it off but Tesla doesn’t give you that option, increasing the frustration factor.

 

[stopcrazypp]

I'll never buy the idea that all the auto wiper problems are simply because it is operating for the cameras and not for the human driver, with the implication it's doing a fantastic job and it's just our expectations that are wrong! Makes no sense! First, this is not an autonomous car - it's a car with ADAS. The human is in full control. Second, I'm inclined to go with the simpler explanation that it just does a lousy job (ymmv). I mean you could say the same for a rain sensor system anyway - it's also operating on and for the patch of glass top centre of the windscreen, but somehow does a better job for the human than Tesla Vision. It's not a matter of location or who/what they are serving - that's stretching too hard to excuse Tesla - it's simply the wrong tool for the job.

Also, to the idea that there MUST be a camera based rain detection system - nonsense! There's really no reason or benefit that the cameras themselves must decide whether or not it is raining. Sure in Tesla's Vision dogma it fits, but in reality the car as an autonomous system can take input from different types of sensor. Simple fact is the system would be better informed it is raining by a rain sensor than its own 'eyes' which can't even focus on the screen they need to determine the status of and a NN.

That isn't an issue. The camera only needs to know if it is obscured or not, just like how when you are driving, you don't need to refocus your eyes onto your windshield to know that it is being obscured.

Just as I am better informed of the time or temperature, for example, by looking at a clock or thermometer than relying on my eyes or other less suited sensors and a NN to figure it out. A rain sensor can be literally an inch away from the screen the cameras need to see out of - arguments the conditions can be different are rubbish.

My own 'cameras' keep themselves clean by blinking. Not because they see any obstructions and figure it out with the meat NN, but because they directly sense contact on the protective outer screen and operate the wipers. Yes - dedicated dumb sensors hard wired to the wipers - no Vision needed!

Think of it like having a wetness/dust sensor on your nose, and your eyes never blinking unless that sensor detects water/dust. That's obviously not how your eyes work, it can feel if there is dust or wetness on the eye itself and it can blink in response. If a rain sensor is added it won't be where the camera is, it'll typically be lower.