bjorn nyland's test of tesla vision

[yessuz]

It 'just works' with human input, which to me is why Tesla chose not to go that way as they are single minded about self driving. All I'm doing is answering the much repeated question about why they didn't include a $5 rain sensor. It wouldn't allow them to reach a self-driving car, seems logical to me.

They have attempted a solution that remove the requirement for a human to operate the sensitivity control that's required with these rain sensors on even the most advanced alternative cars. I doubt even the most vocal pro-Tesla user (might be me) would argue that it is 100% perfect, some may find it adequate, some not.

By making it mandatory with Tesla Vision it has to be perfect.

well, let's look at this:
you might have same sensor (which is not in the car at all) being adjusted with already existing computer. no human involvement, happy days, win win, fsd or something.

same applied camera for high beam - others have it working with 99.999% accuracy. while in tesla with all its bloody cameras all over the place it's like roll of dice every time. why it cannot get it straight? it's almost as simple as:

if LightsInFront(1) {
  SwitchOffHighBeam(1);
}
else {
    SwitchOffHighBeam(0);
}

sh!t, need to sell this to Tesla AI team...

 

[mtndrew1]

I had a Mazda in 2008 with better auto lights and wipers than my Tesla.

It would be really nice if Tesla would finish a project before starting a new one. The wipers and auto lights have been “beta” or whatever for four years.

 

[GalacticHero]

The worrying thing is that auto headlights and wipers (two vision controlled systems) are some of the most complained about for poor performance on the cars and are now integral to a vision controlled system attempting to control the whole vehicle
What could possibly go wrong....

Same worry here. Yesterday on the way home from work I ran into a deluge. Yet, the auto wipers did not turn on. I had to hit the manual wipe button to get the wiper pop-up and set it to max. I could still barely see with max wipers and yet the auto setting did not recognize it.

 

[GeorgeSymonds]

I had a Mazda in 2008 with better auto lights and wipers than my Tesla.

It would be really nice if Tesla would finish a project before starting a new one. The wipers and auto lights have been “beta” or whatever for four years.

I think it's nearly 6 - Tesla hardware was introduced in late 2016

I get the long term thinking that one awesome mega core that knows everything thats going on around the car would be a great thing and essential for self driving. You get that core working and everything falls out from that - you know all the cars around you and whether they're being driven or parked etc, and you can then control auto headlinghts better than any other system, you know the condition of the road, how grippy it might be, make allowances for spray and rain... and as a consequence auto wipers would be a doddle (or simply not needed if the car is driving). Get that central core working which understands all this and AP, EAP and FSD are all next logical steps.

The issue is the core isn't there, and Tesla are between a rock and a hard place given the million or two cars they've put on the road with "all the hardware you need for self driving". They're already into pixel peaking the camera, one might wager a better camera would now be fitted, they're dropping the radar because maybe the radar they spec'd back in 2016 wasn't that good and trying to make it all work without a radar seems to be an easier option than upgrading an aweful lot of cars to a new better radar. But I think it's safe to deduce that Tesla Vision was a restart of building the core. I've no doubt there were lots or reusable bits in the software kitbag, but its a pretty big shift.

Bjorns experience seems to reinforce this. Lets say they had a relatively crude lane and speed keep working with a mash up of video and radar, and remember when they first started EAP they launched only using 4 cameras so we know it wasn't, at least initially, the full monty and probably they made a few some short cuts to get some features out there once Mobileye departed to save face.

So it's possible, and none of us really know, we just speculate and join the dots in our mind differently, that AP features were developed down a development track that reached a fairly limited performance, and they're now trying to switch over to a new stack that uses 8 camera, has a better core (as mentioned above) but the core isn't ready for prime time and it's introducing new problems.

As for the wipers, I'd have included the rain sensor and included it as a secondary input. Best of both worlds.

 

[mtndrew1]

I think it's nearly 6 - Tesla hardware was introduced in late 2016

I get the long term thinking that one awesome mega core that knows everything thats going on around the car would be a great thing and essential for self driving. You get that core working and everything falls out from that - you know all the cars around you and whether they're being driven or parked etc, and you can then control auto headlinghts better than any other system, you know the condition of the road, how grippy it might be, make allowances for spray and rain... and as a consequence auto wipers would be a doddle (or simply not needed if the car is driving). Get that central core working which understands all this and AP, EAP and FSD are all next logical steps.

The issue is the core isn't there, and Tesla have are between a rock and a hard place given the million or two cars they've put on the road with "all the hardware you need for self driving". They're already into pixel peaking the camera, one might wager a better camera would now be fitted, they're dropping the radar because maybe the radar they spec'd back in 2016 wasn't that good and trying to make it all work without a radar seems to be an easier option than upgrading an aweful lot of cars to a new better radar. But I think it's safe to deduce that Tesla Vision was a restart of building the core. I've no doubt there were lots or reusable bits in the software kitbag, but its a pretty big shift.

Bjorns epxerience seems to reinforce this. Lets say they had a relatively crude lane and speed keep working with a mash up of video and radar, and remember when they first started EAP they launched only using 4 cameras so we know it wasn't, at least initiallythe full month and probably they made a few some short cuts to get some features out there once Mobileye departed to save face.

So it's possible, and none of us really know, we just speculate and join the dots in our mind differently, that AP features were developed down a development track that reached a fairly limited performance, and they're now trying to switch over to a new stack that uses 8 camera, has a better core (as mentioned above) but the core isn't ready for prime time and it's introducing new problems.

As for the wipers, I'd have included the rain sensor and included it as a secondary input. Best of both worlds.

I couldn’t have said it better.

It’s great that they’re trying to solve complex unprotected left turns with medians. I often dream of that capability while people are flashing me because my high beams are blasting them in the face.

 

[browellm]

This is a sensitivity control for the rain sensor, it is needed to make the auto-wipers find the correct speed and it needs a human to operate it. It's not fully-automatic.

I did it once in 2013, found the sensitivity that suited me and never touched it again in 7-8 years of use.

The implication that you "fiddle" with it is patent nonsense.

 

[Wol747]

>>AP1 cars had rain sensors, and I'm reliably informed that their auto wipers work as you'd expect - i.e. properly. Somehow, cars built years later are orders of magnitude worse. It is long past the time Tesla accepted that the "cameras & AI detects rain" gambit has failed.<<

I wouldn’t be at all surprised if this syndrome is the reason that Karpathy has left Tesla: he told Musk that so much of the magic technology is suspect and just isn’t going to be fixable.

 

[Durzel]

Let me refer you to the manual of the new BMW i4, a brand new car with all the latest technology

View attachment 836130

This is a sensitivity control for the rain sensor, it is needed to make the auto-wipers find the correct speed and it needs a human to operate it. It's not fully-automatic.

Ok, I will concede that you've found a brand new car that gives you fine tuning of the automatic rain sensor.

But, as said, in my experience on previous cars the adjustment simply made the manual interval faster or slower, it didn't attentuate the sensitivity of the rain sensor.

The way you describe it, that "it is needed", etc - this sensitivity wheel is a required part of the tech otherwise it basically doesn't work. That patently is not the case since plenty of cars have functional auto wiper settings without any calibration or sensitivity settings. I'd go as far as to say the i4 is an anomaly in that regard, not evidence of the fact the tech requires user calibration (otherwise every car with auto wipers would have the same system, which they don't).

Case in point - the Peugeot e-2008 I mentioned:

Anyway.. this is going off topic, suffice to say that a proper rain sensor absolutely does not need require end user calibration or adjustment. I can't explain why the i4 allows it, or what the end result is, since the sensor ought to be a "set and forget" thing in automatic mode and - in every car I care to remember owning or driving - it was.

 

[browellm]

I can't explain why the i4 allows it, or what the end result is, since the sensor ought to be a "set and forget" thing in automatic mode and - in every car I care to remember owning or driving - it was.

It's been that way forever on BMW. It allows you to set the sensitivity of the auto operation to user preference.

They use the same principle with their "Auto" A/C. It has 5 levels of strength. It's still automatic but operates within different parameters depending on your preference for the range of blower strength.

 

[Durzel]

Good to know, thanks. My experience has mostly been with Audi, although I had an i3. I can't remember how well the auto wipers worked on that so I presume they worked perfectly fine.

 

[Zilla91]

Premium brands like BMW and Audi give you the option to customise your car just the way you like it instead of deciding what’s best for you

 

[GRiLLA]

Ok, I will concede that you've found a brand new car that gives you fine tuning of the automatic rain sensor.

But, as said, in my experience on previous cars the adjustment simply made the manual interval faster or slower, it didn't attentuate the sensitivity of the rain sensor.

The way you describe it, that "it is needed", etc - this sensitivity wheel is a required part of the tech otherwise it basically doesn't work. That patently is not the case since plenty of cars have functional auto wiper settings without any calibration or sensitivity settings. I'd go as far as to say the i4 is an anomaly in that regard, not evidence of the fact the tech requires user calibration (otherwise every car with auto wipers would have the same system, which they don't).

Case in point - the Peugeot e-2008 I mentioned:

View attachment 836199

Anyway.. this is going off topic, suffice to say that a proper rain sensor absolutely does not need require end user calibration or adjustment. I can't explain why the i4 allows it, or what the end result is, since the sensor ought to be a "set and forget" thing in automatic mode and - in every car I care to remember owning or driving - it was.

Also from the e2008 manual

so basically you need to manually initiate the 'auto wipers' each time. My old BMW was the same, I assume they aren't confident enough that it won't mistakenly start wiping, and they are cheaping out on providing a sensitivity adjustment which they can do as it doesn't generally auto start. Maybe you were lucky that the speed felt 'about right' without adjustment, reading their forums its not always the case and they use some OBD connection app to adjust the sensitivity. Auto Wipers - Frustrating

 

[browellm]

I assume they aren't confident enough that it won't mistakenly start wiping, and they are cheaping out on providing a sensitivity adjustment which they can do as it doesn't generally auto start.

No, once again, it's good ergonomics. You don't want auto wipers starting in a car wash.

 

[guru1]

No, once again, it's good ergonomics. You don't want auto wipers starting in a car wash.

The other typical scenario is if you have an icy/snowy windscreen when you get to your car, you don't want the wipers coming on right away till you've cleared your screen.

 

[GeorgeSymonds]

Also from the e2008 manual

View attachment 836221

so basically you need to manually initiate the 'auto wipers' each time.

Yep - last thing you want to do is get in a car with a frozen windscreen and the wipers grind themselves into oblivion. They don’t do it like that because they can’t do it any other way.

With preconditioning that may be less likely but not guaranteed

 

[mtndrew1]

I wouldn’t be at all surprised if this syndrome is the reason that Karpathy has left Tesla: he told Musk that so much of the magic technology is suspect and just isn’t going to be fixable.

If Musk really believed that automatic wipers could be accomplished with vision they could have left the IR sensor in for a model year to train the network against a known value.

If they were seeing good results they could have deprecated the sensor in subsequent model years.

It all feels very Steve Jobs. Make the magic happen or you’re fired.

 

[DoricJim]

In all my BMWs with auto wipe there was a little button on the end of the wiper stalk that activated the auto wipers. This lit a little led on the stalk to let you know it was on auto. No need to touch again. And yes, every model I’ve had also allowed sensitivity adjustment using the up and down toggle.

Much better solution than the unpredictable Tesla wipers that you can’t fine tune…

Oh and the fully adaptive matrix LED headlamps on my 2017 3 series were pretty much turn on and forget. Aside from the occasional lorry coming towards you over the brow of a hill.

 

[GRiLLA]

The other typical scenario is if you have an icy/snowy windscreen when you get to your car, you don't want the wipers coming on right away till you've cleared your screen.

So these systems are not as automatic as Tesla's, and they rely on human drivers making decisions about starting and typically adjusting the sensitivity. I can see how with Tesla's self driving mission the use of a rain sensor didn't offer a solution and an alternative was implemented that could meet their design goal.

I'm not saying it works, just that for a car being built to be self driving the industry normal auto wiper technology didn't suit.

 

[mtndrew1]

So these systems are not as automatic as Tesla's, and they rely on human drivers making decisions about starting and typically adjusting the sensitivity. I can see how with Tesla's self driving mission the use of a rain sensor didn't offer a solution and an alternative was implemented that could meet their design goal.

I'm not saying it works, just that for a car being built to be self driving the industry normal auto wiper technology didn't suit.

If that’s the constraint the car could just do as it pleases when FSD is enabled.

 

[Battpower]

I'm not saying it works

There's a basis for quite a lot of agreement then!

Of course this isn't anything like an intentional strategy on Tesla's part, but it sure looks like it could be one...

Set the goal posts astronomically (? impossibly) high, claim technical superiority based on those aspirations that you haven't yet achieved, while being covered for failing to deliver because your goals are so high (bound to take time) and everything is still (stuck) in Beta - which of course you knew when you bought the car.

Yet as an engineer I 100% applaud people / companies for pushing the envelope, but it does feel to me like doing it to this extent with such basic features that can be delivered better / cheaper using other / proven technologies is stretching it too far. And of course Tesla have done so much that is staggering in scale and performance too.

It feels like while updates keep coming and coming, tweeking this and that, trying not to damage anything too significant and giving us all something to talk about, there are just too many significant weaknesses that should get fixed way sooner and with no drama.