Discussion: Tesla Vision system for Model 3/Y

[Tam]

...Maybe they already have...

Already started with new May 2021 radarless Model 3 and Y in North America.

Apparently the safety council has concerns and dropped Tesla from #1 in its safety rankings.

I am not sure about "concerns" but Tesla didn't submit the passing grades of radarless Model 3 and Y to them so they just downgrade the ratings.

The absence of "passing" does not mean "flunking".

Does anyone know more about this?

Tesla should.

Will this affect those cars which now have the radar system?

No, not immediately but eventually, Tesla will go radarless for all of its fleet and not just North America Model 3 and Y. The software will be able to disable the existing physical radars so we'll all have the radarless feature in the end.

Any info about this will be appreciated.

See Tesla blog as cited by @Gimmethecash

Why would Tesla do such a thing?

Tesla believes Tesla Vision is best and others such as radar and LIDAR are just crutches, not solutions.

Did the recent accidents have anything to do with this decision?

Which ones? The "brake failure" protest in China? The sudden acceleration lawsuit?

 

[Sackmonkey]

I picked up my radar-less M3P (no FSD) yesterday and already have 100 miles on it. Prior to this, I spent 18 months in a company-owned Model 3 LR RWD (2017) w/FSD and the updated FSD HW, so I have a lot of experience with AP for comparison.

The weather has been sunny at times and cloudy, but no rain since I picked the car up, so I can't comment on how bad the performance is in those compromised conditions. I will say this though, I typically don't trust AP in heavy rains anyway, so I'm not as concerned about this as others seem to be. I'm sure if I lived in Seattle or something, it would be a big deal to me. I haven't driven in the complete dark yet either, but the headlights did seem to come on earlier at dusk than I remember with the old car. I actually like that change, but agree that requiring auto-high beams will suck.

So far, the AP has worked exactly the same as my radar-equipped experience, including the same flaws. It seems to track the center of the lane exactly as well as before. When a merge/right turn lane opens up, the car wants to re-center itself across the 1 1/2 lane width (so it shifts to the right for a bit, annoyingly). As expected, I don't see two-cars-ahead anymore since the camera doesn't seem capable of doing that radar magic. Stop and go behind other cars feels just as smooth as before. The increased following distance (basically you can't set it to less than '3') doesn't matter to me in country driving because that a fairly rude following distance anyway. But in heavy traffic or on highways, I typically run at '1', so I'm hoping this issue is fixed very soon. Also, the 75mph speed limit sucks, but I couldn't care less about "emergency lane departure".

There is one very tricky corner nearby.. a very tight, decreasing radius turn on a country road that takes a typical driver down from 50 mph all the way to about 10mph. AP has about a 50/50 success rate around this corner. I've only tried it once with the new system and it failed - I had to take control to avoid overshooting the corner, even though the car had slowed WAY down. I can't imagine how radar could help on a corner like this though so I really doubt it's a factor. I'll try to hit it a few more times though and see if it gets better.

Last summer, with the old car, I suddenly started getting a lot of phantom braking events. I went from one event every 2 months to one per week. My wife refused to even ride in the car if I engaged AP because it was getting so bad. It seemed to happen mostly on very sunny days with very sharp tree shadows. It got better over the next year and was mostly fixed, but still about 1/month. I haven't experienced any phantom braking on the new car, but it's only been 100 miles so far, so too soon to tell.

Last night, Tesla pushed me the software update to enable the cabin camera. I'm curious how finicky this will be. Is it just looking for a warm body in the seat? Or will it freak out if I close my eyes or turn my head? I realize the camera is quite capable of all this, but not sure if Tesla has started using it to nanny up the AP. I sure hope not.

Anyway, I just wanted to share one person's perspective on the radar-less situation.

 

[id436eg9045]

I picked up my radar-less M3P (no FSD) yesterday and already have 100 miles on it. Prior to this, I spent 18 months in a company-owned Model 3 LR RWD (2017) w/FSD and the updated FSD HW, so I have a lot of experience with AP for comparison.

The weather has been sunny at times and cloudy, but no rain since I picked the car up, so I can't comment on how bad the performance is in those compromised conditions. I will say this though, I typically don't trust AP in heavy rains anyway, so I'm not as concerned about this as others seem to be. I'm sure if I lived in Seattle or something, it would be a big deal to me. I haven't driven in the complete dark yet either, but the headlights did seem to come on earlier at dusk than I remember with the old car. I actually like that change, but agree that requiring auto-high beams will suck.

So far, the AP has worked exactly the same as my radar-equipped experience, including the same flaws. It seems to track the center of the lane exactly as well as before. When a merge/right turn lane opens up, the car wants to re-center itself across the 1 1/2 lane width (so it shifts to the right for a bit, annoyingly). As expected, I don't see two-cars-ahead anymore since the camera doesn't seem capable of doing that radar magic. Stop and go behind other cars feels just as smooth as before. The increased following distance (basically you can't set it to less than '3') doesn't matter to me in country driving because that a fairly rude following distance anyway. But in heavy traffic or on highways, I typically run at '1', so I'm hoping this issue is fixed very soon. Also, the 75mph speed limit sucks, but I couldn't care less about "emergency lane departure".

There is one very tricky corner nearby.. a very tight, decreasing radius turn on a country road that takes a typical driver down from 50 mph all the way to about 10mph. AP has about a 50/50 success rate around this corner. I've only tried it once with the new system and it failed - I had to take control to avoid overshooting the corner, even though the car had slowed WAY down. I can't imagine how radar could help on a corner like this though so I really doubt it's a factor. I'll try to hit it a few more times though and see if it gets better.

Last summer, with the old car, I suddenly started getting a lot of phantom braking events. I went from one event every 2 months to one per week. My wife refused to even ride in the car if I engaged AP because it was getting so bad. It seemed to happen mostly on very sunny days with very sharp tree shadows. It got better over the next year and was mostly fixed, but still about 1/month. I haven't experienced any phantom braking on the new car, but it's only been 100 miles so far, so too soon to tell.

Last night, Tesla pushed me the software update to enable the cabin camera. I'm curious how finicky this will be. Is it just looking for a warm body in the seat? Or will it freak out if I close my eyes or turn my head? I realize the camera is quite capable of all this, but not sure if Tesla has started using it to nanny up the AP. I sure hope not.

Anyway, I just wanted to share one person's perspective on the radar-less situation.

Thank you for sharing your experiences so far!

 

[RDoc]

Coming from a data engineer who works with ML I can tell you that the radar feature set is just noise to a visual feature set. (Except for the cars in front of other cars.)

The real issue Tesla is most likely having is overpasses. The radar doesn't give you a Z coordinate only X and Y or 2 dimensions. Thus the radar is going to have features that tell the ML model that you are approaching an obstruction. The visual features are most likely compressed to the point where the radar is overriding. It sounds like even with millions of miles of training they still can't train the model to ignore overpasses. Which makes sense if the images it uses are compressed.

Thus dropping radar will remove this hinderance at the cost of losing 'cars in front of cars' features from radar.

I saw this exact issue last weekend in MD on the Turnpike in Western MD. Each overpass would cause the car to start breaking. The solution is to drop the radar.

Actually the radar does give Z (vertical) coordinates, it's much lower resolution though. At 100 yds the vertical resolution is about 6 ft.
Why would they compress the images? Do you have anything to back that up because all the literature I've ever read about artificial vision has emphasized feeding it raw video date without any preprocessing.
If the system can't recognize a feature as large and distinctive as an overpass it's pretty much useless.
IMHO a more believable reason for phantom braking at overpasses is that Tesla has never been able to get sensor fusion to work. OTOH, our car with Mobileye and radar has never braked for an overpass the entire time we've had it so apparently the Mobileye engineers are just a lot better.

 

[id436eg9045]

More relevant testing from Kevin Smith with the most recent software update.

 

[guzzijones12]

Actually the radar does give Z (vertical) coordinates, it's much lower resolution though. At 100 yds the vertical resolution is about 6 ft.
Why would they compress the images? Do you have anything to back that up because all the literature I've ever read about artificial vision has emphasized feeding it raw video date without any preprocessing.
If the system can't recognize a feature as large and distinctive as an overpass it's pretty much useless.
IMHO a more believable reason for phantom braking at overpasses is that Tesla has never been able to get sensor fusion to work. OTOH, our car with Mobileye and radar has never braked for an overpass the entire time we've had it so apparently the Mobileye engineers are just a lot better.

PCA or SVD is commonly used to compress images. IE: ML doesn't recognize a face is a face by looking at each pixel. It only needs to see the two eyes and the mouth.

 

[DGSteig]

Any rumors whether NHTSA testing will be completed soon? I am anxious that my insurance rates will be higher with those safety features disabled (according to them). Just got my VIN and have setup delivery on the 20th, wondering if I should call Erie today to beat any change of rates or wait until the last minute hoping things get reset.

 

[Accentcreate]

Personally I think you'd be mad to rely on autopilot or cruise control in the weather in the video. Full, two hands on wheel and 100% driver attention is required in those conditions.

 

[Gasaraki]

...

Last night, Tesla pushed me the software update to enable the cabin camera. I'm curious how finicky this will be. Is it just looking for a warm body in the seat? Or will it freak out if I close my eyes or turn my head? I realize the camera is quite capable of all this, but not sure if Tesla has started using it to nanny up the AP. I sure hope not.

...

Just don't blink. That's not allowed.

 

[Rattletrap]

Just don't blink. That's not allowed.

Can we block the nanny cam if we don't want Elon or anyone else watching us but still be able to use TACC/autosteer? Some of us are white people who drive around rapping along to our favorite beats. I max tint every car all the way around so no one can see this happening. I'm not allowing a live stream from the cabin of my car to Tesla but I'd still like to have basic autopilot...

 

[id436eg9045]

Can we block the nanny cam if we don't want Elon or anyone else watching us but still be able to use TACC/autosteer? Some of us are white people who drive around rapping along to our favorite beats. I max tint every car all the way around so no one can see this happening. I'm not allowing a live stream from the cabin of my car to Tesla but I'd still like to have basic autopilot...

According to the release notes turning off Data Sharing prevents the camera data from leaving the vehicle.

 

[techlogik]

View attachment 671730

According to the release notes turning off Data Sharing prevents the camera data from leaving the vehicle.

Someone else posted in the FSD section about this verbiage....tells me it's a double edge sword here.


"Please note that if you opt out from the collection of telematics log data or any other data from your Tesla vehicle, we will not be able to notify you of issues applicable to your vehicle in real time. This may result in your vehicle suffering from reduced functionality, serious damage, or inoperability, and it may also disable many features of your vehicle..."


Classic Tesla!!! Hey, we might turn your autopilot off if you cover your camera.....take that!

 

[Sackmonkey]

Two more days of evaluation under my belt and I'm not super impressed with TV. I went back to that difficult corner and tried it a couple more times. It basically drives off the road in one direction and goes across the centerline when going the opposite direction!! I can't imagine how radar could help to maneuver this intersection at all (no other cars around), but the car definitely seems less capable compared to the 2017 I was driving. I'm not sure why. There were a couple other roads that it handled okay, but it scared me a bit compared to prior experience. But I have also seen many times over the past couple years where AP regresses after an update and kind of sucks for a few weeks before yet another update gets it back on track, so personally I'm not super concerned yet.

As far as the cabin camera (spy), I couldn't seem to anger it (eyes closed; turning head away, etc.). I think it's just looking for a warm body in the seat to enable AP.

 

[stopcrazypp]

Someone else posted in the FSD section about this verbiage....tells me it's a double edge sword here.


"Please note that if you opt out from the collection of telematics log data or any other data from your Tesla vehicle, we will not be able to notify you of issues applicable to your vehicle in real time. This may result in your vehicle suffering from reduced functionality, serious damage, or inoperability, and it may also disable many features of your vehicle..."


Classic Tesla!!! Hey, we might turn your autopilot off if you cover your camera.....take that!

That's just standard CYA and my take is Tesla is referring to the data that is sent to Tesla in an accident. Sometimes Tesla will call you to see if you are ok or sometimes that data can be used in investigations where the EDR is destroyed.

 

[oreobbq]

Two more days of evaluation under my belt and I'm not super impressed with TV. I went back to that difficult corner and tried it a couple more times. It basically drives off the road in one direction and goes across the centerline when going the opposite direction!! I can't imagine how radar could help to maneuver this intersection at all (no other cars around), but the car definitely seems less capable compared to the 2017 I was driving. I'm not sure why. There were a couple other roads that it handled okay, but it scared me a bit compared to prior experience. But I have also seen many times over the past couple years where AP regresses after an update and kind of sucks for a few weeks before yet another update gets it back on track, so personally I'm not super concerned yet.

As far as the cabin camera (spy), I couldn't seem to anger it (eyes closed; turning head away, etc.). I think it's just looking for a warm body in the seat to enable AP.

Thanks for sharing. It's a bummer about not seeing two cars ahead. I want my radar to have radar so I can see everything 360 like a submarine.

 

[st_o_p]

Wake up and smell the roses, Tesla is going to disable your radar in previous models with a software update in the near future.

Almost certainly at some point they are going to disable the radar in cars that have it, so it will essentially just be just extra weight at the front

And when they try there will be a massive class action lawsuit against Tesla. And I will join it. And if not - I will start it.

They can gimp new cars all they want - it's a boneheaded move but if they'd rather sell sub-par cars rather than no cars due to chip shortage - so be it. But removing radar from existing car owners - that we purchased, paid for, and expect to be used in safety features like front collision warning - that is a BIG NO-NO. We are still the car owners, not Tesla.

 

[Willy’sEV]

And when they try there will be a massive class action lawsuit against Tesla. And I will join it. And if not - I will start it.

They can gimp new cars all they want - it's a boneheaded move but if they'd rather sell sub-par cars rather than no cars due to chip shortage - so be it. But removing radar from existing car owners - that we purchased, paid for, and expect to be used in safety features like front collision warning - that is a BIG NO-NO. We are still the car owners, not Tesla.

Tesla states that their Tesla Vision is superior to radar. That is not my statement, that is theirs.
Let us know when you sue them, I need a good chuckle.

 

[Lightbulb]

Having just picked up my 2021 model 3 in the 3rd week in March, mine is equipped with the radar. I have experienced phantom braking but not so much with under passes as with corners and seemingly random on the highway . I really don’t think Tesla vision is the problem, as radar cars experience this, I think it comes down to their code and software. Hopefully the next updates fix those issues regardless if it’s vision or radar based.

My biggest concern is needing to have auto brights on for auto pilot to work. Auto pilot was a huge selling feature and I use it multiple times a week on my communute. In my experience the auto brights have not worked 95/100 times. Everything seems to affect it, to where driving down the highway seems to just be a flashing session. I really hate using it as I have to pay so much attention to manually turning it off each time, that is defeats the purpose of auto pilot at all

Makes we want to not do future updates in fear of the car being downgraded

 

[st_o_p]

Tesla states that their Tesla Vision is superior to radar. That is not my statement, that is theirs.
Let us know when you sue them, I need a good chuckle.

Sure Willy, "Tesla states...". Full self-driving will be here before the end of year 2016 or whatever it was.

You know - I'm a big Tesla fan. But that doesn't mean I turn off my brain and start parroting ridiculous marketing statements. There are things that radar can see and camera cannot (particularly in low visibility conditions) - it's simply physics, whatever the Tesla sales guys say. That's why they use radar in military installations and not cameras.

The fact remains - we bought and paid for cars with specific safety features, including radar. If Tesla attempts to remove that from the cars we own - they will be in serious trouble, Microsoft may claim they own your Windows (and you're just leasing it) so they can disable features as they want. Tesla has no claim it owns the cars it sold - and it has zero right to disable anything on them.

Edit: And BTW - the guy few pages back was right. Tesla software updates were one of the greatest features, and a great reason to buy the cars. We all were looking forward to the cars getting better over time. But now - they are making people get scared of the next software update, and what it will disable. Looking for ways to turn software updates off. Turning one of the best reasons to buy the car into a reason NOT to buy it. At least when I buy a BMW I know it will always have its radar. It is a really bad move for Tesla, If they have some sense - they will come out and very clearly state that software updates will never disable existing features - otherwise they are simply doing massive damage to themselves.

 

[cylonmode]

Sure Willy, "Tesla states...". Full self-driving will be here before the end of year 2016 or whatever it was.

You know - I'm a big Tesla fan. But that doesn't mean I turn off my brain and start parroting ridiculous marketing statements. There are things that radar can see and camera cannot (particularly in low visibility conditions) - it's simply physics, whatever the Tesla sales guys say. That's why they use radar in military installations and not cameras.

The fact remains - we bought and paid for cars with specific safety features, including radar. If Tesla attempts to remove that from the cars we own - they will be in serious trouble, Microsoft may claim they own your Windows (and you're just leasing it) so they can disable features as they want. Tesla has no claim it owns the cars it sold - and it has zero right to disable anything on them.

Exactly... I hope Tesla Vision is successful and it surpasses the need for radar. But until then... Leave my car alone, if I want a newer Tesla with no radar I'll buy one. This is like Ford or Chevy knocking on your door telling you they need to take something off your vehicle, it doesn't happen.