Tesla.com - "Transitioning to Tesla Vision"

[diplomat33]

I mostly agree with this but to play devils advocate, tesla vision successfully ignored a box while the radar thought the box was a car. It demonstrates the weakness of radar, it has no idea what it is looking at! But also the strength of radar, it can cover for something the vision system, can't classify, just with poor probabilities and insufficient confidence to actually take action other than a warning.

Yes, camera vision and radar have different roles. Radar cannot classify objects. Camera Vision can. But good radar can give you very accurate position and velocity, even in low visibility conditions. So camera vision and radar can be complementary.

It should also be noted that Tesla's radar is poor. I don't think all radar would be that bad at detecting that object. There are higher resolution radars that would do a better job, I think. But to improve radar, Tesla would need to upgrade to better radar which would require either retrofitting all existing cars like they did with the FSD computer (expensive and time consuming and ties up service centers) or adding new radar to new cars and rendering existing cars obsolete. But with camera vision, they don't need to upgrade the cameras and can improve the performance with just better software. So focusing on Tesla Vision makes sense from that angle.

Tesla vision also successfully errors on the side of caution and didn't hit a fake pedestrian, so in an actionable instance no change. I wonder how much general object detection will be in V9? Or if the current vision engine is basically V9, just without the city streets driving policy.

I am guessing V9 will have more object detection since it is designed to be more "FSD". And if V9 is super good at object detection and such, it would make sense to use V9 for basic AP and just remove the city street stuff. After all, it should be the same underlining software.

 

[powertoold]

I'm in awe at this new vision-only system. The car:

1) recognizes this completely made up pedestrian / box / obstacle about 20m away
2) is able to gauge the distance of it, despite the NN was probably never trained on random fake pedestrian boxes
3) smoothly slows down the car and keeps the box object stable and persistent throughout the slow down.

https://twitter.com/x/status/1399192853234995201

 

[stopcrazypp]

Yeah, you nailed it. There are other factors at play: cars on the road.
The anecdotes I saw were the mist/ cloud kicked up by vehicles ahead of the Tesla impeded the AP’s vision such that it disabled. Not that it can’t keep active on an empty straight highway in the rain.

Aka without radar it lost “lock” / location awareness of those vehicles due to road mist.

That doesn't explain it though, given the video in question of it working fine has a car in the road kicking up mist also.

 

[Knightshade]

I'm in awe at this new vision-only system.

Were you equally in awe when he did the same test in manual mode and it didn't warn him or attempt to slow him down from running the pedestrian over?

(same test done with a radar car produced emergency braking on manual driving)

https://twitter.com/x/status/1399192915243569152

 

[MP3Mike]

Were you equally in awe when he did the same test in manual mode and it didn't warn him or attempt to slow him down from running the pedestrian over?

(same test done with a radar car produced emergency braking on manual driving)

https://twitter.com/x/status/1399192915243569152

Other than he wasn't driving at a constant speed. He was slowing down as he approached it, so the system may have thought the driver was aware and would handle it.

 

[powertoold]

Were you equally in awe when he did the same test in manual mode and it didn't warn him or attempt to slow him down from running the pedestrian over?

I'm mostly pointing out the advancement in vision-only distance and position predictions, not about pedestrian AEB.

Do you think the vision-only car had no ped warning in manual mode because of a weakness in vision vs radar?

 

[daktari]

https://twitter.com/x/status/1399101535775121409

Translation per Twitter:

"Tesla Vision" test video in the heavy rain. The autopilot works fine.

Is this a chinese made example, driving in China? Do China cars also lack radar? I can't see the location on the map.

 

[Mobile3228]

No, they have radar, but my MIC SR+ shows everything upside down because I'm located in the northern hemisphere.

 

[powertoold]

Is this a chinese made example, driving in China? Do China cars also lack radar? I can't see the location on the map.

That's Japanese language, and I think it's in New Jersey (interstate 78 east).

According to Tesla, MIC cars should still have radar.

 

[powertoold]

Tesla Vision, first build 2021.4.15.11, seems to be at least on par with radar performance in good weather with the 75mph limitation. I do notice the cars popping into and out of existence near the b-pillar and fisheye camera blindspot (although this shouldn't be a radar limitation b/c radar doesn't see there either):

 

[Knightshade]

I'm mostly pointing out the advancement in vision-only distance and position predictions, not about pedestrian AEB.

Do you think the vision-only car had no ped warning in manual mode because of a weakness in vision vs radar?

Honestly it's confusing- you'd THINK if it can know "THIS IS SOMETHING I SHOULD STOP FOR INSTEAD OF RUN OVER" in AP mode, it would also at least give an FCW or AEB activity in manual mode.... that's what the radar vehicle does.

It seems like that's more bad programming to know that in one mode but not do anything with the other but we can't be sure.

 

[Cheburashka]

I hit a box on the freeway the other day in AP. Cracked my bumper too. No warnings at all.. hoping this vision stuff works out for things like this. Theres lots of garbage out on the freeway.

 

[orion2001]

The forced auto-highbeam with AP enabled for Tesla-vision cars is absurdly bad judging from this video. I can understand now why some folks are up in arms over this given that they have no way to disable this without also being forced to turn off AP.

 

[DayTrippin]

Those high beams flash so much in traffic it could give people seizures. Just kidding but that is terrible. How soon until they remove AP too? Is the car so blind in low light conditions that the AP can't function without them on? How was it able to work before at night?

Or is it just Tesla playing it safe so to speak after making everyone safety feature beta testers?

 

[rjpjnk]

Use of high beams to support AP has to go. Totally unacceptable.
Can you even run basic non AP cruise control without auto HB?

 

[texas_star_TM3]

Tesla Vision, first build 2021.4.15.11, seems to be at least on par with radar performance in good weather with the 75mph limitation. I do notice the cars popping into and out of existence near the b-pillar and fisheye camera blindspot (although this shouldn't be a radar limitation b/c radar doesn't see there either):

in perfect weather, on a perfectly straight road with perfect vision in broad sunlight the cameras work well.... wow.... I'm blown away. I don't think anyone here ever doubted *this* use case when questioning the removal of radar which tends to shine at night / low visibility / heavy rain downpours...

 

[texas_star_TM3]

How was it able to work before at night?

... Radar + Vision

 

[mikes_fsd]

in perfect weather, on a perfectly straight road with perfect vision in broad sunlight the cameras work well.... wow.... I'm blown away. I don't think anyone here ever doubted *this* use case when questioning the removal of radar which tends to shine at night / low visibility / heavy rain downpours...

Take it easy there grasshopper, he is just posting what is available right now. As more ppl get their new Model 3/Y's we will see all those scenarios as well!

 

[clydeiii]

I have not yet. Near the end of the ride home it had stopped raining and still required it though. The wording of the warning makes me believe that the requirement has nothing to do with weather so I would be VERY surprised if it allowed them to be turned off at any time. I did just drive to Costco and noticed even during the day it switches auto on as soon as engaging AP. That being said, no I have not explicitly been able to test clear weather at night.
I am almost definitely going to have to make the same trip tomorrow though and it’s supposed to be clear so will report back if I remember how it handles it.

It's also worth considering that this requirement will be removed within Two Week(TM), which is the time they'll be validating Tesla Vision Only works. Upon validation, they remove the highbeam requirement.

 

[gearchruncher]

Upon validation, they remove the highbeam requirement.

Source?

Vision only is first principles- it drives like a human does. That means it relies on the same things humans do to drive safely as well.

My understanding was this is a permanent change. With a visual wavelength, vision only system, you have to have light for the system to work. How can a camera see far ahead if it has no headlights to illuminate the road, and why would Tesla trust drivers to set the highbeams correctly? People will just leave them off when AP is active because they "don't need them".