Tesla Cameras' Coverage and Field-of-View for AP

[Three3]

Hi All,

One thing that makes me a bit nervous is the AP blind spots that Tesla seems to have at the front two corners. Cars sometimes "disappear" (albeit less frequently now with the latest software update) from my screen as I pass them, and I think it's due at times to these blindspots. If you look at the videos posted by Green, or on the Tesla Autonomy Day video, you'll note that the actual image that the fisheye sees is much less than what the autopilot paints as driveable space. In other words, the fisheye camera's field of view is constrained by the camera housing on the sides and bottom (and you have to look very closely to see this as at first it looks like the camera is seeing more than it really is). So for objects close to the car and low, such as wheels of cars, these objects will "disappear" when they pass by the fish eye camera and before they get to the pillar cams.

It is curious as to why Tesla didn't widen and extend the bottom of the camera housing slightly as this would have improved the field of view, particularly on the sides. And I realize that the camera still sees "most" of the car, but in the case of very small cars (smart cars) or other objects, it's still curious and disconcerting.

I'm no fan of XPeng's, but they seem to have much better coverage, as evinced by their video.

I don't know why I raise this other than it's curious (or perhaps someone has video that shows that Tesla's cameras' field-of-view is somehow wider than what is actually being portrayed in these videos??)

Tesla's cameras, via greentheonly (many other great examples on his youtube channel):

Xpeng:

 

[thewishmaster]

There isn't much that is unseen by the overlapping fields of view:

(see around 22:09)

There are some better examples of still photos in some threads here that show how much overlap there ends up being. There's a little bit of a blind spot maybe directly in front of the front bumper and right next to the front fenders, but not many things can spontaneously appear there (in most driving situations).

 

[Three3]

I guess I just question why are there any blind spots, and why the camera housing winds up taking up valuable pixels that could have been used for the road.

I assume it’s an aesthetic design decision, but, if you’re going to base your entire AP approach on cameras, and if you want a level of safety above that of humans, it seems to me you’d want at least complete coverage, and likely even a degree of overlap at that. I’m impressed how with the same number of cameras as Tesla (at least the ones that are visible in the above video) Xpeng clearly has better coverage and even a high degree of overlap. And I’m surprised Tesla does not.

Things like this make me wonder if when HW4 is released if Tesla doesn’t at least slightly adjust the cameras as well.

 

[animorph]

Since there is no camera in the front bumper, there's a small blindspot that might be nice to fill for parking situations.

Otherwise, in the greentheonly video I'm not seeing any blindspots, nor the obstruction of the wide angle camera, though I have seen that before. Looking at signs going by, they appear in the B-pillar camera before disappearing from the front camera.

It looks pretty much like my normal visibility from within any car without camera aids, only better.

 

[Joel]

I guess I just question why are there any blind spots, and why the camera housing winds up taking up valuable pixels that could have been used for the road.

I assume it’s an aesthetic design decision, but, if you’re going to base your entire AP approach on cameras, and if you want a level of safety above that of humans, it seems to me you’d want at least complete coverage, and likely even a degree of overlap at that. I’m impressed how with the same number of cameras as Tesla (at least the ones that are visible in the above video) Xpeng clearly has better coverage and even a high degree of overlap. And I’m surprised Tesla does not.

Things like this make me wonder if when HW4 is released if Tesla doesn’t at least slightly adjust the cameras as well.

Perhaps if you view the B pillar cameras, that may answer your question. You posted Green’s video with 5 cameras (tri-camera and two rear facing fender cameras). You are not accounting for the B pillar cameras and the rear facing camera.

 

[Three3]

Perhaps if you view the B pillar cameras, that may answer your question. You posted Green’s video with 5 cameras (tri-camera and two rear facing fender cameras). You are not accounting for the B pillar cameras and the rear facing camera.

Look again, video of his that I posted is of the fisheye, the medium camera, the B Pillars, and the repeater cams.

There is much better coverage of the rear of the car, in fact, given the position of the rear facing fisheye and how the repeater cams jut out.

You can see the blind spots, animorph, if you go to about a minute into the video and do some freeze frames of the barrels on the left hand side. There are definitely frames where barrels are mostly, if not completely, occluded, from when they go to the fisheye to the B pillars.

 

[ramonneke]

Don't forget that there is also sonar. These sensors are also used by AP to map its world.

Also, the fact that there is a 'blindspot' doesn't mean AP can't know what there should be. If you see something entering the blindspot you know where it is. So even it the vision sensors and the sonor sensors don't see something anymore doesn't mean that the system can't be aware of it.

However, it is very unlikely that the current firmware knows but I assume that the upcoming 360 view will have these sensors combined and does know.

 

[heltok]

Tesla has 360° camera coverage:

 

[Three3]

The cameras do not have consistent 360 coverage. If you watch the video I posted, and watch for cones (there’s a stretch of them toward the end to the right side of the vehicle) you will definitely find frames where a cone disappears into the blind spots.

it is easy to imagine scenarios in which this becomes an issue. Not necessarily driving past static cones on the highway, but if the car is parked, and for smaller moving objects. I also wonder if depending on how the car is angled (both on the x but also the y axis) it is possible for cars approaching on that angle to be more or less be invisible to the Tesla. I have had a few instances, and have seen a few online, in which the car tries to switch lanes and doesn’t “see” the car it is about to smash into the corner of. This has admittedly become a rarer occurrence, however, so fingers crossed these blind spots are not insurmountable. My doubts remain.

Here’s an example from Scott Kubo of where dynamic moving objects may “find” and live in that blind spot. (Perhaps this was more a case of the software needing to be adjusted, and maybe it’s been fixed, but I do wonder.)

 

[Mo City]

Question: Do the repeater cameras really overlap as illustrations of their coverage seem to indicate? If so, how far behind the car? My concern is the rear camera often becomes useless during rain and drizzle. When this happens, do the repeaters provide enough rear vision on their own?

 

[heltok]

The cameras do not have consistent 360 coverage. If you watch the video I posted, and watch for cones (there’s a stretch of them toward the end to the right side of the vehicle) you will definitely find frames where a cone disappears into the blind spots.

Car has 360° coverage. HW2.5 neural networks don’t use all cameras though and each camera is labelled individually and thus it might struggle with detecting cars in some angles. With 3D labelling and recurrent neural networks this should be much improved.

 

[Watts_Up]

The cameras do not have consistent 360 coverage. If you watch the video I posted, and watch for cones
(there’s a stretch of them toward the end to the right side of the vehicle)
you will definitely find frames where a cone disappears into the blind spots.
it is easy to imagine scenarios in which this becomes an issue. Not necessarily driving past static cones on the highway, but if the car is parked, and for smaller moving objects. I also wonder if depending on how the car is angled (both on the x but also the y axis)
it is possible for cars approaching on that angle to be more or less be invisible to the Tesla.
I have had a few instances, and have seen a few online, in which the car tries to switch lanes
and doesn’t “see” the car it is about to smash into the corner of.
This has admittedly become a rarer occurrence, however, so fingers crossed these blind spots are not insurmountable. My doubts remain.

Here’s an example from Scott Kubo of where dynamic moving objects may “find” and live in that blind spot.
(Perhaps this was more a case of the software needing to be adjusted, and maybe it’s been fixed, but I do wonder.)

There is absolutely a front side corner blind spot.
I experience this problem everyday when I dry to exit from my driveway and I try to turn right to enter into the street.

The car parked on the side are blocking the view and to be able to see the traffic coming from my left,
I need to move the front of my car into the traffic lane.

To avoid to move the front of my car into the traffic, I need to see through the windshield of the car parked on my right.
If someone put a windshield cover, exiting from my driveway become hazardous.
Also try to teach a Self Diving Car to look through a windshield to see cars approaching from the left...

Note: To solve this issue, I installed a small camera under my front license plate, looking perpendicularly on the side of the car.
Using this camera, I can stop my car at the edge of the street lane, and check the coming traffic.
This avoided me in particular some collision with electric bicycles or electrical scooter going very fast and barely noticeable.

 

[moa999]

Bjorn with a video of the Xpeng, including 360degree overhead,. And parking in a vacant carpark

 

[SilverSp33d3r]

The cameras do not have consistent 360 coverage. If you watch the video I posted, and watch for cones (there’s a stretch of them toward the end to the right side of the vehicle) you will definitely find frames where a cone disappears into the blind spots.

it is easy to imagine scenarios in which this becomes an issue. Not necessarily driving past static cones on the highway, but if the car is parked, and for smaller moving objects. I also wonder if depending on how the car is angled (both on the x but also the y axis) it is possible for cars approaching on that angle to be more or less be invisible to the Tesla. I have had a few instances, and have seen a few online, in which the car tries to switch lanes and doesn’t “see” the car it is about to smash into the corner of. This has admittedly become a rarer occurrence, however, so fingers crossed these blind spots are not insurmountable. My doubts remain.

Here’s an example from Scott Kubo of where dynamic moving objects may “find” and live in that blind spot. (Perhaps this was more a case of the software needing to be adjusted, and maybe it’s been fixed, but I do wonder.)

Tesla has 3 forward facing cameras, currently only mid and long range views are used. If you look at all posted by tesla on this you can clearly see this.

 

[mikes_fsd]

Tesla has 3 forward facing cameras

a small nit...
Tesla has 5 forward facing cameras.
The B-pillar cameras are forward facing and expand the FOV (field of view)

 

[alseT Motors]

a small nit...
Tesla has 5 forward facing cameras.
The B-pillar cameras are forward facing and expand the FOV (field of view)

Hi All,

One thing that makes me a bit nervous is the AP blind spots that Tesla seems to have at the front two corners. Cars sometimes "disappear" (albeit less frequently now with the latest software update) from my screen as I pass them, and I think it's due at times to these blindspots. If you look at the videos posted by Green, or on the Tesla Autonomy Day video, you'll note that the actual image that the fisheye sees is much less than what the autopilot paints as driveable space. In other words, the fisheye camera's field of view is constrained by the camera housing on the sides and bottom (and you have to look very closely to see this as at first it looks like the camera is seeing more than it really is). So for objects close to the car and low, such as wheels of cars, these objects will "disappear" when they pass by the fish eye camera and before they get to the pillar cams.

It is curious as to why Tesla didn't widen and extend the bottom of the camera housing slightly as this would have improved the field of view, particularly on the sides. And I realize that the camera still sees "most" of the car, but in the case of very small cars (smart cars) or other objects, it's still curious and disconcerting.

I'm no fan of XPeng's, but they seem to have much better coverage, as evinced by their video.

I don't know why I raise this other than it's curious (or perhaps someone has video that shows that Tesla's cameras' field-of-view is somehow wider than what is actually being portrayed in these videos??)

Tesla's cameras, via greentheonly (many other great examples on his youtube channel):

Xpeng:

a small nit...
Tesla has 5 forward facing cameras.
The B-pillar cameras are forward facing and expand the FOV (field of view)

a small nit...
Tesla has 5 forward facing cameras.
The B-pillar cameras are forward facing and expand the FOV (field of view)

Tesla Cameras' Coverage and Field-of-View for AP Does not cover enough distance to determine safe entrance from stop signs to State Route highways, and other situations with traffic that is traveling multiple times the FSD Tesla's speed. Additional camera coverage distance is needed to sense these situations sooner. Camera hardware and position will need to be updated and until then the steering wheel will be needed to take over. The sensors are not able to look down the road as far as needed. I have to look down the road much further than the sensors are capable