Major FSD camera suite oversight

[powertoold]

Yesterday while driving home, I encountered a left turn from a main street into a residential street that would be unsafe with the current FSD cameras suite.

Essentially, the left turn is made in a left turn only lane. However, there is another left turn lane for the opposing traffic that is facing me but shifted about half a car width to the left.

When there is another car in the opposing left turn lane, it blocks the view of the opposing traffic, and in order to see the opposing traffic, I have to put my head close to the driver's side window.

With the current camera suite, there are only cameras in the middle of the car that face forward. There's no way for the current Tesla cameras to see forward from the sides of the car, whereas with the rear repeater cameras, they can see all the way behind the car.

Does anyone here have a solution to this problem with the current camera suite?

 

[Tam]

...Does anyone here have a solution to this problem with the current camera suite?

Right now, it's just speculation.

Others say Tesla needs to have LIDAR.

Tesla says no way, LIDAR is a crutch.

Both sides can argue all they want but we won't know until we can actually use them in real life and post them on Youtube.

 

[neroden]

Good catch. I expect in the long run they'll need additional cameras. (Which are fairly cheap, but it does mean another hardware revision.)

 

[diplomat33]

Yesterday while driving home, I encountered a left turn from a main street into a residential street that would be unsafe with the current FSD cameras suite.

Essentially, the left turn is made in a left turn only lane. However, there is another left turn lane for the opposing traffic that is facing me but shifted about half a car width to the left.

When there is another car in the opposing left turn lane, it blocks the view of the opposing traffic, and in order to see the opposing traffic, I have to put my head close to the driver's side window.

With the current camera suite, there are only cameras in the middle of the car that face forward. There's no way for the current Tesla cameras to see forward from the sides of the car, whereas with the rear repeater cameras, they can see all the way behind the car.

Does anyone here have a solution to this problem with the current camera suite?

The forward wide angle camera can take care of this. And, the car can wait a bit until the view is clear (blocking car moves out of the way).

Not a problem.

 

[MattMatt]

I think you present a valid point and there are many more scenarios when the center cameras forward also are questionable. Another example being if there is a semi-truck in front of you the amount of lane markings in the forward direction the car can see is reduced, a curve wouldn't be identified until the truck already starts to make it.

I've been puzzling a bit on why there aren't cameras on the extreme left and right upper windshield. Seems like logical places to see lane markings, see around the car in front of you, and would address the scenario you describe.

 

[diplomat33]

Another example being if there is a semi-truck in front of you the amount of lane markings in the forward direction the car can see is reduced, a curve wouldn't be identified until the truck already starts to make it.

I encountered this issue with AP. AP did ping pong a bit but corrected itself in time. (Note: this was before 24.4, the issue may be fixed now).

I do think that HD maps could probably solve this problem too instead of extra cameras.

 

[destructure00]

Isn't the front facing radar capable of seeing around the car in front of you already?

 

[Az_Rael]

Yesterday while driving home, I encountered a left turn from a main street into a residential street that would be unsafe with the current FSD cameras suite.

Essentially, the left turn is made in a left turn only lane. However, there is another left turn lane for the opposing traffic that is facing me but shifted about half a car width to the left.

When there is another car in the opposing left turn lane, it blocks the view of the opposing traffic, and in order to see the opposing traffic, I have to put my head close to the driver's side window.

With the current camera suite, there are only cameras in the middle of the car that face forward. There's no way for the current Tesla cameras to see forward from the sides of the car, whereas with the rear repeater cameras, they can see all the way behind the car.

Does anyone here have a solution to this problem with the current camera suite?

How far forward do the B pillar cameras see? Those are farther over than you in the driver seat, so they may work.

 

[willow_hiller]

Isn't the front facing radar capable of seeing around the car in front of you already?

Yep. It actually bounces the radar underneath the car in front of you to see vehicles in front of the vehicle in front of you: Tesla Autopilot under v8.0 really can see two if not three cars ahead

 

[Cirrus MS100D]

3 rights make a left?

 

[cucubits]

I've seen similar intersections with two turning lanes from each side and no clear lane markings. I don't think the current hardware will ever be capable of properly navigating these, no matter how much they improve the software and no matter how hard Elon or his 'believers' want to.

 

[boonedocks]

3 rights make a left?

Just in...Tesla taking a cue from UPS for FSD.

Why UPS drivers don't turn left and you probably shouldn't either

 

[willow_hiller]

Just in...Tesla taking a cue from UPS for FSD.

Why UPS drivers don't turn left and you probably shouldn't either

It's actually not a completely bad point. It would be a good way to get to level 4 autonomy:

"The car can operate without human input or oversight but only under select conditions defined by factors such as road type or geographic area."

 

[powertoold]

The difference in the UPS case is that they can turn left, but in the interest of time, they don't. In my example, there's no real safe way for the car to turn left.

As for radar, I'm not sure the sweeping angle of the radar can adequately catch the opposing cars. Plus, I don't think the radar can see underneath the car in front when their car in front is not really close by. Take a look at the sensor suite from the Tesla autopilot webpage attached below.

As for the b-pillar cameras, they don't see in front of the car or on the front side of the car. Again, check out the sensor suite picture below.

I think the only safe way to turn left in this case is to wait until the opposing left turn lane is clear.

 

[ChrisPDX]

This is a case of a poorly designed and therefore unsafe intersection. If it's not safe for human drivers, it probably won't be safe for the FSD system. As noted above, the FSD will probably just have to sit and wait until there is no traffic in the other turn lane before confirming if it can proceed or not. Hopefully the FSD team adds smart learning so once a few FSD cars encounter this, the intersection can be tagged as unsafe and all future routing is done to avoid it. Google already does this now in that it usually avoids having you turn left through a busy intersection unless there's a light.

 

[willow_hiller]

The difference in the UPS case is that they can turn left, but in the interest of time, they don't. In my example, there's no real safe way for the car to turn left.

As for radar, I'm not sure the sweeping angle of the radar can adequately catch the opposing cars. Plus, I don't think the radar can see underneath the car in front when their car in front is not really close by. Take a look at the sensor suite from the Tesla autopilot webpage attached below.

As for the b-pillar cameras, they don't see in front of the car or on the front side of the car. Again, check out the sensor suite picture below.

I think the only safe way to turn left in this case is to wait until the opposing left turn lane is clear.

You don't think the wide-angle forward facing camera would be able to see to the left and right of a vehicle blocking the immediate view in front?

 

[powertoold]

You don't think the wide-angle forward facing camera would be able to see to the left and right of a vehicle blocking the immediate view in front?

No, it's not possible to see the opposing traffic from the middle vantage point of the camera. You need to move the camera to the left side of the car facing forward.

 

[powertoold]

This is a case of a poorly designed and therefore unsafe intersection. If it's not safe for human drivers, it probably won't be safe for the FSD system. As noted above, the FSD will probably just have to sit and wait until there is no traffic in the other turn lane before confirming if it can proceed or not. Hopefully the FSD team adds smart learning so once a few FSD cars encounter this, the intersection can be tagged as unsafe and all future routing is done to avoid it. Google already does this now in that it usually avoids having you turn left through a busy intersection unless there's a light.

Unfortunately, FSD has to work well in an imperfect world

 

[Twiglett]

good point - some folks seem to be expecting that FSD will drive like them.
FSD is just as likely to NOT drive like a human.
If you aren't driving why does it matter what route the car takes or how fast it travels around a particular section of road.
So in this case to obvious fix is - don't drive that route, making sure the FSD response to a difficult situation is to drive past and come back the other way.
In the same way that FSD may not break road rules or traffic law etc

 

[powertoold]

good point - some folks seem to be expecting that FSD will drive like them.
FSD is just as likely to NOT drive like a human.
If you aren't driving why does it matter what route the car takes or how fast it travels around a particular section of road.
So in this case to obvious fix is - don't drive that route, making sure the FSD response to a difficult situation is to drive past and come back the other way.
In the same way that FSD may not break road rules or traffic law etc

I think the flaw here is that in terms of perception, FSD has to be superior to a human in every way. But in this case, a human can see what the car can't. And the example presented here may not be the only case where this may cause problems.

And this is why pretty much every other FSD testing platform out there has sensors on the edges of the car that can see forward.