AP2.0 Cameras: Capabilities and Limitations?

[_jal_]

Combined video of main camera replays from all rain snapshot

(Remember that the quality of compressed replay videos is bad, significantly worse than the uncompressed still images from the same camera, this is just more interesting to watch.)

That movie reminds me of The Ring movie for some reason

 

[Saghost]

Repeaters face backwards, B pillar cameras face forwards... they are further out than the drivers head can possibly be unless you stick your head out of the window. I'm not seeing any problem.

Cruise Automation demo
2:30

(and this wasn't just because it had lidar)

Bounce radar under the truck to see oncoming traffic?

TACC already watches the car ahead of the car in front of you that way.

Edge left in the lane until the camera clears the truck side?

You may be absolutely confident in your conclusions, but the rest of us aren't.

 

[chillaban]

That movie reminds me of The Ring movie for some reason

I know right? Did you catch a couple of frames with the upside down chair and the horse? Hold on my phone is ringing...

 

[Saghost]

Bounce radar under the truck to see oncoming traffic?

TACC already watches the car ahead of the car in front of you that way.

Edge left in the lane until the camera clears the truck side?

You may be absolutely confident in your conclusions, but the rest of us aren't.

Oops. Sorry, JeffK. I was doing this from my phone, and quoted the wrong post. It was really intended to be a reply to the post you were quoting. :-/

 

[Bladerskb]

Repeaters face backwards, B pillar cameras face forwards... they are further out than the drivers head can possibly be unless you stick your head out of the window. I'm not seeing any problem.

Cruise Automation demo
2:30

(and this wasn't just because it had lidar)

This is incorrect. The B pillar cameras face the side not forward. They can't see around the truck in that scenario.

Why in the world would you post a video of a cruise automation which have diversity of sensors and different sensor location.

They have lidar, radar and camera on the side of the cars and can see out and around the car.

 

[JeffK]

This is incorrect. The B pillar cameras face the side not forward. They can't see around the truck in that scenario.

Read the part that says Forward Looking Side Cameras (the green part clearly extends far past the front of the car too.)

Assuming the car doesn't pull too close, if the main forward camera is blocked, the wide forward camera or the driver's side forward facing side camera might be able pick up the next lane enough so that the car can safely pivot to get a better look.

 

[bswn1]

There are honestly 2 - 3 situations per week where I ask myself how Tesla will solve for this with current sensor suite.

The situation you described is far from one of those situations.

 

[Bladerskb]

Read the part that says Forward Looking Side Cameras (the green part clearly extends far past the front of the car too.)

Assuming the car doesn't pull too close, if the main forward camera is blocked, the wide forward camera or the driver's side forward facing side camera might be able pick up the next lane enough so that the car can safely pivot to get a better look.

I never contested what tesla is calling it. I'm simply showing you its FOV and direction. its not forward looking. its side looking and simply slanted a lil one way.

This camera will not be any help in the situation that was brought up which is parked behind a large truck.
because it won't be able to see upcoming cars, its not even seeing lanes that's the problem.
Also it won't take a lil peeking either since the side looking cams aren't slanted enough and the forward looking cameras are in the center.

no matter how you try to twist it. it won't work.
It will literally have to turn into the other lane till the forward facing cameras have a direct LOS

 

[JeffK]

I never contested what tesla is calling it. I'm simply showing you its FOV and direction. its not forward looking. its side looking and simply slanted a lil one way.

That one way is forward.
see this post for absolute confirmation it's pointing both forward and to the side
AP2.0 Cameras: Capabilities and Limitations?
You can clearly see overlap in the fisheye and pillar cameras.

This camera will not be any help in the situation that was brought up which is parked behind a large truck.
because it won't be able to see upcoming cars, its not even seeing lanes that's the problem.

My point is that it can see what's in the other lane better than a human as is (unless you stick your head out of the window as I mentioned).

 

[verygreen]

My point is that it can see what's in the other lane better than a human as is (unless you stick your head out of the window as I mentioned).

I don't think so, it's not tilted forward as much for that.
E.g. in this one: View attachment 245201 we can see there's chevy car in the right lane a bit closer to us than the truck.

Now looking at the right pillar camera: View attachment 245200
That car is not seen.

Now imagine the trick is much closer to us and a semi - if we don't see that chevy, we won't see anything in front of it either even if we partially see the chevy on the forward-looking cam somewhat.

This is not the argument that fsd could not be done, just the camera positioning thing.

Datapoint, in Russia (left hand drive) it was very popular to import cars from Japan (right hand drive) because they were super cheap. Also in Russia there are a lot of highways that only have two lines. Do you think people were not able to overtake? Nope, they just shift a bit to the left until they are able to see enough forward to confidently overtake, all while being prepared to hide behind a semi if obstacle appears.
If people are able to do it, computer should be able to do it as well with much better reaction time (eventually).
Also, it's illegal to use those Japanese cars in Russia now on sefety grounds and there are still lots of people very upset about it

 

[JeffK]

E.g. in this one: View attachment 245201 we can see there's chevy car in the right lane a bit closer to us than the truck.

I see what you are saying. That side camera can't see the dark vehicle. might have to turn slightly and rely on radar.
The white car, however, is visible in both shots.

The cruise automation vehicle Albatross (the model in the video) has corner radars

I've always thought the Model 3 needed corner radars. I'm kind of disappointed they didn't add them.

 

[arnis]

Repeaters face backwards, B pillar cameras face forwards...

Somewhat forwards but not straight ahead where car in the opposite lane is positioned.

Bounce radar under the truck to see oncoming traffic?
Does not work that way reliably.

TACC already watches the car ahead of the car in front of you that way.

Edge left in the lane until the camera clears the truck side?

You may be absolutely confident in your conclusions, but the rest of us aren't.

40% over the middle lane? That is a lot.
I have thought about that for weeks. I think that others are just assuming

Read the part that says Forward Looking Side Cameras (the green part clearly extends far past the front of the car too.)

Green part does overlap with two main frontal cameras.

 

[stopcrazypp]

I see what you are saying. That side camera can't see the dark vehicle. might have to turn slightly and rely on radar.

Actually there's another way that's kind of counter intuitive. If you turn the car to the right, the pillar cameras can probably see the dark vehicle.

So instead of how us humans to it (pull left and stick head out as far as possible), the car needs to turn right to get the pillar camera aligned with the ajacent lane.

Otherwise it has to rely on fisheye.

 

[arnis]

Otherwise it has to rely on fisheye.

Turning right will result in being further away from dividing line. To overtake a truck program must see at least 5 seconds of opposite side traffic. That is around 75meters ahead in case of 50kmh traffic.
Fisheye can't see as well as a driver can. Next time in traffic put your head in the middle of the vehicle and try to see oncoming traffic while being behind a van with moderate following distance.

 

[Saghost]

Somewhat forwards but not straight ahead where car in the opposite lane is positioned.


40% over the middle lane? That is a lot.
I have thought about that for weeks. I think that others are just assuming


Green part does overlap with two main frontal cameras.

No, 40% to the left of the truck. Unless the truck is hugging the centerline for some reason, it'll be much less than that over the line - and the car should already have an idea of the danger level from bounced radar returns.

 

[arnis]

Unless the truck is hugging the centerline for some reason, it'll be much less than that over the line - and the car should already have an idea of the danger level from bounced radar returns.

Are you sure? Because this is not what I see in real world. Truck dimensions vs lane width.

 

[stopcrazypp]

Turning right will result in being further away from dividing line. To overtake a truck program must see at least 5 seconds of opposite side traffic. That is around 75meters ahead in case of 50kmh traffic.
Fisheye can't see as well as a driver can. Next time in traffic put your head in the middle of the vehicle and try to see oncoming traffic while being behind a van with moderate following distance.

Turn right then back up until pillar gets a good view.

If using Fisheye, the car will have to creep out from behind the truck.

 

[stopcrazypp]

Are you sure? Because this is not what I see in real world. Truck dimensions vs lane width.

Did you look at your own picture? The truck is quite a bit to the right of the center line.

 

[verygreen]

So, I got a hint today that the NN model is Caffe and you can run it outside the car too, just need GTX 1060 or better I guess.
Instructions here: Using a Trained Network: Deploy · BVLC/caffe Wiki · GitHub

Not that I have any time for that myself, of course, but might be somebody is motivated enough (and knows what they are doing) to try what is it the car actually "sees".

 

[tedsk]

Pm-ed.