Seeing the world in Autopilot V9 (part three?)

[verygreen]

Side camera still shows the distance of side objects (non-stereo vision).
I would say the distance isn't accurate at all. Just a rough number.
Say you can simply define an 30-pixel-wide object as 60 meters.

Note when a vehicle is in motion, stereovision is possible with just one camera as well by looking at two consecutive frames for example.

 

[EvanLin]

AP seems to draw a left turn lane in main camera even before camera can see it. Any idea?

 

[Tezlanian]

I don't have a model3, and these bits are not included for S/X, at least not in the v8, I did not check in v9, but I suspect it's the same

I take it the X and the S are usually identical binaries and not different builds from your comment?

 

[Tezlanian]

Note when a vehicle is in motion, stereovision is possible with just one camera as well by looking at two consecutive frames for example.

A lot can happen between those frames that some freaky stuff could happen if you consider them the same point in time. I would hope Tesla would not do this.

 

[verygreen]

AP seems to draw a left turn lane in main camera even before camera can see it. Any idea?

View attachment 347947

you can see the double yellow lane to the left of that car (just move forward a dosen frames to make it more visible and then go backwards to your original position) - so perhaps the car is seeing it too.

I take it the X and the S are usually identical binaries and not different builds from your comment?

Yeah, they are 100% identical. the only difference is between hw2.5 and hw2.0 builds.

 

[Anner J. Bonilla]

A lot can happen between those frames that some freaky stuff could happen if you consider them the same point in time. I would hope Tesla would not do this.

Can you elaborate on what you mean by "a lot" and "freaky stuff"?
Do you have a more concrete example? I really doubt they are relying on this "stereo" images/frame data to a really accurate precision. You just need enough to know there's something and its been there for more than 5 frames to not do a lane change.
Explains why a lane change does not start instantly and it has a slight delay. Verifying that the data between frame is consistent and when in doubt wait a few more milliseconds/frames and continue showing lane obstructed.

 

[Tezlanian]

Can you elaborate on what you mean by "a lot" and "freaky stuff"?
Do you have a more concrete example? I really doubt they are relying on this "stereo" images/frame data to a really accurate precision. You just need enough to know there's something and its been there for more than 5 frames to not do a lane change.
Explains why a lane change does not start instantly and it has a slight delay. Verifying that the data between frame is consistent and when in doubt wait a few more milliseconds/frames and continue showing lane obstructed.

5 frames at high velocity can yield very different results. Trying to equate frames near one another seems like a bad idea.

This introduces time series data into the mix. Current AP code doesn't deal with this as far as I am aware. Also, doing such with discrete algorithms interpreting the machine vision results instead of utilizing RNNs as a layer on top of this would seem odd. @jimmy_d can correct me on this if there is evidence to the contrary and they are already doing this but I don't recall reading as such on this forum. If I recall there was an academic effort recently to incorporate motion prediction into Torch (a fork) utilizing both for this purpose.

 

[Anner J. Bonilla]

Agree 5 frames is a lot ideally you would want 2 consecutive frames for high speed / reaction areas like the front. But for lane change you can be more forgiving since if you have doubt you just wait for a few more frames to make sure that the NN classified object is still there / does not go away.

 

[Tezlanian]

Agree 5 frames is a lot ideally you would want 2 consecutive frames for high speed / reaction areas like the front. But for lane change you can be more forgiving since if you have doubt you just wait for a few more frames to make sure that the NN classified object is still there / does not go away.

I agree with you on the sampling to determine the existence of the object as separate inputs over time. Inferring motion and depth as amalgamated inputs, however, I do not.

 

[EvanLin]

Note when a vehicle is in motion, stereovision is possible with just one camera as well by looking at two consecutive frames for example.

Just saw that Mitsubishi had similar tech. Didn't mention how they achieve it.

MITSUBISHI ELECTRIC News Releases Mitsubishi Electric Develops Object-recognition Camera Technology Using Proprietary AI for Coming Mirrorless Cars