Will substantial changes in vehicle height affect NN accuracy?

[willow_hiller]

I was looking at one of the fan renders of the Tesla Pickup and a thought occured to me. We know Tesla has an absolute advantage in gathering real world training data for their neural networks, but will networks trained by Model 3 camera feeds work as well on a vehicle with a substantially taller frame? I'm sure the road will look quite a bit different to a neural network after it's been fed millions of hours of footage from a Model 3 height.

Could Tesla compensate for this by positioning cameras lower on the truck/semi?

 

[boaterva]

How do you account for the X? It’s a bit higher than the S and 3 (which aren’t identical). I’m sure each feed is ID’d by source and if it is enough to matter then the height differential is compensated for in any calculations needed.

 

[willow_hiller]

How do you account for the X? It’s a bit higher than the S and 3 (which aren’t identical). I’m sure each feed is ID’d by source and if it is enough to matter then the height differential is compensated for in any calculations needed.

Do we know which camera feeds are used for which tasks right now? Lane identification could be via the forward facing cameras right now, which I imagine would be pretty similar regardless of height.

I'm thinking when we get to FSD, and more feeds are used, the models might need to be carefully tuned to make sure they're not overfit for the 3 angles.

And then the Semi is an entirely new can of worms when it comes to height!

 

[EVNow]

I'm sure the road will look quite a bit different to a neural network after it's been fed millions of hours of footage from a Model 3 height.

NN is a lot more robust than you think. A few degrees change will not make a difference. Think about all the slightly sloping roads, stop signs not perfectly verticle etc etc. NN needs to handle all this - part of the training process is to transform the images (including rotation in 3D) and using those to train.

 

[heltok]

I was looking at one of the fan renders of the Tesla Pickup and a thought occured to me. We know Tesla has an absolute advantage in gathering real world training data for their neural networks, but will networks trained by Model 3 camera feeds work as well on a vehicle with a substantially taller frame? I'm sure the road will look quite a bit different to a neural network after it's been fed millions of hours of footage from a Model 3 height.

Could Tesla compensate for this by positioning cameras lower on the truck/semi?

If a human can label the images, ie decide what is lane line what is a vehicle etc, then the neural network will eventually be able to. Having seen the world from very different perspectives will likely make some tasks more robust.

Once the images is labeled it is straightforward to move the objects into vector space, from the taller vehicles camera it will be a different transformation than from the lower vehicles. Once this is done you are back to vector space where pretty similar rules will apply. There might be some different signs to be followed and some different behavior of other drivers near you, but nothing impossible to learn.