Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

FSD Beta Videos (and questions for FSD Beta drivers)

This site may earn commission on affiliate links.
So, you are saying “cloud NN” only when the car is moving very slowly ?
Just an idea. As other have suggested, perhaps a bad idea. When is odd objects detection most needed? Since a freeway is controlled and doesn't allow pedestrians and other traffic, one can theorized it is less needed when a car is moving very fast. In residential neighborhoods, 25 mph, one can think almost anything can happen. For example google published a video of a lady in a wheelchair chasing a duck. Also in the context was a parking lot. How fast can cloud network respond? It would just be another avenue to help. Multiple ways are needed for fault tolerance. If a car doesn't understand what is in front of it, it makes sense to slow down or even stop and analyze the situation carefully. Seems to me a cloud NN could work well in this case.

I wouldn't be surprised if Waymo already has this implemented. Thanks for asking.
 
Last edited:
If a car doesn't understand what is in front of it
How do you train a neural network to know what it doesn't understand? I suppose the orange netting might lower the confidence that the path in front of the car is "drivable space" but reacting to that is how you get phantom braking.
I wouldn't be surprised if Waymo already has this implemented.
Here's a similar example in a Waymo though It seems like this obstruction might be easier to detect. The car gets confused and then requests remote assistance.
 
How do you train a neural network to know what it doesn't understand? ...
Also as I mentioned earlier, having a depth map of the scene helps with knowing there are obstructions in the path. Did Karpathy say depth map was easy using deep neural networks? I disagree that it is easy in a reliable fashion. Much easier if you have hardware helping you like low cost dual pixel pdaf. Works surprisingly well on things like gates with vertical bars that Elons says is difficult. The default pdaf setup only works on vertical edge detection.

> How do you train a neural network to know what it doesn't understand?
In other words you train a neural network (different than main NN), to detect obstructions in your path using pixel mapping techniques that map pixels to locations / depth in the scene. If the pixel(s) are in your way, that is a problem. The NN doesn't understand what the pixel is, it just knows it is an obstruction.
Yes, this may result in more phantom braking. For example will the car slow down for insects, birds, dirty air, an empty bag flying in the air.
 
Last edited:
n other words you train a neural network (different than main NN), to detect obstructions in your path using pixel mapping techniques that map pixels to locations / depth in the scene. If the pixel(s) are in your way, that is a problem. The NN doesn't understand what the pixel is, it just knows it is an obstruction.
Yes, this may result in more phantom braking. For example will the car slow down for insects, birds, dirty air, an empty bag flying in the air.
That’s not what Elon was saying. Basically once the NN has placed the road surface in 3D space, it can deduce bumps and debris on the road using another NN trained for that task. This wont tell the car what the bump is (object), just roughly what its severity is (height and size/location). But this is a special case and cannot be applied to arbitrary pixels, and certainly not to deduce depth. Gates and barriers (e.g. at a parking lot) thus need the full NN object treatment to be handled correctly, and they are hard since they often “float” in the FOV.
 
Recognizing objects IN the road is to me the big needed improvement for FSD. When on the highway at 80 mph (speed limit here is 80) avoiding those objects is always a bit tricky, meaning recognizing the size, shape, mobility, and nature of the object and the best evasive maneuvers is often a last-minute decision. So far FSD/Beta/NOA doesn't even show those objects unless on the side of the road.
 
Last edited: