Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Almost ready with FSD Beta V9

This site may earn commission on affiliate links.
yes, I had jiggling, wiggling, and spinning cars for 18 months, and not long after it was fixed, now I have jiggling and wiggling lines. It's distracting. :eek:
I am not asking to see all the NN probabilities. I just want nicer looking visualizations, with cleaner lines that don't jiggle and wiggle.

Here is a leak of what the final FSD Beta visualizations might look like:

8U1ZFb4.png


I got it from this facebook video:


Elektrek also has an article about it:


IMO, if that is that the production version of the FSD Beta visualization that we get, I will be very happy. I think it looks great. It is much more refined and clean than the current beta visualizations. The intersection looks great. It looks sharp. The pedestrian looks good too. I like that they have the 1 blue line like NOA uses. I think that will be great to show the intended path the car intends to take.
 
  • Like
Reactions: helvio and edseloh
Here is a leak of what the final FSD Beta visualizations might look like:

8U1ZFb4.png


I got it from this facebook video:


Elektrek also has an article about it:


IMO, if that is that the production version of the FSD Beta visualization that we get, I will be very happy. I think it looks great. It is much more refined and clean than the current beta visualizations. The intersection looks great. It looks sharp. The pedestrian looks good too. I like that they have the 1 blue line like NOA uses. I think that will be great to show the intended path the car intends to take.
BINGO.... THIS ^^^
 
Depth percepting cameras would know. For example cameras that use PDAF for depth perception.
A couple of links after googling the subject:
  1. https://www.osapublishing.org/Direc...01-39B2A8904F795762_344306/oe-24-12-12868.pdf
  2. Learning to Predict Depth on the Pixel 3 Phones
Sony sells PDAF depth percepting image sensors for a few dollars in large quantities.
I don't think the PDAF part of the camera can do per pixel depth. It sounds to me like PDAF (phase detection auto-focus) is just what my SLRs have been doing for years. It can determine which direction and approximately how much focus needs to be adjust to get whatever is under the auto-focus point. Based on the Google AI blog, it's a NN that does the depth per pixel depth perception Again, it sounds similar to the Pseudo-Lidar approach.
 
  • Disagree
Reactions: mikes_fsd
Yes it can as an approximation. The sensor generations stereo images and the differences are used to approximate depth. A neural network is not required.
Seems my original link didn't work. Here it is again:
Or if you have ~1.6 million cars already with hardware, and you can update the NN's on it to do the depth approximation (like Karpathy already demonstrated/spoke about) then WTF would you want to add more hardware, when the software is "good enough"?
 
  • Like
Reactions: insaneoctane
Or if you have ~1.6 million cars already with hardware, and you can update the NN's on it to do the depth approximation (like Karpathy already demonstrated/spoke about) then WTF would you want to add more hardware, when the software is "good enough"?
I'd have to agree with this. Of course if they find Pseudo-Lidar / cameras still can't cut it, then perhaps. While they're at it, might as well put in cross traffic cameras.

Yes it can as an approximation. The sensor generations stereo images and the differences are used to approximate depth. A neural network is not required.
Seems my original link didn't work. Here it is again:
Link still doesn't work. Why not give the authors and article title so we can Google it?
 
Or if you have ~1.6 million cars already with hardware, and you can update the NN's on it to do the depth approximation (like Karpathy already demonstrated/spoke about) then WTF would you want to add more hardware, when the software is "good enough"?
I do have a question on this, and I apologize if it has been discussed before (too many posts to read through). Why not use hardware/sensors that are designed to measure distance, and let vision system do the object identification that it excel at? It seems that radar/lidar can measure distance with very small errors and small amount of power, whereas visual recognition requires a lot more power and resource to detect distance, and likely without the accuracy of radar/lidar.
 
I do have a question on this, and I apologize if it has been discussed before (too many posts to read through). Why not use hardware/sensors that are designed to measure distance, and let vision system do the object identification that it excel at? It seems that radar/lidar can measure distance with very small errors and small amount of power, whereas visual recognition requires a lot more power and resource to detect distance, and likely without the accuracy of radar/lidar.
Cost and precision. When Tesla originally launched Autopilot 2 / Hardware 2, Lidar systems cost about the same as a car, so would be prohibitively expensive. Radar is currently used for distance, but what they fielded / available for the price at the time doesn't have nearly the resolution required to distinguish individual cars from things like road signs, much less pedestrians.

Lidar has come down in price, and Radar has increased in resolution, but that would be very expensive for Tesla to retrofit all the cars already sold with the system to deliver on their promise of "Full Self Drive." That, and Elon Musk is confident they aren't needed.
 
Link still doesn't work. Why not give the authors and article title so we can Google it?
I googled "pdaf depth map". It is the second link that comes up for me.
Title: Depth map generation using a single image sensor with phase masks
Authors: Jinbeum Jang, Sangwoo Park, Jieun Jo, and Joonki Paik

If my memory is correct, the sony sdk for the image sensor comes with software to compute a depth map. Didn't use Neural Network 4 years ago when I looked at it quickly. Now all the rage is using NNs: https://arxiv.org/pdf/1806.04171.pdf
Quote:
DEPTH FROM A DUAL-PIXEL CAMERA
Dual-pixel (DP) auto-focus systems work by splitting pixels in half,
such that the left half integrates light over the right half of the
aperture and vice versa (Fig. 5). Because image content is optically
blurred based on distance from the focal plane, there is a shift, or
disparity, between the two views that depends on depth and on
the shape of the blur kernel.
 
Last edited:
I do have a question on this, and I apologize if it has been discussed before (too many posts to read through). Why not use hardware/sensors that are designed to measure distance, and let vision system do the object identification that it excel at? It seems that radar/lidar can measure distance with very small errors and small amount of power, whereas visual recognition requires a lot more power and resource to detect distance, and likely without the accuracy of radar/lidar.
Radar doesn't know what it is looking at. Lidar is expense relative to cameras. Cameras cost a few dollars. Distance measurement with cameras is very low power and resources with various technologies like depth map using pdaf sensor. Accuracy with a camera can be considered high when you know what you are looking at. Lidar has a bunch of issues, like can bounce off of smoke from car exhaust.
 
In theory there is no data LIDAR can provide that you can't already get with vision.

There is data RADAR can provide you can't get with vision. But it'd require significantly more, and better, radars than typically go on cars.

And fusion of that data with a fully working 4D vision system would be exceedingly challenging.

Tesla is currently moving forward with the idea it'd be better to have a vision-only system sometime "soon" than to theoretically have a better-in-specific-cases fusion radar/vision system some much further in the future time.

But LIDAR gets you nothing except spending money to hide that your vision system isn't very good.
 
But LIDAR gets you nothing except spending money to hide that your vision system isn't very good.
I assume you mean that LIDAR doesn't get you anything you need to drive a car that can't be done with vision (first principals, humans drive with vision and all)? There are visual illusions that fool binocular vision that would not fool LIDAR (like the broad side of a truck with a low contrast background). LIDAR also excels at distance estimation over large dynamic ranges, on static/slowly moving objects, or in situations without external illumination like an obstacle in the road ahead of headlight range. Like radar they do add data, but like radar it is clear physics does not required them to drive a car at the same safety level as a human.
 
Last edited:
  • Like
Reactions: diplomat33
The question asks about limited release. So it will probably just be to early access users, not wide to the public.

Yup, V9 is very exciting because it's supposed to have surround video inference. I hope Elon and Tesla deliver V9 with the features they've been talking about. I hope V9 isn't just a rebranded 8.3 with a new visualization.

Even a limited release of V9 with surround video means they've tackled some of the challenges involving video labeling and NNs. This would be great news for their approach.
 
Last edited: