LiDAR - NASA confirms cameras better than LiDAR - Musk is correct

[Brando]

Lead Scientist Behind NASA's Asteroid Mission Talks About The Biggest Problems They Solved

From Scott Manley YouTube Channel:
Scott Manley

Very interesting interview, IMHO.

Especially note how LiDAR failed, and NASA moved to a camera only solution. Musk opinion validated? I think so.

 

[VT_EE]

It's funny listening to the LIDAR fanboys who insist that FSD is impossible with just cameras.

 

[diplomat33]

Lead Scientist Behind NASA's Asteroid Mission Talks About The Biggest Problems They Solved

From Scott Manley YouTube Channel:
Scott Manley

Very interesting interview, IMHO.

Especially note how LiDAR failed, and NASA moved to a camera only solution. Musk opinion validated? I think so.

I think your title is a bit misleading and click bait. The scientist never said that lidar failed or that cameras are inherently better. He simply said that in this particular instance lidar did not provide the resolution they needed and that cameras were a more ideal solution to this particular problem. That does not mean that lidar never works. Also, if I am not mistaken, Elon uses lidar for SpaceX. He just does not use lidar for Tesla's FSD. But Elon believes in lidar for space maneuvers.

A satellite orbiting an asteroid in the emptiness of space is very different from a car that has to navigate a road system with other fast moving cars and/or pedestrians while following strict road rules. So I am not sure we can infer that lidar is doomed for FSD based on this one case.

But I am not sure why "camera fanboys" get so worked up about this. Companies that use lidar for FSD, also use cameras too. It's not either or. Everybody, even "lidar fanboys", agree that cameras are needed for FSD. It's just that most companies get some benefit from adding lidar to the mix.

It's funny listening to the LIDAR fanboys who insist that FSD is impossible with just cameras.

To be clear, I don't think anybody is arguing FSD with cameras only is impossible. Yes, you can do self-driving with just cameras. The issue is how safely and how reliably. I would argue that FSD with no driver supervision is difficult with just cameras so adding lidar as a back-up makes things easier by adding extra reliability and redundancy. For example, your camera vision might be very good and correctly determine that an object is 300 meters away but a lidar can double check that number and help confirm if the object really is 300 meters or not, in case your camera vision got it wrong. When doing FSD, no matter how good your camera vision might be, it can't hurt to get a second opinion. You can't afford a wrong distance measure. Lidar can also help avoid unnecessary phantom braking by confirming if the camera vision is correct or not about seeing some object on the road.

Mobileye has a 12 camera system that can do FSD but they still plan to add lidar as a back-up to increase the reliability high enough to meet their standard for safe enough to remove the driver.

It is worth noting that when Waymo started on FSD about 10 years ago, camera vision was not good enough. So Waymo needed to use lidar. Now, machine learning, neural networks and camera vision have greatly improved. If now, camera vision only is good enough to do safe L5 autonomous driving with no driver supervision, then great. I don't think anybody would be upset if that turns out to be the case. I think we all want the same thing: safe autonomous driving. People just have different opinions on the best way to get there.

 

[Brando]

@diplomat33 - Here is more click bait.
Tesla Autonomy Day 2019 - Full Self-Driving Autopilot

Tesla team (and Elon) explain in detail why LiDAR is not used.
As you pointed out (in fact Elon claims to have designed) LiDAR system used by Dragon to dock with ISS.
Nothing wrong with LiDAR - just the application. Both NASA Asteroid & Tesla FSD dropped LiDAR in favor of camera only.
You can listen/research for yourself.

 

[diplomat33]

@diplomat33 - Here is more click bait.
Tesla Autonomy Day 2019 - Full Self-Driving Autopilot

Tesla team (and Elon) explain in detail why LiDAR is not used.
As you pointed out (in fact Elon claims to have designed) LiDAR system used by Dragon to dock with ISS.
Nothing wrong with LiDAR - just the application. Both NASA Asteroid & Tesla FSD dropped LiDAR in favor of camera only.
You can listen/research for yourself.

I watched Autonomy Day. I am well informed on why Elon does not use lidar for Tesla's FSD: lidar is too expensive for Tesla and neural networks and machine learning have advanced to the point where Elon is convinced that camera vision can do FSD without lidar.

But why is Elon automatically right? Maybe he is right but maybe he is wrong. I know about a dozen of companies at least, including FSD leaders like Waymo, Cruise, Mobileye, Aurora, and Baidoo, who disagree with Elon and they have autonomous driving prototypes that work with lidar. If you do some research, you will find that there are good reasons to use lidar for FSD.

Who should I believe? The dozen companies that have FSD prototypes with lidar or the 1 company with no FSD prototype that claims that lidar is not needed?

If Tesla is right though, great. I won't be upset. But if Tesla does end up needing lidar, then hopefully they will agree to add lidar. Ultimately, I hope we all want the same thing: safe autonomous driving.

 

[bxr140]

Dude, take it easy with your reason and logic.

 

[NHK X]

Dude, take it easy with your reason and logic.

haha have to agree with this a little

Diplomat33: You used to be the biggest Tesla apologetic. Staunchly defending Elon Musk, his intent, timeline, his decision to not use Lidar ect. What happened?

You are now arguing with the same logic you argued against not so long ago. Is the change due to your Tesla experience? Or you just trying to be more balanced in your opinion and not Tesla fanboyish? I mean no ill intent, just curious with the distinct change in your opinion.

 

[Brando]

Logic? Basic Principle thinking? Autonomy Day, I think, did a very good job of explaining how Tesla approach had developed. The things they tried and which things they dropped and why. GPS, detailed maps, LiDAR are just a couple of examples. And they explained how & why they were solving the problem of self driving and the tools they were developing. FSD computer just one of them. Other things they didn't have time to explore such as dojo and improved labeling ... I'm too slow to get it all in one pass.

I found it worth while to watch more than once. I learned or better understood the problems and the solution approaches being worked on.
Listen at 2x playback speed to save your time. key board arrow keys allow 5 second backups to replay as needed.
You all know these tricks/tips ??

 

[malcolm]

Jury is still out on a possible LiDAR/camera FSD preference.

Same with Elon is correct / giddy optimist.

 

[RandomX]

I did not yet view the video... sorry I will. Still I have some input.

First, Tesla has built their cars with cameras not lidar. They have to and will stand behind that decision and their current product (no matter what).

Next, it is arguable as to which technology is better for FSD. Mostly because we are not there yet; So nothing has been proven.
The easy answer is that both would be best, maybe so... but is that cost effective or would it be a financial Achilles heel? How about from a processing data from the sensors (at real time) standpoint? Would it not be best to use all of the available processing power on the best sensors?

In my opinion lidar systems and cameras are not the same. They are looking at different wavelengths of light and don't even function the same way. From what I understand, lidar has a better 3D view of what it can see ...based on the timing of reflections of IR light back to its sensors. Rain, fog, and solid objects can and will get in the way. So it basically only sees shapes... which to me would seem limited and even scary to rely on for driving.

Still, lidar is cool and when it was first used in robot vs robot combat it was awesome, it gained attention, and scored many wins.
In robot wars you have a square arena and not to many dynamic factors such as weather and everything else that can happen in the big world. It was easy for robots to use the sensor and some basic code to zero in on their opp and destroy it.
Unlike lidar, a camera can detect light in many wavelengths by a very big factor. This larger set of sensor feedback is probably more suited for the real world. By that I mean that with a larger feedback set of data from the sensor it seems logical that it would eventually have more potential, over time, when coupled with AI and machine learning as we know it.

To look at it from another completely different standpoint. In nature almost all creatures on the earth have a "camera like" vision system where they see colors to take in their surroundings and survive successfully. There are lidar (sonar) type systems in use by some animals but only by a small group and mostly for specialized low light conditions.

For our vehicles, even in the worst conditions, we at least have headlights beaming forward. Also, most driving is done during daylight and even at night typically there is much more light than just headlights (ie reflective stripes, streetlights, taillights... ect). So cameras have the opportunity to be more effective overall.

 

[mikes_fsd]

To be clear, I don't think anybody is arguing FSD with cameras only is impossible.

To be clear, that is exactly what you have been arguing across at least 5 different threads.

Good grief! At least try to keep your story consistent.

 

[diplomat33]

To be clear, that is exactly what you have been arguing across at least 5 different threads.

Good grief! At least try to keep your story consistent.

No, I've been consistent. There is a difference between FSD and safe FSD with no driver supervision. I've been consistent that some FSD is possible with just cameras. I've pointed to Mobileye as an example of FSD with cameras only. But I've said that safe FSD with no driver supervision is probably not possible with cameras only.

 

[mikes_fsd]

There is a difference between FSD and safe FSD

No there is not, because a non-safe FSD is not FSD at all!

 

[diplomat33]

Diplomat33: You used to be the biggest Tesla apologetic. Staunchly defending Elon Musk, his intent, timeline, his decision to not use Lidar ect. What happened?

You are now arguing with the same logic you argued against not so long ago. Is the change due to your Tesla experience? Or you just trying to be more balanced in your opinion and not Tesla fanboyish? I mean no ill intent, just curious with the distinct change in your opinion.

It is a combination of my Tesla experience + learning more about autonomous driving in general.

Before, I was actually very uninformed about autonomous driving. When you don't really know autonomous driving works, it is much easier to fall for Elon's PR and timelines.

The more I learned about autonomous driving, the more I realized that Elon is yada yada over some important parts of autonomous driving and that his timelines were unrealistic. Also, when I started looking at what Waymo and Cruise have, I started to realize that Tesla's FSD is not "years ahead of the competition" as the fanboys claim. So that deflated some of my fanboy attitude as well.

Also, as an owner, I've experienced phantom braking, missing exits, red hands on wheel warnings, ping ponging in the lane etc... So that gave me a more realistic view of what is good and bad about Autopilot. Now some of these issues have gotten better over time.

The bottom line is that when you know more how autonomous driving works and you experience some the problems that AP has, it is harder to believe Elon's optimistic timelines about L5 autonomy coming next year.

 

[mikes_fsd]

The bottom line is that when you know more how autonomous driving works

The bottom line is that @diplomat33 claims to know more about autonomous driving than all the teams working on it at Tesla, including, but not limited to, Karpathy, Elon, et al

 

[Jeff Hudson]

key board arrow keys allow 5 second backups to replay as needed

Thanks for that YouTube tip. I did not know of that behavior and now I do.

 

[diplomat33]

The bottom line is that @diplomat33 claims to know more about autonomous driving than all the teams working on it at Tesla, including, but not limited to, Karpathy, Elon, et al

No. I don't claim to know more about autonomous driving than Tesla engineers. But I do trust all the engineers working for dozens of autonomous driving companies that have produced actual autonomous driving. I think they know more about autonomous driving than Elon does. I am merely sharing my opinion based on what the experts in autonomous driving have said.

And I like Karpathy a lot. I think he is very competent in his area of expertise. But he is just following the leader. It is Elon who believes that camera vision and machine learning are all you need to do FSD and has directed the team to make it happen. It's Elon's vision that sets the tone and direction. They are just trying to implement what Elon wants.

 

[ai4px]

It's funny listening to the LIDAR fanboys who insist that FSD is impossible with just cameras.

My head only has two cameras, no lidar and I've been Full Self Driving since I was 16. Sometimes just for fun, I drive with one of my "cameras" covered up.... honestly, I don't have stereoscopic vision, I can only see from one eye....

 

[diplomat33]

Basically, we have two FSD camps:

Camp 1: Elon firmly believes that camera vision and machine learning are all you need to do FSD. He believes this because humans only use 2 eyes and a brain to drive and because he has tremendous confidence that machine learning and AI have improved enough to do the job. Tesla "fanboys" automatically trust and believe Elon is right just because he is Elon.

Camp 2: Everybody else, including Waymo that has over a decade of experience working on autonomous driving, believe that you need sensor fusion of cameras, radar and lidar. They adopted a very logical and methodical approach to autonomous driving. Before you can even attempt to navigate a route, you must first localize the car precisely on a map and secondly, accurately detect, track and predict the movement of objects around the car. So they built a HD map and then using cameras, radar and lidar, they are able to precisely localize the car on the HD map and also precisely track objects around it. That way, they have a precise map with the precise position of the car and precise tracking of everything around the car needed to reliably and safely navigate the car from A to B. But that is only part 1 of FSD. Part 2 is writing the driving policy so that the car follows the rules of the road and also drives in a responsible manner. But if you don't have part 1, you can't do part 2 reliably.

 

[diplomat33]

No there is not, because a non-safe FSD is not FSD at all!

I am making the distinction between a car that can self-drive around the city but needs a safety driver to watch over it (FSD), versus a car that can self-drive so safely that it does not need a safety driver at all (safe FSD without driver supervision).