Will Tesla ever do LIDAR?

[CarlK]

They didn’t use it. Read the report. Their camera/lidar system saw the pedestrian but they were trying to get their disengagement rate down so they ignored it.

Everyone is using cameras! Lidar just provides more data that can not currently be obtained by cameras alone. There is zero evidence that Tesla has made their camera/radar system as good as others camera/radar/lidar systems.

So when there is a problem it's because of the camera?

Watch this Lex Fridman lesson to learn about self driving systems (41:45). You can tell he's already tilting toward Vision + Deep Learning even it was months before Tesla's latest annoucement.

Hope that video can help fix your confusion. There is no such thing as Lidar/camera system if you want to imply that's similar to the vision +DL system. That's just your invention. ICE car with battery and starter does not make it an EV.

 

[Daniel in SD]

So when there is a problem it's because of the camera?

No, the problem was not in the vision system. It was in the driving policy.

There is no such thing as Lidar/camera system if you want to imply that's similar to the vision +DL system. That's just your invention.

So vision+radar is possible but vision+radar+lidar is impossible? Ok....
Here's a talk by the head of computer vision at Cruise. It sure sound like they're using all three and they're using deep learning.
He talks about why cameras are great here:

 

[electronblue]

Vision features are fact too. You have still failed to use those facts to make a compelling reason why LIDAR is needed, how it should be incorporated and why humans are not required to use LIDAR before getting behind the wheel of a deadly machine on public roads.

You simply dismiss my reason, that is not the same as the reason not being there.

Lidar is so far the most reliable tool at saying ”there’s nothing there”. No other technology comes even close. Radar, certainly not, most of Tesla’s shadow braking is trying to decipher the radar signature. Ultrasonics miss pole structures and are useless at speed. Vision is great, but if the AI misses, it misses — and that’s where redundancy comes in.

I guess people forget the Uber crash. Lidar had been switched off. The vision missed, the pedestrian is dead. Redundancy.

Also, since you have such strong opinions on this subject, how many LIDAR do you recommend and where would they be mounted to be most effective?

You misunderstand me. My strong opinion (if it is that even...) is in doubting anyone here making assertive opinions about what the best autonomous sensor suite is — Elon Musk included. I don’t think anyone knows yet. The people who keep an open mind... those I have no need to disagree with at all.

As for Lidar placement, I think 360 degree placement along the lower perimeter of the car could be preferable for aesthetics and to avoid a moving Lidar mechanism. But if the latter offers better views and additional safety I wouldn’t be against a dome either, personally. If it makes the school-run safer, people will want a dome.

That said I don’t claim a good autonomous suite necessarily requires Lidar. I have never claimed that. I simply see it as one useful tool and don’t buy its dismissal...

 

[StealthP3D]

I guess people forget the Uber crash. Lidar had been switched off. The vision missed, the pedestrian is dead. Redundancy.

Well, you are just plain wrong about the facts in the Uber fatality case. All of the sensors were still functioning (LIDAR included), it was the automatic braking that had been switched off. You see, Uber switched off the Automatic Emergency Braking because it brakes erratically and unnecessarily when left on. So much for the superiority of LIDAR you keep pretending exists!

Uber self-driving cars: everything you need to know | TechRadar

Update: The National Transportation Safety Board (NTSB) has issued its preliminary report into the fatal accident involving an Uber self-driving car and a pedestrian in Arizona that happened in March.

The preliminary NTSB report reveals that while the self-driving car had an automatic emergency braking feature, this was disabled because the car was in "computer mode."

Uber disables emergency braking to prevent erratic driving behavior, according to the NTSB report, citing the ride-hailing service.

If Uber had invested their effort in machine vision and had developed a reliable system that could detect a pedestrian stepping into the path of the vehicle and stop in time, then they wouldn't be having all these nuisance braking events that resulted in them disabling the Automatic Emergency Braking. If something is working well, you don't disable it. You disable it because the sensors are returning false positives.

So, it's just like Musk said, LIDAR is a self-driving crutch that will come back to bite those who are using it. He also predicted in no uncertain terms, that those using LIDAR will abandon it (but he didn't advocate them turning it off while using it in public!).

But here's the problem: They have invested billions of dollars of other people's money in complex systems with integrated LIDAR They would be better off ditching the whole ball of yarn but that's an awfully difficult thing to explain to your investors, especially in light of the rapid improvements being made by Tesla's system now that they have the neural net learning at lightspeed with billions of miles being driven.

 

[Saghost]

Everyone is using cameras! Lidar just provides more data that can not currently be obtained by cameras alone. There is zero evidence that Tesla has made their camera/radar system as good as others camera/radar/lidar systems.

On the contrary - I've seen zero evidence that they haven't made an equal solution, but at the investor day they spent a while showing off just how much depth data they were routinely extracting from the cameras alone - even the calibration project they did between the camera depths and the radar ranges.

What evidence do you have that this isn't equal to the performance of these other systems?

 

[iTech]

I enjoy people using Tesla's opinion on how to solve L4+ autonomy as fact. Nobody, including Tesla, has a guaranteed solution. Everyone is taking what they think is their best chance at solving the problem.

 

[electronblue]

I enjoy people using Tesla's opinion on how to solve L4+ autonomy as fact. Nobody, including Tesla, has a guaranteed solution. Everyone is taking what they think is their best chance at solving the problem.

So glad to see someone else say this out loud too. This is key.

 

[CarlK]

No, the problem was not in the vision system. It was in the driving policy.

So vision+radar is possible but vision+radar+lidar is impossible? Ok....

Again straw man argument. Adding two more steering wheels to vision system will work too but where is the logic? Remember all cars with Lidar on top are Lidar primary system with no deep learning tuned vision system. Lidar and (big data) deep learning are mutually exclusive.

Here's a talk by the head of computer vision at Cruise. It sure sound like they're using all three and they're using deep learning.
He talks about why cameras are great here:

Did you understand what he's saying? He only mentioned vision system OTHERS are doing research on, not that Cruise is using it. No way they could do that with few hundreds test cars. That he even mentioned it is a strong sign how even Lidar people feel about it. They just don't have the infrastructure and resource to do it. If Tesla pulls it off, I think it will, you have to thank Elon's great vision for getting the whole plan laid out years ago.

 

[electronblue]

Well, you are just plain wrong about the facts in the Uber fatality case. All of the sensors were still functioning (LIDAR included), it was the automatic braking that had been switched off. You see, Uber switched off the Automatic Emergency Braking because it brakes erratically and unnecessarily when left on. So much for the superiority of LIDAR you keep pretending exists!

Uber self-driving cars: everything you need to know | TechRadar

Update: The National Transportation Safety Board (NTSB) has issued its preliminary report into the fatal accident involving an Uber self-driving car and a pedestrian in Arizona that happened in March.

The preliminary NTSB report reveals that while the self-driving car had an automatic emergency braking feature, this was disabled because the car was in "computer mode."

Uber disables emergency braking to prevent erratic driving behavior, according to the NTSB report, citing the ride-hailing service.

I’ve read a more granular report of the Uber case and I disagree with your summary — but of course I was simplifying too. No matter, we can ignore the Uber case.

If Uber had invested their effort in machine vision and had developed a reliable system that could detect a pedestrian stepping into the path of the vehicle and stop in time, then they wouldn't be having all these nuisance braking events that resulted in them disabling the Automatic Emergency Braking. If something is working well, you don't disable it. You disable it because the sensors are returning false positives.

Yes indeed. Ever gotten shadow braking events in Tesla? All the time, if you actually drive one. That’s because detecting obstacles reliably with vision is hard and if you have no second opinion that’s all you’ve got.

So, it's just like Musk said, LIDAR is a self-driving crutch that will come back to bite those who are using it. He also predicted in no uncertain terms, that those using LIDAR will abandon it (but he didn't advocate them turning it off while using it in public!).

Yes, of course. The problem is there is a large chance Musk is wrong.

But here's the problem: They have invested billions of dollars of other people's money in complex systems with integrated LIDAR They would be better off ditching the whole ball of yarn but that's an awfully difficult thing to explain to your investors, especially in light of the rapid improvements being made by Tesla's system now that they have the neural net learning at lightspeed with billions of miles being driven.

But here’s the problem: Tesla has invested billions of dollars of other people’s money in simple system without integrated LIDAR They would be better off adding more redundancy but that’s an awfully difficult thing to explain to your investors, especially in light of you having pre-sold the thing to a half a million of already shipped cars.

 

[StealthP3D]

I enjoy people using Tesla's opinion on how to solve L4+ autonomy as fact. Nobody, including Tesla, has a guaranteed solution. Everyone is taking what they think is their best chance at solving the problem.

When Kennedy, in 1961, said the USA would put a man on the moon by the end of the decade, there were a minority of scientists who thought it an impossible and foolish goal. Some of them even thought it was impossible in any time frame. But most scientists knew it was possible, it was just a matter of how long it would take and the leading scientists in the necessary fields thought it possible within the decade. Guess which ones were right? Not only did we land a human being on the moon, we brought them all back safely.

FSD is easier by orders of magnitude easier now that we have machine vision and neural nets. When you know what Elon Musk knows about the state of development of these technologies, and you have watched its steady and exponential improvements, you don't have to guess what the solution is, you already know (much like the Apollo scientists in 1961). You still have to guess on the timeframe. Yes, it will happen and it will be sooner than the naysayers think.

 

[electronblue]

@StealthP3D Elon Musk is the naysayer in this thread’s room. In the meanwhile, Waymo is already on the Moon with their Lidar.

 

[Daniel in SD]

Well, you are just plain wrong about the facts in the Uber fatality case. All of the sensors were still functioning (LIDAR included), it was the automatic braking that had been switched off. You see, Uber switched off the Automatic Emergency Braking because it brakes erratically and unnecessarily when left on. So much for the superiority of LIDAR you keep pretending exists!

What’s the difference between LIDAR being switched off and ignoring the data from LIDAR and RADAR? Uber’s software for reacting to the data was getting too many false positives so they turned it off.

According to data obtained from the self-driving system, the system first registered radar and LIDAR observations of the pedestrian about 6 seconds before impact, when the vehicle was traveling at 43 mph. As the vehicle and pedestrian paths converged, the self-driving system software classified the pedestrian as an unknown object, as a vehicle, and then as a bicycle with varying expectations of future travel path. At 1.3 seconds before impact, the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision (see figure 2).2 According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator.

I have no idea what Uber is means by “emergency brake maneuvers are not enabled while vehicle is under computer control”...
The exact same accident would have occurred if the system was using only cameras so I don’t really see how it proves that a camera only system would be superior.

 

[Daniel in SD]

When Kennedy, in 1961, said the USA would put a man on the moon by the end of the decade, there were a minority of scientists who thought it an impossible and foolish goal. Some of them even thought it was impossible in any time frame. But most scientists knew it was possible, it was just a matter of how long it would take and the leading scientists in the necessary fields thought it possible within the decade. Guess which ones were right? Not only did we land a human being on the moon, we brought them all back safely.

FSD is easier by orders of magnitude easier now that we have machine vision and neural nets. When you know what Elon Musk knows about the state of development of these technologies, and you have watched its steady and exponential improvements, you don't have to guess what the solution is, you already know (much like the Apollo scientists in 1961). You still have to guess on the timeframe. Yes, it will happen and it will be sooner than the naysayers think.

Elon Musk is the guy in 1961 saying that going to the Moon is useless and we should instead go to Mars.

 

[StealthP3D]

What’s the difference between LIDAR being switched off and ignoring the data from LIDAR and RADAR? Uber’s software for reacting to the data was getting too many false positives so they turned it off.

The fact that they switched it off is the point. If it's the "silver bullet" then why would they turn it off? Particularly in light that their machine vision is not nearly as developed as Tesla's (because it simply doesn't have enough data, for one).

I have no idea what Uber is means by “emergency brake maneuvers are not enabled while vehicle is under computer control”...
The exact same accident would have occurred if the system was using only cameras so I don’t really see how it proves that a camera only system would be superior.

See above. Yes, by putting their eggs in the LIDAR basket they have a substandard vision system. Their system REQUIRES LIDAR but the LIDAR is too unpredictable to leave it on. So they are in a real dead end.

 

[StealthP3D]

Elon Musk is the guy in 1961 saying that going to the Moon is useless and we should instead go to Mars.

No. He wouldn't have said that in 1961, he would have backed the moon mission 100%. He wasn't even alive then! He made it clear that he believes we learned more than enough by already going to the moon that we shouldn't keep doing the same thing.

 

[electronblue]

The fact that they switched it off is the point. If it's the "silver bullet" then why would they turn it off? Particularly in light that their machine vision is not nearly as developed as Tesla's (because it simply doesn't have enough data, for one).

See above. Yes, by putting their eggs in the LIDAR basket they have a substandard vision system. Their system REQUIRES LIDAR but the LIDAR is too unpredictable to leave it on. So they are in a real dead end.

The lesson of the Uber case is the value of redundancy — and the importance of treating a prototype system responsibly. Recognizing an oddly shaped pedestrian (with a bike full of stuff) emerging from the shadows was missed by the vision system. Lidar does not miss such things. Mind you I’m not saying vision will miss necessarily either, I’m just pointing out the value of redundancy.

As for your assessment that Tesla’s vision system is ahead of Uber’s, I respectfully disagree with any such firm views. We really don’t know enough to say for certain.

I would suggest Tesla currently has a sub-standard vision system and no Lidar. How is that better?

 

[StealthP3D]

I have no idea what Uber is means by “emergency brake maneuvers are not enabled while vehicle is under computer control”...
The exact same accident would have occurred if the system was using only cameras so I don’t really see how it proves that a camera only system would be superior.

It doesn't prove Uber's camera only system is superior, actually the opposite. They took a dead-end path and have ended up with an inferior machine vision solution. That is what Musk has said, LIDAR is an unnecessary distraction that will keep those who use it as a crutch from getting to where they need to be.

 

[electronblue]

No. He wouldn't have said that in 1961, he would have backed the moon mission 100%. He wasn't even alive then! He made it clear that he believes we learned more than enough by already going to the moon that we shouldn't keep doing the same thing.

Musk’s aspirations and designs on going to Mars are perfectly fine.

They are a far cry from calling the current autonomous driving leaders DOOMED thoiugh. That’s naysaying big time. In the meanwhile Waymo is running their robotaxi fleet and Tesla has a PowerPoint.

 

[StealthP3D]

Recognizing an oddly shaped pedestrian (with a bike full of stuff) emerging from the shadows was missed by the vision system. Lidar does not miss such things. Mind you I’m not saying vision will miss necessarily either, I’m just pointing out the value of redundancy.

Redundancy is what caused Uber to turn off the automatic braking that relied on the LIDAR. LIDAR also doesn't miss a plastic grocery bag gliding across the road in the breeze. Is that worth dying over? Machine vision can recognize a bag and ignore it. If the automatic emergency braking is activated every time LIDAR "sees" something, we are in trouble. If the machine vision is smart enough to know to ignore plastic bags, then the LIDAR is not needed in the first place.

Those who argue for layers of redundancy don't understand how artificial intelligence works. They also miss the fact that any system that is better than humans will save thousands of lives, regardless of the remaining flaws. Once FSD is statistically safer than humans, it would be criminal to require additional layers of redundancy before approval because people are slaughtered every day. The systems will naturally get better over time but they should be deployed as soon as they will save lives. And, at the point, it will make more sense to continue to improve the AI than to attempt to add more sensors and a bunch of lines of code to figure out which sensor data over-rides the other (the AI or the sensor data which has been processed and determines there is a threat). At some point, every car will be required to communicate with every other car within striking distance and negotiate a path. So we will avoid the nasty energy robbing LIDAR dome on the top of every car completely.

 

[StealthP3D]

In the meanwhile Waymo is running their robotaxi fleet and Tesla has a PowerPoint.

Holding up Waymo's robotaxi fleet as a good example of FSD is laughable! Tesla is about 10 times closer to FSD in the real world.