Possible scientific vindication of using just cameras, no lidar

[lux_cars]

What if you have a stroke while driving? Can't cover every single possibility in the world.

A stroke would be much rarer than cameras getting dirty/splashed but even if it wasn’t, human driven cars are very unsafe and I doubt regulators/public opinion would allow anything that’s not orders of magnitude safer. Can you imagine self-driving cars causing millions of death worldwide? There would be an uproar and it would be shutdown.

 

[calisnow]

Why would this paper be a vindication for using cameras only? It seems to be a paper on how cameras only can be used, not a comparison of cameras vs. cameras + lidar.

That's what the OP meant. I believe you misunderstood his use of the word "vindication." I'll give you a pass since English isn't your first language - this was a very subtle distinction you missed and you miss almost none.

 

[RDoc]

Why choose? Time of Flight cameras are getting cheap and do both visual and distance.

 

[strangecosmos]

Why choose? Time of Flight cameras are getting cheap and do both visual and distance.

A difference of 1-2 years makes a huge difference. If Tesla waits 1-2 years for even better sensors to become affordable when regular cameras would be enough, it is needlessly letting Tesla owners get injured or killed in crashed that could have been prevented by a primarily camera-based self-driving system. Getting to market 1-2 years sooner than competitors also makes a big difference from a business perspective.

I think Tesla probably will continuously upgrade the sensors used, but it's important to distinguish good enough from better. More sensors is always better, but what's the minimum sensor suite to do full self-driving at 2x average human safety?

 

[lux_cars]

but what's the minimum sensor suite to do full self-driving at 2x average human safety?

How do we know regulators/public opinion will allow FSD at only 2x human safety? I would assume acceptable safety levels would be a lot higher for FSD vehicles. Humans and our societies aren’t purely rational in their decision making.

 

[strangecosmos]

How do we know regulators/public opinion will allow FSD at only 2x human safety?

Well, Tesla's aspiration is 10x safer. It's aiming for at least 2x safer. I think if Tesla can prove its cars are 10x safer, people will find that compelling.

The U.S. Congress looks like it's about to pass a bill that would permit self-driving cars if car companies "could demonstrate they are as safe as current vehicles." Initially each company would be capped in terms of the number of self-driving cars it could sell per year, but after four years the cap would be lifted.

So we're very close to legislation that would legalize self-driving cars in the United States. This bill also overrides states' laws on self-driving cars, so there would one unified law for the whole country.

Tony Seba has an interesting argument about regulation globally. If one large country like the United States or China legalizes self-driving cars, that will make that country's economy so much more productive that other countries will be forced to legalize them too to remain competitive. I think the U.S. law is the first of its kind. Germany has partially legalized self-driving cars, but a safety driver is still required.

 

[lux_cars]

Well, Tesla's aspiration is 10x safer. It's aiming for at least 2x safer. I think if Tesla can prove its cars are 10x safer, people will find that compelling.

The U.S. Congress looks like it's about to pass a bill that would permit self-driving cars if car companies "could demonstrate they are as safe as current vehicles." Initially each company would be capped in terms of the number of self-driving cars it could sell per year, but after four years the cap would be lifted.

So we're very close to legislation that would legalize self-driving cars in the United States. This bill also overrides states' laws on self-driving cars, so there would one unified law for the whole country.

Tony Seba has an interesting argument about regulation globally. If one large country like the United States or China legalizes self-driving cars, that will make that country's economy so much more productive that other countries will be forced to legalize them too to remain competitive. I think the U.S. law is the first of its kind. Germany has partially legalized self-driving cars, but a safety driver is still required.

Yes, 10x safer than humans will definitely be compelling!

I haven’t looked too much into the self-driving legislation but I was under the impression that it primarily dealt with testing? Can you provide some additional details on what this legislation entails?

 

[strangecosmos]

I haven’t looked too much into the self-driving legislation but I was under the impression that it primarily dealt with testing? Can you provide some additional details on what this legislation entails?

Check out this article. I originally thought the bill dealt with testing too, but it sounds like companies will be able to sell self-driving cars to consumers.

 

[AnxietyRanger]

That's what the OP meant. I believe you misunderstood his use of the word "vindication." I'll give you a pass since English isn't your first language - this was a very subtle distinction you missed and you miss almost none.

It is hard to think OP actually somehow meant "cameras + lidar" when the title of the thread says "no lidar" and his whole post was talking of using cameras only.

 

[calisnow]

It is hard to think OP actually somehow meant "cameras + lidar" when the title of the thread says "no lidar" and his whole post was talking of using cameras only.

This from the same guy who jumped in to explain to me that it was "clear" to you that bladerskb meant something other than what he actually said one time when I wasn't even talking to you. Again - English is not your native tongue but you do exceedingly well all things considered - I mean it.

 

[AnxietyRanger]

This from the same guy who jumped in to explain to me that it was "clear" to you that bladerskb meant something other than what he actually said one time when I wasn't even talking to you. Again - English is not your native tongue but you do exceedingly well all things considered - I mean it.

I do genuinely try to understand what people really meant behind words that may fail them. In this case I could not find the meaning you suggest from OPs posts and he did not correct me either. Doesn't mean I can't be wrong on his meaning, I just still don't see it. Any nuance of the meaning of the word vindication does not IMO help explain it, when OP goes on and on about camera vs. lidar. Just telling it like I see it with a best effort.

OP is free to correct either of us, of course.

 

[strangecosmos]

The paper did not compare cameras to lidar. However, we can use the numbers from the paper to compare cameras to lidar. The authors found that at low driving speeds, a four-camera system can achieve a localization accuracy of under 10 cm (3.9 in), just over the length of a credit card. Lidar has an accuracy of 1.5 cm (0.6), about the width of a finger.

Based on these numbers, my hunch is that lidar is overkill and cameras are good enough — crucially, as long as cameras are as accurate at high speeds. (That’s something I’m still trying to figure out.) If the car leaves a margin of error of 45 cm (about 18 inches) and it miscalculates distance by 20 cm, it will still be 25 centimetres (9.8 inches) away from the object.

Further evidence to support my hunch is that one study found human drivers are only able to park with about 10 cm of lateral accuracy (left/right), and much less longitudinal accuracy (forwards/backwards).

With parking, you generally want to be closer than 45 cm to the curb, so the margin of error should be less. However, it isn’t a big deal if the car touches the curb, and in general auto-parking seems like it’s almost already a solved problem. In parking and other low speed situations, a HW2 Tesla can use the ultrasonics as well as the cameras.

Finally, Tesla uses eight cameras, whereas the experiments used only four cameras. Tesla augments its multi-camera system with GPS, unlike the system in the paper. From what I understand, the localization software used in the four-camera system consisted of hand-coded algorithms. It’s possible that deep neural networks could get better performance. At low speeds, Tesla can check its cameras against ultrasonics and radar, and at high speeds can check its cameras against radar.

So, with better hardware and possibly better software, as well as more time, more money, and more engineering talent, it’s likely that Tesla can achieve even better accuracy than was achieved in the experiments. That would close the gap even further between under 10 cm for cameras and 1.5 cm for lidar. The only remaining question is how much this level of accuracy deteriorates at high driving speeds. That’s why I’m currently looking into Tesla’s camera hardware and the software Tesla could be using to correct visual artifacts that occur at high speeds.

 

[calisnow]

The paper did not compare cameras to lidar. However, we can use the numbers from the paper to compare cameras to lidar. The authors found that at low driving speeds, a four-camera system can achieve a localization accuracy of under 10 cm (3.9 in), just over the length of a credit card. Lidar has an accuracy of 1.5 cm (0.6), about the width of a finger.

Based on these numbers, my hunch is that lidar is overkill and cameras are good enough — crucially, as long as cameras are as accurate at high speeds. (That’s something I’m still trying to figure out.) If the car leaves a margin of error of 45 cm (about 18 inches) and it miscalculates distance by 20 cm, it will still be 25 centimetres (9.8 inches) away from the object.

Further evidence to support my hunch is that one study found human drivers are only able to park with about 10 cm of lateral accuracy (left/right), and much less longitudinal accuracy (forwards/backwards).

With parking, you generally want to be closer than 45 cm to the curb, so the margin of error should be less. However, it isn’t a big deal if the car touches the curb, and in general auto-parking seems like it’s almost already a solved problem. In parking and other low speed situations, a HW2 Tesla can use the ultrasonics as well as the cameras.

Finally, Tesla uses eight cameras, whereas the experiments used only four cameras. Tesla augments its multi-camera system with GPS, unlike the system in the paper. From what I understand, the localization software used in the four-camera system consisted of hand-coded algorithms. It’s possible that deep neural networks could get better performance. At low speeds, Tesla can check its cameras against ultrasonics and radar, and at high speeds can check its cameras against radar.

So, with better hardware and possibly better software, as well as more time, more money, and more engineering talent, it’s likely that Tesla can achieve even better accuracy than was achieved in the experiments. That would close the gap even further between under 10 cm for cameras and 1.5 cm for lidar. The only remaining question is how much this level of accuracy deteriorates at high driving speeds. That’s why I’m currently looking into Tesla’s camera hardware and the software Tesla could be using to correct visual artifacts that occur at high speeds.

Bro just shut UP and ZIP it! We don't want your logic, examples and insights. You are stupid! Know why? Cause EVERY other company uses lidar. I watched their demo videos! You think Elon knows something they don't? Get out! End of story. Lidar = everybody. Your "cool story" = TLDR. God I can't wait for my Audi dreammobile in 2019 bombing along at 35 mph (only when surrounded by traffic on both sides tho - cause safety matters).

 

[calisnow]

In the mean time I will spend my waking hours here laughing at your lack of lidar. Hey your fly is unzipped. Made you look! Ha! Ha ha ha ha ha! You Tesla nuts are too much!

 

[AnxietyRanger]

At low speeds, Tesla can check its cameras against ultrasonics

Only at short distances. Cross-traffic detection for example can't rely on ultrasonics.

at high speeds can check its cameras against radar.

Only in a narrow beam at front. Again, cross-traffic detection is not redundant.

 

[calisnow]

The paper did not compare cameras to lidar. However, we can use the numbers from the paper to compare cameras to lidar. The authors found that at low driving speeds, a four-camera system can achieve a localization accuracy of under 10 cm (3.9 in), just over the length of a credit card. Lidar has an accuracy of 1.5 cm (0.6), about the width of a finger.

Based on these numbers, my hunch is that lidar is overkill and cameras are good enough — crucially, as long as cameras are as accurate at high speeds. (That’s something I’m still trying to figure out.) If the car leaves a margin of error of 45 cm (about 18 inches) and it miscalculates distance by 20 cm, it will still be 25 centimetres (9.8 inches) away from the object.

Further evidence to support my hunch is that one study found human drivers are only able to park with about 10 cm of lateral accuracy (left/right), and much less longitudinal accuracy (forwards/backwards).

With parking, you generally want to be closer than 45 cm to the curb, so the margin of error should be less. However, it isn’t a big deal if the car touches the curb, and in general auto-parking seems like it’s almost already a solved problem. In parking and other low speed situations, a HW2 Tesla can use the ultrasonics as well as the cameras.

Finally, Tesla uses eight cameras, whereas the experiments used only four cameras. Tesla augments its multi-camera system with GPS, unlike the system in the paper. From what I understand, the localization software used in the four-camera system consisted of hand-coded algorithms. It’s possible that deep neural networks could get better performance. At low speeds, Tesla can check its cameras against ultrasonics and radar, and at high speeds can check its cameras against radar.

So, with better hardware and possibly better software, as well as more time, more money, and more engineering talent, it’s likely that Tesla can achieve even better accuracy than was achieved in the experiments. That would close the gap even further between under 10 cm for cameras and 1.5 cm for lidar. The only remaining question is how much this level of accuracy deteriorates at high driving speeds. That’s why I’m currently looking into Tesla’s camera hardware and the software Tesla could be using to correct visual artifacts that occur at high speeds.

@Trent Eady - as to the question of whether cameras are accurate at high speed - it is an interesting technical question. However I think we can logic our way to a "yes" by imagining that the answer is no. Let's assume the cameras are not sufficiently accurate at high speed. In that case we must ask ourselves several questions:

1 - Are Tesla's engineers so stupid that it did not occur to anyone on the team that spec'd hardware last year to estimate whether the cameras are sufficiently accurate at high speed to produce a usable image?

2 - If number 1 above is "Yes they are that stupid" - then is the entire autopilot team so dumb that a year later they are still training neural nets with the same not-good-enough cameras, and letting executives hint at public talks that they're very happy with the state of the software direction now - instead of fixing the freaking insufficiently-accurate-cameras?

3 - WHY ARE THEY STAKING THE FUTURE OF THEIR SELF DRIVING EFFORTS WITH THE MODEL 3 on these craptastic cameras?

4 - Is Andrej Karpathy some kind of idiot savant? Has he not said anything about the inadequate cameras?



I think we can reason to a conclusion that the cameras they used are accurate enough - though I'm still curious to see what you find out.

It seems to me this really is an AI software problem at this point - whether it can be solved - we shall see.

 

[strangecosmos]

Only in a narrow beam at front. Again, cross-traffic detection is not redundant.

Do we know exactly how wide the radar beam is?

 

[croman]

Only in a narrow beam at front. Again, cross-traffic detection is not redundant.

FYI, while on AS the other day my car tracked a lead car, appearing white on IC, down a driveway about 25 feet away from the road. I'm not sure if it was tracking it via radar (2.0 Bosch) or visually or extrapolating, but it was accurate. It disappeared as I passed. It still slightly braked. It also had the angle of the car wrong. It was pointed as if it were parallel to me but moved perpendicularly along the IC (horizontal I guess along the ICs axis.). Otherwise it appeared to accurately reflect the movement of that formerly lead car. This latest update sure is impressive. Indicates cross traffic is possible with whatever is in AP2. Clearly not there yet but I've seen other signs even with AP disabled. When I first pull up to an intersection it will briefly show blips of the cross traffic. It stops doing that after 2 seconds or so. Try it and see. You'll be at a red light and be the first car so your car can see the intersection.

 

[croman]

Do we know exactly how wide the radar beam is?

Depends on hardware.

 

[calisnow]

Do we know exactly how wide the radar beam is?

Furthermore redundancy is nice but not necessary if the primary sensor is reliable enough. If the single sensor failure rate is as good as a human or better then the problem is solved (at least for those of us who do not irrationally demand that robot drivers be better than humans).