Unedited Mobileye’s Autonomous Vehicle & other CES 2020 Self Driving ride videos

[willow_hiller]

Camera vision alone will not work for L5 autonomy.

Is this predicated on the idea that autonomous cars will need to be safer than human drivers by several orders of magnitude? Because if we're aiming to achieve performance on par with good drivers, I think surround camera vision and a microphone are all that will technically required (and even the microphone is questionable, as deaf people are perfectly capable of being good drivers).

People have millions of years of evolution behind our ability to use just our eyes and ears to identify things on the road, but those abilities aren't necessarily well suited to traveling at high speeds. Why won't an advanced AI be able to match or outperform human drivers with 8 cameras compared to our two eyes?

 

[diplomat33]

Is this predicated on the idea that autonomous cars will need to be safer than human drivers by several orders of magnitude? Because if we're aiming to achieve performance on par with good drivers, I think surround camera vision and a microphone are all that will technically required (and even the microphone is questionable, as deaf people are perfectly capable of being good drivers).

People have millions of years of evolution behind our ability to use just our eyes and ears to identify things on the road, but those abilities aren't necessarily well suited to traveling at high speeds. Why won't an advanced AI be able to match or outperform human drivers with 8 cameras compared to our two eyes?

Yes, it is predicated on autonomous cars being many orders of magnitude safer than human drivers. If you are aiming for autonomous driving that only works in good visibility and/or is only as good as human drivers, then camera only will work just fine. In fact, this very thread shows that Mobileye has apparently achieved some decent autonomous driving with just cameras. So we know it is doable. But if we want autonomous driving that works in bad weather, low visibility etc and is many times better than humans, then you need extra sensors to compliment each others' weaknesses. Cameras alone, won't be enough.

Yes, I have no doubt that we will get to computers that can "see" with cameras as good as humans. With enough training, we can get computers to detect shape, patterns, colors, etc with cameras and understand what they are seeing, like a human does. And they will probably get better than humans at some point. That's not the issue. The issue is that cameras have physical limitations. They are a passive sensor that requires a certain number of photons to travel from an object to the lens. If not enough photons travel from the object to the lens, it does not matter how good your "brain" is, the camera will fail. That's physics. So what do you do at night when it is too dark to see, or in a dense fog or in another scenario of low visibility? or what if a camera breaks? Or what if a camera is obstructed by mud? In those instances, a camera only autonomous car will not be able to operate at least not at a level safer than a human. So again, if you are ok with that limitation, that's fine. But if you want an autonomous car that can operate in all conditions and is super reliable, many times better than human, then you need more than just cameras.

 

[Eno Deb]

Is this predicated on the idea that autonomous cars will need to be safer than human drivers by several orders of magnitude? Because if we're aiming to achieve performance on par with good drivers, I think surround camera vision and a microphone are all that will technically required (and even the microphone is questionable, as deaf people are perfectly capable of being good drivers).

This is a very superficial argument. For one, today's AI isn't anywhere near as good as the human brain for this kind of task. Also, humans do actually use "maps" to improve driving performance, as evidenced by the fact that driving is more difficult in unfamiliar cities (which we often try to compensate by driving slower to gain more time to understand the environment) ...

 

[willow_hiller]

This is a very superficial argument. For one, today's AI isn't anywhere near as good as the human brain for this kind of task. Also, humans do actually use "maps" to improve driving performance, as evidenced by the fact that driving is more difficult in unfamiliar cities (which we often try to compensate by driving slower to gain more time to understand the environment) ...

That's a really good point. I'm just a little surprised that so many people can firmly conclude that level 5 autonomy is impossible with Tesla's current suite of sensors. I think if HD maps prove necessary, Tesla can include those via firmware.

I always find SAE's writings on their classifications a bit obscure, but I really like the Self-driving car Wikipedia article's summary table of them: Self-driving car - Wikipedia Do you all think that table represents a fair summary? Under Level 5, it says "under all roadway and environmental conditions that can be managed by a human driver." Using that definition, I think just vision-based systems can eventually reach L5.

 

[diplomat33]

I always find SAE's writings on their classifications a bit obscure, but I really like the Self-driving car Wikipedia article's summary table of them: Self-driving car - Wikipedia Do you all think that table represents a fair summary? Under Level 5, it says "under all roadway and environmental conditions that can be managed by a human driver." Using that definition, I think just vision-based systems can eventually reach L5.

The table is good. It is basically copied and pasted from the SAE document.

I don't like the "eyes off", "mind off" terms they use before the table section because they are a bit simplistic. For example, L4 would also be "eyes off" too, "eyes off" is not limited to just L3. Also, L4 can also be "steering wheel optional" not just L5.

Yes "under all roadway and environmental conditions that can be managed by a human driver" is accurate to describe the ODD of L5. And yes, vision alone could probably achieve that, but the question is how reliably. Doing it is one thing, doing it well enough is quite another.

 

[Bladerskb]

I don't know if this clip has been posted but here:

 

[willow_hiller]

I don't know if this clip has been posted but here:

They're honestly really lucky the car obscured behind the bus didn't hit them at around 1:53. But this does demonstrate that they have a good command over the space the vehicle actually occupies. The car was confident maneuvering pretty closely to other vehicles.

 

[Routybouty]

Yikes! That wheel movement must have shaved off 20k miles of tread!

 

[Daniel in SD]

They're honestly really lucky the car obscured behind the bus didn't hit them at around 1:53. But this does demonstrate that they have a good command over the space the vehicle actually occupies. The car was confident maneuvering pretty closely to other vehicles.

It can see that the other vehicle is stopped before getting into its path.

No idea what kind of intersection that is though! Are there lights? Stop signs? Or is it just a free for all?

 

[nepenthe]

They are only short term solutions if you view them as crutches, ie if you think camera only will solve full self-driving and you just need something to temporarily help you out until you finish the camera vision. But that is incorrect. Camera vision alone will not work for L5 autonomy. You need lidar and HD maps, not as a crutch, but as an integral part of the whole full self-driving system.

A relevant quote from "Safety First for Driving Automation":

"As of today, a single sensor is not capable of simultaneously providing reliable and precise detection, classifications, measurements, and robustness to adverse conditions. Therefore, a multimodal approach is required to cover the detectability of relevant entities."

In other words, a single sensor is NOT good enough do full self-driving reliably. So camera only, or lidar only or radar only or HD map only approach will NOT work to do L5 autonomy. So it is not a matter of finding one sensor, like just camera or just lidar, to do full self-driving. No one sensor alone will work. You need cameras, radar, lidar and HD maps working together to make full self-driving really work. Lidar and HD maps are not crutches or short term solutions, they are integral component of the whole system!

Lidar can get you centimeter level accuracy or better, but as I understand it, computer vision can get at least to 10-centimeter accuracy for localization. Both are more than capable of providing the technical requirements for "FSD" or even Level 5 autonomy. HD maps are extra support in relation to localization, you can call it a crutch, support or "part" - this is just semantics, the end result is the same, it's assisting with something a suite of cameras can obviously accomplish as well; where am I in relation to my environment.

I'm not sure anyone is stating FSD is right around the corner utilizing just one camera! Personally, I do not see there's enough evidence to suggest camera vision alone is not and will never be capable of L5 autonomy. Perhaps there's an argument to be made, someone like Waymo will reach that Level 5 ability first, with their sensor suite, but to state with absolute certainty that their approach is the only approach, seems misguided.

My point is, if to achieve Level 5 requires the ability to operate in situations where HD maps are not accurate or present conflicting information any system would have to be able to operate under the assumption the maps are wrong, no? You would not drive into a wall because you're relying on an HD map which says there should be roadway here. Obviously this is extremely simplistic example, but if you have to fall back on not relying on HD maps when there's an issue or disagreement, that implies there's the ability to operate without them.

 

[diplomat33]

Lidar can get you centimeter level accuracy or better, but as I understand it, computer vision can get at least to 10-centimeter accuracy for localization. Both are more than capable of providing the technical requirements for "FSD" or even Level 5 autonomy. HD maps are extra support in relation to localization, you can call it a crutch, support or "part" - this is just semantics, the end result is the same, it's assisting with something a suite of cameras can obviously accomplish as well; where am I in relation to my environment.

I'm not sure anyone is stating FSD is right around the corner utilizing just one camera! Personally, I do not see there's enough evidence to suggest camera vision alone is not and will never be capable of L5 autonomy. Perhaps there's an argument to be made, someone like Waymo will reach that Level 5 ability first, with their sensor suite, but to state with absolute certainty that their approach is the only approach, seems misguided.

Again, the question is not whether cameras can do autonomous driving. Of course, they can. A car can self-drive with just a bunch of cameras. In fact, from the CES presentation, Mobileye says that their camera system can achieve a safety level of 10,000 hours of driving without an accident. But that is not safe enough to deploy as a robotaxi. They want to achieve a safety level of 10 million hours without an accident in order to be safe enough to deploy as a robotaxi. To get to that level of safety, cameras only are probably not enough. Remember, experts want to achieve safety several order of magnitudes better than human drivers.

And the quote I gave is from a paper on safety of autonomous cars where the top experts from over a dozen companies all reach the same conclusion that a single sensor type is not good enough to do autonomous driving many orders safer than a human driver and therefore they strongly recommend using more than one sensor and HD maps to compliment each other to make the car safe enough.

So again, this is not me just guessing that cameras are not good enough, it's the best engineers from over a dozen companies all saying that an autonomous car needs more than one type of sensor if it wants to be a truly safe autonomous car.

My point is, if to achieve Level 5 requires the ability to operate in situations where HD maps are not accurate or present conflicting information any system would have to be able to operate under the assumption the maps are wrong, no? You would not drive into a wall because you're relying on an HD map which says there should be roadway here. Obviously this is extremely simplistic example, but if you have to fall back on not relying on HD maps when there's an issue or disagreement, that implies there's the ability to operate without them.

Like I've said before, HD maps are the back up. You don't drive based on the HD maps. So no, you would never drive into a wall because the HD map was wrong. You would use your camera vision, or radar or lidar to detect the wall and not hit it. But it is useful to have the HD maps for the stuff that the sensors can't do.

 

[diplomat33]

Let's say that you do achieve autonomous driving with just cameras, no lidar, no radar and no HD maps. Great. Let's say that you achieve a safety level of 10,000 hours per accident (that's the number that Mobileye uses for a cameras only system). If you can boost that safety even higher by adding lidar, radar and HD maps to your car that can already do autonomous driving, why not do that? That's my question. It seems silly to me that you would not do that.

In fact, that is Mobileye's approach. They plan to do autonomous driving with just cameras only. And then once it achieves autonomous driving at a safety level around 10,000 hours per accident, they plan to add radar and lidar to boost the safety level to 10 million hours per accident so that the safety is good enough to be deployed without a driver.

 

[nepenthe]

Again, the question is not whether cameras can do autonomous driving. Of course, they can. A car can self-drive with just a bunch of cameras. In fact, from the CES presentation, Mobileye says that their camera system can achieve a safety level of 10,000 hours of driving without an accident. But that is not safe enough to deploy as a robotaxi. They want to achieve a safety level of 10 million hours without an accident in order to be safe enough to deploy as a robotaxi. To get to that level of safety, cameras only are probably not enough. Remember, experts want to achieve safety several order of magnitudes better than human drivers.

And the quote I gave is from a paper on safety of autonomous cars where the top experts from over a dozen companies all reach the same conclusion that a single sensor type is not good enough to do autonomous driving many orders safer than a human driver and therefore they strongly recommend using more than one sensor and HD maps to compliment each other to make the car safe enough.

So again, this is not me just guessing that cameras are not good enough, it's the best engineers from over a dozen companies all saying that an autonomous car needs more than one type of sensor if it wants to be a truly safe autonomous car.



Like I've said before, HD maps are the back up. You don't drive based on the HD maps. So no, you would never drive into a wall because the HD map was wrong. You would use your camera vision, or radar or lidar to detect the wall and not hit it. But it is useful to have the HD maps for the stuff that the sensors can't do.

Safe enough is a relative term. Safe enough comparative to human drivers? Or to Uber drivers? Generally products do not enter the market, or fail to see large penetration/success, because they are either too costly, too cumbersome to operate at scale or do not provide any added benefit over existing solutions. When the cost comparative to an Uber ride is cheaper and marginally safer - you have a successful product. Waymo will not see large scale market penetration and success, until costs are greatly reduced. If a solution is brought to market, minus Lidar and minus HD maps - which is still markedly safer than human drivers and cheaper, why would you not pursue that product? Just because you can add expense, to create something even more "safe" doesn't mean the net result will be less driver/vehicle related accidents when you consider the whole. You have to think about safety at scale relative to the number of deployed Robotaxis in this example.

I appreciate that some experts in the field agree Lidar and HD maps are integral, but that is not the sole opinion, obviously, otherwise there would be no debate here. To reiterate, I think the premise "not good enough" is flawed at the outset, if you can get 10,000 robotaxis on the road, two years earlier than say Waymo, which are 25% less likely to get into an accident compared to a human driver and offer rides at half the cost - you have a winning solution on both fronts; safety and cost. Even if these could be much safer with Lidar and HD maps.

 

[ItsNotAboutTheMoney]

Safe enough is a relative term. Safe enough comparative to human drivers? Or to Uber drivers? Generally products do not enter the market, or fail to see large penetration/success, because they are either too costly, too cumbersome to operate at scale or do not provide any added benefit over existing solutions. When the cost comparative to an Uber ride is cheaper and marginally safer - you have a successful product. Waymo will not see large scale market penetration and success, until costs are greatly reduced. If a solution is brought to market, minus Lidar and minus HD maps - which is still markedly safer than human drivers and cheaper, why would you not pursue that product? Just because you can add expense, to create something even more "safe" doesn't mean the net result will be less driver/vehicle related accidents when you consider the whole. You have to think about safety at scale relative to the number of deployed Robotaxis in this example.

I appreciate that some experts in the field agree Lidar and HD maps are integral, but that is not the sole opinion, obviously, otherwise there would be no debate here. To reiterate, I think the premise "not good enough" is flawed at the outset, if you can get 10,000 robotaxis on the road, two years earlier than say Waymo, which are 25% less likely to get into an accident compared to a human driver and offer rides at half the cost - you have a winning solution on both fronts; safety and cost. Even if these could be much safer with Lidar and HD maps.

They don't have to be safer than a human. They have to be safer than a human with cheap driver assistance. It's a moving target.

 

[Daniel in SD]

They don't have to be safer than a human. They have to be safer than a human with cheap driver assistance. It's a moving target.

Beating Mobileye's technology with a target of one failure every 10,000 hours for a vision only system in combination with a human driver with sophisticated driver monitoring and speed limit enforcement (it's probably coming soon!) seems nearly impossible. The failures of human drivers and machines are probably very uncorrelated.
We could also make the roads way more safe by limiting all cars to 25mph. At some point you have make a tradeoff between safety and convenience.

 

[Eno Deb]

Lidar can get you centimeter level accuracy or better, but as I understand it, computer vision can get at least to 10-centimeter accuracy for localization.

Localizing yourself against known landmarks is an entirely different problem from creating a 3D image of other (possibly unknown) objects around you, which is where Lidar excels. There are various ways of doing that using vision (stereoscopic cameras, monocular but moving cameras, known shape estimation, estimation based on the touch points with the road surface and others), but it is significantly harder than self-localization. This is probably the reason why Tesla still relies on the front Radar for many things (which has the disadvantage that it doesn't work well with stationary objects).

 

[ItsNotAboutTheMoney]

Beating Mobileye's technology with a target of one failure every 10,000 hours for a vision only system in combination with a human driver with sophisticated driver monitoring and speed limit enforcement (it's probably coming soon!) seems nearly impossible. The failures of human drivers and machines are probably very uncorrelated.
We could also make the roads way more safe by limiting all cars to 25mph. At some point you have make a tradeoff between safety and convenience.

Their failure rate is meaningless. What matters outcome from failures.

Annual US VMT is 3,200,000,000,000 miles. There are around 40,000 fatalities per year. That's one death for every 80,000,000 miles driven. Even at 80mph that would be 1,000,000 hours of driving.

A failure every 10,000 hours would mean at 100 failures per 1,000,000 hours. If the fatality rate from those failures was 1 in 100 it'd clearly be worse than humans with the current fleet with current technology.

The clear majority of the current vehicle fleet does not have:
- backup camera
- ACC
- autosteer
- blind-spot detection
- AEB
- rear collision alert
- front collision alert
- rear cross-traffic alert

The target will keep moving.

 

[S4WRXTTCS]

Their failure rate is meaningless. What matters outcome from failures.

Annual US VMT is 3,200,000,000,000 miles. There are around 40,000 fatalities per year. That's one death for every 80,000,000 miles driven. Even at 80mph that would be 1,000,000 hours of driving.

A failure every 10,000 hours would mean at 100 failures per 1,000,000 hours. If the fatality rate from those failures was 1 in 100 it'd clearly be worse than humans with the current fleet with current technology.

The clear majority of the current vehicle fleet does not have:
- backup camera
- ACC
- autosteer
- blind-spot detection
- AEB
- rear collision alert
- front collision alert
- rear cross-traffic alert

The target will keep moving.

One of the biggest obstacles with self-driving cars is determining how safe they need to be.

I agree with you in that the target will keep moving, BUT that's because the target includes terrible drivers. Terrible drivers and people doing really dumb things make up a large component of our fatalities. The driver assistance stuff along with other changes (like Volvo limiting the top speed of their vehicles) is really meant to lower fatalities as a result of terrible drivers. They don't help nearly as much for drivers who don't text, don't drive drowsy, don't follow too closely, etc.

The funny thing is that these technologies haven't helped as much as one would anticipate. That's because people keep using these driver assistance stuff as an excuse to do things like texting. Or they perform so badly they erode confidence in them so people turn them off (like lane keep assist is a common one that people turn off). Or they just come to ignore all the beeping.

The other thing at play with humans is we drive in adverse conditions, and that's where a lot of people get into trouble. The autonomous car simply won't do that so they have a bit of an advantage that has to be accounted for.

To minimize the moving target we should simply establish rigorous testing procedures that demonstrates that it's just as good as professional drivers at handling all kinds of situations.

Where the test is a combination of simulated data, and real life controlled testing.

We absolutely have to allow for failure out of the gate with the expectation that over time it will improve. Where we simply limit it to geo-fenced areas, weather restrictions, speed restrictions and allow it to incrementally grow.

I also think we need to back away from driver safety a bit on it, and focus more on all the other benefits of self-driving cars.

Like Pedestrian safety, traffic improvements (through V2X technology), reducing car ownership to reduce the physical footprint of car ownership, convenience of it, etc. Plus economically it can drive a lot of growth. Humans are inherently lazy so if we cater to that laziness people will buy.

 

[TerrapinThief]

And the quote I gave is from a paper on safety of autonomous cars where the top experts from over a dozen companies all reach the same conclusion that a single sensor type is not good enough to do autonomous driving many orders safer than a human driver and therefore they strongly recommend using more than one sensor and HD maps to compliment each other to make the car safe enough.

So again, this is not me just guessing that cameras are not good enough, it's the best engineers from over a dozen companies all saying that an autonomous car needs more than one type of sensor if it wants to be a truly safe autonomous car.

Several things worth remembering. A lot of the pro-LIDAR people are involved with MobilEye and MobilEye is selling HD maps, they're obviously going to campaign (and probably politically lobby) for consensus that their solution is correct. Tesla isn't using a single sensor or even just "vision", they have RADAR. QUALCOMM's video about their product only shows the cars outfitted with RADAR and cameras, no LIDAR (seems to be LIDAR on the validation vehicle, but not the product vehicles).

 

[diplomat33]

Safe enough is a relative term. Safe enough comparative to human drivers? Or to Uber drivers? Generally products do not enter the market, or fail to see large penetration/success, because they are either too costly, too cumbersome to operate at scale or do not provide any added benefit over existing solutions. When the cost comparative to an Uber ride is cheaper and marginally safer - you have a successful product. Waymo will not see large scale market penetration and success, until costs are greatly reduced. If a solution is brought to market, minus Lidar and minus HD maps - which is still markedly safer than human drivers and cheaper, why would you not pursue that product? Just because you can add expense, to create something even more "safe" doesn't mean the net result will be less driver/vehicle related accidents when you consider the whole. You have to think about safety at scale relative to the number of deployed Robotaxis in this example.

I appreciate that some experts in the field agree Lidar and HD maps are integral, but that is not the sole opinion, obviously, otherwise there would be no debate here. To reiterate, I think the premise "not good enough" is flawed at the outset, if you can get 10,000 robotaxis on the road, two years earlier than say Waymo, which are 25% less likely to get into an accident compared to a human driver and offer rides at half the cost - you have a winning solution on both fronts; safety and cost. Even if these could be much safer with Lidar and HD maps.

I think you might be minimizing the importance of safety. Marginally safer or 25% safer is not good enough because it would still result in hundreds or thousands of deaths depending on how many robotaxis were deployed. And yes, I know that robotaxi deaths are inevitable. But when a company deploys robotaxis in large numbers with no drivers inside, they will be liable for every single at-fault accident. Too many accidents could bankrupt the company, especially if people die in the accidents. Not to mention the bad PR would scare customers away. And if investigations show that the accidents were avoidable with LIDAR or HD Maps but the company chose not to include them, that would be very bad. So no, I don't think you would have a successful product if you had a robotaxi that was cost effective and only marginally safer than human drivers. You might be first to market but you would also go out of business pretty quickly.

That's why companies like Waymo and Cruise who have L4 autonomous cars now are still waiting before deploying them in large quantities. They know the safety has to be much much better than human drivers before they can deploy them. It's why Mobileye is setting a target of 10 million hours of driving per accident. Just saying the safety is marginally better is not good enough.

Actually, there is no real debate. Virtually all experts agree now that LIDAR and HD Maps are required for safe autonomous cars. The only disagreement comes from Tesla because they think cameras can achieve autonomous driving that is "safe enough".

It's about redundancy. Each sensor has pros and cons. So if you only use cameras, your robotaxis will have limitations. If you use multiple sensors then your robotaxi will have fewer limitations since the different sensors can cover for each other's weaknesses. Plus, if a sensor fails, the other sensors can cover and prevent a failure, allowing for higher safety and reliability. Cameras can do autonomous driving but will have too many limitations. If just one of the cameras fail, your robotaxi could crash, if a headlight goes out, your robotaxi can't drive at night, etc... That's why experts argue that having different sensors is not optional.

Several things worth remembering. A lot of the pro-LIDAR people are involved with MobilEye and MobilEye is selling HD maps, they're obviously going to campaign (and probably politically lobby) for consensus that their solution is correct. Tesla isn't using a single sensor or even just "vision", they have RADAR. QUALCOMM's video about their product only shows the cars outfitted with RADAR and cameras, no LIDAR (seems to be LIDAR on the validation vehicle, but not the product vehicles).

They did not just come up with an expensive system and then decide to lobby for it. They are pro-LIDAR because they know LIDAR offers some real advantages in doing safe autonomous driving. If they thought they could do safe autonomous driving without LIDAR, they would.

Yes, Tesla uses radar too but only in the front because they needed it for TACC. Teslas lack radar on the sides or rear.