Everybody’s discussing computer vision and sensor configuration (cameras, radars, lidars, bla bla bla) as if it was the holy grail and fundamental challenge to solve L3, L4, L5 or “FSD”. It’s like people think that the more sensors, the closer you are to driverless.
Let’s not forget that sensing is only 1/3 of the AV equation. Let’s not forget that there are actually
three fundamental tasks to solve here:
1. Sensing, aka perception;
2. Localization, aka reasoning;
3. Driving policy, aka planning/forethought and negotiation.
We’re really just scratching the surface with these "radar vs lidar" discussions. We’re farting around like Statler and Waldorf with stuff that’s really not interesting anymore.
Perception is pretty much solved now (the issues being fine-tuning, cost-lowering and distribution; "technicalities" IMO).
View attachment 252507
Anyway, with
sensing, we all know you can do collision avoidance, lane holding and traffic aware cruise control. Your car sees
all the things – lanes, pedestrians, street lights, signs (“free space”, “objects”, “driveable paths”) – so you can cruise pretty safely. The car won't kill you. Great.
But that’s not self-driving, is it? That’s not all it takes if you want to “read a book” while your car takes you from your garage to work – no controls touched?
You can equip your car with Bill Gates’ worth of lasers, 32K cameras and yottaFLOP GPUs, but your car
won’t have a clue where it is! Without
localization – i.e. super high precision maps – you'll have to tell your car which turns to make (
intervene), which lanes to go to (
intervene), which exits to take (
intervene), where to park (
intervene), etc. Localization – pin-pointing your location; having your car understand and reason about where it is on the road, and in the world – is absolutely critical to achieve true self-driving.
I think most people get this, but I think people often forget that this is a
monumental task.
First off, you need resolution. GPS won't cut it, but with lidar, radar and/or great structure-from-motion algorithms you can get pretty decent res. maps. So, send some cars a few times down the road with decent sensors and you'll get an HD map. Great.
But how useful is a nanometer precision map once the terrain changes? I mean, you got these super fancy sensors and this super fancy map that you made, and then
wham! Construction!
Wham! Temporary detour.
Wham! You can’t go that way anymore!
View attachment 252504
So you see the grand mapping challenge is actually
updates. I mean really, ultra-high refresh rates. A three month old HD map is no good. A month old HD map isn’t reliable either. I think you want at least daily updates to achieve reliability. You have to be able to trust the maps.
The problem is, such extreme refresh rates require an enormous fleet of vehicles doing that mapping work. I know what you’re thinking: Tesla is the OEM with the biggest fleet of HD-mapping-capable vehicles. They’ve got 260 Model 3’s for Gods sakes! (Ok, that was under the belt
I own a Tesla so I can joke about it.)
But the question is – is Tesla’s fleet big enough for the mapping necessary for FSD?
Well Mobileye certainly doesn’t think so. Mobileye believes the real solution is crowd sourced, cross-OEM, truly massive data harvesting from pretty much every car out there. Mobileye thinks you need to leverage ALL the cameras and internet connections that’s standard in every new vehicle built today. So, not a few hundred thousand cars.
Millions.
Quantity is obviously key to the mapping challenge. Will Mobileye succeed before Tesla? Well I tend to think so. They’ve signed 27 OEMs, most notably VW, BMW, Ford, General Motors, Nissan, Volvo, Audi and Hyundai. That. Is. Effing. Huge.
We shall see, shan’t we
@AnxietyRanger?
Now, finally we have
driving policy. Planning. Car interacting with humans. Signaling to other drivers and pedestrians what you’re intending to do next. Receiving and interpreting such signals from other agents. “Giving and taking”, negotiating. This is kind of the event horizon of autonomous driving. Nobody’s got this figured out AFAIK. Probably takes ridiculous amounts of machine learning to get there, and my guess is as good as yours.
But let's stop dwelling on radars and lidars. Let's talk maps and planning. It's going to be fun, I promise.
Now watch this again: