I thought you were done here? The problem is not that the additive input of all of the sensors doesn't provide a superset of any one sensor alone. It does. The point that has been made repeatedly, is that, even though not completely done yet, there are a core set of data that MUST be provided: edge detection, range, object classification, color/position recognition, etc... If the removal of any one sensor does not provided that needed data in the resulting subset, then the rest of the sensor suite is useless. So, nobody is arguing that lidar doesn't give you object and edge detection in the dark. But CV plus headlights also do Nobody is saying HD maps don't provide path planning. But so does CV, with the added benefit of dynamic response. Nobody is saying radar doesn't give range/speed data, but CV appear to be capable of that sufficiently. Nobody is saying ultrasonics don't provide close-proximity data. But CV could as well. In short, CV is the only sensor that has a roadmap to be able to provide all the needed driving data to a sufficient degree. Which isn't surprising, as that's how billions of cars are driven today. So while getting to that point is a work in progress, you have to decide where you want to point your architectural development and balance cost/benefit until CV is sufficient, Do you to plan on needing lidar knowing it can't suffice alone? Not a great development plan if focusing on CV can eliminate it. Do you want to support it in the interim? Not at 50-100x the cost of a camera, with perhaps multiple units needed. Radar in the mean time? It's cheap, so maybe so. Same with ultrasonics. But I wouldn't be surprised to see them eventually get deprecated as well. Bottom line: lidar is a crutch. An expensive one.
Definitely expensive in 2016 when AP2 came out. Less so today and in the future. I can see a world where Tesla "solves" CV to 99.9999%, but maybe others need the "crutch" of Lidar to achieve that margin of safety under normal conditions (but say drop to 99% if either CV or Lidar fails, so it can stop safely). Or maybe Tesla will fail to hit that level with CV alone.
It is still expensive today. You are looking at a solid state LiDAR provider with a headline "$500 SS Lidar to come out in 2022". That is just the sensor itself, and one that only gives a limited field of view, so you'll need multiple sensors to get full coverage. This does not take into account the actual processing power and the Lidar-Vision software that needs to be written and validated. In Tesla's case, for example, you would probably need a second HW3 computer just for processing 360 view of all the lidar sensors (assuming your going with solid state, you will need multiple sensors to have a 360 view all around the car) Then, think of all the power consumption and the wiring harness redesign that would need to go into the cars with that new hardware. So, while Lidar sensors might get cheaper over time, the jump to include Lidar is not a simple slapping on of a few Lidar sensors but a much bigger overhaul of the entire suite. Not worth the headache since you MUST have CV solved anyway.
If Tesla is not able to "solve vision" to enough 9's with cameras, then they will need additional sensors to get those extra 9's.
I didn't say not expensive. I said less expensive. Even 8 solid state Lidars at $1000 is cheaper than the $40,000 Lidar systems available from Velodyne in 2016. I'd expect the lower resolution of Lidar to require less processing power than CV. The chip running the Lidar would have to do the distance measurement, since it's based on time of flight of photons. Everything else to generate the point cloud is simple trigonometry, with no NN required. Karpathy showed off Pseudo-Lidar, which produces a 3D point-cloud compatible with Lidar databases from camera data. If that's going in a future FSD system (HW3 re-write or even HW4 based), then the required NN is already baked in. Not that I expect Tesla to start adding in Lidar into the sensor suite. Depending on the application, say for dedicated robotaxies (not from Tesla), the extra expense could be well worth it to get to market sooner. I wrote it in another thread, but I can see a situation where you have CV + Lidar at 99.9999%, with CV being "solved enough" to say 99% alone in case Lidar fails, so the car pull over.[/quote][/QUOTE]
Yes. According to this article, lidar actually requires less computing power than camera vision: "Finally, LiDAR saves computing power. LiDAR can immediately tell the distance to an object and direction of that object, whereas a camera-based system must first ingest the images and then analyze those images to determine the distance and speed of objects, requiring far more computational power." LiDAR vs. Cameras for Self Driving Cars - What's Best? - AutoPilot Review
That chart is hilariously wrong and should not be used. It was created by Lex Fridman (Elon superfan) . Comparing radar resolution to Lidar is like saying being completely blind and only seeing random dots is only slightly worse than having 20/20 vision. That's their 5th gen radar. Tesla uses their 4th gen radar from 2010 (MRR (AP.20) and the ARS410 (AP 2.5+) and that one definitely CAN'T do object classification and has massive problems with differentiating stationary objects or large objects from small objects. Even that isn't adequate enough which is why Waymo created their own imaging radars.
Everything is relative. So that's 5x less than a Velodyne system. It's also 500x the cost of cameras. You know what else has a bill of material cost of $8000 in an mid-priced car? Nothing, except maybe the battery pack in an EV, and that has been ID'd as the biggest area where cost has to be reduced. Moore's law says the computing capability for a size/cost will double every 18 months... which is faster than non-computer hardware typically declines. It'll be cheaper to add more compute than try to add expensive but less data-intensive sensors. Heck, Tesla could double the amount of AI computing power in their cars today for less than the price of 1 lidar sensor, much less 8. This is why many of us see it as a crutch... sort term gains at the cost of dead-end long term architecture.
Rp = Radar Power consumption CVp = camera power consumption Lp = Lidar power consumption CPUp = in Tesla's case this is HW3 computer, but more generally the FSD computer. Rp + CVp + CPUp << Rp + CVp + CPUp + Lp Same concept for processing/compute power. Just because individually Lidar is less processing intensive, does not mean there is no processing required. And if there is processing required, then it is consuming more processing capacity of the CPU. Since we can't get rid of computer vision from any FSD solution any additional processing required for lidar is overhead.
Except that you don't need as much CV computing power if you are using lidar. For example, you don't need CV to do "pseudo-lidar" if you are using lidar. So we don't know what the actual numbers will be for each variable in your equation.
Absolutely false. Remember when a few posts up you agreed that vision must be solved... well that vision solution will still end up using up compute power. You will still need to recreate an accurate virtual reality of your actual surroundings. That will all take the inputs from all the sensors and create the "birds-eye view" that Karpathy was talking about.
I should clarify, I wasn't talking exclusively about Tesla's FSD solution. For other companies, that don't have access to Tesla's data, or constrained to selling cars private owners can afford, then I see a Lidar + CV system to be a viable path to FSD if you can't solve CV to 99.9999%. A short term gain could be the difference of going out of business / getting shut down or not for a Waymo or Cruise.
But like I tried to explain to you earlier, you don't need to "solve vision" to the same number of 9's with lidar than without lidar. With camera-only, you need to "solve vision" to more 9's than with cameras + lidar. I am thinking that camera-only, since it needs to be solved to more 9's will probably use more computing power than CV that only needs to be solved to less 9's.
We’re really getting into the weeds here trying to justify LIDAR compute + vision using fewer cycles than just vision. An autonomous car can safely drive on vision alone. An autonomous car cannot safely drive on LIDAR alone. This is all anyone needs to know for years to come. If LIDAR is useful at any point in the future it will ADD to a solved vision system. You’ve agreed with this up thread. Honestly you’re looking for every little LIDAR crevice into which you can push string. Your argument is noted and nobody will know for some time whether adding it to a solved vision system will provide enough value to justify its cost, unsightliness, and range hit.
You still have to solve to the same number of 9s, but you might not have to solve as many vision issues. You still have to have the same number of 9s for reading signs/stoplights but maybe you can rely exclusively on LiDAR for lane lines so vision doesn't need to handle that.
Then lidar is going to reliably get a lot of cars up to the intersection only to have them sit there at green lights or blow through red ones significantly more often...