Lidar vs Camera revisited

[heltok]

Here is a very recent video going into Lidar perception:

My takeaway:
Current Lidar approaches do the feature engineering online between the input and the neural network.
Tesla are doing their feature engineering when they generate the labels offline. This saves online computation and no information is lost from the input to the neural network. It also abstracts away the feature engineering which plays into software 2.0 while Lidar inherently has to be more software 1.0.

At some point(no pun intended) the Lidar need to go to pseudo camera to make the data palatable to the neural network, while camera is doing pseudo lidar to generate the labels. If you use range pseudo image directly, you need to discretize the ranges, this loses information. Meanwhile when camera does pseudo Lidar you don’t lose data, but you get holes in the output. These holes can be filled up with using data from the future or different drives offline, but you cannot fill up the holes using the future when you run online.

The Lidar approach seems less clean to me. It must be so easy to work for Tesla, each group can focus on doing what they like doing. The point cloud guys can do point clouds, the neural network guys can do neural networks. These groups don’t really need to care about what the other group does as long as they have agreed on their interface, which just is labels. Meanwhile the Lidar team needs to do both and they will change the interface between the data and the neural network all the time as they try to figure out what is the best interface.

 

[Terminator857]

I wonder if we will get lidar by hw5. Maybe it will cost $20 by then. @Bladerskb says it costs $40 each now?

 

[okcazure]

I wonder if we will get lidar by hw5. Maybe it will cost $20 by then. @Bladerskb says it costs $40 each now?

Doubtful. In a interview this year Elon said no Lidar, even if the hardware was free. I'd link but I don't recall where I saw it. The shareholder meeting is my best guess of where I saw it...

 

[heltok]

There are many costs with adding a sensor besides just the hardware of the sensor. Power, network, compute etc. Integrating into the software stack will cost a lot of man power…

Elon believes the cameras has enough information to solve FSD, it seems likely to be true given that humans with cameras can drive. We will see if his intuition was correct or not once we have a proof of it working.

 

[EVNow]

I wonder if we will get lidar by hw5. Maybe it will cost $20 by then. @Bladerskb says it costs $40 each now?

I think that boat has sailed.

Its possible we'll get a much richer sensor suit sometime down the line - after FSD has gone to general release and Tesla can claim they have fulfilled their obligation.

But Tesla FSD will continue to be mainly Vision, with standard maps augmented by more cameras & other sensors by 2030.

This can play out differently if Tesla hits a wall and can't get to FSD general release with current set of sensors and some other company gets to a general release of FSD for consumer cars with a larger set of sensors.

 

[Daniel in SD]

We will see if his intuition was correct or not once we have a proof of it working.

When will we know if he was incorrect?

 

[DanCar]

When will we know if he was incorrect?

Lidar night time visibility looks hard to dispute. You can make the headlight thing work at night, but I doubt it will work as well if lidar was helping.
When is pothole avoidance going to work? Will lidar help know how deep?
Elon says a lot of things about FSD. Every long term statement has been wrong.

 

[okcazure]

When will we know if he was incorrect?

I guess will know he's correct when all cars with FSD get fsdbeta, not just a small subset in the beta test program. There is a long way to go before this happens but DOJO should speed this up.

The only way we'd know he's incorrect is if Elon says he was wrong and needs Lidar for FSD to work, which I suspect is never!

 

[okcazure]

Lidar night time visibility looks hard to dispute. You can make the headlight thing work at night, but I doubt it will work as well if lidar was helping.

If it's only about nighttime visibility, I would think FLIR would be the way to go. It seams it would integrate into the existing NN much easier than LIDAR. I'm not an expert in the slightest on this, so this is only my speculation.

Advanced Driver Assist Systems | Teledyne FLIR

Advanced Driver Assist Systems (ADAS) will revolutionize travel and transport while improving safety. Thermal cameras, along with other sensors, will be used to quickly detect and classify hazards in the road, increasing safety for passengers and pedestrians.

www.flir.com

 

[EVNow]

When will we know if he was incorrect?

Like here ...

This can play out differently if Tesla hits a wall and can't get to FSD general release with current set of sensors and some other company gets to a general release of FSD for consumer cars with a larger set of sensors.

 

[heltok]

When will we know if he was incorrect?

It takes one example to prove that something is possible, it takes an exhastive list of examples to prove that something is impossible. But for argument’s sake, let’s say in 100years without any example our grand children can probably call it a dud.

 

[powertoold]

At this point, I'm fully convinced vision-only FSD will work and is the best approach to achieve generalized FSD first. In the future, we may have improved and cheap sensors, but for now, Tesla will win this race to a consumer-deployable generalized (90%+ of USA) robotaxi service.

I'm convinced because what Tesla is able to do with their approach and diverse data advantage. Here's one example:

https://twitter.com/x/status/1467198613298114561

Tesla is able to implement state-of-the-art approaches on the fly, test them out in their diverse fleet in all weather conditions and locales, and then re-deploy / fine-tune / unit-test on the fly. Tesla has the best iterative approach to a problem that no one has solved. Their iterations are 100x (my hyperbole) faster and more data-based than any competitor.

 

[S4WRXTTCS]

Tesla will win this race to a consumer-deployable generalized (90%+ of USA) robotaxi service.

The issue with those race parameters is they favor Tesla's generalized approach, and they're US centric.

90% is an exceptionally large amount of roads where the economics simply don't make sense to support robotaxi services, and so they'll likely require generalized AI which were far from achieving.

The limitation to the US is problematic because it doesn't look like we're going to be a leader when it comes to commercial deployment of consumer vehicles capable of L3 or above autonomy.

Germany will soon have multiple L3 vehicles available for purchase
China might end up taking the crown for the first to L4, and most of the Lidar cars Blader talks about are Chinese.

I'd say the goal should be the first to L4 that's capable of traveling 598+ miles (the longest path in Germany as far as I can tell) without requiring the passenger to do anything except plugging it in to charge, and cleaning the sensors (tiny bias for HW3). For weather limitations I'd say it has to deal with mild to moderate weather conditions without resorting to traveling abnormally slow.

The goal I'm stating is something that would give the vast majority of people in the country of deployment a pretty sweet L4 vehicle.

 

[Tam]

...China might end up taking the crown for the first to L4...

Waymo is already proven as L4 robotaxi with no drivers but only in Chandler, AZ.

Generalized L4 is hard. I am not sure China will beat anyone with generalized L4 because:

California halts Pony.ai's driverless testing permit after accident

A California regulator on Monday said it has suspended a driverless testing permit for startup technology firm Pony.ai following an accident - the first time it has issued such a suspension.

www.reuters.com

 

[JHCCAZ]

...The goal I'm stating is something that would give the vast majority of people in the country of deployment a pretty sweet L4 vehicle.

I've been saying for a while that highly useful L4 need not be a nearly-universal i e drive-almost-anywhere-anytime capability. People will want it as long as it works most of the time. I think the key is that the probability of trip-completion success* can be estimated before the trip is initiated (or before the ride is hailed, confirmed & accepted in the case of a RoboTaxi).

Some users, in some circumstances, will be willing to agree to a lower probability of timely completion depending on their circumstances.

*Success mostly meaning timely completion and/or a minimal need to walk (or wheelchair-propel) a bit between L4-car and origin/destination. In cases of bad weather, known road construction issues and so on, there will be increased risk of delay but in most cases, not indefinite stranding or personal danger. And no form of transportation has zero risk of such issues.

If I cannot drive myself (a very possible future scenario) and I need to make a trip, I will probably be satisfied with a car that tells me the estimated probability of trip completion is 75% or better, with the most probable downside being significant delay. If it tells me some trip is unavailable or very likely to fail due to weather or route restrictions, with no acceptable alternate route, I'd still rather have that L4 car or RoboTaxi service than not have it. If a competitor has a known better L4 in these respects, then of course I'll be motivated to buy that car instead / use that service instead.

A lot of people in these discussions simply look at it as a potential convenience if it works wonderfully well, with the alternative being "I'll just drive it myself". In this context the automation needs to be pretty good so it's less annoying to ride than to drive. People without a license, or who need to move other non-drivers around, will be motivated to put up with a less-perfect AV as long as it's not a matter of personal danger.

 

[Fernand]

The people who think Lidar should be added are imagining adding a waffle iron to a microwave oven.

That said, I think that any level of "FSD", with ANY sensor set, becomes impractical in heavy enough Weather. Even the most advanced aircraft are grounded at times.

Humans take truly extraordinary risks with significantly elevated chance of collision or loss of control when they drive out into a storm.

Since our regulators seem to think in absolutes, and aren't taken with the demonstrated 10:1 accident rate improvement with Tesla's current base autopilot, it's hard to comprehend how they allow humans to engage in criminally negligent severe weather drives. Dear Missy, we need a weather interlock on traditional automobiles, trucks and tractors!

 

[Terminator857]

The issue with those race parameters is they favor Tesla's generalized approach, and they're US centric.

Elon is eager to release in Canada. I suspect it will happen soon, Q1'2022?

Germany will soon have multiple L3 vehicles available for purchase

Heard that a few years back and wasn't true then.

China might end up taking the crown for the first to L4, and most of the Lidar cars Blader talks about are Chinese.

Elon is quite the competitor. I doubt he will allow anyone to get far ahead. If someone does release he will be close behind even if it is a polished turd. But maybe that is only true in the U.S.

I'd say the goal should be ...

General FSD beta released would be nice. Then the next rung on the ladder might be L3 in stop and go traffic.

The goal I'm stating is something that would give the vast majority of people in the country of deployment a pretty sweet L4 vehicle.

5+ years down the road is my estimate for a sweet L4.

 

[linux-works]

Elon believes the cameras has enough information to solve FSD, it seems likely to be true given that humans with cameras can drive. We will see if his intuition was correct or not once we have a proof of it working.

such a gross GROSS oversimplifiation, and yet it gets taken out and shown the light every week, it seems.

the human brain still does processing that we have no idea about, yet. we dont even have half of what we need to know, in order to 'build a new brain' that is even 10% as smart as a human, just in sensor processing and world-view, alone.

this wont ever work in our lifetimes. hubris does not do anyone any good when the science has decades yet to go before we even scratch the surface of what it means to THINK.

and cars that run statistics on huge data models is NOT AT ALL how the brain works. not even close. we think we know about neural nets but that's not at all a sufficient model for how the brain truly processes info.

vision will not work until we have a handle on how machines can THINK. processing data is not thinking.

keep crashing, fanboys. just stay away from my car, ok?

 

[EVNow]

this wont ever work in our lifetimes. hubris does not do anyone any good when the science has decades yet to go before we even scratch the surface of what it means to THINK.

Lots of unstated assumptions here.

Human brain may be great but tends to overestimate what can be done in short term (FSD this year !), but, underestimate in the long run (No FSD this century or only 4 computers needed the world over).

 

[Terminator857]

I've never seen someone, Elon, being so wrong so many times on FSD predictions, greater than 1 year out. You would think he would learn from his mistakes. Anyone quoting Elon and saying because Elon says, doesn't have much credibility from my perspecitive, because you are quoting someone who has negative credibility with respect to FSD long term. In other words everything he says greater than one year out, the opposite is true.