Too much interference? Maybe one of the reasons Tesla is trying to abandon radar?

[MP3Mike]

Maybe the proliferation of Teslas, and other vehicles with radar, is one of the reasons Tesla is trying to abandon radar?

AV Radar: Don't Cross The Streams - CleanTechnica

Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News! In Ghostbusters, Egon gives the other ghost hunters an important warning for the operation of their proton packs: don’t cross the streams. Why? Because it would be bad. Even “all life as you know ...

cleantechnica.com

In the case of traffic on a two-lane highway, assuming that the radars use randomly selected carrier frequencies, NHTSA predicted that “an automotive radar would encounter power from other radars far greater than the echoes of its own transmissions needed to track other vehicles. The interference approaches four orders of magnitude, or nearly 40 dB, greater than echoes typical of a reference target, as specified for the system.”

In radars that face rearward (as in blind-spot detection systems), “these units are vulnerable to the direct arrival of forward collision avoidance radars that utilize higher power and antenna gain.” The study said, “Our analysis shows these units could experience interfering power from a forward collision avoidance radar that is nearly five orders of magnitude, or 50 dB, greater than the reflections from their specified reference target.”

Similar interference can happen with LIDAR as well, but generally will be less of a problem.

 

[diplomat33]

Green has a thread on Twitter about how Tesla's software has a bug where it struggles with correctly assign radar returns with the right object. If true, this would seem to suggest that Tesla is abandoning radar because they don't know how to do sensor fusion.

https://twitter.com/x/status/1391171736654188545

https://twitter.com/x/status/1391171738294235145

 

[WattPwr]

It's a bit silly to say they don't know how to do sensor fusion. Instead, perhaps Elon is in favour of rapid iteration and he's always looking two steps ahead (for better or for worse). Then, it would be fairly logical to conclude they chose not to fix this bug because they expected to rapidly evolve to vision only. This would fit with what we know of Elon and his various ventures.

 

[drtimhill]

Maybe the proliferation of Teslas, and other vehicles with radar, is one of the reasons Tesla is trying to abandon radar?

This certainly going to be a problem with all "active" sensors (including LIDAR), as there is potential for interference (there were a couple of research articles on this, but can't recall where atm). Cameras, being passive, dont suffer from this (and in fact benefit from other cars headlights in general).

 

[Nuclear Fusion]

Surely radar would need to be maintained for high speed driving (German autobahn)

 

[MP3Mike]

Surely radar would need to be maintained for high speed driving (German autobahn)

Actually the radar is too short range to be effective for that. The cameras would be more useful. (At least from everything I have seen.)

 

[Tam]

Actually the radar is too short range to be effective for that. The cameras would be more useful. (At least from everything I have seen.)

How much longer range does the RADAR need? Tesla's RADAR is 160m (525 ft) while its main camera is 150m although its narrow camera is 250m.

 

[drtimhill]

Surely radar would need to be maintained for high speed driving (German autobahn)

My guess is radar will be used just for follow distance type stuff.

 

[gearchruncher]

How much longer range does the RADAR need? Tesla's RADAR is 160m (525 ft) while its main camera is 150m although its narrow camera is 250m.

Be careful using any of that data as particularly useful. "Max range" doesn't mean max useful range, and what does max range mean for a camera anyway? It assumes some specific image processing or algorithm. It's like asking how far a human can see. I mean, I can see a star from light years away.

Also, that image is from a page that says Radar is an important dual modality to have meanwhile the company is saying it's totally not needed. So not sure how much to believe anything else that is on their too.

 

[emmz0r]

My car has started to break for bridges again. They can never seem to get this right

 

[Earl]

"Max range" doesn't mean max useful range

Good points.
Max range is somewhat meaningful for an active sensor using the classic radar range equation where you determine how much signal power/energy is needed to detect something. You start with the transmitted power (or energy if you have time to integrate), then you subtract twice the distance (out and back) to the object, subtract the portion of the reflected from the object, then subtract out any interference, and see if you still have enough power/energy to actually detect something. <= this part is easy.
In the real world, however, it is very tough:
The reflected part from an object that can still be dangerous can be small and vary greatly with the object and the angle the signal it hits the object.
The interference increases with the number of other transmitters around sending energy into the frequency channel. For the first few Google kids slowly driving a few goofy-mobiles around Mountain View with LIDAR, it was easy. The software geeks could just believe the LIDAR data, plug it into the basic formulas, and get a fairly good idea of what was around them. Tesla, with a million cars on the road needs to deal with the physics problems of interference which gets bigger with the number of cars. Today, in the Silicon Valley, there are times when you'll be in range of hundreds to thousands of Teslas, all blasting their radar signals out. What was a rare corner case 5 years ago is now a common occurrence that, if not handled correctly, will kill someone. What happens when all of the cars on CA101 are transmitting?
High speed driving only reduces the amount of time the detectors can integrate (average) the returned signal to get better detection but, at high speeds, the car may need to take action before there's been enough integration time.
A common solution is to put coding onto the radar signals. This helps to distinguish the the energy from one's own radar from others, however, this either takes longer to transmit (see High speed driving above) or increases the required signal bandwidth. Requiring more signal bandwidth doesn't scale with capacity since frequency is a finite resource.
I'm sure Tesla has enough necessary data to extrapolate to what would happen if all cars on the road try to use radar or LIDAR and realize the need diverges to a problem instead of converging to a solution.

 

[bateau]

Unfortunately I see nothing but downsides in the removal of radar.

How many times has my car told me 1 or more cameras were "obstructed", read: blinded by the sun. This is easily 80% of the trips I have in the morning or evening in winter when the sun shines. And that means no AP. I even had basic cruise control totally shut off because the sun was low in front of the car. Not even non-adaptive cruise control. Nada. Nice on that 100km trip.
And no, the cameras are not dirty, and no, I don't have condensation.
Or what about the phantom braking caused by shadows from trucks or bridges?
Vision alone does not work and will not work.
Please let the car at least use radar as a backup to vision, or better, as additional input.

 

[gearchruncher]

I even had basic cruise control totally shut off because the sun was low in front of the car. Not even non-adaptive cruise control. Nada. Nice on that 100km trip.

The irony is that a little bit of snow will do the same thing when it blocks the radar.

How many times has my car told me 1 or more cameras were "obstructed", read: blinded by the sun

We'll never be able to drive safely just using radar, so this will always cause an issue. The interesting thing is that humans can drive in this case, but at least Tesla's cameras have an issue. It seems like this would be an obvious place that Tesla's current hardware won't work for L3+.

 

[mark95476]

Yup. Elon already stated a while ago that they're removing radar so it makes little sense to keep patching it.

It's a bit silly to say they don't know how to do sensor fusion. Instead, perhaps Elon is in favour of rapid iteration and he's always looking two steps ahead (for better or for worse). Then, it would be fairly logical to conclude they chose not to fix this bug because they expected to rapidly evolve to vision only. This would fit with what we know of Elon and his various ventures.

 

[stopcrazypp]

Unfortunately I see nothing but downsides in the removal of radar.

How many times has my car told me 1 or more cameras were "obstructed", read: blinded by the sun. This is easily 80% of the trips I have in the morning or evening in winter when the sun shines. And that means no AP. I even had basic cruise control totally shut off because the sun was low in front of the car. Not even non-adaptive cruise control. Nada. Nice on that 100km trip.

That unfortunately is a design choice by Tesla, a lot of people with phantom braking wish there is a non-adaptive cruise control available. Ironically it's only available in cars with no AP (back when it's an option), so it's purely a design choice.

And no, the cameras are not dirty, and no, I don't have condensation.
Or what about the phantom braking caused by shadows from trucks or bridges?
Vision alone does not work and will not work.
Please let the car at least use radar as a backup to vision, or better, as additional input.

Radar may be at fault for phantom braking as much as vision. It's not necessarily a shadow, but when the road dips/rises, the radar may get a return from a truck or bridge and then cause braking. I remember Tesla working on a whitelist for a while to address this, back when they relied more on radar.
The current issue may also be an alignment issue like mentioned above, where a stationary radar return gets incorrectly matched to a moving car.

 

[gearchruncher]

Yup. Elon already stated a while ago that they're removing radar so it makes little sense to keep patching it.

Except it's existed for 1.5+ years for the whole fleet of Teslas which are not getting vision only anytime soon. This is not how you do safety important systems, nor how you make customers happy.

If you committed to vision only years ago to the point that you purposefully ignore bugs in it, you also remove this from your website:

A forward-facing radar with enhanced processing provides additional data about the world on a redundant wavelength that is able to see through heavy rain, fog, dust and even the car ahead.

You also maybe don't tweet this a year ago, indicating that radar is part of what makes your system "superhuman":

https://twitter.com/x/status/1247695291739852800

It's clear that just a year ago Elon was not on the vision only path. This is not a long term plan of his, it's very recent.

 

[stopcrazypp]

Green has a thread on Twitter about how Tesla's software has a bug where it struggles with correctly assign radar returns with the right object. If true, this would seem to suggest that Tesla is abandoning radar because they don't know how to do sensor fusion.

https://twitter.com/x/status/1391171736654188545

https://twitter.com/x/status/1391171738294235145

Hard to say this is a fault of vision or that it's because they "don't know how to do sensor fusion". For example, in one of the pictures the radar return of one of the vehicles is very misaligned with the image. That could be from a radar return that gives the wrong location (even though speed may be correct). Not a radar expert so don't know how prevalent such errors can happen, but given originally when they were relying on radar for ACC, it was the cause of lots of phantom braking from misidentifying overpasses as cars (and this isn't only a problem with Teslas, also happens with other cars and with AEB), I imagine it's probably existent.

 

[Earl]

use radar as a backup to vision

I assume that A/P will turn control over to you or stop if a particular level of degradation is detected. It, most likely would turn over control if an optical sensor is blocked whether or not some other sensor (eg. radar) may be covering the same area. I'm quite sure they look at the whole picture.
Given the billions of miles of data collected, you can be sure that, if Tesla removes the radar, it's because, after reviewing all of the sensor data, they are extremely sure it is not germane to the sensor fusion picture they put together. It most likely means they haven't been using it for some time already.
The tweet from 'green' could be evidence that they're already ignoring the radar return and may have been ignoring it for the past 1.5 years. Or it could just be more drivel from another geek on the internet (like my posting here ).
The great thing about machine learning is that the computer can replay a suspect or challenging scenario (such as phantom braking episodes and real emergency braking episodes) many times, weighing all of the sensor data in many ways to determine which weightings of which data would have worked well enough or best.
I suspect that, before A/P goes full FSD, they'll need washers, wipers, and shades protecting all cameras just as we have multiple layers of for the human optical sensors (Car: windshield washers, wipers, defroster, defogger, visor; Human: eye lids, lashes, brow, tears, and irises).
There will also, undoubtedly be times that the FSD pulls off the road and stops because it determines the driving conditions to be unsafe. It will undoubtedly do this at times, upsetting people who might have made a different decision (whether it was a good call or not). There will definitely be grousing on the internet because "FSD won't drive me to Tahoe in a snowstorm so I had to do it myself".
Interesting times are ahead for A/P and FSD.

 

[gearchruncher]

I assume that A/P will turn control over to you or stop if a particular level of degradation is detected. It, most likely would turn over control if an optical sensor is blocked whether or not some other sensor (eg. radar) may be covering the same area. I'm quite sure they look at the whole picture.

The issue is that turning control over to you is just an L2 task (or L3 if it can wait 10-30 seconds). Most people want a car that can actually drive itself with no human intervention, and this is what Tesla has advertised. A car which still requires a driver that can take over at a moment's notice isn't really autonomous, it's just an advanced driver assistance system. Sounds like "autopilot" to me, not "full self driving".

Given the billions of miles of data collected, you can be sure that, if Tesla removes the radar, it's because, after reviewing all of the sensor data, they are extremely sure it is not germane to the sensor fusion picture they put together. It most likely means they haven't been using it for some time already.

Don't be so sure that Tesla has tested things well, historically they have released some pretty untested stuff. This is the company that threw objects at glass windows in public to show they couldn't be broken, and they broke. You think that culture doesn't exist in other places in the company?

The one thing we are 100% sure of though- radar is still in use in every "autopilot" Tesla, including those in the city streets beta. Elon has said this. Behavior of the car says this. If snow is on your bumper everything shuts off. They are flat out highly dependent on radar today, so dropping the use of it is a big deal. Given the fact that they are simultaneously advertising Radar as an important, redundant sensor while their CEO tweets about how they are totally going to get rid of it once they just manage to make it reliable enough should tell you that this is not a consistent plan inside Tesla.

There will also, undoubtedly be times that the FSD pulls off the road and stops because it determines the driving conditions to be unsafe. It will undoubtedly do this at times, upsetting people who might have made a different decision (whether it was a good call or not). There will definitely be grousing on the internet because "FSD won't drive me to Tahoe in a snowstorm so I had to do it myself".
Interesting times are ahead for A/P and FSD.

FSD that pulls over and stops is not FSD. That is not what a human needs to do, and pulling over while still on the roadbed is not safe. What is "full self driving" if it needs a human in the car and can't handle weather like a human does? What would you call a system that doesn't do this if "full self driving" doesn't need to? "All weather no human needed full self driving v3.8, pinky swear"?

 

[Earl]

The issue is that turning control over to you is just an L2 task (or L3 if it can wait 10-30 seconds).

This is a logical misnomer. No autopilot level can always drive under all conditions (eg white-out blizzard or pea-soup fog). Nothing can, not even a snowmobile. At some point the A/P will give up, just as a person will. In the case of A/P, 'giving up' is essentially 'turning control over to you' and it always will be, even if you happen to be 1000 miles away or sleeping in the back. Ideally, it will have had time to drive to a safe location before giving up instead of leaving your car in the middle of a road somewhere. As with human drivers both will happen sometimes.

Most people want a car that can actually drive itself with no human intervention, and this is what Tesla has advertised.

Yes, I want that too. However, there will always be times and places where it won't work. A human driver probably won't be able to drive there either.
I hope you aren't one of those people who think that FSD is better than any human driver and, if you get into a tough situation believe that FSD will be able to get you out. If so, I guess we know who will be the one:

grousing on the internet because "FSD won't drive me to Tahoe in a snowstorm so I had to do it myself"

Don't be so sure that Tesla has tested things well, historically they have released some pretty untested stuff. This is the company that threw objects at glass windows in public to show they couldn't be broken, and they broke. You think that culture doesn't exist in other places in the company?

True. However, I find that some Tesla systems fail and others don't. We haven't seen serious safety issues (such as ABS or electric-assist steering) caused by software faults even though we know there is software there. We have seen the touchscreen and audio software fail fairly often. Therefore, someone within Tesla is definitely prioritizing what must work and what is more valuable if released sooner or with less reliability in order to keep the company moving forward.
That glass episode arguably came out better than if the glass had not broken because of the immense amount of PR buzz it generated.

The one thing we are 100% sure of though- radar is still in use in every "autopilot" Tesla, including those in the city streets beta. Elon has said this. Behavior of the car says this. If snow is on your bumper everything shuts off. They are flat out highly dependent on radar today, so dropping the use of it is a big deal.

No. All you know is that, if Radar can't work, the autopilot shuts off. You don't know whether it is actually used for driving or safety. You certainly have zero idea how "dependent" they are on it. It is dangerous to assert things that you don't know.

FSD that pulls over and stops is not FSD.

You are set to be very disappointed. There are times it is unsafe to drive. At rare times, everything will have to pull over and stop because that is less dangerous than driving. I've been in blizzards in NY, "gully-washer" downpours in Georgia, sandstorms in New Mexico, Fog in CA, etc when it was just plain too dangerous to drive. Even with my 20:10 vision and a robust vehicle, I had to pull over and stop. I've also driven in these kinds of conditions when, perhaps, I shouldn't have. FSD will have to pull over and stop too. Note that no human is needed to do this, therefore, it is still Full Safe Driving. You may rightly argue that it is missing the caveat that this applies only when "it is safe to drive".
Musk is definitely guilty of overestimating humans and the dumb things they will do. I guess his habit of firing people even when they make small mistakes means he doesn't spend time around really dumb people. One would think that he spends enough time on Twitter that he'd understand by now. I guess, he's in 'write only' mode and doesn't read much.
Perhaps the Boring Company can solve this problem eventually but for now, we sometimes have extreme weather in our Solar System.

What would you call a system that doesn't do this if "full self driving" doesn't need to?

If it works in most decent weather, I'd call it AWESOME! I don't need 100.00000%
I haven't seen the ". . . v3.8, pinky swear" but I know enough to know there will be exceptions.
Even if there are 6-sigma weather events that it can't handle, doesn't, mean that FSD won't be awesome and life changing. It can be good enough to be safe, make life much better, and even be awesome without having to be perfect.