Autonomous Car Progress

[powertoold]

I don't know the details of how Waymo and Cruise software works, however, long term I assume they will expand with drivable space (even if map restricts) using onboard sensors including Lidar... if they haven't done so already. I'd actually be surprised if they aren't doing this already.

They don't do this. That's the whole point of using HD maps. You predefine the HD map static layer, which includes drivable space (essentially all road space the car can drive on, doesn't yet include traffic rules). Once the car is driving autonomously and localizes itself within the HD map, the LIDAR is used to detect dynamic objects (objects that may need to be maneuvered around within the drivable space) like cones, obstructions, cars, peds, etc.

 

[powertoold]

He also said that SuperVision, the productization of their camera-only L4 system will be able to do everything on the clip they showed.

Do you have the source for this (like an article)? From my previous experience with you, you tend to make deceptive assertions with links or information that doesn't match the facts you're stating (like posting a foreign language video to prove that bmw already deployed the traffic light feature, when they hadn't after I used Google translate on the description text).

 

[Bladerskb]

Do you have the source for this (like an article)? From my previous experience with you, you tend to make deceptive assertions with links or information that doesn't match the facts you're stating (like posting a foreign language video to prove that bmw already deployed the traffic light feature, when they hadn't after I used Google translate on the description text).

Says the guy who falsely claimed that geely with supervision was next year and is low volume.

That early sensor fusion isn’t industry standard.

That Lidar doesn’t detect free space.

And that’s just 3 out of the hundreds of overtly false things you have said over the years.

Secondly I already proved to you that BMW had red light warning well before Tesla in consumers hands and that the video I posted was of a journalist testing the new urban cruise control that was readying release in November according to BMW’s own press release and their previous history of software updates. (March, July and November)

 

[Snuffysasa]

They don't do this. That's the whole point of using HD maps. You predefine the HD map static layer, which includes drivable space (essentially all road space the car can drive on, doesn't yet include traffic rules). Once the car is driving autonomously and localizes itself within the HD map, the LIDAR is used to detect dynamic objects (objects that may need to be maneuvered around within the drivable space) like cones, obstructions, cars, peds, etc.

You have a source with Waymo/Cruise engineers or just assuming based on videos and what you understand about how their system works?

You misunderstand HD maps. Yes they are used to localize and give you a predefined space, but that is a starting point and not the end all of end all. The maps are very helpful to understand the space beyond what the sensors see, or when sensors are occluded, and to increase confidence and redundancy even when the sensors do see. But you are suggesting that onboard sensing cannot override the HD map data which is of course not true.

Lidar does more than just localization and dynamic obstacle detection.

Do you have the source for this (like an article)? From my previous experience with you, you tend to make deceptive assertions with links or information that doesn't match the facts you're stating (like posting a foreign language video to prove that bmw already deployed the traffic light feature, when they hadn't after I used Google translate on the description text).

Amnon has said many times that it is their full camera subsystem from their L4 system and has said it will support all the features. It will be a much better version of FSD, I'll also travel to China sometime to try it. Also, again the BMW traffic light feature did launch, perhaps not in the US though.

 

[daniel]

That is the entire debate summed up.
Tesla is saying no need for lidar as radar is better at the important stuff (rain, snow, fog, dust, and soft debris like plastic bags) and good enough on resolution and camera vision gives you the rest.
Waymo and their copy cats are saying lidar is more detailed than radar and we will toss radar in for the fog, rain, dust, snow.

I am on team Tesla!

I want Tesla to be the first, so that I don't have to buy a non-Tesla when the first driverless car for the consumer comes out. That's why I don't want them to paint themselves into a corner by committing themselves to only using the present hardware. I want them to be open to using whatever hardware can best do the job, and I don't believe that anybody knows that yet, since the software has not yet been developed.

Wow, that went WwwwwaaaaaaaaaaaaaaaayyyyyyyyyyyyyyyyyyYYY over your head.

The cameras do not determine the free space.
The cameras do not determine anything for the FSD car -- they provide data.

Lidar does not determine free space.
Lidar provides data to the computer that determines the free space.

Come on, that's just playing with words. Everybody knows that for fsd, sensors do nothing without processing by a computer and software. Any and all systems for fsd are going to have computers and software. When discussing the pros and cons of different kinds of sensors it's an acceptable shortcut to refer to the sensor+computer+software suite by naming the sensor.

Tesla makes the best cars on the road. I want my first driverless car to be a Tesla. I am 100% on Team Tesla. But that doesn't mean I need to assert that their approach will be the one that wins. Nobody can know yet which approach will win.

 

[powertoold]

You have a source with Waymo/Cruise engineers or just assuming based on videos and what you understand about how their system works?

I posted an article from Cruise. You can find another that supports your point.

 

[diplomat33]

Tesla has filed with the FCC to use a millimeter wave radar for FSD:

"The equipment under test (EUT) was an Vehicle Millimeter-wave Radar Sensor operating in 60 GHz band (60-64 GHz)"

Tesla files to use new 'millimeter-wave radar' on its 'full self-driving' electric cars - Electrek

IMO, this is a good sign that Tesla will probably add imaging radar to Tesla cars soon.

 

[Buckminster]

Tesla has filed with the FCC to use a millimeter wave radar for FSD:

"The equipment under test (EUT) was an Vehicle Millimeter-wave Radar Sensor operating in 60 GHz band (60-64 GHz)"

Tesla files to use new 'millimeter-wave radar' on its 'full self-driving' electric cars - Electrek

IMO, this is a good sign that Tesla will probably add imaging radar to Tesla cars soon.

Is this to replace the existing radar or in addition? Just one per vehicle?

Could the timing of this info suggest that the refreshed S/X will include this? Or will this come further down the road? (excuse the pun..)

 

[diplomat33]

Is this to replace the existing radar or in addition? Just one per vehicle?

Could the timing of this info suggest that the refreshed S/X will include this? Or will this come further down the road? (excuse the pun..)

I don't think we know. Yes, presumably it would replace the existing front radar. But we don't know if Tesla plans to add more than 1 per vehicle.

And yes, it could go on the refresh S and X. We don't know.

 

[Mardak]

Could the timing of this info suggest that the refreshed S/X will include this?

Even if this is only for the lower volume S/X, the additional fleet data generating training data with more accurate distance/height/depth/speed information without requiring human labelers would benefit the vision neural networks used by the existing fleet. Notably Arbe radar has 100° width/azimuth, 30° height/elevation and 300m range whereas currently the Continental radars have 45° width, 18° height and 160m range, so the new radar would provide better data for training perception of most of the wide fisheye camera 120°, some of pillar cameras and all of the narrow forward camera 250m.

Phoenix Imaging Radar - Arbe

Here's a quick mockup with red lines for what a single forward Arbe radar could cover. (Looks like 250m for narrow forward was shortened, but I estimated 300m of the radar based on 2x of main camera distance.)

If there's one at each front corner, most of the pillar cameras, which point back from 30° to 120° relative of the front 0°, would be covered as well.

 

[PhantomX]

Apology if this has been addressed already or if this is the wrong thread, but I am wondering how any of these autonomous driving technologies deal with busy pedestrians areas such as school zones? Often time we human look at body motions and gestures to understand the intent of crossing guards and people trying to cross the street. Do any of the systems work in these scenarios?

 

[diplomat33]

Apology if this has been addressed already or if this is the wrong thread, but I am wondering how any of these autonomous driving technologies deal with busy pedestrians areas such as school zones? Often time we human look at body motions and gestures to understand the intent of crossing guards and people trying to cross the street. Do any of the systems work in these scenarios?

Yes. Waymo and others are able to read hand gestures. They can handle these situations.

 

[Mardak]

New video from CES today that goes into detail on Mobileye's FSD solution, including how they build the scalable maps:

Starting from Mobileye's 10KB/km REM data generation and assuming Tesla is 3 orders of magnitude worse in data efficiency, a vehicle driving at 30mph would generate data at less than 1Mbps which is well within 3G upload speeds.

Seems like this type of data could be just constantly streamed back to the Tesla mothership without explicit shadow mode triggers. Tesla vehicles already uploads all trip details when put into park including when Autopilot is available, Autosteer/TACC active, accelerator override, etc. https://twitter.com/greentheonly/status/1168201533126840321

 

[PhantomX]

Yes. Waymo and others are able to read hand gestures. They can handle these situations.

Awesome. Thanks for the info!

 

[heltok]

The radar will be super useful. See the mobileye video. Another cool benefit of using it, it can be used to label their dataset making the neural network better at seeing things in images even for those cameras facing other directions than the radar.

So whenever the cameras fails to see a vehicle far away that the radar sees, they send that data to their data engine. Then in a T-intersection when doing a left turn, the side camera can use the same tiny pixel movements to predict vehicles.

 

[DDotJ]

Tesla has filed with the FCC to use a millimeter wave radar for FSD:

"The equipment under test (EUT) was an Vehicle Millimeter-wave Radar Sensor operating in 60 GHz band (60-64 GHz)"

Tesla files to use new 'millimeter-wave radar' on its 'full self-driving' electric cars - Electrek

IMO, this is a good sign that Tesla will probably add imaging radar to Tesla cars soon.

I have a suspicion that this is not for Autopilot. It lines up with this filing with the FCC for a short range radar in the 57-71 GHz band for a short range mmWave interactive motion sensing device.

In that FCC filing it says it would be integrated in passenger vehicle interiors to detect movement and classify vehicle occupants. I assume it's a similar technology to Google's Soli chip in the Pixel 4 and Nest devices.

The device could be used, inter alia, to reduce the risk of pediatric vehicular heatstroke, protect vehicle occupants from injury through advanced airbag deployment and seatbelt reminders, and enhance theft prevention systems.

Until recently, most of these technologies have focused on sensing outside the vehicle (e.g., cameras, sensors, and radar used for forward collision warning, automatic emergency breaking, lane keep assist). However, there are numerous safety and security benefits of using RF devices for sensing inside the vehicle, both while the vehicle is stationary and while it is in motion.

mmWave radar technology can detect a child left in a vehicle and has some advantages over other sensing systems, including camera-based systems or in-seat occupant detection systems. Unlike cameras, mmWave provides depth perception and can “see” through soft materials, such as a blanket covering a child in a child restraint. And, unlike in-seat sensors, mmWave can differentiate between a child and an object left on the seat, reducing the likelihood of false alarms.

In addition, mmWave is capable of detecting micromovements like breathing patterns and heart rates, neither of which can be captured by cameras or in-seat sensors alone. In-cabin mmWave can provide other safety benefits, such as advanced airbag deployment and seat belt reminders. Radar imaging can assess body size to discriminate between adults and children and optimize airbag deployment in a crash – more effectively than existing weightbased, in-seat sensor systems. In addition, the technology’s ability to detect breathing patterns and heart rates permits discrimination between people and inanimate objects to more accurately limit system responses – e.g., seatbelt reminders when people are detected, passenger airbag suppression when objects are detected.

mmWave radar can also provide vehicle security benefits. It can be used to enhance theft prevention systems by detecting a broken window or vehicle intrusion. While other sensors may also be used for this purpose, mmWave is more efficient.

 

[Dan D.]

Is FSD a forward-only driving system? What about reversing?

It's a rainy day. You're reversing down your driveway into the street and some yahoo zips past you at 60mph. Or your driveway is on an angle so the yahoo approaches you at 60mph. If radar is a better way of detecting this, and your car doesn't have rear & side radar, then isn't that a problem?

Do we have tunnel vision and just say, well nobody would reverse onto a street where traffic is going 60mph, so that's that.

Seems like 360° radar is needed if you need it at all.

 

[Mardak]

I have a suspicion that this is not for Autopilot.

Oh interesting. Arbe's data sheet says 76-81GHz. https://arberobotics.com/wp-content/uploads/2019/12/Arbe-Phoenix-Data-Sheet-Eval-Kit-Gamma_Web.pdf

 

[daniel]

Tesla has filed with the FCC to use a millimeter wave radar for FSD:

"The equipment under test (EUT) was an Vehicle Millimeter-wave Radar Sensor operating in 60 GHz band (60-64 GHz)"

Tesla files to use new 'millimeter-wave radar' on its 'full self-driving' electric cars - Electrek

IMO, this is a good sign that Tesla will probably add imaging radar to Tesla cars soon.

Agreed! This is a good sign.

And one more hardware upgrade they'll have to provide for people who paid for FSD before Tesla really knew what hardware would be needed but assured buyers that their cars had all the necessary hardware. Boy, maybe I should have paid for FSD because by the time it comes it will be cheaper for them to give those legacy FSD buyers a whole new car than retrofit the old cars with all the needed hardware.

Now all they need to do is make that radar 360° for when a maniac (which is to say a normal driver in some parts of the country) comes speeding at them from the side or from behind. Speed limits set allowable speeds; they don't limit actual speeds.

 

[powertoold]

FSD isn't about perfection or dealing with every extreme case.