HW2.5 capabilities

[AnxietyRanger]

Unlike sunny California, I feel like people in regions with varying weather conditions are going to find themselves reaching to the top of their cars trying to clean the lidar when the wipers fail to do an adequate job. It'll be super fun when the wiper fluid nozzles freeze up and the wipers get frozen. Perhaps the lidar dome is heated as well? Winter is going to suck for people with lidar and without garages.

I would still expect a good redundant system will suck less for people with anything, than a system with no redundancy.

Whether lidar exactly is needed, who knows. But winter and the lack of redundancy in AP2 certainly is a reasonable concern.

 

[verygreen]

A dark highway or rural road (no urban lights) would be interesting. Anyone?

We discussed this a bit in the 2.0 camera thread I think. Basically on a dark highway you light the road in front of you with headlights (and can even use high beams) and if there's no other traffic, then you don't even care what's behind you (and if there are other cars, they'd work as a light source for us, or the oncoming traffic might be doing that for us too).

 

[AnxietyRanger]

We discussed this a bit in the 2.0 camera thread I think. Basically on a dark highway you light the road in front of you with headlights (and can even use high beams) and if there's no other traffic, then you don't even care what's behind you (and if there are other cars, they'd work as a light source for us, or the oncoming traffic might be doing that for us too).

Still, I am interested. That is a scenario where lidar can see.

 

[verygreen]

Still, I am interested. That is a scenario where lidar can see.

This is going to be tricky to find I am afraid the way roads are lighted in US. and I don't really want to go too far into the boonies and hit a deer or something

Are none of the existing night snapshots at least close to what you envision, do you really need total pitch black surrounding?

 

[lunitiks]

I think we've seen alot now.

But still there are a couple of pretty important scenarios left:

- Night time + rain
- Snowfall

Then of course there are some "moments" that one could assume is challenging:

- Glaring sunshine straight on the fwd cameras
- The transition when entering/exiting a dark tunnel from/to daylight

 

[AnxietyRanger]

Are none of the existing night snapshots at least close to what you envision, do you really need total pitch black surrounding?

I don't NEED it. I'd just be interested in seeing the dynamic range in a situation where the only lights are that of the car.

 

[lunitiks]

I wish moderator could move this discussion to the AP2.0 Cameras thread...

 

[verygreen]

I think we've seen alot now.

But still there are a couple of pretty important scenarios left:

- Night time + rain
- Snowfall

Then of course there are some "moments" that one could assume is challenging:

- Glaring sunshine straight on the fwd cameras
- The transition when entering/exiting a dark tunnel from/to daylight

I would not hold my breath on a snowfall anytime soon

The only few tunnels I know around here are lighted.

I might be able to get a night + rain set eventually. Glaring sunshine should also be possible, I assume it's the kind that also blinds the driver we are after?

 

[verygreen]

I wish moderator could move this discussion to the AP2.0 Cameras thread...

We already had it there, and there was even some processing of night shots when I first published that link over there, we even had a bit of a same discussion with same arguments there

 

[stopcrazypp]

I think there was a more dark location in one of the later snapshots, if you find where this stash is referenced in the AP 2.0 cameras thread there were some descriptions.

Also perhaps more importantly there were a few highway snapshots that probably more realistically represent what cameras would be against during intended usage.

And yes, dynamic range is pretty good. Also do try other HDR processing options in photoshop, I think. They don't look natural, but that's just because our monitors have a limited dynamic range that we need to compress for

Yeah, I didn't have time to go through all the snapshots as there's a lot of files just in one of them. I just picked the darkest set of images from that snapshot. When I have the time I will try to find the reference.

I did try some of the other options, and as you say they don't look natural. Yeah, our monitors have no chance to display the images natively. Even the most state of the art ones can't do it.

 

[AnxietyRanger]

We already had it there, and there was even some processing of night shots when I first published that link over there, we even had a bit of a same discussion with same arguments there

Am I smelling the opportunity for some repetition here? I'm GAME!

 

[lunitiks]

Glaring sunshine should also be possible, I assume it's the kind that also blinds the driver we are after?

Sure!

But not necessarily. I've driven lots of stretches with my AP1 where the sun obviously is the cause of AP fail, but still my vision is intact.

Has to do with where the sun stands in the sky. Relative to the super massive black hole in the center of the galaxy u know.

Difficult to exhaust all possibilities on this one

 

[conman]

The night footage is here, feel free to process it (there's also a dashcam video for comparison): https://app.box.com/s/jah6ovf6y5eoq9faejeylt7fm2j2od4i

Sorry I know @stopcrazypp already processed them but I'd love to play with them. What format are the img.data files meant to be? If either of you could answer I'd be grateful.

 

[stopcrazypp]

I wish moderator could move this discussion to the AP2.0 Cameras thread...

To keep things on topic, I posted the processed pictures of the deliberate stationary test @verygreen did in an area with no streetlights in the AP2.0 Cameras thread:
AP2.0 Cameras: Capabilities and Limitations?

 

[stopcrazypp]

Sorry I know @stopcrazypp already processed them but I'd love to play with them. What format are the img.data files meant to be? If either of you could answer I'd be grateful.

You can use this tool to convert to tiff (or other formats, but only the tiff version would give you the full information):
AP2.0 Cameras: Capabilities and Limitations?

Then you can load into Photoshop or other image editors.

 

[JeffK]

So no streetlights, but are the headlights on or off in these?

 

[croman]

They clearly look on to me. Look at the street signs -- they are illuminated. Also there is a large light source off the road illuminating a tree to the right about 30-40 feet down the road (might be a cross street).

 

[daktari]

If you have time some day, please grab some snapshots going over a crest or bottom of a hill or an undulation in the road. Would be interesting to see if the narrow can see past the blind spot of the normal cam.

 

[Bladerskb]

Didn't we discuss this a while back? Tesla's AP2 Bosch radar range of 160m matches MRR, which has the following specs (double the numbers if talking about FOV as we use colloquially):

Field of view (horizontal)
Main antenna
±6° (160 m)
±9° (100 m)
±10° (60 m)
Elevation antenna
±25° (36 m)
±42° (12 m)

Mid-range radar sensor (MRR)
Frustrated with FSD timeline

A max of 84 degrees FOV at 12m is definitely enough for pedestrian detection. In fact, that's what it's advertised for:
"Thanks to the elevation antenna, the system achieves an opening angle of ± 42 degrees at close range – so a pedestrian stepping out into the road from behind a parked car, for example, is detected at an early stage."

We don't have the specs on the new Continential radar in HW2.5, but the PR suggests it's even more capable:
HW2.5 capabilities

The FOV of the radar still misses over 95% of all driving tasks.

This is not true as far as I can tell. Automotive radar can tell the difference between a moving subject (like a pedestrian or a deer) and a small static object (like a rock) by using information from the doppler frequency image. This has been the subject of a lot of the automotive radar research and why there are systems out there that can do pedestrian recognition using only radar.

This is MAJOR incorrect.

There's a reason radars are configured to only pick up moving objects.. its because they can't differentiate a static object from the environment.

 

[Bladerskb]

Volvo used camera/radar combos to detect large animals. No one suggested using radar only even though it is possible to do some basic pedestrian, large animal, huge rock, or whatever detection. It doesn't really matter much. If there's an object in front of the car you don't want to hit it. It doesn't matter if it's an Alaskan moose, North-Western moose, or a big deer you aren't going to want to hit it. The best sensor for confirmation of what the object is will always be a camera which is better than both radar and lidar for such a task.

Not true. A camera can't see a object, it can't tell the difference between a pedestrian, animal, rock, whatever.
A picture is a collection of RGB pixels (aka numbers). So no camera is no where near the best sensor for confirmation of what an object is.

Lidar works with corrective algorithms in rain, snow, or dust, lidar does not see through snow whether on the sensor or on the ground

Everything is based on corrective algorithm. EVERYTHING.

, unlike radar. lidar will also fail if it gets dirty, you'd need something like Waymo's lidar wipers, you'll definitely need the wiper fluid if it gets oily spray from other drivers.

AP1 radar fails all the time when it gets dirty or when it gets snow/ice build up on it.
Stop praising Elon's and Tesla's terrible engineering job.

In a really heavy rain where humans typically pull over, forget about it, it'll be next to useless (same would be true with cameras, but not for radar).

Again incorrect, lidar works in heavy rain and snow.
You keep mentioning radar. how is it you don't understand that radar is useless.

RADAR is NOT a primary sensor? how can you not understand that?
seriously, that baffles me.