FSD v12.x (end to end AI)

[mongo]

Thinking a self driving car needs to hear sirens is anthropomorphism. A siren is used to catch our attention so we are looking in all directions and paying full attention. An autonomous car is ALWYAS looking in all directions and paying full attention. So hearing a siren brings NOTHING to the table.

Assuming the vehicle with the siren is visible...

 

[JulienW]

Assuming the vehicle with the siren is visible...

.....and if it is not visible what is it that you do are want the car to do?

 

[dramsey]

Uh... If the car is moving, every object in the images moves from frame to frame, except for those moving directly toward or away from ego, which instead only bloom or shrink.

This frame to frame parallax shift gives solid distance clues, so it is not a bad thing, but your C code just got way more complicated. But, hey, a 3 gram humming bird brain does this, corrects for winds, and adjusts flapping all while flying at speed through 3D trees and landing on a twig. Oh, right, that is a NN, not C code.

Evolution develops these things by killing off the ones which make mistakes, which explains why Telsa's NNs are trained in simulations, at great expense of compute power and time.

My post was intended as an off-the-cuff, very simplified outline of how this problem could be tackled with non-NN programming. Of course the code would take the car's movement into account, probably try to discern object vectors by looking at the different camera images, et cetera so on and such forth. The point was that it's possible to do this in C.

 

[mongo]

.....and if it is not visible what is it that you do are want the car to do?

Not drive into the path of where the vehicle will be.

 

[mongo]

Not drive into the path of where the vehicle will be.

See also: uncontrolled train crossings.

 

[aronth5]

Today drove 14 miles (11.4.9) and disengaged many times due to tire eating pot holes. We've had a lot of rain storms mixed in with snow in January and after freezing and thawing roads are a mess. Hoping that V12.x can address this significant problem.

 

[AlanSubie4Life]

Today drove 14 miles (11.4.9) and disengaged many times due to tire eating pot holes. We've had a lot of rain storms mixed in with snow in January and after freezing and thawing roads are a mess. Hoping that V12.x can address this significant problem.

Given that is difficult for a human, I think it is impossible for FSD, unless they include a Super-sensitive Pothole Avoidance Maneuvers (SPAM) mode.

This mode would swerve all over the road all the time, avoiding anything that looks out of place.

The problem is false positives. Distinguishing depth vs simple pavement markings is really difficult. Potholes can vary significantly in how they look.

They could map them of course, but that is also going to be sad for some people. Would need a rapidly responding system not based on Tesla driving data. Waze has this but is of course hit or miss.

 

[willow_hiller]

They could map them of course, but that is also going to be sad for some people.

Tesla has actually been mapping them for over a year: Tesla vehicles are now scanning for potholes and rough roads to help avoid them

I think the mechanism was described in an earnings call, but instead of vision they were using the car's accelerometer to measure rough roads. But it's not for avoidance, just for adjusting air suspension.

That being said, they could leverage the rough road maps, and only have vision be on the lookout for avoidance in areas that other cars report bumps.

 

[AlanSubie4Life]

Tesla has actually been mapping them for over a year: Tesla vehicles are now scanning for potholes and rough roads to help avoid them

I think the mechanism was described in an earnings call, but instead of vision they were using the car's accelerometer to measure rough roads. But it's not for avoidance, just for adjusting air suspension.

That being said, they could leverage the rough road maps, and only have vision be on the lookout for avoidance in areas that other cars report bumps.

Yes that is all very well in some situations which are more static. That is not useless.

But the issue is that literally overnight a huge number of potholes can appear. That is just the reality. That is the situation that would need to be rapidly responded to without using Tesla data (otherwise Teslas will be damaged).

This massive sudden appearance of potholes lasts a couple weeks. Then they disappear (in the absence of continued rain).

These ones are typically the real wheel killers.

San Diego has some incredibly rough roads and I think the somewhat sandy subsurface tends to lead to rapid deterioration. It is bad right now and will be worse next week.

 

[powertoold]

I'm good at predicting the future: V12 will initially avoid potholes really well but NHTSA will ask Tesla to recall FSD to fix lane drifting without due caution

 

[AlanSubie4Life]

V12 will initially avoid potholes really well

Initially V12 will do nothing about potholes. Maybe on a future revision it will initially avoid a lot of things that look sus.

 

[AndrewZ]

Dodging potholes and such you'd think would be crucial in the pursuit of taking liability, because damages from that kind of stuff would really add up over a large fleet

 

[AlanSubie4Life]

Dodging potholes and such you'd think would be crucial in the pursuit of taking liability, because damages from that kind of stuff would really add up over a large fleet

It is almost like they will not take liability.

 

[powertoold]

Initially V12 will do nothing about potholes. Maybe on a future revision it will initially avoid a lot of things that look sus.

I think it's most likely that V12 will initially avoid problematic potholes, but we only need to wait a "few" weeks to find out!

 

[JHCCAZ]

They could map them of course, but that is also going to be sad for some people. ... Waze has this but is of course hit or miss.

You hit I miss

 

[zoomer0056]

Dodging potholes and such you'd think would be crucial in the pursuit of taking liability

Unless it's written that the driver is responsible. To take liability for pothole damage is not gonna happen.

 

[Tronguy]

.....and how is this accomplished?

Actually, I know the answer to this one. The basics are pretty straightforward.

It's all about the Ears and Brain. If one has a constant, steady tone like, say, a sine wave, or a pair of sine waves, it's possible for a human to triangulate by turning the head. But it's very rough: What one is doing is detecting whether one ear is picking up a noise louder than another ear. Turn the head until both ears get the same loudness and one is either facing roughly at the source or directly away from the source.

US sirens are pretty much like that. Constant, high amplitude, with (from the perspective of the human auditory system) relative slow changes over time.

Now, try letting off a blast from a shotgun. One's head will turn and directly aim at the source, no ifs, buts, or maybes. Or let some (bad guy?) crack a twig on the forest floor, same result. Reason that humans are good at this: Time delay. A sharp rise time sound (like a snap, or blam) gets picked up by the two ears at different times. Yes, it's milliseconds. But we have an evolution-driven ability that helps to (a) not get eaten and (b) find something when hunting.

If you think about it: Just how easy is it to aim your eyeballs at an airliner going by at altitude? Yes, it takes time for the audio to get to where one is located, so the visible airplane is always well ahead of the sound. But just getting one's head aimed in the right rough direction isn't easy. Helicopters, with their whop-whop-whop noises, are far easier to determine the location of. It's all about easily discernible, rapid changes in amplitude and frequency.

The Europeans have their standard BLAW-blaw-BLAW-blaw two tone thingummy that, on each frequency shift, has shifts in both amplitude and frequency, and each shift has high-frequency, fast rise/fall times that were designed to be localizeable by humans, with testing and malice aforethought. (Mind you, one doesn't get the effect from watching a TV set playing it on some show or other because, well, all the noise is coming from the TV, straight ahead.)

At one time I heard of an attempt at a US-researched siren of this type; it sounds like a dozen burbling chimes going off, loudly, and has that same your-head-turns-that-way effect on humans. I may have heard it on fire engines in some city or other.

 

[aronth5]

Given that is difficult for a human, I think it is impossible for FSD, unless they include a Super-sensitive Pothole Avoidance Maneuvers (SPAM) mode.

This mode would swerve all over the road all the time, avoiding anything that looks out of place.

The problem is false positives. Distinguishing depth vs simple pavement markings is really difficult. Potholes can vary significantly in how they look.

They could map them of course, but that is also going to be sad for some people. Would need a rapidly responding system not based on Tesla driving data. Waze has this but is of course hit or miss.

The front cameras have a better view of potholes then humans do since they are higher so it should be doable for the really large potholes. Biggest challenge for humans is avoiding potholes at night in the rain so those are the real edge cases. Pothole locations will never be mapped reliably since they pop up suddenly and when large get patched pretty quickly since towns want to avoid liability.

 

[lzolman]

Tesla has actually been mapping them for over a year: Tesla vehicles are now scanning for potholes and rough roads to help avoid them

I think the mechanism was described in an earnings call, but instead of vision they were using the car's accelerometer to measure rough roads. But it's not for avoidance, just for adjusting air suspension.

That being said, they could leverage the rough road maps, and only have vision be on the lookout for avoidance in areas that other cars report bumps.

Ha, and as a result of this, my suspension keeps raising in a few specific spots now for no apparent reason. There are no road features that would result in the car getting any benefit from it.

 

[Todd Burch]

Ha, and as a result of this, my suspension keeps raising in a few specific spots now for no apparent reason. There are no road features that would result in the car getting any benefit from it.

My Model X raises its suspension "for comfort" every day when I cross a bumpy suspension bridge.