Autonomous Car Progress

[malcolm]

Anyways, I could live with a car looking like that if it really could drive my son to the karate all by itself!

But I think your son might choose to re-program the drop-off and pick-up points a block away and walk the rest.

 

[mongo]

Hi, using you post as a spring-board. You had replied to a "vision only" post, so much is this is outside the scope of your response due to other sensors on Tesla so I call out AP sensors that are not cameras.

Vision only would work if the expectation of safety from an autonomous car wasn't orders of magnitudes greater than a human driver.

Two factors involved, the availible data and the use of that data. There are human drivers who are safer than other human drivers with the same data. Remove distractions and excessive speed for conditions and you should be down to a zero "at-fault" accident rate.

Hitting a kid because the cameras were blinded by the sun won't be acceptable

Cameras can have a greater dynamic range than the eye, along with faster response. In addition, for frontal impacts, there are multiple cameras covering. Lastly, if everyone used the boating safety requirement : "always be able to stop in half your visible distance" then things are slow, but safe.
(Off scope: Tesla also has radar)

Crashing into cars in the fog won't be acceptable

If cars do not overdrive their vision, fog and white outs are not an issue. (Off scope: ultrasound and radar)

Well meaning drivers get into accidents every day because they just couldn't see. Why would we give autonomous cars this handicap?

Those drivers are not safe (well meaning or not), if you can't see, don't move.
(Off scope: radar, or at really low speed ultrasound)

Even non-autonomous cars need have better sensing capability to help drivers out. One of these is cross-traffic rear radar because the backup camera by itself isn't good enough.

Is this for reversing out of perpendicular parking spaces/ dead ends? Extending past human POV, a 180 degree fish eye rear camera helps a lot (wider view than standard reverse cam (hypothetical: why the 3 backup looked so bad to start). Side view rear mounted cameras would also show oncoming traffic. (out of scope: but not as much as cross traffic being more aware of encroaching vehicles. Or backing into the spot (not always possible) ) Other option: safe cars don't (normally) park in high risk spots.

 

[banned-66611]

Human vision is enough to drive. Tesla has low quality camera vision. It would fail the sight test in many European countries. Plus the cameras can't move, and don't have stereo vision for depth perception.

Try closing one eye and diving around a car park. Make sure you have insurance.

 

[malcolm]

Try closing one eye and diving around a car park. Make sure you have insurance.

Good news for Emilio Largo: Driving with one eye (LostEye.com)

Pretty much all of the states allow one-eyed drivers, so long as their eye is correctable to close to 20/20 vision.

 

[mongo]

Plus the cameras can't move, and don't have stereo vision for depth perception.

Try closing one eye and diving around a car park. Make sure you have insurance.

If you want to catch a baseball, stereo is helpful. If you want to judge how close an object on the ground is to you it is a bonus, not a requirement. The amount of pavement tells you how far it is.
Multiple close intersections with hanging lights are more of a problem though, need to track those on approach.

So says the guy who mostly uses one eye at a time (sort of weird: can switch the primary eye and can also turn in the other)
And yeah, if you are driving a car in public, make sure you have insurance.

 

[croman]

@oktane had some very compelling arguments as to why monocular vision was totally acceptable and how pidgeons use it and gain depth insight.

Anyways Ap2 has stereo vision forward facing using main and narrow.

 

[lunitiks]

It even has smell (air quality sensor!), hearing (sonars!) and touch (collision sensors!).

Now all it lacks is some good reasoning

 

[scottf200]

Human vision is enough to drive. Tesla has low quality camera vision. It would fail the sight test in many European countries. Plus the cameras can't move, and don't have stereo vision for depth perception.
Try closing one eye and diving around a car park. Make sure you have insurance.

I have been driving with one eye since last summer (2017) when a deer came through my Model X windshield. I drive regularly now (bought service loaner in Dec 2017). It was certainly uncomfortable at first because you are more used to peripheral vision and your distance judging had to be reprogrammed for different clues. Stereo vision is MOSTLY really useful when things are close ... so less than arms length. Your eyes are only a few inches apart!!! Big whoop at long distances. Out to 20-25' you have to retrain your brain. Playing racquet and ball sports has been a challenge but my brain is adjusting (pickleball, racquetball, tennis, etc). I'm finding out MANY people have problems with one eye and do about everything under the sun still. People who have one eye usage since they were born or at a very young age have adapted unbelievably well. I have a friend with one eye since a very young age and he plays hockey and racquetball !!!! -- very well I might add.

 

[BinaryField]

I have been driving with one eye since last summer (2017) when a deer came through my Model X windshield. I drive regularly now (bought service loaner in Dec 2017). It was certainly uncomfortable at first because you are more used to peripheral vision and your distance judging had to be reprogrammed for different clues. Stereo vision is MOSTLY really useful when things are close ... so less than arms length. Your eyes are only a few inches apart!!! Big whoop at long distances. Out to 20-25' you have to retrain your brain. Playing racquet and ball sports has been a challenge but my brain is adjusting (pickleball, racquetball, tennis, etc). I'm finding out MANY people have problems with one eye and do about everything under the sun still. People who have one eye usage since they were born or at a very young age have adapted unbelievably well. I have a friend with one eye since a very young age and he plays hockey and racquetball !!!! -- very well I might add.

I have a private pilot buddy who is effectively blind in one eye. Has no problems flying. There's also the famous airline captain Carlos Dardano, who only has one eye. He was in command of TACA Flight 110 - Wikipedia, where he made an emergency landing on a levee after his airliner lost its engines.

 

[mblakele]

I have been driving with one eye since last summer (2017) when a deer came through my Model X windshield. I drive regularly now (bought service loaner in Dec 2017). It was certainly uncomfortable at first because you are more used to peripheral vision and your distance judging had to be reprogrammed for different clues. Stereo vision is MOSTLY really useful when things are close ... so less than arms length. Your eyes are only a few inches apart!!! Big whoop at long distances. Out to 20-25' you have to retrain your brain. Playing racquet and ball sports has been a challenge but my brain is adjusting (pickleball, racquetball, tennis, etc). I'm finding out MANY people have problems with one eye and do about everything under the sun still. People who have one eye usage since they were born or at a very young age have adapted unbelievably well. I have a friend with one eye since a very young age and he plays hockey and racquetball !!!! -- very well I might add.

I have a private pilot buddy who is effectively blind in one eye. Has no problems flying. There's also the famous airline captain Carlos Dardano, who only has one eye. He was in command of TACA Flight 110 - Wikipedia, where he made an emergency landing on a levee after his airliner lost its engines.

Oliver Sacks has described how persons born with stereo vision can learn to compensate for its loss. I found previews on google books:

• Musicophilia
• The Mind's Eye

Depending on who you ask, anywhere from 3-10% of the population lacks full stereopsis. This wikipedia page also talks about prevalence and impact: Stereopsis - Wikipedia

It seems possible that a computer could do something similar, perhaps by tracking changes as the vehicle moves. However I'm not sure if it'd be practical to implement that with HW2 or 2.5. That's the billion-dollar question for a lidar-free system, stereo or not.

 

[mongo]

It seems possible that a computer could do something similar, perhaps by tracking changes as the vehicle moves. However I'm not sure if it'd be practical to implement that with HW2 or 2.5. That's the billion-dollar question for a lidar-free system, stereo or not.

Due to my interest in autonomous driving, and poor stereo vision, I've been contemplating such things on my commute. It seems fairly simple, if you can differentiate road from objects. Perspective makes things further away go toward the center of vision and increase the amount of road in the bottom of scene to the object.

Using the same technique that 360 overhead camera systems use by mapping 4 fisheye cameras onto a virtual ground plane, you can derive an approximate location for things with one camera.

From a normal single frame image, one can pick out where things are in relation to each other. By comparing time separated frames, velocity can be reduced, and three sets gives you acceleration. Demo SW with Nvidia GPUs does full screen motion vectors faster than real time (same type of thing used in video compression).

Lidar gives a mapping of current locations of the first surface, but does not provide correlation to specific objects. Is that just a car, or a car with a person/child next to it?

I think a hard case to solve is neighborhood driving and having a child running from the side and then become occulded by a car before popping out in the road. Other one is detecting a driver (inside or outside) with a parked car who may open the door into traffic.

Crosswalks will be tougher for Lidar, I think. If you have to yield to a pedestrian, you need to determine what objects on the side if the road are pedestrians, shrubbery, or loiterers. This is easier if you can tell which way they are facing.

 

[scottf200]

Due to my interest in autonomous driving, and poor stereo vision, I've been contemplating such things on my commute. It seems fairly simple, if you can differentiate road from objects. Perspective makes things further away go toward the center of vision and increase the amount of road in the bottom of scene to the object.

Using the same technique that 360 overhead camera systems use by mapping 4 fisheye cameras onto a virtual ground plane, you can derive an approximate location for things with one camera.

From a normal single frame image, one can pick out where things are in relation to each other. By comparing time separated frames, velocity can be reduced, and three sets gives you acceleration. Demo SW with Nvidia GPUs does full screen motion vectors faster than real time (same type of thing used in video compression).

Lidar gives a mapping of current locations of the first surface, but does not provide correlation to specific objects. Is that just a car, or a car with a person/child next to it?

I think a hard case to solve is neighborhood driving and having a child running from the side and then become occulded by a car before popping out in the road. Other one is detecting a driver (inside or outside) with a parked car who may open the door into traffic.

Crosswalks will be tougher for Lidar, I think. If you have to yield to a pedestrian, you need to determine what objects on the side if the road are pedestrians, shrubbery, or loiterers. This is easier if you can tell which way they are facing.

I think this example is relevant as I believe they are using a single camera lens. comma.ai referenced these guys and displayed their usage in some of their testing. They are using open source software called ORB-SLAM2. I would guess this would be more for HD mapping so you would know all permanent reference data points (signs, trees, fire hydrant, buildings, guard rails, etc, etc) for localization.

Below is one way that cameras are being used for 3D mapping. There are several ways already. Imagine multiple passes. Imagine multiple methods and just syncing the ECEF points to create the 3D world and on-going variations (construction) when the ECEF points deviate.

ORB-SLAM Project Webpage
webdiis.unizar.es/~raulmur/orbslam/
ORB-SLAM is a versatile and accurate SLAM solution for Monocular, Stereo and RGB-D cameras. It is able to compute in real-time the camera trajectory and a sparse 3D reconstruction of the scene in a wide variety of environments, ranging from small hand-held sequences of a desk to a car driven around several city ...

Check out a couple of ORB-SLAM2 example videos. Amazing (!!!) with basic cameras.

Driving and mapping

Walking through hallways and staircases to get a two-floor view

 

[roblab]

I think a hard case to solve is neighborhood driving and having a child running from the side and then become occulded by a car before popping out in the road. Other one is detecting a driver (inside or outside) with a parked car who may open the door into traffic.

Crosswalks will be tougher for Lidar, I think. If you have to yield to a pedestrian, you need to determine what objects on the side if the road are pedestrians, shrubbery, or loiterers. This is easier if you can tell which way they are facing.

I've been thinking about this as I drive. I come to a stop sign on a road that tees into a main road, and I have to check traffic both ways before turning left. I spot a moving hole approaching a few hundred yard away in traffic to my right, and then look for a matching hole in traffic approaching to my left. When the holes look like they will arrive at my location near the same time, I have about a half second to jump and get up to speed. I can't see how any self-driving car could do this.

 

[S4WRXTTCS]

Hi, using you post as a spring-board. You had replied to a "vision only" post, so much is this is outside the scope of your response due to other sensors on Tesla so I call out AP sensors that are not cameras.


Two factors involved, the availible data and the use of that data. There are human drivers who are safer than other human drivers with the same data. Remove distractions and excessive speed for conditions and you should be down to a zero "at-fault" accident rate.


Cameras can have a greater dynamic range than the eye, along with faster response. In addition, for frontal impacts, there are multiple cameras covering. Lastly, if everyone used the boating safety requirement : "always be able to stop in half your visible distance" then things are slow, but safe.
(Off scope: Tesla also has radar)



If cars do not overdrive their vision, fog and white outs are not an issue. (Off scope: ultrasound and radar)


Those drivers are not safe (well meaning or not), if you can't see, don't move.
(Off scope: radar, or at really low speed ultrasound)


Is this for reversing out of perpendicular parking spaces/ dead ends? Extending past human POV, a 180 degree fish eye rear camera helps a lot (wider view than standard reverse cam (hypothetical: why the 3 backup looked so bad to start). Side view rear mounted cameras would also show oncoming traffic. (out of scope: but not as much as cross traffic being more aware of encroaching vehicles. Or backing into the spot (not always possible) ) Other option: safe cars don't (normally) park in high risk spots.

That's kind of a funny post because you keep mentioning other sensors as off-post.

So it's like you're trying to defend a vision only system, but then you're kind of suggesting that yeah maybe having some other sensors would be a good thing. So feel free to mention other sensor, but please avoid mentioning ultrasonics. That just makes everyone cringe because we're mostly Tesla owners. As Tesla owners we know first hand that ultrasonics suck, and that's why our blindspot monitoring system doesn't work some/most of the time. That's something that AP2 owners are looking forwards to vision being added to solve that problem.

My point was that Vision only wasn't enough.

There is an inescapable fact that the safety requirement for autonomous driving will be magnitudes of degrees greater than a human driver. That you have to have redundancy in not just the number of sensors, but the type of sensor. It's not just eliminating at fault accidents, but it's also avoiding no fault accidents. Even with lots of sensors the state of the art in autonomous driving seems to be causing accidents because it's overly cautious. It's going to be made massively worse if the autonomous car is given less information, and then forced to slow down excessively in adverse conditions.

You can't escape the irrefutable fact that some sensing systems are better than other sensing systems for specific situations, and weather conditions.

it's also really tough to achieve vision that exceeds the human vision system if you include every part of the human vision system. Not just in the dynamic range, but in the ability to process the image. So we use other sensing systems to give the computer data to use to validate data from the vision system or the other way around. Like if you read up on it you'll see some articles talking about the need for bike to car communication because vision only systems seem to have issues detecting cyclist. The bike to car communication is a way to solve that problem for now until the computer vision system is at a point that solves that problem. Keep in mind were only a couple years from when Subaru (a vision only system) was slowing down for shadows in the road. We Tesla owners are lucky in that we have radar.

There is also the need for autonomous driving to improve the efficiency of our roads. Now you argued that the proper way to deal with every situation was to simply slow down or to stop all together. That's fine for a human driver who is limited to vision only, but it seems like a massive artificial limit for autonomous cars when the one of the primary benefits of them over a human is to use additional sensors.

 

[mongo]

I've been thinking about this as I drive. I come to a stop sign on a road that tees into a main road, and I have to check traffic both ways before turning left. I spot a moving hole approaching a few hundred yard away in traffic to my right, and then look for a matching hole in traffic approaching to my left. When the holes look like they will arrive at my location near the same time, I have about a half second to jump and get up to speed. I can't see how any self-driving car could do this.

Which part of your process do you feel is the hardest to implement?

Unoptomized approach.
Cameras that cover left and right.
After a time of say 3 car lengths/road speed, you can detect cars vs background.
Look for car sized hole based on road angle and vision angle both directions. Track cars to check for acceleration/ deceleration.
Once the spots overlap and provide a hole based on physics model of road type/lane numbers / weather conditions, initiate turn.

In sone ways, it's the inverse of the robots that play ping pong.

 

[mspohr]

I've been thinking about this as I drive. I come to a stop sign on a road that tees into a main road, and I have to check traffic both ways before turning left. I spot a moving hole approaching a few hundred yard away in traffic to my right, and then look for a matching hole in traffic approaching to my left. When the holes look like they will arrive at my location near the same time, I have about a half second to jump and get up to speed. I can't see how any self-driving car could do this.

I don't see why an autonomous car with just cameras couldn't do this. The car could easily watch traffic in both directions (at the same time, even) and make these simple calculations.

 

[Reciprocity]

That's kind of a funny post because you keep mentioning other sensors as off-post.

So it's like you're trying to defend a vision only system, but then you're kind of suggesting that yeah maybe having some other sensors would be a good thing. So feel free to mention other sensor, but please avoid mentioning ultrasonics. That just makes everyone cringe because we're mostly Tesla owners. As Tesla owners we know first hand that ultrasonics suck, and that's why our blindspot monitoring system doesn't work some/most of the time. That's something that AP2 owners are looking forwards to vision being added to solve that problem.

My point was that Vision only wasn't enough.

There is an inescapable fact that the safety requirement for autonomous driving will be magnitudes of degrees greater than a human driver. That you have to have redundancy in not just the number of sensors, but the type of sensor. It's not just eliminating at fault accidents, but it's also avoiding no fault accidents. Even with lots of sensors the state of the art in autonomous driving seems to be causing accidents because it's overly cautious. It's going to be made massively worse if the autonomous car is given less information, and then forced to slow down excessively in adverse conditions.

You can't escape the irrefutable fact that some sensing systems are better than other sensing systems for specific situations, and weather conditions.

it's also really tough to achieve vision that exceeds the human vision system if you include every part of the human vision system. Not just in the dynamic range, but in the ability to process the image. So we use other sensing systems to give the computer data to use to validate data from the vision system or the other way around. Like if you read up on it you'll see some articles talking about the need for bike to car communication because vision only systems seem to have issues detecting cyclist. The bike to car communication is a way to solve that problem for now until the computer vision system is at a point that solves that problem. Keep in mind were only a couple years from when Subaru (a vision only system) was slowing down for shadows in the road. We Tesla owners are lucky in that we have radar.

There is also the need for autonomous driving to improve the efficiency of our roads. Now you argued that the proper way to deal with every situation was to simply slow down or to stop all together. That's fine for a human driver who is limited to vision only, but it seems like a massive artificial limit for autonomous cars when the one of the primary benefits of them over a human is to use additional sensors.

More sensors is not always better. Better sensors are better. If you have 3 different ways to detect an object, which one is right when they disagree? This is why Elon says it's critical to get vision right and not rely on multiple potentially conflicting sensors.

A lot of talk about redundancy. People highly over estimate the need because you assume that it cant ever fail or the car will go berserk and start mowing people down. If for some reason a camera fails, the car just stops and calls for help. Just like you would if your engine seized. No time to elaborate more now as I have to play tickle monster with my son. But use your imagination and think of scenarios where a human driven car is disabled and how an autonomous car would handle situations. I think you will find that there aren't huge differences and that you don't need 2-3x redundancy on every system to match or excede humans in safety.

 

[mspohr]

As Tesla owners we know first hand that ultrasonics suck, and that's why our blindspot monitoring system doesn't work some/most of the time.

My blindspot monitoring system works all of the time when parking and when changing lanes.

There is an inescapable fact that the safety requirement for autonomous driving will be magnitudes of degrees greater than a human driver. That you have to have redundancy in not just the number of sensors, but the type of sensor.

I don't see why autonomous driving has to be "magnitudes of degrees" (whatever that is) greater than a human driver.
It just has to be as good or better.

it's also really tough to achieve vision that exceeds the human vision system if you include every part of the human vision system. Not just in the dynamic range, but in the ability to process the image. So we use other sensing systems to give the computer data to use to validate data from the vision system or the other way around. Like if you read up on it you'll see some articles talking about the need for bike to car communication because vision only systems seem to have issues detecting cyclist. The bike to car communication is a way to solve that problem for now until the computer vision system is at a point that solves that problem. Keep in mind were only a couple years from when Subaru (a vision only system) was slowing down for shadows in the road. We Tesla owners are lucky in that we have radar.

Cars have been navigated by the human vision system forever. Human vision can be thought of as camera connected to a very sophisticated visual processing system. It's only a matter of time until artificial visual processing systems surpass the brain.

 

[S4WRXTTCS]

My blindspot monitoring system works all of the time when parking and when changing lanes.

I don't see why autonomous driving has to be "magnitudes of degrees" (whatever that is) greater than a human driver.
It just has to be as good or better.

Cars have been navigated by the human vision system forever. Human vision can be thought of as camera connected to a very sophisticated visual processing system. It's only a matter of time until artificial visual processing systems surpass the brain.

Let's start with what I agree with. What I agree with is it's only a matter of time until artificial visual processing systems surpass the brain. This is a true statement, but what people seem to think is that we're anywhere close. We're not so we use other sensors for redundancy where the sensor data requires less processing, and is less prone to false positives or false negatives.

As to the safety of autonomous driving we can agree that it SHOULDN'T have to be significant safer, but that's not how things are. Across various industries the safety requirements of a machine are much higher than they would be for a person.

This is partly because of higher expectations when it comes to machines, and how things are perceived in the media. The Tesla Autopilot system isn't even autonomous yet it became famous for being the first autonomous car someone died in. Somehow there was the expectation that the accident wouldn't have happened. It's also a question of liability. If a robot screws up just once the maker of the robot gets sued for millions. Can you imagine an autonomous car manufacture being sued by thousands? There are a lot of things we could have if the safety of margins for an autonomous machine wasn't so incredibly high. Autonomous cars is really a non-starters if it can't be demonstrated to be much higher. Now I don't know what this number is, and I don't think anyone does. It certainly isn't equal to 2x greater though. Probably more like 10X or greater.

The poor blindspot monitoring in the Tesla Model S/X has been well covered on the TMC forums. I would advise you not to trust the ultrasonics too much or you'll end up posting a self-park disaster thread. We already have too many self-park disaster threads on TMC. Part of the reason they don't work well is they miss things that are vertically higher/lower than it. So it won't detect a semi trailer next to you. The other problem with the blindspot monitoring is where they put the indicator, and how slow the indicator seems to respond to what's occurring on the road.

 

[Bladerskb]

A lot of talk about redundancy. People highly over estimate the need because you assume that it cant ever fail or the car will go berserk and start mowing people down. If for some reason a camera fails, the car just stops and calls for help. Just like you would if your engine seized.

No its equivalent to having someone put an unexpected blindfold on you while going 90 mph.

It has absolutely no comparison to a car's mechanical failure. My goodness can u ppl actually try to have a logical evidence driven discussion?