Model X Crash on US-101 (Mountain View, CA)

[sinv]

There will always be confusing marks, or partly even unmarked sections. In such cases, the driver must be alerted immideately. This the current system doesn't seem to do.
As the system is additionally unable to detect stationary obstacles, it must, in case the driver doesn't immideately react, stop down. This the current system also doesn't seem to do.

 

[bhzmark]

There will always be confusing marks, or partly even unmarked sections. In such cases, the driver must be alerted immideately. This the current system doesn't seem to do.
As the system is additionally unable to detect stationary obstacles, it must, in case the driver doesn't immideately react, stop down. This the current system also doesn't seem to do.

I believe it can detect stationary objects, but so often we just drive past stationary objects that it is reluctant to brake for every single one of them. That functionality disregarding stationary objects ahead is essential when using just TACC where the system has to assume that you will steer or manually brake to avoid hitting stationary objects, e.g., drive past parked cars on a curving road. But I agree that when the AP is in Autosteer mode, it seems that it ought to be able to both detect those stationary objects and plot whether they are in the predicted of path that the AP is steering into and if so to brake. It seems that they do that for semis crosswise blocking the highway, as well as certain low overpasses that appear as false positive semi trailers. It would be good if the collision alert system could at least sense those situations and ring the alarm bells if not actually brake or swerve.

I guess meanwhile it means we just have to pay attention with hands on the wheel and ready to take action. . . . In other words drive the AP as designed.

 

[mongo]

Another possible theory is, that if the car instead lost both lines and locked onto following a car that headed SB101 while he was either in the SB85 off-ramp or in the gore zone, the car may have followed the other car, crossing over into the gore zone as it attempted to get behind that lead vehicle

Does that align with the 150 meter clear line of sight report from Tesla?

So what can be done to improve AP?
- additional front cameras on each side of the car to give a better stereoscopic vision.
- implement a more precise telemetry measurement to detect obstacles.
- increase obedience from the driver to keep the hands on the steering wheel.

Not sure how you can do much better than the three cameras that it already has in the front. That should be plenty to get a reliable depth map and recognize the difference between dips in the road and actual obstacles.

@NeverFollow has a point in that a larger baseline between the cameras creates a larger shift in the images if you are doing stereoscopic analysis. Stereo is handy if you are starting with a static scene and someone is trying to fool you. However, I don't think stereo vision is needed (partly due to my bias being fairly crosseyed). For example, have we been determining distance to the barrier in all these single frame images people have posted?

This. Actually use the stereoscopic data.

Or use more of the available vision data (which is what they are working on).

Change in object size and position indicate an object's distance and relative motion. Once the software can tell what is pavement/ road and what isn't, it is a small step to determine how close the first non pavement object is using perspective. (Other than things like kangaroos, which look like they are touching the ground when they aren't)

More cameras = more data = heavier compute load and slower reaction time, so I would expect that to make things worse, not better.

Only if the NN is too large. Fortunately, a NN is just a lot of add and multiply instructions without any branching (until you get to the actual decision code). So they can accurately model whether the NNs use more than 100% computing power. At that point they can either reduce NN size, or change how often they run.

For example: the front camera could process at 100Hz (every 0.88ft at 60MPH) while the side cameras run at 50Hz. And this can be dynamic, at a red light it could reduce the forward camera rate and increase the side to better catch cars running the yellow.

 

[22522]

Or use more of the available vision data (which is what they are working on).

Change in object size and position indicate an object's distance and relative motion. .

Honda uses the increasing distance between tail lights to process closing speed
(frame to frame vector on features known to be on the same object).

For that to work on these barriers with radiuses on the front corners or little background contrast, assembly of pixels into a single object needs to be very fast.

Seems like if you use the pixel vectors from two frames you could do it. With enough pixels... more pixels might be more important than higher frame rate.

I have not figured it out yet.

 

[Krazaak]

@NeverFollow has a point in that a larger baseline between the cameras creates a larger shift in the images if you are doing stereoscopic analysis. Stereo is handy if you are starting with a static scene and someone is trying to fool you. However, I don't think stereo vision is needed (partly due to my bias being fairly crosseyed). For example, have we been determining distance to the barrier in all these single frame images people have posted?

Or use more of the available vision data (which is what they are working on).

Change in object size and position indicate an object's distance and relative motion. Once the software can tell what is pavement/ road and what isn't, it is a small step to determine how close the first non pavement object is using perspective. (Other than things like kangaroos, which look like they are touching the ground when they aren't)

I don't think stereo vision is required, but I think it would help significantly. In some cases, it can probably be achieved with a combination of cameras. For instance, in order for EAP to automatically change lanes without driver input, it's going to have to know if another vehicle is approaching at high speed. It's harder to gauge that with only size and position than with parallax, since the slope of the road and the size of the vehicle can play tricks with mono vision.

 

[mongo]

Honda uses the increasing distance between tail lights to process closing speed
(frame to frame vector on features known to be on the same object).

For that to work on these barriers with radiuses on the front corners or little background contrast, assembly of pixels into a single object needs to be very fast.

Seems like if you use the pixel vectors from two frames you could do it. With enough pixels... more pixels might be more important than higher frame rate.

I have not figured it out yet.

Nvidia has an included library function for motion estimation for video compression. The demo does full scene processing in 3x real time (>100 frames per second).
High frame rate would make growth detection harder due to less pixel change per image, if the comparison is only adjacent frames. By running it on 1, 5, 10 etc frame time sets, slower objects can be detected.
Seems like a tricky way to go since vehicle angle also changes the apparent distance.

I don't think stereo vision is required, but I think it would help significantly. In some cases, it can probably be achieved with a combination of cameras. For instance, in order for EAP to automatically change lanes without driver input, it's going to have to know if another vehicle is approaching at high speed. It's harder to gauge that with only size and position than with parallax, since the slope of the road and the size of the vehicle can play tricks with mono vision.

For lane change, how do you get two cameras with appropriate baseline that are not blocked by the surrounding cars?
I agree that road slope needs to be taken into account, but there is still the horizontal angle of the target vehicle to work with. If the road were effectively shorten due to a dip, that would place the object closer, not further leading to a more restrictive lane change enabling. Unless I am missing a way to make it appear there is more road between the camera and an object.

There is also the issue of creating an effective stereo processing algorithm. It seems more that 2x as hard as making a mono-camera algorithm, but I haven't messed around with that as much, Nvidia also has a demo of that.

 

[Barklikeadog]

There will always be confusing marks, or partly even unmarked sections. In such cases, the driver must be alerted immideately. This the current system doesn't seem to do.
As the system is additionally unable to detect stationary obstacles, it must, in case the driver doesn't immideately react, stop down. This the current system also doesn't seem to do.

Does this mean that the FSD on currently sold vehicles is impossible?

 

[sillydriver]

Does this mean that the FSD on currently sold vehicles is impossible?

Yes. My argument why is upthread #1510.

 

[MP3Mike]

And if you make the mistake of accidentally double tapping the stalk to activate autosteer.

Not if you haven't enabled Autosteer in the settings.

 

[Doug_G]

Potentially yes. His hands were off the steering wheel for six seconds. I doubt they were folded neatly in his lap. A distraction of some type occurred and held his attention for that six seconds. iPhone?

Tesla's wording was very telling: "driver’s hands were not detected on the wheel for six seconds prior to the collision." They didn't say his hands were not on the wheel. This is an important distinction. You can easily get the hands on wheel warning when your hands are already on the wheel. Happens to me all the time. So you can't draw any firm conclusions from this.

 

[mongo]

Does this mean that the FSD on currently sold vehicles is impossible?

Yes. My argument why is upthread #1510.

Isn't your argument in 1510 that current EAP software lacks some functionality needed to do FSD?
To say that FSD is impossible on currently sold vehicles means that the hardware is incapable of FSD regardless of software updates.
If you are saying the current hardware can't ever do FSD, I'll probably write a rebuttal.

 

[bro1999]

I have a feeling Elon's "fully autonomous x-country drive in 3-6 months" pledge earlier this year is gonna bust again. Maybe next year (again).

 

[Nosken]

Nvidia has an included library function for motion estimation for video compression. The demo does full scene processing in 3x real time (>100 frames per second).
High frame rate would make growth detection harder due to less pixel change per image, if the comparison is only adjacent frames. By running it on 1, 5, 10 etc frame time sets, slower objects can be detected.
Seems like a tricky way to go since vehicle angle also changes the apparent distance.



For lane change, how do you get two cameras with appropriate baseline that are not blocked by the surrounding cars?
I agree that road slope needs to be taken into account, but there is still the horizontal angle of the target vehicle to work with. If the road were effectively shorten due to a dip, that would place the object closer, not further leading to a more restrictive lane change enabling. Unless I am missing a way to make it appear there is more road between the camera and an object.

There is also the issue of creating an effective stereo processing algorithm. It seems more that 2x as hard as making a mono-camera algorithm, but I haven't messed around with that as much, Nvidia also has a demo of that.

My single camera, ThinkWare Dashcam F770 has forward collision warning and lane keep warning. Of course, I disabled it for my Model S, but for the few days I tried it, it was pretty effective.

 

[dgatwood]

Nvidia has an included library function for motion estimation for video compression. The demo does full scene processing in 3x real time (>100 frames per second).
High frame rate would make growth detection harder due to less pixel change per image, if the comparison is only adjacent frames. By running it on 1, 5, 10 etc frame time sets, slower objects can be detected.
Seems like a tricky way to go since vehicle angle also changes the apparent distance.



For lane change, how do you get two cameras with appropriate baseline that are not blocked by the surrounding cars?

You put them a few inches apart, like human eyes are, just like Tesla did. If one or the other is blocked by cars, you're following too closely, and you won't be able to avoid obstacles in the road, but then again, if you hit any, the objects will already have been thoroughly pulped by the car in front of you, so it doesn't make any difference whether you hit them, really. If you're far enough back to see the object in the road, then they can't realistically be blocked by other cars.

 

[MXWing]

Not if you haven't enabled Autosteer in the settings.

TRUE! I turned that on the first 5 seconds it was available so I forgot that fact.

 

[MXWing]

I have a feeling Elon's "fully autonomous x-country drive in 3-6 months" pledge earlier this year is gonna bust again. Maybe next year (again).

Asking if Elon is wildly optimistic and if he will miss his self imposed deadlines is like asking if the pope is catholic.

No story here at all.

 

[mongo]

You put them a few inches apart, like human eyes are, just like Tesla did. If one or the other is blocked by cars, you're following too closely, and you won't be able to avoid obstacles in the road, but then again, if you hit any, the objects will already have been thoroughly pulped by the car in front of you, so it doesn't make any difference whether you hit them, really. If you're far enough back to see the object in the road, then they can't realistically be blocked by other cars.

That sound like in lane object detection. I was referring to:

For instance, in order for EAP to automatically change lanes without driver input, it's going to have to know if another vehicle is approaching at high speed.

If they are reffering to a car approaching from the rear (overtake), or from the other direction (2 lane road), then your placement is possible (although being centrally located make car shadowing worse). However, for checking the lane the car is moving into with stereo would require double offset cameras on all four corners.

 

[sillydriver]

Isn't your argument in 1510 that current EAP software lacks some functionality needed to do FSD?
To say that FSD is impossible on currently sold vehicles means that the hardware is incapable of FSD regardless of software updates.
If you are saying the current hardware can't ever do FSD, I'll probably write a rebuttal.

My argument is that current installed sensor hardware consisting of the current camera placement, radar and ultrasonics is unable to do stationary object recognition without false positives or negatives within a few years, which is the timescale relevant to the cars that have shipped. One argument is that doing this is an important enough matter, which has been a high priority for a long enough time, that if the current sensor suite could do it, or if Tesla knew how to do it, Tesla would have already deployed the software to do it. The fact that the functionality is not here even though there are multiple high-profile instances of running into stationary objects implies they don't know how to do it currently or (in my opinion) within a timeframe that's reasonable for current owners.

Yes, strictly speaking, stationary object recognition can be done with the current sensors: take the current camera images and give them to a system with all the relevant capabilities of a competent human being. Human beings have object detection and scene analysis abilities vastly ahead of the best current systems I have heard of (case in point: the Google "That's a Gorilla" embarrassment) and can place objects in a scene without distance data in most cases. My opinion is that software won't do this reliably within the timeframe relevant to current car owners.

Upthread I gave my opinion, which is just my opinion, that it would require a single forward looking LIDAR around hood level, or stereo cameras and compute power for interpreting binary vision, or something like cameras and a projected grid of infrared points that could be seen by the cameras and interpreted by software. All of these involve new sensor hardware that is not easy to retrofit. Obviously, your opinion may differ.

 

[mongo]

My argument is that current installed sensor hardware consisting of the current camera placement, radar and ultrasonics is unable to do stationary object recognition without false positives or negatives within a few years, which is the timescale relevant to the cars that have shipped. One argument is that doing this is an important enough matter, which has been a high priority for a long enough time, that if the current sensor suite could do it, or if Tesla knew how to do it, Tesla would have already deployed the software to do it. The fact that the functionality is not here even though there are multiple high-profile instances of running into stationary objects implies they don't know how to do it currently or (in my opinion) within a timeframe that's reasonable for current owners.

Yes, strictly speaking, stationary object recognition can be done with the current sensors: take the current camera images and give them to a system with all the relevant capabilities of a competent human being. Human beings have object detection and scene analysis abilities vastly ahead of the best current systems I have heard of (case in point: the Google "That's a Gorilla" embarrassment) and can place objects in a scene without distance data in most cases. My opinion is that software won't do this reliably within the timeframe relevant to current car owners.

Upthread I gave my opinion, which is just my opinion, that it would require a single forward looking LIDAR around hood level, or stereo cameras and compute power for interpreting binary vision, or something like cameras and a projected grid of infrared points that could be seen by the cameras and interpreted by software. All of these involve new sensor hardware that is not easy to retrofit. Obviously, your opinion may differ.

I think the time it has taken to implement is due partly to the sudden MobilEye split and having the wrong people in the internal AP group. The addition of Andrej appears to already be bearing much fruit as seen in the recent release. So I would not take the delay as indication that the sensor suite is incapable. (even if is taking longer than many would prefer)

My single camera, ThinkWare Dashcam F770 has forward collision warning and lane keep warning. Of course, I disabled it for my Model S, but for the few days I tried it, it was pretty effective.

Consider one of the suggested potential solutions: stereo cameras. To function, the system (mono or stereo) needs to:

1. Identify an object
2. Classify the object (shadow vs solid vs ???)
3. Determine position, relative motion, and risk.

Stereo also needs to

• Correlate the objects
• Find reference feature

To do the position processing, the system needs to determine a vector to the object from each sensor. For stereo, it then uses trig to get the position and distance. Stereo positioning is highly dependent on identifying the exact same feature point on each sensor's image The further away, the few pixels are available to determine displacement. Going with just a single camera, you get a path of position/distance pairs for where the object could be. Integrating multiple samples of a dynamic scene provides localization of which sets of points are rational.

The other thing to consider is that in many cases the system down't care about exact distances or position, it cares about trends. If an object is moving away from you, it is not a threat. If it is moving toward you, does it have sideways velocity so that it will pass by? If the object is centered in the image or is to the right or left, it is less of a concern. If it is near the bottom of the image, look out.

 

[Az_Rael]

I am trying to find out if I lose TACC if I don’t buy EAP

You don’t get to keep TACC without EAP. However, the good news is you don’t ever have to enable autosteer. It comes turned off by default on the car (and TACC functions normally with autosteer disabled). We drove my husband’s S for quite a while with Autosteer disabled until he felt comfortable with enabling it after reading the manual and learning its limitations. I enabled it on my 3, but wasn’t using it at all prior to the latest software update because it was too squirrelly.

To enable autosteer, you also have to stop the car and I think put it in Park, so it’s not something that can be done on the fly or anything either (if you are worried about accidentally enabling it).