AP2 Lateral Turn Across Path detection

[Tam]

It may not be a big deal as a car was waiting in the driveway on my right would run across my path to make its left turn.

However, this is what Mobileye described that its hardware supplied to Tesla was lacking "Lateral Turn Across Path detection" (LTAP) in 2016 fatal Autopilot accident in Florida.

That feature works fine in this version 8.1 (2018.12 5eadc71):

1) The black car in the driveway was not detected and not displayed on the Instrument Clusters:

2) As the car in the driveway started to protrude more into the main road, it's detected and displayed on Instrument Clusters as a regular car on the same direction not as a car with its side facing me. No automatic brake at this time:

3) Automatic brake now's applied:

4) Automatic brake was released after a reduction of speed from 44 MPH to 27 MPH and the when the crossing car was clear:

I love this example as it reacted well in advance and I don't have to guess whether it would brake at a very last minute or not!


On the other hand, this feature seemed to work well in 2015 AP1 for a Uber in a rainy night:

 

[croman]

I've noticed that several times. The reaction isn't as fast as it needs to be and it resumes moving too slowly. It's still nice to see.

 

[Tam]

Another major departure from the past practice of following the lane markers into an obstacle seemed to be resolved in 2018.14.2.

In previous versions, Autopilot would not recognize traffic cones and would follow that white lane marker right into obstacles but not on this youtube clip:





Another report from youtube below, AP2 did not follow the faded yellow lane marker on the right to collide with the cement barrier:






In contrast, this was how AP1 performed by following lane markers into a collision as reported by Electrek--followed the yellow line on the left to collide with median barrier:

The trick of lane-marker-following vs anti-lane-marker-following to avoid obstacles seems to be so much improved!

 

[bebbiXpress]

In previous versions, Autopilot would not recognize traffic cones and would follow that white lane marker right into obstacles but not on this youtube clip

Here's another one I was able to capture:

 

[Tam]

Here's another one I was able to capture...

It did not follow the white line to collide with traffic cones on the right.

It correctly ran over the white crosswalk pattern and it correctly stayed within 2 yellow lines:

It appropriately stopped following the white crosswalk pattern and it avoided hitting the traffic cones on the left in order to get into a correct new lane on the right:

This is mind blowing!!!!

That kind of Tesla Automation competency has never been recorded before!

Does this mean Vision Recognition has now been improved? And if so, with how many cameras?

@bebbiXpress Thanks so much for sharing an amazing clip!

 

[bebbiXpress]

I have to say, on another occasion without a lead car I wasn't confident AP will make this exact same merge to the right and I took over. I think this could be one of the features Elon was tweeting about not ready to be released: Early access build and especially Dev build are feature-rich, but the features aren’t reliable enough yet

I see AP prioritising and following temporary orange markings in construction zones instead of the white lines, but not every time. I know AP is not meant to be used there but with very low to no other traffic I try to find the limits of AP always ready to take over immediately if necessary.

Yesterday I observed another thing when AP slowed the car down right after an oncoming car turned on the left turn signal and only slightly moved towards the center line but not crossing it while slowing down to make the turn. I can't tell for sure if those two actions are related but I couldn't see another reason for braking.

 

[bebbiXpress]

It correctly ran over the white crosswalk pattern and it correctly stayed within 2 yellow lines:

I think it stayed between the left orange and the right white lines.

 

[mongo]

I've noticed that several times. The reaction isn't as fast as it needs to be and it resumes moving too slowly. It's still nice to see.

My understanding is that you can use the accelerator to get going sooner (doesn't cancel AP). Detecting that it is safe to accelerate is something they are going to be conservative on.

 

[croman]

My understanding is that you can use the accelerator to get going sooner (doesn't cancel AP). Detecting that it is safe to accelerate is something they are going to be conservative on.

I do that but it also slows in a slightly delayed manner. They need to increase the reaction time to make it like a safe human driver. It's great it does something but it's not 10x safer than a human driver or whatever Elon mentioned.

 

[daktari]

I think it stayed between the left orange and the right white lines.

You are sure it did not only followed a tracked car on your video and the previous?

It sure looks like a better marked gore point on The 101-crash would not have helped...

 

[bebbiXpress]

You are sure it did not only followed a tracked car on your video and the previous?

On another occasion AP drove this section without a leading car, so yes I'm sure for the part where it drove on the barred area (white diagonal lines) between orange (and white) lines. As for the merging back into the regular lane with the help of the cones I'm not confident that AP recognises the path reliably, with the leading car in the video though it worked. I think it's a combination - or cumulation if you will - of cones and a leading car.

It sure looks like a better marked gore point on The 101-crash would not have helped...

I don't think it's a yes or no decision with neural networks. It's more a weighing of multiple factors that results in a drivable path. Like in this case the process for finding a path would be:

• There are regular white lines
• There are orange lines used in temporary marking
• There is an area marked as barred
• There is a merge left sign
• There are cones
• Usually I mustn't drive on barred areas
• If I follow the regular lines I would crash into cones
• Orange lines are usually considered prioritized

All factors combined gives a human the simple solution "there's a construction zone and I must respect and follow the orange lane markings and cones even if I have to drive on the usually prohibited barred area.

For the 101 crash I think it was a wrong conclusion what the drivable path is and that could and did happen to humans too. It's still a level 2 system and monitoring the environment is job of the driver. AP is like a student driver and the human is the driving instructor. If the student makes an error the instructor has to correct it and for that constant attention is required.

 

[Tam]

...a lead car...

A lead car is useful to predict a driveable path if that path is straight.

In early version, it would have a hard time to cross an intersection that had no lane markers. Later on, it was able to lock onto a lead car and it would be represented as a blue car icon on the Instrument Clusters and would be able to cross a straight laneless intersection fine.

On your clip, the curve path is predicted on the Instrument Clusters and the straight-line-of-sight lead car is represented as outside of the curve path, to the left outside of the path. May be the radar is on the right so the straight-line-of-sight would place the lead car outside of the curve path.

If it followed the lead car as appeared on the Instrument clusters, the car would hit the left traffic cones.

 

[bebbiXpress]

I reviewed the footage and the path is changing quite rapidly (actually jumps) in matter of frames. I tried to export some single frames and combined them into a animated gif:

 

[Tam]

Whatever that have caused this version improvement, its ability to avoid hitting traffic cones is really good!

Comparing to 2016 AP1 when it encountered a small patch of white cement that covered up lane markers and AP1 would slightly veered toward the right and hit traffic cones shortly after that: