Autonomous Car Progress

[mikes_fsd]

Does that mean it's Tesla's software issue?

I know you are trolling, but clearly it is a software issue!
It would be nice if you would just state your point clearly.

Things we know.
In some markets (read EU) we know that they've gimped how fast an ADAS can make a turn (turn the wheel) -- to the point where switching lanes on a curve fails (as by the rules design)
We also, know that this accident happened in China - and I am not sure of what kind of regulations they have re ADAS performance.

I know in the US this actually has improved greatly. I take regular trips on the Blue Ridge Parkway and Autopilot handles 99% of the twists and turns today compared to roughly 60-70% just ~2 years ago.

In the end, it is a software issue, but as mentioned above, it depends if just applying the US solution is even allowed depending on what kind of regulation is in place for ADAS.

 

[daniel]

I'll never understand these questions. There are laws, rules, regulations. Is it serious anybody would want a car (or another robot like device) to break the law? No speed limit, run through red lights and so on?

There is a simple solution: fine/ticket all speeders.

Currently the few US states that have pre-approved (before their existence) of self driving cars all explicitly require them to follow all traffic rules/laws.

Which would include not exceeding the posted speed limit.

Yes that's going to annoy a lot of folks

I would fully support aggressive enforcement of all traffic laws. But the fact is that speed limits are not enforced anywhere I know of except the Indian reservation in New Mexico, where one mph over the limit will get you a ticket. Elsewhere the police give drivers five or ten mph or more over the limit before they ticket. And this results in the de facto speed limit being that much higher than the posted limit.

And it's not just a matter of annoying drivers, though it would do that for FSD cars to drive at exactly or below the limit. It is dangerous for one car to be driving 40 when every other car on the road is driving 55.

We need rational speed limits, strictly enforced. But we don't have that. So how are autonomous cars going to deal with the reality, when their first priority has to be safety? Drive at the legal speed, or drive at the safe speed?

 

[JHCCAZ]

You mean the traffic light control that was released over a year ago and still requires driver confirmation on green lights. LOL.

If it is a such a low bar, why hasn't Tesla's solved it yet? Why does Tesla still require driver supervision on such an easy feature?

Mobileye does not release traffic light control with driver supervision because they are past it. Mobileye is focused on real FSD!

Don't switch the subject princess.
It is such a low bar in the context of your comment --- and yet no one has matched it in the end-user personal vehicle space.

I see you guys engaging this in this particular argument point about traffic-light behavior. Permit me to referee just a little by saying that in Tesla's case, I think it's not really a demonstration of a "milestone on the way to FSD" that you can compare to someone else's progress.

Tesla appears to have at least two different historical feature-development tracks, though some of us hope they are about to merge:

1. Historical AutoPilot features that emphasize assisted driving and is safest on limited-access highways, but which have been increasingly used by Tesla owners for suburban and city driving.
2. City Streets aka "FSD beta" that seems to be a very different starting point and development track.

It seems evident to me that Traffic Control awareness has been added on to both standard and optional historical-AP, originally as a safety-net feature to warn about red-light and stop-sign running. Tesla chose not to actually restrict L2 AP's ODD to limited-access divided highways, though NoA does so. Traffic Control recognition in historical-AP mostly recognizes lights and stop signs correctly, but the ability to predictively watch for hazardous left-turning vehicles (failing to yield, lagging intersection-clearing etc.) or cross-traffic red-light/sop-sign runners, is almost nonexistent in AP. If you don't anticipate or take control, you're basically dependent on last-second emergency braking to save you from cars that are in your path. The Confirm-on-Green requirement (without a lead vehicle to establish that the current forward right-of-way has already been claimed) is absolutely consistent with this predictive limitation. Deriding it for not being able to "see" the lights is missing the point, it usually sees them fine but cannot safely proceed without the very capabilities that prevent it from being a full-blown autonomous L4 system.

City Streets / FSD-beta is fundamentally different, and prediction of traffic, pedestrians and other hazards from all directions is clearly the goal and requirement. It's also a lot more autonomous and a lot harder, which is why its development is coming from a newer AI concept and so on.

So, I think that for Tesla at least, and probably also when benchmarking Intel MobilEye, GM and others in L2 vs. prototype-L4 products, you can't properly consider Traffic-control behavior as a general indicator of comparative advancement. You have to take into account the design goal and the advertised capabilities.

Both Tesla and MobilEye can see traffic lights, but Tesla chose to work that into L2 support. Both have some impressive but unfinished prototype-L4 autonomous systems that surely are far more intelligent when it comes to safely proceeding through green lights.

 

[Tam]

I know you are trolling, but clearly it is a software issue!
It would be nice if you would just state your point clearly.

Another possibility is: It is not just software, it's hardware too.

An example for the sharp turn to the right below: There is an iron fence obstacle in front and Tesla system still wants to drive straight right into that fence as indicated by the solid straight line emanating from the front of the Tesla car icon on the instrument cluster:

Only if Tesla adds a LIDAR, that iron fence would be easily detected as in other companies' systems. But again, Tesla might say that adding LIDAR would create sensor fusion issue

...Things we know.
In some markets (read EU) we know that they've gimped how fast an ADAS can make a turn (turn the wheel) -- to the point where switching lanes on a curve fails (as by the rules design)
We also, know that this accident happened in China - and I am not sure of what kind of regulations they have re ADAS performance.

I know in the US this actually has improved greatly. I take regular trips on the Blue Ridge Parkway and Autopilot handles 99% of the twists and turns today compared to roughly 60-70% just ~2 years ago.

In the end, it is a software issue, but as mentioned above, it depends if just applying the US solution is even allowed depending on what kind of regulation is in place for ADAS.

I don't understand what have EU/China/US laws have got to do with preventing a competent sharp turn maneuver.

Sharp turns accidents with AP have happened before but what relevant here is China seems to attract more attention than outside of China (Similarly, China got more attention on sudden acceleration with TACC which is not something new outside of China if it's set to accelerate automatically to the posted speed sign).

 

[mikes_fsd]

An example for the sharp turn to the right below: There is an iron fence obstacle in front and Tesla system still wants to drive into that fence as indicated by the solid straight line emanating from the front of the Tesla car icon on the instrument cluster:

And clearly the vision system is showing that the road edges are turning right.
This seems a problem of driving policy (ahem, software) not properly determining what to do. Vision read the road correctly as is show my the red edges in the same instrument cluster.

Only if Tesla adds a LIDAR, that iron fence would be easily detected in other companies' systems. But again, Tesla might say that adding LIDAR would create sensor fusion issue

As shown above, LIDAR would provide no additional help here, as the actual perception is not the problem but the driving policy in that example.

 

[mikes_fsd]

I don't understand what has EU/China/US laws have got to do with preventing a competent sharp turn maneuver.

Probably worth looking up the EU regulations on how Autopilot was made useless (at least NoA) for the EU market, would help understand why a lot of the complaints from Europe are so much worse than US.

 

[powertoold]

So, I think that for Tesla at least, and probably also when benchmarking Intel MobilEye, GM and others in L2 vs. prototype-L4 products, you can't properly consider Traffic-control behavior as a general indicator of comparative advancement. You have to take into account the design goal and the advertised capabilities.

Both Tesla and MobilEye can see traffic lights, but Tesla chose to work that into L2 support. Both have some impressive but unfinished prototype-L4 autonomous systems that surely are far more intelligent when it comes to safely proceeding through green lights.

So you're saying Mobileye / GM / etc. don't want to deploy a traffic control feature into their ADAS fleet?

 

[linux-works]

It has LIDAR so theoretically it should be able to see road debris Of course "seeing" and responding are two different things.
Here's a first look: We Try the Lexus Teammate Hands-Free Driver-Assist System
They still require hands on the wheel every "tens of minutes."

you can't (usually) swerve fast and safely enough to avoid hitting stuff when its debris and if the traffic is dense, with not a lot of advance visual depth.

best thing is to just BLAST IT with something like a laser. wonder if the texas factory will include that option?

 

[JHCCAZ]

So you're saying Mobileye / GM / etc. don't want to deploy a traffic control feature into their ADAS fleet?

Well, want is not quite the correct or clearly-interpretable word. I 'll try to clarify further, and perhaps convince you that my comment wasn't exactly on on one side or the other (and not because I'm trying to be so peacefully pure and above the fray, just because I think it's honestly a little more complicated as I laid out in the prior post)*.

Tesla and its consumer-ADAS competitors both deploy L2 supervised driver-assist systems. Of these, Tesla's is arguably the most advanced in that it operates in a wider ODD - I believe Tesla is the only one that permits engagement of AutoSteer when on lane-marked city/suburban streets, i.e. the kinds of streets that have traffic lights. The competitors, as far as I'm aware, are more specific about auto-steering aka "Lane Keeping" as being designed for use only on real highways.

Given this difference, the existing competitors would be undermining their own ODD definition if they were to advertise Traffic Control response along with Lane Keeping auto-steering.

Indeed they may want to get to a Tesla-like L2 (supervised) ODD, but they or their automaker clients' definition of a releasable suburban-ODD auto-steering feature may well be more conservative than Tesla's (here, conservatism is not clearly a good or a bad thing, it's different. Giving drivers a heads-up as Tesla does can be considered safer, OTOH deploying it as less than a foolproof autonomous capability, as we know, makes some drivers overly confident it it despite the Owner's Manual insistence that they not rely on it).

I have the impression that Tesla originally intended AutoSteer as a limited-access highway Lane-Keeping feature, but hey it works well enough, most all the time, that people commonly use it on traffic-light and stop-sign roads. Tesla has chosen to further assist this ODD with Traffic Control recognition. Probably some competitors have such recognition - I don't think it's at all the hardest ADAS perception task - but at this time they'd use it to enforce (restrict ) the ODD rather than assisting it.

These are choices, and no I don't think it proves who actually has the "best" Traffic Control recognition code. This is not taking sides against Tesla nor its competitors!

If I sit with you while you're driving, and I say to you with 100% reliability "that light is green", will you use that fact to power through the intersection with no other considerations? Not if you're any kind of a good driver. And presently, Tesla requires a second factor, either your goosing of the pedal or a lead-car confidence-booster, to proceed through. I hope this gets better but it'll need Karpathy's NN-Vision stuff to do so. But even lacking that, this is not a complaint about Tesla! I for one prefer Tesla's approach here, and to my knowledge no other OE ADAS package on the market offers this. But again, I'm guessing the reasons are not technical inability to recognize the traffic signal itself. It's the different companies' decisions about what to do with the information, coupled with Tesla's arguable advantage in AutoSteer control on non-highway streets using affordable deployed hardware.

*I've said before that I find TMC to be an unusually good forum for detailed and thoughtful discussions/arguments, way above the usual internet-comment fare. But still there's this "whose side are you on?" overlay. It's harder to make people appreciate a point about a topic than it is to make one for Tesla or against Tesla, or for/against Mobileye, Waymo etc. If I think @diplomat33 makes a good point or a useful post and I Like/Agree with it, will I be put on @mike_fsd's suspicion list? If I choose (even cautiously) to contradict @gearcruncher or @Bladerskb in any way, will I my point be twisted and blasted with a torrent of "You're saying Tesla is perfect and you understand nothing"?​

 

[stopcrazypp]

Another possibility is: It is not just software, it's hardware too.

An example for the sharp turn to the right below: There is an iron fence obstacle in front and Tesla system still wants to drive straight right into that fence as indicated by the solid straight line emanating from the front of the Tesla car icon on the instrument cluster:

Only if Tesla adds a LIDAR, that iron fence would be easily detected as in other companies' systems. But again, Tesla might say that adding LIDAR would create sensor fusion issue

I disagree on this take. Sensor hardware will only help if there is a perception problem. However there isn't: the roadway is drawn correctly in the picture shown. It's the path planning that went wrong, and that's purely a software issue.

 

[JHCCAZ]

I disagree on this take. Sensor hardware will only help if there is a perception problem. However there isn't: the roadway is drawn correctly in the picture shown. It's the path planning that went wrong, and that's purely a software issue.

That seems sensible and also somewhat bothersome, as this sharp bend is not exactly a one-in-a-million edge case.

Is there any way to tell from the very short trouble sequence, whether the driver may have actually overridden braking/slowing ahead of time, or auto-steering in the curve? The video view doesn't show the driver's feet, nor actually his hands right before the trouble.

Otherwise, why would the car drive right off the road that it knew was bending sharply to the right?

If it's the car, then indeed it looks like the path-planning module ignoring the perception module. But if the driver was distracted and confusedly intervened in a counter-productive way, that could produce this result. I guess the answer is "in the logs"...

 

[thewishmaster]

I disagree on this take. Sensor hardware will only help if there is a perception problem. However there isn't: the roadway is drawn correctly in the picture shown. It's the path planning that went wrong, and that's purely a software issue.

+1, definitely a software/localization issue. Didn’t the driver recently enable FSD in the screenshotted part of the video, and the software was still trying to localize itself and figure out the route? You can see on the nav that the directions say to go straight, so the car assumes that straight is where it should go, gate or no gate. I recall green mentioning increased reliance on drivable space in more recent software versions, so I’m guessing V9 might not make as many such rookie mistakes

 

[JHCCAZ]

Unless I'm confused I think we're talking about two different examples here. The Chinese sharp bend in the road / accident video is not the same as the screenshot of the sharp right turn directly in front of a metal gate. Both are potentially examples of failures, the first perhaps in path planning (or as I suggested, maybe confused input). The second (gate photo) seems to be a perception or decision/interpretation failure - perhaps as you say a dangerous settling-in interval.

 

[thewishmaster]

Unless I'm confused I think we're talking about two different examples here. The Chinese sharp bend in the road / accident video is not the same as the screenshot of the sharp right turn directly in front of a metal gate. Both are potentially examples of failures, the first perhaps in path planning (or as I suggested, maybe confused input). The second (gate photo) seems to be a perception or decision/interpretation failure - perhaps as you say a dangerous settling-in interval.

Shoot, I 100% was talking about the gate case, not the sharp bend case. Let me edit to quote the post you were responding to instead.

 

[Knightshade]

I have the impression that Tesla originally intended AutoSteer as a limited-access highway Lane-Keeping feature

I mean- it literally says that in the owners manual, so it's more than an impression.

Tesla just chooses not to enforce it.

If I think @diplomat33 makes a good point or a useful post and I Like/Agree with it, will I be put on @mike_fsd's suspicion list? If I choose (even cautiously) to contradict @gearcruncher or @Bladerskb in any way, will I my point be twisted and blasted with a torrent of "You're saying Tesla is perfect and you understand nothing"?

The great part is if you try and stick to the middle ground where many of the facts live, you get disagrees and arguments from both

 

[powertoold]

The competitors, as far as I'm aware, are more specific about auto-steering aka "Lane Keeping" as being designed for use only on real highways.

Totally not true, look at bmw, which has blog articles about a soon to be released traffic light feature, also this Toyota video:

Tesla competitors definitely want to deploy a traffic control feature if they had the technical means.

 

[JHCCAZ]

Totally not true, look at bmw, which has blog articles about a soon to be released traffic light feature, also this Toyota video:

Tesla competitors definitely want to deploy a traffic control feature if they had the technical means.

Thanks for the video, and I think I found the related BMW blog. Well I don't think what I wrote is "totally not true". First of all, as you noted the BMW thing is a planned/advertised upcoming feature, it's hard to decipher the exact meaning but I think they're saying many BMWs post-mid-2020 are equipped - to download this upgrade when it becomes available. But you are correct that when it happens, this is a non-Tesla example of an urban-streets ODD with red-light recognition and (like Tesla currently) requiring green-light Go confirmation. So, is this happening because until now BMW was technically unable to recognize traffic lights? Or that until (someday soon) they weren't confident enough to explicitly support a non-highway lane-tracking / adaptive-cruise ODD? My engineering judgment is that the latter has been the barrier.

Regarding the Toyota video, maybe I missed it but I didn't see a scene or a caption relating to traffic-signal recognition. Is there a red/green traffic-light icon on the visualization display? Or maybe this is just Traffic Jam Assist, basically car-following ACC + Lane-Centering down to 0 mph but no reaction to signals. In any case, I grant you that it's showing both highway and non-highway speed control and lane-centering capability in an official video.

Look, we're both Tesla fans. It's a fascinating car. I ordered mine largely on the basis of its best-in-class ADAS features, including that it's the only one that I know will try to assist me on my normal traffic-light-heavy commute. I'm keenly following this topic as I think someday I may actually need L4 to remain fully independent, and I don't prefer to call taxis whether manned or unmanned. But my engineering judgment is that recognition of the traffic signal is one of the easier sub-tasks and within the capabilities of any serious ADAS dev team in 2021. Nowhere do I read that it's a particular perceptual difficulty.

Recognizing all the other goings-on at the intersection and figuring out what to do with the green light, that's the challenge. So far, only Tesla seems to have been confident enough to give owners access to this signal-phase info (another reason to be a Tesla customer) but I'll agree it may begin to appear elsewhere. That it's coming late and slowly but not "never" - you feel that validates that competitors just couldn't code it before; I still feel it's because they were too cautious, their vehicles weren't well-enough equipped or they lacked confidence in their other general urban-ADAS capabilities.

Can we agree on this one, I think more fundamental: Tesla is pushing the car industry - towards electrification, towards charging infrastructure, towards new manufacturing methods, towards autonomy. Most of the established players are being pushed, and they'd better pick up speed or they'll get pushed aside.

 

[dgatwood]

Currently the few US states that have pre-approved (before their existence) of self driving cars all explicitly require them to follow all traffic rules/laws.

Which would include not exceeding the posted speed limit.

Yes that's going to annoy a lot of folks

That's going to end up causing a few dozen deaths in a twenty-car pileup in Arizona, and suddenly they'll change their minds. Ever wonder why Waymo doesn't drive on the freeway in Chandler? One reason is probably because someone would likely get killed if you tried to drive the speed limit through Phoenix on a freeway. There's just no way to do it safely when the average speed of traffic is 15–20 MPH over the limit. ⁠

 

[powertoold]

Regarding the Toyota video, maybe I missed it but I didn't see a scene or a caption relating to traffic-signal recognition. Is there a red/green traffic-light icon on the visualization display? Or maybe this is just Traffic Jam Assist, basically car-following ACC + Lane-Centering down to 0 mph but no reaction to signals. In any case, I grant you that it's showing both highway and non-highway speed control and lane-centering capability in an official video.

I was using the Toyota video as an example of an "autopilot" competitor being used on city streets. I think your prior point was that Tesla competitors aren't focused on city streets (can only be used on highway), and that's why they're not deploying traffic control features.

 

[Knightshade]

Totally not true, look at bmw, which has blog articles about a soon to be released traffic light feature

if it's soon to be released it doesn't exist as far as consumers are concerned.

Remember when Audi was "soon to be" releasing a 2018 model with L3? Never happened.

Hell Tesla has been "soon" to drive coast to coast without needing a single touch (including for charging!) since what 2017?