What are the guidelines for using Autopilot on roads with cross traffic?

[turnem]

Now that we are hopefully back on topic and past those silly side-bars, what behavior would you guys like to see in these situations? Personally, I wish the car took the crossing vehicle's speed and trajectory in to account and if it calculated with reasonable certainty that it won't be a collision hazard, it simply colored that car yellow on the screen to let the driver know the car sees what is going on and the car just continues on it's normal course exactly like a human would do.

Yes - something along these lines would be nice. They could incorporate similar logic for when I car traveling ahead of you in your same direction makes a left or right turn out of your lane. Most of the time the car will almost come to a complete stop and then hesitantly resume forward... the delay is very exaggerated and usually frustrates drivers behind me. And sometimes I've had other cars think I'm pausing to let them pull in front of me which creates a whole new problem.

 

[Knightshade]

Now that we are hopefully back on topic and past those silly side-bars, what behavior would you guys like to see in these situations?

Situations where people are complaining that AP doesn't "work right" in places Tesla themselves explicitly tells you it's not intended to work right?

I'd like to see the behavior of those complaining to change to actually reading, understanding, and accepting both the owners manual and the well documented facts about the design intent and functionality of the system.

That would likely stop them from continuing to complain about something that's working 100% as it's supposed to work.


Even better would be, once nav-in-city FSD is released, that those folks would purchase FSD since that would provide them a system actually intended to be used in the circumstances they seem very upset that AP does not work as they wish it did.

Such a system would obviously behave quite differently in a cross traffic environment since it's actually designed to be used in one unlike the current AP system which explicitly is not.

Personally, I wish the car took the crossing vehicle's speed and trajectory in to account

I'm not actually sure that's actually possible.

Radar is generally very good at being able to tell if an object moving toward, or away, from you is moving at a given relative speed.

But a sideways car is neither moving toward or away from you.

And a single front-mount radar is generally pretty bad at figuring out WTF a sideways car, moving sideways, is doing.


Which is why the system today thinks it's a stopped object in your way and reacts accordingly.

Especially since its fundamental assumption is cross-traffic is not a thing- all cars are going the same way.




Obviously that needs to be solved for FSD in-city driving.... and it might be something they can figure out with vision- but probably NOT with HW2.5 as it doesn't have enough power to actually process every single frame from every single camera so you'd have to be ignoring some OTHER data to maybe figure this out.


Which brings us to an interesting situation.

Those with HW2.5, who did not buy FSD, won't ever get "in city nav" driving features anyway.

So if the AP code remains something that needs to work on both 2.5 and 3, dealing with cross traffic would remain a feature exclusive to only cars with FSD (which will all be HW3 in the nearish future anyway).

And all AP-only cars would continue to operate "properly" only when used where intended- which is divided highways without cross traffic in in the first place.

Yes - something along these lines would be nice. They could incorporate similar logic for when I car traveling ahead of you in your same direction makes a left or right turn out of your lane. Most of the time the car will almost come to a complete stop and then hesitantly resume forward... the delay is very exaggerated and usually frustrates drivers behind me. And sometimes I've had other cars think I'm pausing to let them pull in front of me which creates a whole new problem.

Again it does that now because it's fundamentally assuming you're on a limited access highway that uses on/off ramps.

Not turn lanes and intersections.

 

[t-rizzle]

This scenario is quite different than your examples.

This is something that you have to actively engage AND the car has to allow you to engage it. If I try to engage Autopilot when there are no lane lines it won't let me. It does this because it doesn't feel that it has enough information to properly steer the car. So if they are TELLING us in the manual that the system isn't capable of of using autopilot on city streets then why does it let us engage autopilot on city streets? I'd understand this if it doesn't know the difference between city streets and a divided highways but it clearly does as it automagically limits our TACC speed on city streets differently than it does on divided highways.

If they didn't WANT us to be able to do this because they felt the car wasn't able to do it then they would simply not allow us to engage autopilot in these situations. By allowing us to engage autopilot in these situations one could reasonably assume that Tesla intends for you to do this.

I'd assume that Tesla 100% want us to use AP in every possible scenario where AP can be engaged, if they didn't how else would the NN learn (maybe I'm not understanding fully how NN / ML works). They want you to use AP in these situations and disengage it because it's acting in a manner, that we as experienced drivers would never act. Again to help train the NN to get in city FSD ready for the initial release.

I use AP everywhere I can engaged it and know that on city streets it's going to do things I'd never do and I always correct the wrong driving behavior by taking over the steering wheel (my understanding is that when disengaging AP by taking over the steering wheel it will send this information back, but I could be wrong here). I would rather take this feedback over running AP in shadow mode because you're directly correcting wrong behavior, but maybe shadow mode is better. This all may be a mute point though if they are not using any AP code for FSD code but I'd like to think they are using AP code as a base for FSD.

 

[diezel_dave]

Situations where people are complaining that AP doesn't "work right" in places Tesla themselves explicitly tells you it's not intended to work right?

I'd like to see the behavior of those complaining to change to actually reading, understanding, and accepting both the owners manual and the well documented facts about the design intent and functionality of the system.

That would likely stop them from continuing to complain about something that's working 100% as it's supposed to work.


Even better would be, once nav-in-city FSD is released, that those folks would purchase FSD since that would provide them a system actually intended to be used in the circumstances they seem very upset that AP does not work as they wish it did.

Such a system would obviously behave quite differently in a cross traffic environment since it's actually designed to be used in one unlike the current AP system which explicitly is not.





I'm not actually sure that's actually possible.

Radar is generally very good at being able to tell if an object moving toward, or away, from you is moving at a given relative speed.

But a sideways car is neither moving toward or away from you.

And a single front-mount radar is generally pretty bad at figuring out WTF a sideways car, moving sideways, is doing.


Which is why the system today thinks it's a stopped object in your way and reacts accordingly.

Especially since its fundamental assumption is cross-traffic is not a thing- all cars are going the same way.




Obviously that needs to be solved for FSD in-city driving.... and it might be something they can figure out with vision- but probably NOT with HW2.5 as it doesn't have enough power to actually process every single frame from every single camera so you'd have to be ignoring some OTHER data to maybe figure this out.


Which brings us to an interesting situation.

Those with HW2.5, who did not buy FSD, won't ever get "in city nav" driving features anyway.

So if the AP code remains something that needs to work on both 2.5 and 3, dealing with cross traffic would remain a feature exclusive to only cars with FSD (which will all be HW3 in the nearish future anyway).

And all AP-only cars would continue to operate "properly" only when used where intended- which is divided highways without cross traffic in in the first place.








Again it does that now because it's fundamentally assuming you're on a limited access highway that uses on/off ramps.

Not turn lanes and intersections.

Please stop responding until you actually have something useful to contribute to this conversation.

 

[diezel_dave]

I'd assume that Tesla 100% want us to use AP in every possible scenario where AP can be engaged, if they didn't how else would the NN learn (maybe I'm not understanding fully how NN / ML works). They want you to use AP in these situations and disengage it because it's acting in a manner, that we as experienced drivers would never act. Again to help train the NN to get in city FSD ready for the initial release.

I use AP everywhere I can engaged it and know that on city streets it's going to do things I'd never do and I always correct the wrong driving behavior by taking over the steering wheel (my understanding is that when disengaging AP by taking over the steering wheel it will send this information back, but I could be wrong here). I would rather take this feedback over running AP in shadow mode because you're directly correcting wrong behavior, but maybe shadow mode is better. This all may be a mute point though if they are not using any AP code for FSD code but I'd like to think they are using AP code as a base for FSD.

Couldn't agree more. I continue to use AP in those situations in the hope that I can help Tesla with training data from my area.

 

[turnem]

Couldn't agree more. I continue to use AP in those situations in the hope that I can help Tesla with training data from my area.

Yep. Me as well. I'll continue my course of reckless human error.

 

[Knightshade]

Please stop responding until you actually have something useful to contribute to this conversation.

I did. A fairly detailed and specific discussion on the shortcomings of radar, the base programming asumptions of the current AP system, and how HW 2.5 vs HW3 might impact future capabilities.

If you want to explain which parts you didn't understand I can try and simplify further for you?

That seems more useful than "I wish" but YMMV.

I'd assume that Tesla 100% want us to use AP in every possible scenario where AP can be engaged, if they didn't how else would the NN learn (maybe I'm not understanding fully how NN / ML works).

Nope, not at all.


The NN is not trained in the car. Ever.

It's trained back on Teslas servers.

And largely with data captured passively regardless of AP being in use or not.

Scroll back for the discussion linked to by Green- who has direct access to the AP computer feeds and data explaining this.

I use AP everywhere I can engaged it and know that on city streets it's going to do things I'd never do and I always correct the wrong driving behavior by taking over the steering wheel (my understanding is that when disengaging AP by taking over the steering wheel it will send this information back, but I could be wrong here)

It sends back your location, how you disengaged, the speed, direction, and time. That's it.

It does not send back any video or still shots, or radar data, on what the car was seeing or "thinking" at the time.

it's a tiny amount of data, less than 1kb in size. Basically a text message.

. I would rather take this feedback over running AP in shadow mode because you're directly correcting wrong behavior

You very definitely aren't. If you were it'd be sending back all the sensor data from radar, cameras, ultrasonics, etc... it does none of that.

Couldn't agree more. I continue to use AP in those situations in the hope that I can help Tesla with training data from my area.

You can't. Tesla doesn't "train" that way. See above.

I already cited Green explaining exactly why using AP in those situations doesn't do what you think it does and that shadow mode allows them to collect large amounts of data without you needing to use AP in places it's not intended to be used.

 

[Mardak]

I use AP everywhere I can engaged it and know that on city streets it's going to do things I'd never do and I always correct the wrong driving behavior by taking over the steering wheel (my understanding is that when disengaging AP by taking over the steering wheel it will send this information back, but I could be wrong here).

Yup, the car is definitely sending back more data because you took over. What Tesla does with this data is not clear, but even very basic analysis could show an average number of miles driven before disengagement, so Tesla knows if there has been improvement or regression with a software rollout. Slightly more complex analysis could show that average miles split up by road type, e.g., interstate vs highway vs city. And because these disengagements have location data, Tesla could figure out what locations have the most disengagements and classify the types of failures to perhaps create a trigger to capture additional data.

 

[bmah]

Moderator note: Moved a half dozen posts to Snippiness. Some violated forum rules for conduct, others quoted said messages and came along as collateral damage.

It's fine to disagree, debate, and argue. But name-calling and insults run contrary to TMC rules. Fair warning: Repeat behavior will lead to a ban.

Thank you,

Bruce.

 

[t-rizzle]

Nope, not at all.


The NN is not trained in the car. Ever.

It's trained back on Teslas servers.

Yes, I understand it's going back to the servers. I'm going to start reading up on the posts by Green. I've read some things and it's interesting (everything NN / ML is cool to me).

It sends back your location, how you disengaged, the speed, direction, and time. That's it.

It does not send back any video or still shots, or radar data, on what the car was seeing or "thinking" at the time.

it's a tiny amount of data, less than 1kb in size. Basically a text message.

If this is the case then technically it could use this as a marker for specific scenarios they may want to force into the NN to learn from and pull video, maybe not from my specific car but others maybe that have encountered the same thing?

For the record I agree with you on when AP should be used and then complaining BTW.

Question for you about data going back to the Tesla NN / servers. I do notice when I use AP on specific drives and if I disengage a bunch, both highway and / or street driving, an increase in data being uploaded back to Tesla. I've had some nights where the car uploaded ~4GB of data. Seems like a lot if it's just sending back what would essentially be meta data and no pixel data. I can go back and look at my upload history and see totals for the past few weeks / month on my router. Just would seem crazy to think that I'm always hitting on specific scenarios they are requesting to pull from cars. Either way it's an interesting discussion to have and fun time to be a part of this technology age we're in.

 

[drtimhill]

There is nothing that states that TACC is not to be used around town, especially considering it is specifically "traffic aware".

Nothing at all ... except the user manual. But what do the idiots who wrote that know? Thank goodness we have you to correct us all!

 

[diezel_dave]

Nothing at all ... except the user manual. But what do the idiots who wrote that know? Thank goodness we have you to correct us all!

The user manual states that it isn't to be used on roads with "constantly changing traffic conditions" which would preclude literally every public road.

 

[linux-works]

Nothing at all ... except the user manual. But what do the idiots who wrote that know? Thank goodness we have you to correct us all!

lets be honest. there were many who were part of the effort that produced what you and I see as the 'user manual'.

tech writers, SMEs (subject matter experts, often the coders/designers), managers, editors (well, they used to; I bet like most companies now, they scrapped the 'editor' job) and finally, LEGAL.

a lot of text is put there because 'legal' (dept) wants them to.

this is not a 'pure and direct gospel from engineers'. its a carefully worded legal document and should be thought of that way.

 

[diezel_dave]

lets be honest. there were many who were part of the effort that produced what you and I see as the 'user manual'.

tech writers, SMEs (subject matter experts, often the coders/designers), managers, editors (well, they used to; I bet like most companies now, they scrapped the 'editor' job) and finally, LEGAL.

a lot of text is put there because 'legal' (dept) wants them to.

this is not a 'pure and direct gospel from engineers'. its a carefully worded legal document and should be thought of that way.

Exactly this. If Tesla didn't want drivers to use these features under certain conditions, they have multiple tools to disable said features.

 

[drtimhill]

Exactly this. If Tesla didn't want drivers to use these features under certain conditions, they have multiple tools to disable said features.

Just like all cars have tools to prevent you driving over the speed limit? And disable the car when you are drunk and try to drive it? Or stop you from texting while driving? You seem to think its the makers responsibility to make sure that you can't do anything foolhardy with the car. Since when? You are the driver, and you are responsible for driving the car safely. Not Tesla. Not Autopilot. YOU. If you choose not to read the user manual, or pretend "its just a bunch of lawyers stuff" that you can ignore, then you are driving irresponsibly.

The user manual is very clear about what AP/TACC can and cannot do. If you choose to imagine it can do something else, that's your problem, not Teslas. And it's absurd to argue otherwise.

 

[diezel_dave]

Just like all cars have tools to prevent you driving over the speed limit? And disable the car when you are drunk and try to drive it? Or stop you from texting while driving? You seem to think its the makers responsibility to make sure that you can't do anything foolhardy with the car. Since when? You are the driver, and you are responsible for driving the car safely. Not Tesla. Not Autopilot. YOU. If you choose not to read the user manual, or pretend "its just a bunch of lawyers stuff" that you can ignore, then you are driving irresponsibly.

The user manual is very clear about what AP/TACC can and cannot do. If you choose to imagine it can do something else, that's your problem, not Teslas. And it's absurd to argue otherwise.

Its pretty clear that Tesla wants us to drive around the city on AP so they can gather data on it's performance. They obviously have to make seems like we shouldn't though from a liability standpoint.

They shouldn't disable Netflix or YouTube while I'm driving because it's not their responsibility to protect me from myself and yet they do it anyways?

 

[Knightshade]

Its pretty clear that Tesla wants us to drive around the city

Except for where the manual repeatedly says exactly the opposite of course.

And as explained they aren't "gathering data" by you driving on AP and having the car brake when you don't want it to anyway.

Any data they'd gather from that situation they can gather regardless of AP being on or not via shadow collection.

The only time they "gather data" in general specifically related to AP being on is when you specifically disengage it. And even then the only thing they gather is basically where and when and by what method you disengaged.

You keep thinking you are "helping" by ignoring the directions in the manual. You factually are not- as explained by the guy with direct access to the computer data.

 

[HighZ]

I wish the car took the crossing vehicle's speed and trajectory in to account and if it calculated with reasonable certainty that it won't be a collision hazard, it simply colored that car yellow on the screen to let the driver know the car sees what is going on and the car just continues on it's normal course exactly like a human would do.

I agree - right now, it just sees an obstacle in the path and reacts accordingly. I am fine also with the car slowing, as I would do if someone tried to cut in front, perhaps based on distance.

 

[diezel_dave]

I agree - right now, it just sees an obstacle in the path and reacts accordingly. I am fine also with the car slowing, as I would do if someone tried to cut in front, perhaps based on distance.

Yes, I think slowing a bit is acceptable. Like when I'm driving, I would generally let my foot off the gas when a car is crossing ahead and just coast a bit where right now, the car brakes pretty hard while using TACC because it doesn't seem to use any object persistence to determine where that crossing car came from or where it's going. It just sees something in the way, in that particular sample of video and radar data and it acts on that. I would really like them to make the system smarter and it will be necessary for them to do so when full city support is released.

 

[Mardak]

I am fine also with the car slowing, as I would do if someone tried to cut in front, perhaps based on distance.

Often times I'll press on the accelerator slightly when a car is turning across the road that I shouldn't need to brake hard or at all because the crossing vehicle will pass in time. The extra accelerator press isn't enough to speed up my car, but it will lessen the amount of braking, and it seems like these acceleration overrides can result in extra data sent to Tesla: green on Twitter

So instead of just letting Autopilot do its normal braking on city streets, doing something else while not disengaging will probably help Tesla improve the feature.