Stopping at traffic lights and stop signs now in "shadow mode"

[diplomat33]

Tesla confirmed in the Q2 update letter that "traffic light and stop sign response", is now operating in shadow mode:

"We are making progress towards stopping at stop signs and traffic lights. This feature is currently operating in “shadow mode” in the fleet, which compares our software algorithm to real-world driver behavior across tens of millions of instances around the world,” Tesla wrote."
Tesla Autopilot's stop sign, traffic light recognition and response is operating in 'Shadow Mode'

Good news! A clear sign that Tesla is making progress with their FSD features. Shadow mode means that the feature is one step closer to general release.

Any guess when the feature might roll out wide? End of year probably?

 

[follow.johnny]

Thanks for the information, this is great news and a step closer to FSD eventually.

I'm guessing this feature won't roll out to fleet until it's fully tested by (Early Access Program) owners and their internal staff.

 

[diplomat33]

Thanks for the information, this is great news and a step closer to FSD eventually.

I'm guessing this feature won't roll out to fleet until it's fully tested by (Early Access Program) owners and their internal staff.

According to the autonomy investor day presentation, Tesla uses the following procedure for developing and rolling out a new AP/FSD feature:
1) Learn from humans
Tesla starts with data from human driving to develop the software feature.
2) Shadow Mode
Tesla uploads the software feature to test how it would work in the real world. Tesla uses the feedback to fine tune the software.
3) Early Access
When Tesla feels good about the software, they release it to actual owners in active mode and continue to get feedback on how it works in the real world.
4) Wide roll out
Tesla releases the feature to all cars.

So right now, the feature is at stage 2. So yes, early access should be the next stage.

it is worth noting that it was about 1 month between shadow mode for NOA and wide release of NOA with confirmation. It was about 6 months between shadow mode NOA and wide release of NOA without confirmation. if the timeline holds, we could see "traffic light stop" in a few months.

 

[cucubits]

This traffic light stop thing... won't it just be a party trick since AP is supposed to be used only highways where there are no traffic lights? Or you actually believe AP on city streets will also roll out so soon?

...or they're considering it as an added safety feature? (which with the current track record will choose randomly when it will work)

 

[diplomat33]

This traffic light stop thing... won't it just be a party trick since AP is supposed to be used only highways where there are no traffic lights? Or you actually believe AP on city streets will also roll out so soon?

...or they're considering it as an added safety feature? (which with the current track record will choose randomly when it will work)

I think "traffic light stop" is an incremental first step towards the full "automatic city driving". It's why Tesla has it listed separately from "automatic city driving" even though it is a sub set feature.

It will be more than a party trick. "Traffic light stop" will make AP a bit more useful on city streets. Right now, we can use AP on city streets but we have to disengage when we approach an intersection if there is a red light with no lead car or if we need to turn. "Traffic light stop" will remove one of of those disengagements, allowing us to keep AP on when we approach a red light with no lead car and we intend to go straight.

 

[MorrisonHiker]

This traffic light stop thing... won't it just be a party trick since AP is supposed to be used only highways where there are no traffic lights? Or you actually believe AP on city streets will also roll out so soon?

...or they're considering it as an added safety feature? (which with the current track record will choose randomly when it will work)

Since it is operating in shadow mode, I would assume it is always on, even when AP isn't enabled. It could be recording every single time you are stopped in traffic.

 

[diplomat33]

Since it is operating in shadow mode, I would assume it is always on, even when AP isn't enabled. It could be recording every single time you are stopped in traffic.

I imagine it will record specifically at traffic lights and stop signs and compare what it would have done to what the driver did. It will look at how it would have braked and whether it would have stopped at the stop line correctly.

I am hoping with the size of the Tesla fleet, that Tesla will collect enough data that shadow mode won't take very long. I imagine 1-2 months should be enough to validate the feature and release it to early access.

 

[Tam]

...This traffic light stop thing...

No charge for AP and EAP in shadow mode.

However, once you see others can do this trick on Youtube, those will be the ones who paid up FSD.

And at that time AP and EAP will still be officially for limited-access highway.

 

[MrBadger]

For proper traffic light autonomy, Tesla will need to incorporate V2I in their kit. There are things going on with traffic lights that you cannot detect visually - green waving is one such feature that cannot be predicted visually. Without the ability to gather information from infrastructure, Tesla will be left out in the cold pretty quickly and following (literally) cars from other manufacturers.

 

[Daniel in SD]

...or they're considering it as an added safety feature? (which with the current track record will choose randomly when it will work)

Airbags also work randomly (sometimes they save your life and sometimes they don't) and they're a great safety feature. Even a feature that prevents 50% of running of red lights would be a huge breakthrough.
I don't think they'll release stop light and sign detection as an active feature until it's better than a human (which could be never). Imagine the headlines when someone blows through a stop sign because they thought the car would stop (because it did the last 100 times).

For proper traffic light autonomy, Tesla will need to incorporate V2I in their kit. There are things going on with traffic lights that you cannot detect visually - green waving is one such feature that cannot be predicted visually. Without the ability to gather information from infrastructure, Tesla will be left out in the cold pretty quickly and following (literally) cars from other manufacturers.

If they can't be detected visually then how can humans navigate them?

 

[jerry33]

Airbags also work randomly (sometimes they save your life and sometimes they don't) and they're a great safety feature. Even a feature that prevents 50% of running of red lights would be a huge breakthrough.
I don't think they'll release stop light and sign detection as an active feature until it's better than a human (which could be never). Imagine the headlines when someone blows through a stop sign because they thought the car would stop (because it did the last 100 times).

If they can't be detected visually then how can humans navigate them?

Humans do it by just following the car in front. Tesla would do it the same way. (For you lucky people who have working green waves).

 

[diplomat33]

I don't think they'll release stop light and sign detection as an active feature until it's better than a human (which could be never). Imagine the headlines when someone blows through a stop sign because they thought the car would stop (because it did the last 100 times).

If they can't be detected visually then how can humans navigate them?

Why are people assuming that vision detection of traffic lights can't be done reliably? Like everything else, it just requires training the neural net with enough data. Plus, I am pretty sure other companies like Mobileye and Waymo already have reliable vision detection of traffic lights. It can be done.

Sure there might be some edge cases like traffic lights being broken or out due to a power outage but there are workarounds that Tesla can employ.

 

[sreehyd]

Its interesting to see how AutoPilot would categorize when a driver doesn't fully stop at STOP sign and/or makes a rolling stop at STOP sign & RED light for right turn ..

 

[MrBadger]

If they can't be detected visually then how can humans navigate them?

Humans don't optimally navigate them. They (should) visually obey them, but there is much more going on behind the scenes that a human is oblivious too which means their navigation is dumb. V2I would allow cars to learn from infrastructure to allow them to optimally navigate them. Green waving for instance allows vehicles to traverse traffic lights without stopping greatly reducing queues. You cannot visually predict or learn this - many traffic lights are constantly optimising and even they do not know exactly when they are going to start changing until the second they start. V2I will start to become fairly common place in near future - its already possible in specific locations. Without it, Tesla will be well behind the normal curve.

 

[Daniel in SD]

Humans don't optimally navigate them. They (should) visually obey them, but there is much more going on behind the scenes that a human is oblivious too which means their navigation is dumb. V2I would allow cars to learn from infrastructure to allow them to optimally navigate them. Green waving for instance allows vehicles to traverse traffic lights without stopping greatly reducing queues. You cannot visually predict or learn this - many traffic lights are constantly optimising and even they do not know exactly when they are going to start changing until the second they start. V2I will start to become fairly common place in near future - its already possible in specific locations. Without it, Tesla will be well behind the normal curve.

Doesn't this only really work once the vast majority of cars are autonomous?

 

[brkaus]

Humans don't optimally navigate them. They (should) visually obey them, but there is much more going on behind the scenes that a human is oblivious too which means their navigation is dumb. V2I would allow cars to learn from infrastructure to allow them to optimally navigate them. Green waving for instance allows vehicles to traverse traffic lights without stopping greatly reducing queues. You cannot visually predict or learn this - many traffic lights are constantly optimising and even they do not know exactly when they are going to start changing until the second they start. V2I will start to become fairly common place in near future - its already possible in specific locations. Without it, Tesla will be well behind the normal curve.

That's great for the future - but not sure how all those humans mixed in will drive in that environment?

 

[Silicon Desert]

Humans do it by just following the car in front. Tesla would do it the same way..

Hmmm, Daniel has a good point.. But what if you ARE the car in front and have no one to follow?

 

[diplomat33]

For proper traffic light autonomy, Tesla will need to incorporate V2I in their kit. There are things going on with traffic lights that you cannot detect visually - green waving is one such feature that cannot be predicted visually. Without the ability to gather information from infrastructure, Tesla will be left out in the cold pretty quickly and following (literally) cars from other manufacturers.

Vision is enough. It just required enough data to train the neural net. After all, humans do just fine visually tracking traffic light patterns. A computer can be taught to do the same.

V2I might be good for the future but it requires a big infrastructure cost and would take a lot of time.

 

[MrBadger]

No. Its something that even human drivers can benefit from with V2I instrumentation in the car, And this will start to be common place long before self driving becomes common place.

Knowing a speed range to travel along a section of road to avoid having to slow or stop for traffic lights will hugely benefit congestion build up, reduce travel times and greatly reduce pollution from stop/start ICE vehicles. Lots of other scenarios that V2I also can be applied to, but this thread specifically about traffic lights.

 

[jerry33]

Hmmm, Daniel has a good point.. But what if you ARE the car in front and have no one to follow?

Presumably the car will drive the speed limit and that will get you in the next green wave. Although some green waves I've encountered only work if you are in a pack of cars, not the first car, and some are set so that you have to really speed (10 mph faster) so they can generate revenue.