Doesn't bode well for FSD...

[tls]

I had one of those "AP moments" yesterday and it really got me thinking. We measured the kid -- she's been waiting literally all summer -- at just over 54" in her sneakers and off we went to Coney Island to ride the Cyclone, since she's finally "this tall".

Traffic was awful but both Waze and the Tesla nav took us down NY 9A and through the Brooklyn-Battery Tunnel. A route I'm pretty familiar with since that's how you get to the local Service Center -- it's basically at the Brooklyn tunnel exit. So I know AP is not really happening on 9A (stoplights, unpredictable local traffic) but is good in the tunnel itself unless traffic is really moving at 40+ MPH (tunnel speed limit is 25) which only happens at the off-est of off-hours.

Yesterday one of the two tunnel tubes was closed for construction so they had traffic going both directions in the other tube and it was very slow -- 15-20MPH the whole way. I set Autopilot and kept an eye out for anything unusual. Which I was expecting to see outside the car, not inside!

Now, this tunnel goes from the tip of Manhattan, under the harbor, passes under one corner of Governor's Island where there's a ventilation shaft, then under more of the harbor to Brooklyn. With traffic in both directions we were in the left-hand tube for our direction of travel, which is unusual but happens a few times a week. I assume that may play some part in what follows, since our heading and position entering the tunnel were 40' or so off the usual path through the right-hand tube.

About halfway across something really funny happened:

• First, the IC display snapped from the in-tunnel view with a countdown of the tunnel distance to "turn right on Barry Road in 250 feet".
• Then the 15" display warped our position to show us about halfway between the tunnel and the shore.
• Then the dread "RED HANDS" appeared with no warning
• Then the IC began to frantically issue us directions all as if we were onshore on the island -- or, more properly, perhaps, had just driven off the island and into the water... turn on Andes Road, Kimmel Road, Craig Road, etc, all the old military roads on the island you can't even legally drive on any more.
• Finally it just gave up and stopped issuing directions.

All this time our position continued to move on the big map, parallel to the tunnel but a noticeable distance to one side. Interestingly, I glanced at Waze on my phone and it showed roughly the same approximate position -- generated I assume from our heading and speed when we entered the tunnel.

When we exited the tunnel all was back to normal. But what would have happened in an FSD car? All the cameras, radar, lidar, etc. in the world will not help if the software can fool itself into thinking you're floating in the water when you're in a tunnel, snaps your position to a surface road you couldn't possibly have accessed given your prior path through an underwater tunnel with no exits, etc.

Yes, I understand they have to handle cases where the car is on a ferry, etc. but in those cases it won't be in "drive" and won't have AP engaged, so I can't see a way properly functioning AP software should have been so easily fooled. The car drove a mile "in the water". The software ought to know there's a tunnel there, and adjust its expectations accordingly -- particularly, not be fooled into thinking the car's magically popped up on an island.

I assume what happened was:

• The car's map database may show the northeast (left-hand, for our direction of travel) tube as one-way north/westbound -- which it sometimes is, but sometimes isn't. This could be fixed.
• This probably put our path too far from the nearest lane going in the "right" direction (right-hand tube, right lane?) for the nav system to snap to it with high confidence. Shouldn't they special-case tunnels?!
• Passing under the ventilation shaft on the island the car got a momentary gps or cellular position fix (hard to believe, but...)
• Suddenly the car decided we were on the island
• When it saw us drive "off" the island "into" the water it freaked out and red-handsed us

This stuff seems pretty basic. The nav in my Prius got it right a decade ago -- I drove through this and other tunnels plenty of times with no warping -- yes, including at off-hours when the tubes were switched, yes, including tunnels with ventilation shafts that went up to things with roads on them. In an FSD car, I assume this freak-out would have resulted in FSD disengaging and dumping control on the driver -- not good, right?

I'm not saying I don't think Tesla can get it right, in this and other cases were limitations of the current nav would pose huge FSD problems. But it's always fascinating to find a new one. Let me just say I hope some of the necessary improvements make it back into older cars like mine!

 

[boaterva]

Interesting... you would think it needs to 'double check' location somehow, but what other way is there besides GPS and/or cellular comms? Inertial navigation? Not at consumer prices!

 

[BradWI]

I'd be shocked if any of the current vehicles (Model 3 included) ever have true FSD. There's likely to be increased HW requirements that just won't be able to be retrofitted. Anyone who bought the FSD option will either be refunded or have that $ applied to a future "true" FSD Tesla when they finally have one which could be 4-5 years realistically based on the progress of relatively simple AP features.

 

[MarcusMaximus]

I had one of those "AP moments" yesterday and it really got me thinking. We measured the kid -- she's been waiting literally all summer -- at just over 54" in her sneakers and off we went to Coney Island to ride the Cyclone, since she's finally "this tall".

Traffic was awful but both Waze and the Tesla nav took us down NY 9A and through the Brooklyn-Battery Tunnel. A route I'm pretty familiar with since that's how you get to the local Service Center -- it's basically at the Brooklyn tunnel exit. So I know AP is not really happening on 9A (stoplights, unpredictable local traffic) but is good in the tunnel itself unless traffic is really moving at 40+ MPH (tunnel speed limit is 25) which only happens at the off-est of off-hours.

Yesterday one of the two tunnel tubes was closed for construction so they had traffic going both directions in the other tube and it was very slow -- 15-20MPH the whole way. I set Autopilot and kept an eye out for anything unusual. Which I was expecting to see outside the car, not inside!

Now, this tunnel goes from the tip of Manhattan, under the harbor, passes under one corner of Governor's Island where there's a ventilation shaft, then under more of the harbor to Brooklyn. With traffic in both directions we were in the left-hand tube for our direction of travel, which is unusual but happens a few times a week. I assume that may play some part in what follows, since our heading and position entering the tunnel were 40' or so off the usual path through the right-hand tube.

About halfway across something really funny happened:

• First, the IC display snapped from the in-tunnel view with a countdown of the tunnel distance to "turn right on Barry Road in 250 feet".
• Then the 15" display warped our position to show us about halfway between the tunnel and the shore.
• Then the dread "RED HANDS" appeared with no warning
• Then the IC began to frantically issue us directions all as if we were onshore on the island -- or, more properly, perhaps, had just driven off the island and into the water... turn on Andes Road, Kimmel Road, Craig Road, etc, all the old military roads on the island you can't even legally drive on any more.
• Finally it just gave up and stopped issuing directions.

All this time our position continued to move on the big map, parallel to the tunnel but a noticeable distance to one side. Interestingly, I glanced at Waze on my phone and it showed roughly the same approximate position -- generated I assume from our heading and speed when we entered the tunnel.

When we exited the tunnel all was back to normal. But what would have happened in an FSD car? All the cameras, radar, lidar, etc. in the world will not help if the software can fool itself into thinking you're floating in the water when you're in a tunnel, snaps your position to a surface road you couldn't possibly have accessed given your prior path through an underwater tunnel with no exits, etc.

Yes, I understand they have to handle cases where the car is on a ferry, etc. but in those cases it won't be in "drive" and won't have AP engaged, so I can't see a way properly functioning AP software should have been so easily fooled. The car drove a mile "in the water". The software ought to know there's a tunnel there, and adjust its expectations accordingly -- particularly, not be fooled into thinking the car's magically popped up on an island.

I assume what happened was:

• The car's map database may show the northeast (left-hand, for our direction of travel) tube as one-way north/westbound -- which it sometimes is, but sometimes isn't. This could be fixed.
• This probably put our path too far from the nearest lane going in the "right" direction (right-hand tube, right lane?) for the nav system to snap to it with high confidence. Shouldn't they special-case tunnels?!
• Passing under the ventilation shaft on the island the car got a momentary gps or cellular position fix (hard to believe, but...)
• Suddenly the car decided we were on the island
• When it saw us drive "off" the island "into" the water it freaked out and red-handsed us

This stuff seems pretty basic. The nav in my Prius got it right a decade ago -- I drove through this and other tunnels plenty of times with no warping -- yes, including at off-hours when the tubes were switched, yes, including tunnels with ventilation shafts that went up to things with roads on them. In an FSD car, I assume this freak-out would have resulted in FSD disengaging and dumping control on the driver -- not good, right?

I'm not saying I don't think Tesla can get it right, in this and other cases were limitations of the current nav would pose huge FSD problems. But it's always fascinating to find a new one. Let me just say I hope some of the necessary improvements make it back into older cars like mine!

Right now, AP doesn't use the map at all, and only uses GPS to establish speed limit, so there's no real reason to worry about such things in the software. Even if it did have to in such a scenario, presumably it'd continue following the road looking for the streets it expects, not find them, and then find the correct streets once it exits the tunnel and GPS position is corrected.

There's not much of a reason to suspect it'll completely forget how to drive if it loses track of where it is on the map.

 

[tls]

Right now, AP doesn't use the map at all, and only uses GPS to establish speed limit, so there's no real reason to worry about such things in the software.

So the red-hands I observed immediately after the map position and AP went bonkers was a coincidence?

My guess would have been that -- at least -- since it's trying to use position (and thus some map data, if not "the map"), to establish speed limit, it went bonkers when the car seemed to transition between an impossible sequence of positions (in tunnel, estimating position from speed and initial heading; "on the island"; "in an impossible place" since there was no path from "on the island" to in the middle of the water) because it decided it couldn't know the speed limit, and thus the red-hands. No?

 

[georgetg]

Instrument cluster navigation is GPS only.

Center display is GPS with inertial backup.

Instrument cluster will freeze with lack of GPS signal.
Center display will continue on internal inertial navigation only.

Inertial navigation drift is pretty high, around 1 mile in 10-15 minutes.

Cheers
George

 

[dhanson865]

Maybe they'll just put GPS repeaters inside all tunnels to make sure self driving cars have a proper GPS fix.

GPS Repeaters for Tunnels was at the top of the google search.

 

[boaterva]

Instrument cluster navigation is GPS only.

Center display is GPS with inertial backup.

Instrument cluster will freeze with lack of GPS signal.
Center display will continue on internal inertial navigation only.

Inertial navigation drift is pretty high, around 1 mile in 10-15 minutes.

Cheers
George

So it does use INS? Or do you mean 'dead reckoning' from knowing 'how far it went in a certain direction' which is sorta but not quite the same thing unless inertial hardware is a lot cheaper than I thought it was!

 

[brianman]

But what would have happened in an FSD car?

"Construction data from 'the cloud' has arrived. Recommended route not available. There are alternate routes that add approximately 45 minutes to your trip time. As FSD is enabled, a detour to the alternate route will be taken in 5 minutes. If you'd prefer to take the recommended route and deal with the construction manually, <instructions>."

 

[Max*]

I think you're overreacting.

This problem has been solved, years ago. A simple kalman filter to couple the INS and GPS.

If you're driving in a GPS denied environment for long enough (a tunnel), the car has no idea where you are. Having FSD doesn't mean it needs to follow the nav. It means it needs to not crash into the wall or the car in front it, until it gets out of the tunnel.

There are probably a million edge cases that need to be programmed into it, but showing one of them and saying "FSD is a long way off" doesn't prove anything*.


* I agree FSD is a long ways away, but not due to this scenario.

 

[MarcusMaximus]

So the red-hands I observed immediately after the map position and AP went bonkers was a coincidence?

My guess would have been that -- at least -- since it's trying to use position (and thus some map data, if not "the map"), to establish speed limit, it went bonkers when the car seemed to transition between an impossible sequence of positions (in tunnel, estimating position from speed and initial heading; "on the island"; "in an impossible place" since there was no path from "on the island" to in the middle of the water) because it decided it couldn't know the speed limit, and thus the red-hands. No?

Most likely, yes. Even if it looses track of speed limit entirely, it defaults to a speed limit of 45mph(this happens near my house where there literally is no speed limit yet; new construction).

 

[RDoc]

That's odd. I've specifically done the same thing in the Ted Williams Tunnel in Boston, which is curved, and the in car navigation was very accurate while the phone's GPS didn't work at all.

I believe they use the steering wheel angle and distance driven to get a close fix, then put the car on the tunnel road. Perhaps it got messed up because you were in the wrong tube?

 

[georgetg]

So it does use INS? Or do you mean 'dead reckoning' from knowing 'how far it went in a certain direction' which is sorta but not quite the same thing unless inertial hardware is a lot cheaper than I thought it was!

Well who knows... I'm sure there's no ring laser gyro but solid state accelerometers are a commodity these days. Integrating the speed steering angle, compass and the accelerometers would make for a simple "INS" type system good enough for tunes and parking garages.

The drift is so bad though that when my GPS sensor was intermittent it drifted overnight so far that in the morning the center map showed me halfway around the world, in Iran, while the IC screen still showed me at home in CA...

Cheers
George

 

[Matias]

Interesting... you would think it needs to 'double check' location somehow, but what other way is there besides GPS and/or cellular comms? Inertial navigation? Not at consumer prices!

Actually inertial navigation is a basic feature with navigation chips nowadays. Of course it is not based on gimbals but IC.
Trimble - Aardvark DR + GPS

 

[boaterva]

Actually inertial navigation is a basic feature with navigation chips nowadays. Of course it is not based on gimbals but IC.
Trimble - Aardvark DR + GPS

Good to know, didn't know it was *that* cost effective and integrated! I wonder then which nav system has what priority in calculations to cause the issues like OP's.

 

[eye.surgeon]

Here's the thing....we don't really need FSD. What I've decided in 3 years of ownership with AP1 is simply assisted driving greatly improves safety and makes driving dramatically less demanding and stressful. 90% FSD is very achievable and really all that's necessary until we wait for the next leap in technology. Trying for 100% FSD right now is just going to disappoint.

 

[Max*]

Here's the thing....we don't really need FSD. What I've decided in 3 years of ownership with AP1 is simply assisted driving greatly improves safety and makes driving dramatically less demanding and stressful. 90% FSD is very achievable and really all that's necessary until we wait for the next leap in technology. Trying for 100% FSD right now is just going to disappoint.

I love my AP1, but I disagree.

I don't need "FSD" (L4/L5), but I'd love L3 highway autonomous. If I could zone out/nap/etc. on those 150 mile stretches between superchargers, I'd be golden.

 

[tls]

That's odd. I've specifically done the same thing in the Ted Williams Tunnel in Boston, which is curved, and the in car navigation was very accurate while the phone's GPS didn't work at all.

I believe they use the steering wheel angle and distance driven to get a close fix, then put the car on the tunnel road. Perhaps it got messed up because you were in the wrong tube?

I would think so, yes. But what I meant to highlight was that the Tesla nav screwed up something the nav in my Prius got right pretty much every time a decade ago (tunnel tubes are switched for night and weekend maintenance all the time around here; I drive the Battery Tunnel a few times a month; never seen warping like that before) -- and then, as far as I can tell, the car disabled AP because of it.

That'd be a pretty frustrating experience in an FSD car. I'm just saying, while they're over there at the skunk works with the 15-camera prototype augmented by Skinner's torpedo pigeon in a lidar housing on the roof, building the next, latest, and greatest, some more attention to the not-quite-latest and less exciting may be required and might yield dividends for all.

 

[Max*]

as far as I can tell, the car disabled AP because of it.

I doubt it.

I've had my AP/TACC disabled in the tunnels before. It doesn't happen every time, and has nothing to do with GPS drift (this was before AP, this was when we had just pure TACC, with a message about radar obstructed or something like that). I'm guessing multipath from the tunnel walls, but who knows.

 

[Doug_G]

You don't need an inertial guidance system. Dead reckoning is fine for situations like this. You know the steering angle and the number of rotations of the wheels. Good enough for a tunnel.

Also GPS and autopilot are separate systems. It's surprising that you got red hands just because the nav system was confused. After all if nav wasn't running the autopilot would have worked just fine. So either it was an "abundance of caution" thing or the nav hysterics overloaded the processor.