FSD Beta Videos (and questions for FSD Beta drivers)

[AlanSubie4Life]

You have to remember that what's on the UI doesn't match what's being displayed by the gopro out the windshield. It lags.

Of course. Always very clear when watching these videos.

that the car starts to slow down (green regen bar) just as the bike begins to cross the rear of the white car ahead of Brandon's.

Yeah, but hard to know whether that was Brandon or the car. It's very sudden slowing - could have been the brake (immediately reflected) applied by Brandon, and it may take a fraction of a second for the blue steering wheel to disappear, due to lags in that portion of the display (someone would have to actually test it to know whether this was more likely Brandon or the car braking).

 

[TIppy]

It's interesting. Or maybe it is mundane. Any ideas what is happening here?

It sees her reflection in the building windows to the right of the car?. The forward looking cameras were tracking her, and the b-pillar camera reflections were correlated with forward camera images. When she was no longer visible in the forward cameras the b-pillar cameras continued to track her. Maybe?

I also noticed that she wasn't show in the UI until she turned to enter the crosswalk, therefor becoming relevant.

 

[TIppy]

could have been the brake (immediately reflected) applied by Brandon

I doubt a human would have reacted that quickly. That seems like less than 0.3 secs between the bike turning and the brakes being applied.

 

[AlanSubie4Life]

I doubt a human would have reacted that quickly. That seems like less than 0.3 secs between the bike turning and the brakes being applied.

Haha. That's funny. As outlined above, it was obvious that the bike was turning about 4 seconds before the brakes were applied.

It sees her reflection in the building windows to the right of the car?.

Maybe. That would be impressive, if that were the reason. Superhuman abilities, if so.

On one hand, dumb as a box of rocks. On the other hand, superhuman! Pretty typical for a computer I guess.

 

[TIppy]

it was obvious that the bike was turning about 4 seconds before the brakes were applied.

Looks like less than a second between when he obviously leans into the turn and power goes to zero.

 

[TIppy]

if that were the reason. Superhuman abilities

I think we would obviously know it was her if we looked over at her reflection in the windows after she disappeared beyond the truck.

 

[AlanSubie4Life]

Looks like less than a second between when he obviously leans into the turn and power goes to zero.

I'm just saying that Brandon knew the cyclist was crossing his path before it happened. So reaction times aren't relevant here - it wasn't an unexpected event that he had to react to. So we can't use this as a metric for whether or not it was him braking, or the car.

 

[Daniel in SD]

A problem with these videos is that they're nowhere near the resolution you see in real life (or that FSD sees). The cyclist does a track stand and signals with their left hand. Them raising their arm is not visible because of low resolution and compression artifacts but you can clearly see them lower their arm as they begin the turn.

 

[AlanSubie4Life]

A problem with these videos is that they're nowhere near the resolution you see in real life (or that FSD sees). The cyclist does a track stand and signals with their left hand. Them raising their arm is not visible because of low resolution and compression artifacts but you can clearly see them lower their arm as they begin the turn.

View attachment 618636

Yep. Actually the cyclist is really clearly visible to the driver, in the left turn lane, 8 seconds before he turns in front of the car, not just 4. It's just hard to see in the video due to all the artifacts, etc. Brandon saw this coming a long way off, I am pretty sure. He probably wasn't sure how the car would react, but he did know it was something he would have to deal with shortly. This stuff is likely all going on in his mind in the background as he talks in the video about the bad behavior at the prior intersection. Eyes picking up cyclist, brain processing, etc. I assume FSD picked up the cyclist with similar timing (maybe slightly later due to resolution limits?), but didn't display it on the GUI due to the distance.

I think it was fine for him to not brake much earlier in this case, as it wasn't really that close, but the bad steering necessitated the disengagement.

These videos are pretty hard to interpret with all the compression, that's for sure.

I imagine in future builds Tesla will continue to enhance the ability to pick up hand signals and such at greater distances. Just not sure when that exactly will occur.

 

[Daniel in SD]

This discussion reminded me of the articles from 5 years ago about Google's self driving car getting confused by a cyclist doing a track stand.
https://www.washingtonpost.com/news...-bikes-can-confuse-googles-self-driving-cars/

 

[TIppy]

I'm just saying that Brandon knew the cyclist was crossing his path before it happened

How did Brandon know that? When I fist watched this video, it seemed like the guy came out of nowhere and was heading straight. Then he turned abruptly. You can see the expressions on both of their faces change after he's leaned well into the turn. The regen is way into green before the steering wheel turns to the right and Brandon's hands counteract it.

 

[TIppy]

The problem with these videos is that they're nowhere near the resolution you see in real life (or that FSD sees)

The problem with these videos is that they capture a wide field of view compared to the 6 degrees or so that our vision focuses on. Then they are analyzed frame by frame after the fact when we can scan our 6 degrees over the whole scene at our leisure while at the same time knowing what we're looking for. I mean come on. That guy flapped his arm like a wing to signal his turn. That could easily be missed unless you were looking right at him at the instant he did it.

 

[AlanSubie4Life]

When I first watched this video, it seemed like the guy came out of nowhere and was heading straight.

I think you're just thrown off by the poor quality of the video. Reality is a lot more clear. How, I don't know why you'd be expecting him to head straight! He was in a left turn lane (dedicated, but unprotected) at an intersection where the light had just turned green!!! (It's an odd intersection, but that's where he was clearly coming from. This would be really obvious to any driver - you can see a lot more of the exact road markings than are visible in the video.)

That's why Brandon (hopefully) (and hopefully the car) knew that the cyclist was turning left. Did not need to interpret the hand signals (though clearly that's going to be needed relatively soon, as we go down the march of 9s - not sure whether we're there yet).

The regen is way into green before the steering wheel turns to the right and Brandon's hands counteract it.

Typically people hit the brakes before applying steering correction in emergencies. It's not always the best idea, steering can work really well, but braking first does tend to be the natural response. As discussed, I don't know whether it was Brandon or FSD doing the slowing - some experimenting would be needed to see if we could get a definitive answer. If the blue steering wheel disappears instantly (no lag) when the brakes are applied, then this was FSD doing the slowing. There's also the disengage sound, which does come pretty late, but again, not sure how much that lags.

I can't say for sure that Brandon was expecting this 8 seconds beforehand, of course - we can't know what he actually saw - but most alert drivers would have anticipated this well in advance, with similar timing. It was not a difficult situation. Very common.

That guy flapped his arm like a wing to signal his turn. That could easily be missed unless you were looking right at him at the instant he did it.

You definitely are underestimating the extreme capabilities of the human visual system. Regarding the 6 degrees thing - the awesome thing about our brains is they have an uncanny ability to pick out what matters from a massive visual clutter. It's really quite amazing. We do have to be able to see the object somewhere in our visual field, but if it's there, it's remarkable what we can do.

 

[TIppy]

He was in a left turn lane

No he was on the left side of the straight lane and close to the cars. This made him difficult to pick out of the scene. You can use the entire turn lane to make a left turn on a bicycle.

Typically people hit the brakes before applying steering correction in emergencies

But the steering wheel hadn't even made a turn to the right yet to make him think the car was going to turn into the bike. And when it did turn to the right (so still on fsd), the regen was way into the green.

 

[Daniel in SD]

That guy flapped his arm like a wing to signal his turn.

I think he just held his arm out and I don't even see in the video where he raised it, there's not enough resolution. I didn't see any flapping. Anyway, it's not really relevant.

it seemed like the guy came out of nowhere and was heading straight.

Why would they head straight when they're in the left turn lane with an opposing left turn lane ahead of them?

The only problem here is the car did a very poor job of predicting the path of the cyclist. All it had to do is head straight and perhaps slow down a little bit to be a little bit to be defensive. Turning towards the path of the cyclist was what was responsible for Brandon's reaction. If the car had gone straight I suspect that there would have been no reaction.

 

[Daniel in SD]

No he was on the left side of the straight lane and close to the cars.

Haha. What? Now you're just trolling. There's only one straight through lane. He's in the middle of the left turn lane. Note all the cars that are in the straight through lane are driving by.

 

[TIppy]

the awesome thing about our brains is they have an uncanny ability to pick out what matters from a massive visual clutter

If it's in the 6 degrees. Most of our field of view is peripheral vision. It detects thing like motion or flashes which causes us to move our 6 degrees there to comprehend what the motion is. We're constantly scanning the 6 degrees over a larger field of view to build a mosaic in our memory. But if something changes in one of the tiles while we're scanning something else we'll miss it.

 

[TIppy]

He's in the middle of the left turn lane.

Looks like a complete turn lane between him and the white car as he starts to lean.

 

[AlanSubie4Life]

View attachment 618665

Looks like a complete turn lane between him and the white car as he starts to lean.

I think your visual system may need inspection. First the 6 degrees, now this.

 

[TIppy]

I think your visual system may need inspection. First the 6 degrees, now this.

What's all that grey stuff between the bike and the white car, Donald?