FSD Beta Videos (and questions for FSD Beta drivers)

[PDX-Y]

So there's no reason to say that FSD can't be able to figure out what a hole is.

Of course not. I'm just saying that the only feasible autonomy solution here is to stop and scream about the holes in the road (and also the construction vehicle, and confusingly placed cones), and that an autonomous vehicle just isn't ever going to be able to handle situations like this, for the same reason it won't be able to handle being t-boned in an intersection. It's just fundamentally unsafe.

And in this case, it doesn't stop and scream because it knows it has a safety driver. But the path from this behavior here to a shipping product is really pretty straight, I think.

But yeah, they need a pothole detector.

 

[Daniel in SD]

I think that's part of it. Would you consider other driver behavior an edge case? Would you demand that autonomy be able to avoid another car running a red light into the intersection, or merging directly into the side of the autonomous vehicle? I mean, we'd both agree that we'd want the vehicle to be able to do that. But I think we'd also both agree that this is just out of scope: there are some things in the universe that just can't be avoided.

And I view hostile environments like that construction zone as on that list. They need about 3x as many cones, and flaggers to block traffic for the tracked vehicle in the road. Frankly if FSD was allowed to drive unbidden into that hole, I bet the state would have been on the hook and that guy in the backhoe would have been fired.

Well, as an average human driver, those are all collisions that I have avoided. I think an autonomous vehicle would need to do that to achieve greater than human safety. Think about how many accidents you've avoided that could have been caused by another driver, poorly placed cones, or substandard roads.
The car tried to drive in the parking strip which had a hole dug into it that was marked by cones. Even if you ignore the giant hole there wasn't enough space to drive there. If a human had driven into that hole it would get posted on YouTube and everyone would have a good laugh. Absolutely no one would say it wasn't marked well enough. Look at the situation again, anyone who drives into that hole really shouldn't be driving...

 

[Daniel in SD]

Well, I wouldn’t count any road with standard markings an edge case. But I think all construction sites could be considered an edge case, in the sense that preprogramming and rules won’t really work for them...

The don't drive in the parking strip or into holes rule is pretty simple though.
It's the interaction with the backhoe operator and drivers coming the other direction which seems impossible... that's the real world AI problem.

The true edge case would be when the right decision is to drive into the hole.

 

[Chazman92]

FSDBeta 9.0 - 2021.4.18.12 - Memorial Park Drive #2 after Two Weeks on v9

 

[mhan00]

FSDBeta 9.0 - 2021.4.18.12 - Memorial Park Drive #2 after Two Weeks on v9

6:50 - car tries to run a red light. Disengagement.

26:10 - car randomly decides to swerve towards the opposit lane at an intersection. Looked like it was pathing to pass the car in front of it for some reason.

Two showstoppers. Not good.

 

[cbrigante2]

FSDBeta 9.0 - 2021.4.18.12 - Memorial Park Drive #2 after Two Weeks on v9

Thanks for the time you are spending testing this out. Good video. Curious to see how much changes with first v9 update.

 

[Mardak]

26:10 - car randomly decides to swerve towards the opposit lane at an intersection. Looked like it was pathing to pass the car in front of it for some reason.

The intersection prediction and path prediction believed the current lane ends with a wide median ahead:

Notice the purple lines directly ahead of the lead vehicle. That probably resulted in a related prediction of this being a double left turn lane. Getting closer, it also predicted the path should immediately switch to the adjacent lane to avoid the imaginary median:

 

[rxlawdude]

The intersection prediction and path prediction believed the current lane ends with a wide median ahead:
View attachment 687466

Notice the purple lines directly ahead of the lead vehicle. That probably resulted in a related prediction of this being a double left turn lane. Getting closer, it also predicted the path should immediately switch to the adjacent lane to avoid the imaginary median:
View attachment 687468

That six inch curb median (narrow but a median) doesn't look imaginary to me.

 

[linux-works]

I really wonder how much better 'pure vision' would be if you networked the cars (tesla's, for the first obvious test) and had them V2V to each other to share info.

in the bay area, it really is very likely that a car in front or behind you is a tesla. I know, sounds funny to those not from here, but its kind of cool, in a way, to see so many of them on the road. its not unusual to have 3 near each other, 1 car away, at a line-up, front and left and right.

cars could detect each other (maybe wifi beaconing?) and share some object info.

I would hope that tesla is working on this and testing this, or at least thinking about it.

imagine how good things can be if you can sneak peeks via other friendly cars that are ahead or behind or along side you. postionally AND temporally.

yes, security issues and trust issues. solvable. getting vendors to buy in and customers to have it installed and working; much harder problem. but just think of how good things could be if the vision field was a truly distributed camera source, from MORE than 1 vehicle.

 

[rijc99]

If that’s true why wouldn’t it either turn left or slow and attempt to merge right instead of doing a header into opposing traffic?

…sorry. Was referring to the possibility Tesla thought it was a double left

 

[stopcrazypp]

The intersection prediction and path prediction believed the current lane ends with a wide median ahead:


Notice the purple lines directly ahead of the lead vehicle. That probably resulted in a related prediction of this being a double left turn lane. Getting closer, it also predicted the path should immediately switch to the adjacent lane to avoid the imaginary median:

This may have changed somewhat in the latest versions, but note Tesla is doing some NN magic for path predictions, where the NN may not even need to see where the path ends up to make a prediction. That would explain some of the more bizarre path predictions.

 

[stopcrazypp]

That six inch curb median (narrow but a median) doesn't look imaginary to me.

I think he's talking about how the visualization shows the car approaching a median that is right in the middle of the current lane (thus the left prediction), when in fact the median actually would be to the left of the current lane.

 

[scottf200]

6:50 - car tries to run a red light. Disengagement.

26:10 - car randomly decides to swerve towards the opposit lane at an intersection. Looked like it was pathing to pass the car in front of it for some reason.

Two showstoppers. Not good.

To be fair the lanes are right between two lights. The first is a pretty goofy intersection. The second one is pretty bad stoplight placement too. Interns putting those up?

 

[Mardak]

Yes, there's a real narrow median, but the neural network predicted an imaginary wide median probably because it saw the left edge of the median but not the right edge due to the lead vehicle blocking the video from the short follow distance. Here's an intersection that is right next to Tesla Palo Alto HQ, so it's not that unlikely this specific intersection was part of the training data:

This direction has two left turn lanes while the opposite side only has one with a median taking up space directly in front of the outer turn lane. Also notice how these California roads have solid white lines for the left turn lanes while straight lanes have dashed white lines. But it looks like Florida roads near Chuck have solid white lines for all lanes just before the intersection, so that could have additionally confused the network.

To be clear, the network did predict the correct thing most of the time, but perhaps the high density of traffic temporarily blocked direct views of certain key aspects of the intersection leading to a low confidence path prediction to cross into oncoming traffic. Looks like some software 1.0 safety check kicked in to determine that the predicted path crosses another vehicle to trigger the forward collision warning.

 

[Matias]

I think he's talking about how the visualization shows the car approaching a median that is right in the middle of the current lane (thus the left prediction), when in fact the median actually would be to the left of the current lane.

There’s purple line in correct place, but what does that faint yellow line further right mean?

FSD should notice that the grey lead car is going forward without problems, so the line obviously continues. That is how humans drive.

 

[AndreP]

I really wonder how much better 'pure vision' would be if you networked the cars (tesla's, for the first obvious test) and had them V2V to each other to share info.

in the bay area, it really is very likely that a car in front or behind you is a tesla. I know, sounds funny to those not from here, but its kind of cool, in a way, to see so many of them on the road. its not unusual to have 3 near each other, 1 car away, at a line-up, front and left and right.

cars could detect each other (maybe wifi beaconing?) and share some object info.

I would hope that tesla is working on this and testing this, or at least thinking about it.

imagine how good things can be if you can sneak peeks via other friendly cars that are ahead or behind or along side you. postionally AND temporally.

yes, security issues and trust issues. solvable. getting vendors to buy in and customers to have it installed and working; much harder problem. but just think of how good things could be if the vision field was a truly distributed camera source, from MORE than 1 vehicle.

We need many vehicles scanning the environment and feeding data into a centralized computer that processes the data and routes all vehicles, that would enable a truly powerful/organized/safe traffic system with eyes on everything at once.

Compared to a system like this ^^, having many vehicles individually assessing environments and determining paths around each other feels like it barely makes sense and leaves a lot on the table in terms of harnessing the true benefits of computer processing power and connectivity.

 

[linux-works]

that's the point of v2x. poles with cameras, moving vehicles with cameras, its all shared knowledge.

it requires a little thing called cooperation, but that's what makes us people; we understand the principles of how things can co-operate.

there is work and specs and even early chips for v2x but its not talked about nearly enough and not given enough seriousness.

 

[PDX-Y]

If that’s true why wouldn’t it either turn left or slow and attempt to merge right instead of doing a header into opposing traffic?

I with people would be more careful with language here. The car didn't "do a header into opposing traffic", nor "swerve" (as described above). It really didn't.

What it did was begin to execute a left turn in front of the left turn lane of the opposite direction. That's an unobstructed lane; the car was never closer than 10' or so to the opposing traffic lanes. And immediately (like, within 300ms or so) it detected a car in the path, drew it red, and flagged a collision alert. And presumably it would have stopped too, but for the disengagement.

So to answer the question: it did turn left, that's what it was doing. But there was traffic there and it couldn't. Really the low level driving was fine here.

I think it's clear that as surmised above, the root problem was that it saw the left edge of the small median ahead (the right was blocked by the car ahead) and assumed it blocked the whole lane, and therefore pathed out a left turn.

 

[rijc99]

I with people would be more careful with language here. The car didn't "do a header into opposing traffic", nor "swerve" (as described above). It really didn't.

What it did was begin to execute a left turn in front of the left turn lane of the opposite direction. That's an unobstructed lane; the car was never closer than 10' or so to the opposing traffic lanes. And immediately (like, within 300ms or so) it detected a car in the path, drew it red, and flagged a collision alert. And presumably it would have stopped too, but for the disengagement.

So to answer the question: it did turn left, that's what it was doing. But there was traffic there and it couldn't. Really the low level driving was fine here.

I think it's clear that as surmised above, the root problem was that it saw the left edge of the small median ahead (the right was blocked by the car ahead) and assumed it blocked the whole lane, and therefore pathed out a left turn.

Perhaps follow lead car... or slow and merge right? There is no excuse for it to make an attempt to go left there considering the vehicle was not in a turning lane and that turning lane had a solid red arrow on the light. The right thing to do if it thought that lead car was going drive into a median would be to slow down, signal right and attempt to merge.

 

[rijc99]

Also keep in mind... we have the benefit of slowdown/play/pause during playback. It happens much faster in real time. Especially if people get accustomed to FSD and are just driving. They aren't going to be expecting moves like this or the random red light runs. They may not have enough time to properly intervene.