FSD Beta 10.13

[johnmodely]

I still think camera/hardware is the main issues for more rapid progression, but now they have to solve for their 5+ year old FSD chip plus inferior camera setup because they have cornered themselfs here with 3+ million "FSD capable" cars on the road soon with this hardware.

Imagine logistics nightmare of releasing HW4 + new camera setup that WILL solve FSD but you now have to deal with upgrading millions of cars to this as well.

 

[mtndrew1]

I still think camera/hardware is the main issues for more rapid progression, but now they have to solve for their 5+ year old FSD chip plus inferior camera setup because they have cornered themselfs here with 3+ million "FSD capable" cars on the road soon with this hardware.

Imagine logistics nightmare of releasing HW4 + new camera setup that WILL solve FSD but you now have to deal with upgrading millions of cars to this as well.

This is my pet theory as to why they keep raising the price on something which is functionally useless.

The insane price of FSD keeps the take rate low while allowing huge margins to pay for future retrofits. Once they get to HW4 or whatever and FSD works the price might be justified. I’m very curious to see how it all plays out.

 

[Oil Freedom]

They stopped in front of an oncoming car while making a left turn. What human would do that?

I personally saw a human driver attempt to cross traffic for a turn when the cross traffic had the right-of-way. The driver apparently underestimated the speed of the crossing traffic and freaked out mid-crossing. Instead of accelerating out of the way, the driver completely stopped their car, thereby blocking both lanes in one direction of traffic. Que the mahem of hard braking and swerving this created!

 

[Supcom]

Wondering about the somewhat sudden change in Tesla's FSD update policy. In the past updates have been fairly frequent with improvements in many areas in the 5%-20% range ( as stated in the release notes). This release appears to be held up (at least partially) waiting for a 90% (ish) success in Chuck's ULT. Have to wonder if the departure of AK has changed the upgrade philosophy for FSD. Also wondering why the update policy appears to different for the non-FSD versions.

As progress is made, it gets harder to make additional progress. The easy stuff tends to be developed first, leaving the hard stuff for later.

 

[TravelFree]

This is most definitely not why Chuck’s turn is failing, FWIW. It’s just not driving the turn correctly. Visibility is perfect on this turn.

Interesting how you know that location has perfect visibility. I live here and know what he has there isn't the worst I have seen but it is bad, mainly because too often cars hidden from view of the B-Pillar camera are traveling 60mph and when the Tesla creeps into the lane the fast car can't see it until too late and just begins to creep until into his lane. A driver can lean forward and has a bit better advantage. But to be better than human many of us who have tested this problem with front mounted cameras that can see while the car is not even in the lane of traffic.

There are other problems such as speed once Tesla commits and use of median that can help but for FSD to ever solve this and be better than human the camera has to be able to see where the B-Pillar camera and the human driver can't.

I don't know why people are afraid to suggest we may need a camera change for this problem to ever be fixed. Tesla has made radical hardware changes in the past to fix a problem.

 

[Daniel in SD]

Interesting how you know that location has perfect visibility. I live here and know what he has there isn't the worst I have seen but it is bad, mainly because too often cars hidden from view of the B-Pillar camera are traveling 60mph and when the Tesla creeps into the lane the fast car can't see it until too late and just begins to creep until into his lane. A driver can lean forward and has a bit better advantage. But to be better than human many of us who have tested this problem with front mounted cameras that can see while the car is not even in the lane of traffic.

There are other problems such as speed once Tesla commits and use of median that can help but for FSD to ever solve this and be better than human the camera has to be able to see where the B-Pillar camera and the human driver can't.

I don't know why people are afraid to suggest we may need a camera change for this problem to ever be fixed. Tesla has made radical hardware changes in the past to fix a problem.

We have video of the visibility and eyes. That’s how we know.

I think Tesla needs an A pillar camera but it’s not necessary for this turn.

 

[AlanSubie4Life]

Also, the camera view is from a wide angle cam mounted on the car roof, aligned even with the B pillar.

So, it should be a good representation of the lines of sight of the B-pillar camera (obviously the extents of the field of view will be different since the one on the top is pretty wide angle)

Therefore the issue is not the camera placement or the intersection design. The position of the car with the B pillar camera in the center of the crosswalk looks plenty safe to me.

100% agreed.

Interesting how you know that location has perfect visibility. I live here and know what he has there isn't the worst I have seen but it is bad, mainly because too often cars hidden from view of the B-Pillar camera are traveling 60mph and when the Tesla creeps into the lane the fast car can't see it until too late and just begins to creep until into his lane. A driver can lean forward and has a bit better advantage. But to be better than human many of us who have tested this problem with front mounted cameras that can see while the car is not even in the lane of traffic.

I have a very comprehensive on-the-ground video to reference, taken with a high quality camera, from Chuck. Also drone video from Chuck, which helps us with distances! As we well know, cameras (especially wide angle ones) have distortion, which can make judging distances difficult. However, please see the analysis below.

You should go back to this intersection and look again. Detection RANGE may be an issue (I have no idea), given vehicle speeds, but obscuration is not an issue, even with the current limited sensor placements.

I don't know why people are afraid to suggest we may need a camera change for this problem to ever be fixed.

There is zero resistance here, if that were actually the issue. But I don't think it's a good idea to change something if it's not the problem. Gotta root cause things!

I'm absolutely willing to say that improved cameras and more sensors may well be necessary to achieve better than human performance, in general (with the current status of compute and AI). But I have no idea! However, that's completely different than saying that that is the problem here. Again, I am simply trying to state that the root cause of the issues here is not the camera placement or the visibility.

So, can we please stop saying that visibility is an issue for this corner? It's really not, as long as the car's stopping pose is correct! And there is TONS of margin. There is 20' or more from the main stop line to the edge of the first lane. The Model Y is 15.5' long. Visibility begins to be sufficient about 5' forward of the thick main stop line. The position of the B-pillar camera (I'd estimate 8' from the front of the vehicle) is therefore definitely far enough forward to have an unobstructed view.

In the screen capture below, you can see the distances marked off relative to the known Model Y size. And you can see that in this pose, the Model Y will have great visibility (in the overhead view, the b-pillar camera is at least 2-3 feet forward of the first crosswalk line).

So for this specific turn, as I calculate it, there is approximately a 7' window of positioning for the car. This seems ample! 5' if you want to be 2' back from the edge of the traffic lane. This is why Tesla is working on improvements in stopping pose (and presumably consistency)...

Adding one more picture showing the view corresponding to the overhead measurements above (this is after Chuck disengaged). Visibility is excellent as others have pointed out! This is at 10:22. It takes 8 seconds for the vehicle with headlights on in the closest lane to reach Chuck's position, at 10:30, so that is about 200 meters:

 

[buyleonard]

I won't vouch for the calculations in this video

In the video they put the pillar camera in the middle of the car, which is incorrect

 

[buyleonard]

Imagine logistics nightmare of releasing HW4 + new camera setup that WILL solve FSD but you now have to deal with upgrading millions of cars to this as well.

They only have to upgrade the car if the customer paid for FSD

 

[tmoz]

So, it should be a good representation of the lines of sight of the B-pillar camera (obviously the extents of the field of view will be different since the one on the top is pretty wide angle)



100% agreed.



I have a very comprehensive on-the-ground video to reference, taken with a high quality camera, from Chuck. Also drone video from Chuck, which helps us with distances! As we well know, cameras (especially wide angle ones) have distortion, which can make judging distances difficult. However, please see the analysis below.

You should go back to this intersection and look again. Detection RANGE may be an issue (I have no idea), given vehicle speeds, but obscuration is not an issue, even with the current limited sensor placements.


There is zero resistance here, if that were actually the issue. But I don't think it's a good idea to change something if it's not the problem. Gotta root cause things!

I'm absolutely willing to say that improved cameras and more sensors may well be necessary to achieve better than human performance, in general (with the current status of compute and AI). But I have no idea! However, that's completely different than saying that that is the problem here. Again, I am simply trying to state that the root cause of the issues here is not the camera placement or the visibility.

So, can we please stop saying that visibility is an issue for this corner? It's really not, as long as the car's stopping pose is correct! And there is TONS of margin. There is 20' or more from the main stop line to the edge of the first lane. The Model Y is 15.5' long. Visibility begins to be sufficient about 5' forward of the thick main stop line. The position of the B-pillar camera (I'd estimate 8' from the front of the vehicle) is therefore definitely far enough forward to have an unobstructed view.

In the screen capture below, you can see the distances marked off relative to the known Model Y size. And you can see that in this pose, the Model Y will have great visibility (in the overhead view, the b-pillar camera is at least 2-3 feet forward of the first crosswalk line).

So for this specific turn, as I calculate it, there is approximately a 7' window of positioning for the car. This seems ample! 5' if you want to be 2' back from the edge of the traffic lane. This is why Tesla is working on improvements in stopping pose (and presumably consistency)...

View attachment 838228View attachment 838229

Adding one more picture showing the view corresponding to the overhead measurements above (this is after Chuck disengaged). Visibility is excellent as others have pointed out! This is at 10:22. It takes 8 seconds for the vehicle with headlights on in the closest lane to reach Chuck's position, at 10:30, so that is about 200 meters:
View attachment 838232

When the traffic is coming directly at the camera, how do they determine speed? Especially if only one camera can see the object?

 

[AlanSubie4Life]

When the traffic is coming directly at the camera, how do they determine speed? Especially if only one camera can see the object?

I can do it with a single eye, because I have visual cues, and know how things work. It’s very easy, unless someone is deliberately tricking me with an optical illusion. So they’ll presumably do something similar? Definitely it is possible.

My only point is that they can definitely receive and collect adequate numbers of photons bouncing from (and generated by) oncoming vehicles, certainly out to 200m (even though we lack the feed, obviously we can be quite certain of this, given the similarity with other cameras), with existing sensors, for this corner. Whether or not they can do anything useful with them doesn’t really matter.

You know how I’ve placed my bets. And obviously I hope I am wrong.

 

[Daniel in SD]

When the traffic is coming directly at the camera, how do they determine speed? Especially if only one camera can see the object?

How does a human do it? 100 million years of evolution. (It's not binocular vision, that doesn't work nearly that far and people with one eye can drive just fine)

Nobody knows how an artificial neural net does it. All they do is put a bunch of video training data in and hope it works. I think this is the actual problem with Chuck's ULT, the perception stack isn't good enough and nobody knows if it ever will be. The partial occlusion of cars by the trees is a huge issue with these systems too.

 

[AlanSubie4Life]

by the trees

And the light pole, which I previously mentioned. Such a small angle though that it is irrelevant to a human. Neural net just needs to remember that there’s a car partially obscured by it and all will be well.

Everything important is visible. Now they just need to make it work.

 

[AlanSubie4Life]

the perception stack isn't good enough and nobody knows if it ever will be.

It’s interesting to me that you think it isn’t good enough, yet you’re willing to bet 90% success rate, even though current success rates with that same perception stack are much much lower (less than 30%, so at least 7x worse). Magical thinking?????

I really want to understand what you think Tesla has up their sleeve, so I can back out of this bet prior to release.

 

[Daniel in SD]

It’s interesting to me that you think it isn’t good enough, yet you’re willing to bet 90% success rate, even though current success rates with that same perception stack are much much lower (less than 30%, so at least 7x worse). Magical thinking?????

I really want to understand what you think Tesla has up their sleeve, so I can back out of this bet prior to release.

Haha. Overfitting!
90% isn't even close to good enough. I didn't mean to imply that the perception stack never works in this scenario, just that it doesn't work 99.999% or whatever the human success rate for the turn is.

Or maybe there are usually clusters of oncoming vehicles that make the turn a no go. Therefore reliably detecting any individual vehicle isn't necessary to a achieve 90%. Really just gut feeling.

 

[AlanSubie4Life]

90% isn't even close to good enough.

Ah, I see, I thought you were implying that was why they were having problems right now (hence not good enough for 90%). Of course 90% is nowhere near good enough.

 

[Daniel in SD]

Ah, I see, I thought you were implying that was why they were having problems right now (hence not good enough for 90%). Of course 90% is nowhere near good enough.

It might be but I bet they can just add more video training data from this particular location to get to 90%.

 

[AlanSubie4Life]

It might be but I bet they can just add more video training data from this particular location to get to 90%.

But seems like there's some limit to the range at which these things can work reliably due to limitations other than the training data, and we're seeing flickering at 100m now (whatever that means in the visualization...). I'm just not sure that simply more training will do it. I don't see why they couldn't extend that range substantially with some work or tweaks to how the neural net looks for vehicles, but seems like it would maybe take more than just additional training & overfitting. (That being said, I have no idea how the underlying neural nets are really "searching" the images for these objects and whether there is a limit to the minimum pixel count that is "searched.")

I guess that's the question. I mean, this is obviously a car (partially obscuring a trailing truck; pretty obviously) at 200m (note this is not from the vehicle's sensor, and there are compression artifacts, so not sure exactly what it will look like to the computer). Just need to train it to detect this reliably, to get 200m range. I'll definitely lose the bet if they do that.

Apparently the stated goal of the Autopilot team is that if you can see it in the image, they can detect it. So we'll see.

 

[tmoz]

I can do it with a single eye, because I have visual cues, and know how things work. It’s very easy, unless someone is deliberately tricking me with an optical illusion. So they’ll presumably do something similar? Definitely it is possible.

My only point is that they can definitely receive and collect adequate numbers of photons bouncing from (and generated by) oncoming vehicles, certainly out to 200m (even though we lack the feed, obviously we can be quite certain of this, given the similarity with other cameras), with existing sensors, for this corner. Whether or not they can do anything useful with them doesn’t really matter.

You know how I’ve placed my bets. And obviously I hope I am wrong

I think that problem is hard in that there is no lateral object motion, or very little motion left-right. The number of pixels the object occupies slowly increases, but that would lead to very coarse velocity estimations with larger error bars. The same problem exists regardless of where the camera is placed. It may even be more difficult if the camera is in the bumper as it would be looking at the on coming traffic at a very low angle. So I might be coming around to agree that the camera is seeing as well as the camera can see, but that the obstructions are making velocity calculations imprecise due to the head on view of the camera-object. If this is the case, more pixels are needed to see fine motion at high speeds at shallow angles.

 

[AlanSubie4Life]

If this is the case, more pixels are needed to see fine motion at high speeds at shallow angles.

Just leave it up to the neural nets. I bet I could do the velocity estimation at 200m with the B-pillar camera video feed alone.

but that the obstructions are making velocity calculations imprecise

These are no problem for me when looking at Chuck's top-mounted video feed.

The only issue with obstructions I saw was at 10:07 in Chuck's video where the vehicle stopped in obviously the wrong place (1-2 feet back of the start of adequate vision, so with at least 8 feet still to advance), and missed a vehicle mixed into branches (maybe - it was tough for me to see with high confidence). But it wasn't even close to being in the correct spot.

Spend some time watching the video oncoming traffic. It's amazing how good the human brain is.