Alright everybody, here we go!
Awesome! Chuck is a machine.
I haven't watched all of Chuck's v12 videos, but I gather from his first video (not unprotected left specific) he had an easy roller on his UPL. So we'll call that 1/1.
From the new UPL-focused video:
This was a
medium to light traffic situation. There was a decent amount of traffic (but plenty of gaps) in the close lanes, but very very little in the far lanes.
Hopefully he does another one with more traffic soon! Plenty of easy rollers here, where it just had to wait for a break in near-side traffic and then it was all clear.
Overall: 6 out of 11. (Plus 1/1 for the initial video, so 7/12)
Chuck's UPL:
1 Pass Easy roller
2. Fail. Came to near stop in traffic lanes; this is incorrect and wrong. Fail!
3. Pass Easy roller
4. Pass Easy roller
5. Pass Easy roller
6. Pass Easy roller
7. Fail. Left butt out in lanes for a while. This is a failure, because it is wrong. Then had wrong pose in median. Not clear it could see, though it behaved correctly by not going, so perhaps it could see further than the view that is displayed.
8. Pass Easy roller
Other UPL:
1 Fail. Caused traffic to slow down to avoid near miss or collision (there was construction, but traffic had to slow for Chuck even though he said it was for the construction). I would definitely have disengaged, but this was an obvious failure. Chuck has nerves of steel; must be the Navy training!
2 Fail. Stopped in traffic lanes for left-turning truck in median. First Disengagement.
3 Fail. Missed a six-second gap in traffic (I would give this a pass...but then subsequently it paused in near-side traffic lanes again due to traffic on the far side). Anyway, the pause is incorrect. I think missing a six-second gap followed by a huge gap could be argued to be fine, but it's a close call. But it's a fail anyway.
So Chuck UPL alone:
6/8 + 1/1 =
7/9. (Have not reached minimum number of attempts on that specific turn, but would take 11/11 on the next video on 12.3 for him to get it up to 18/20, and for me to lose.)
Overall:
7/9 + 0/3 = 7/12. One disengagement.
For the Unprotected Lefts:
The NHTSA stop at the exact (legally required) stop line is annoying. But more annoying than that is how long it takes to resume and go to the creep limit. Tesla should fix this! There are big problems with pausing in traffic lanes. That to some extent has existed before but it is back with a vengeance here. I didn't really evaluate pace closely, but it seemed a bit slow to cross, and it still was taking a little bit of time to really push it when entering traffic on the far side.
For 12.3 I nearly have a lock (arguably I've already won), but I'd like to see more difficult situations tested. There's a serious regression here as Chuck said, with the stopping in traffic lanes, etc. And we saw no examples of left turns with significant traffic from the right, far-side lanes
We really need to see that threading in - it's not clear it can handle it reliably without stopping in the near-side lanes.
I'm willing to bet again on the next version, as long as it's not a bunch of easy rollers. It really has to be tested with some traffic. It failed even the easy case this time, but next time with all the special training, we need to have situations where it usually has to wait for (or time the gaps) on far-side traffic.
Usually Chuck always tests higher traffic situations, so I expect unless there's another release very soon, he'll get a chance to try 12.3 in busier traffic. That'll be exciting.
But on 12.4 or whatever is next, I think there's a chance I'll lose. I still am betting against FSD though. I think it will not do 9/10 or better (with traffic).
Overall, it's good to hear that people are generally describing this as a "step change" in utility. That's a bit more than I expected, which was incremental improvement. I guess I'll see, when I get it. So far, a regression on unprotected lefts, but hopefully that is cleaned up soon!