Neural networks learn things in different ways, and it's not surprising for 12.x to have misunderstood the behavior. End-to-end has needed to learn to make a turn on green… except yield when there's a pedestrian… except continue if the pedestrian hasn't moved for a while… except if the walk signal just changed. This could also be complicated by the variance of how quickly pedestrians respond to signals, but presumably something similar happens at stop signs with pedestrians waiting for an opportunity to enter, so there could be potential for shared learning signals.
Potentially the current neural networks already have an internal perception and understanding of walk signals, but control predictions don't give enough weight to that information. The training process to adjust the weights to pay more attention to walk signals could boost the value from say 10% to 50% does not change the size of the network, so it doesn't affect the runtime compute and memory requirements of the system. Hopefully "just" more disengagements and training data should address this issue.
So weird though - why did it go to 8mph without stopping at the line? I do often press the accelerator (otherwise I would have been much closer behind the lead car), but I didn’t here, because all of a sudden (after just lazing about) it was off to the races!
As a human, you proceed to the line quickly, stop, then if the light still hasn’t changed, go, leaving plenty of margin to the pedestrian (tricky here because there are two right turn lanes). And if the light turns, you don’t go!
In all cases you exhibit a bit of caution because it is obvious there is a pedestrian waiting to cross.
I can't see any way to fix this issue unless it has instant reaction time. If it can't anticipate, it has to be able to react immediately. 0.5 seconds is not going to cut it.
By the looks of it on X, Chuck's left turn seems to be yet another fail. It doesn't even use the median at all.
No, this is not a regression - it's hard to regress on something that was failing before. This is the right way to do the turn! You should not use the median (who does this (???), it's borderline insanity).
6/8, scoring extremely generously (if you count the not changing other vehicles behavior - a key part of the scoring system - it's more like 4/8 but I didn't pay attention to detailed scoring). About the same as before. (I think 7/9 if you count the trial run on the prior video, even though it made traffic move on that video from yesterday, so that would be a 0/1. So ~4/9 per proper scoring.)
This will be the start of the march of 9s, now that they're using the right strategy of rolling every turn.
All they have to do is increase assertiveness to give 0.6-0.9g of acceleration, fit comfortably through 4-5 second gaps, and they'll be fine. They also have to fix the creep behavior. This is the way.
I was once again right about the failure of 12.3.4. Unfortunately no beer scored because
@Daniel in SD is scared and has no faith and is waiting for the arrival of FSD Beta.
I don't think any retraining has occurred here, they've just altered the caution thresholds (which are somehow inputs to the NN). I've noticed in
my left turn today that 12.3.4 was MUCH more conservative on traffic from the left. I'd have to check the video but it looked like it waited on traffic 6-7 seconds away. I don't think I've seen it quite that conservative before. However, other than that, it drove the turn in an identical fashion. I haven't seen it stop in the road recently but I'm sure it can. It seems this happens when it's insufficiently aggressive and starts waiting on traffic from the right after committing. That's why they need 0.8g of acceleration. If you don't use it you lose it.
v12.4.x will also fail, at least initially. (I think maybe I'll finally be wrong! It depends on whether they incorporate new training. I actually think that end-to-end is surprisingly good, better than I thought, and maybe there is more hope for training on this sort of problem than I thought. They just need a bunch of training data from people driving the turn assertively, not waiting in the median, and consistently hitting 30mph by the apex (currently at 15mph or so). I originally thought that this problem would be way too complex for E2E, but maybe if they can simplify the trained approach there is hope.)
Good news, no regression. Bad news, still have never successfully met the unprotected left challenge.
Fix the creep! And stop at the stop line. And fix the USB (this key reason for 12.3.4 seems to have also failed to work).