Thanks for such an informitive post. I like to ask a few questions that will help me understand FSDb better.
Has the comfort of the drive improved?
Yes, comfort has improved, but as is typical with FSDb and neural net training, you see regressions in unexpected places. So a turn that was historically flawless can suddenly get jerky in the next release, or the car no longer makes the turn because it now incorrectly picks the wrong lane to be in, etc. These regressions make it very difficult to objectively assess how FSDb is improving. Not to mention variations in behavior within the same day. Different traffic pattern, shadows, time-of-day, weather, etc. can all change FSD behavior at a given spot.
While overall comfort has improved, I would not say FSD is comfortable, particularly in denser traffic conditions. It also slows for stops signs way early, then slowly creeps to the stop sign, stops again, then creeps slowly to make a turn, even if there are no cars around. The overall maneuver is very comfortable, but it's also super annoying if you're paying attention and wondering why it requires 20 seconds to stop for a sign and make a right turn. Not to mention this would piss off whoever is following you. In my area, roads and intersections can be extremely weird, and FSDb can't handle these at all. Such is driving life in the northeast US. FSDb should work better in most other areas of the US.
Is phantom braking a problem for you still?
It's important to remember that phantom braking is not a single discreet problem to be eliminated. It represents thousands of conditions where the car thinks it ought to brake for safety when there was no actual risk. I'll go through some examples:
1) the transition from radar to vision-only: a lot of people like to scapegoat the loss of radar as the cause for phantom braking on the highway. This has not been my experience. There was plenty of PB during the years I used AP on the highway with radar, particularly at overpasses and big highway overhead signage. These were uncomfortable and hard brakes because the car thought I was about to hit a wall. All of that went away for me as soon as I got onto FSDb, which doesn't rely on the radar. These days I have maybe 1 phantom braking incident per month, and based on my commute, that comes out to 1 PB per ~2000 miles. PB on the highway is now typically caused by a car in an adjacent lane drifting toward my lane, and my car thinks it might pass into my lane, so it brakes. It can happen before you even notice or feel like there's a threat. To me, this is just a tuning issue. It's probably a bit sensitive.
2) off the highway, PB is much higher. It was quite bad in the earlier versions of FSD, but it has gotten better. But as I said before, at any version, you can have regressions, and the latest 10.69.x versions have introduced a lot of PB, specifically on 2-lane roads when there is oncoming traffic. Car just gets nervous at the oncoming car, even if they are well within their lane. Car brakes a lot more for pedestrians and other vulnerable road users. Musk did come out on twitter to say that due to huge code changes, FSD is basically set in super-timid mode, which explains this particular class of PB. But what is more common on local roads are "whimsical hesitations," a subclass of phantom braking where the car eases off its constant speed as if it's nervous about something, then resumes its speed. Sometimes these could be attributed to tree shadows, but they happened on overcast days too. Sometimes FSDb will mistaken a mailbox as a person and brakes hard. A low hanging tree branch can confuse it sometimes too. Overall, things are improving though. But you should expect a decent amount of PB even when FSDb is released wide.
How many disengagement do you have only a daily basis and has that improved?
I don't find disengagements to be a good measure of FSD driving performance, mainly because it depends on how tolerant I feel on a given day for FSDb's idiosyncracies, or how many people are around me. For example, if I'm driving at night, and not a lot of people are around, I'll let FSD take a 20-second right turn without intervening. If there's someone behind me, I'll use the accel to make the car go. If FSDb cuts the corner while turning, I might not do anything if there's no one around, but if there's someone already on the road I'm turning on, I may disengage if FSDb actions would make that other driver nervous. FSDb also likes to excessively use the blinkers and then not act on them. Unfortunately I can't do much about those in terms of pissing off other drivers. As far as lane changes go, I'll disengage if I feel like FSD is picking the wrong lane, or if it's picking the correct lane too quickly. A lot of this will depend on your local driving culture. In the Boston metro area, people don't have a lot of tolerance for stupidity, and FSD is frequently way too timid (and stupid).
But to try to answer your question: for easy drives, I would say disengagements are def trending in the right direction. In tricky areas, like anywhere inside the I-95 perimeter of Boston, expect tons of disengagements on the local roads. Multiple disengagements per mile. FWIW, having to disengage for safety-level events (like collisions) is dropping.
The other reason I'm not so bullish on FSD for robotaxi operation is that the demand for robotaxis are highest in the cities. That's also where driving the the most crazy and aggressive. It needs to know exactly what lane to be in at the right time; it cannot wait to read the lines painted on a road to make a decision. Otherwise, the car will be stuck in the wrong lane and you'll miss your turn. It need to know how to deal with other drivers that are actively working against you just because. If you're not in the right lane, you have to fight to get into it. FSD will likely need to learn aggressive maneuvers, like how to wedge itself in to a lane full of stationary cars.
