10.4 installing now, not expecting any major improvements.
FSD is still a long way from wide release.
Many people say how well it works on their drives, but most of the praise come from people who don't drive in busy cities.
I do live in a middle size city's downtown and FSD is virtually unusable here. I still try to use it as much as possible so that my feedback helps the AI improve, but there's rarely a minute when I don't need to interfere with what the car is trying to do. Sometimes I end up yelling at the car when it keeps on trying to commit the same mistake over and over again every time I engage FSD. I can report with confidence: yelling doesn't make it work better.
In my experience, the followings need the most improvements in city driving:
#1 Lane selection logic
This is a biggie. On multi lane streets the car rarely selects the correct lane. It's literally almost always in the wrong lane.
I often drive on one-way streets with 4-5-6 lanes, some of them have parking cars on them, some are bus lanes.
The car loves bus lanes. If I'm in a slow and packed lane adjacent to a bus lane which is open, the AI always wants to switch to the bus lane. Even if the bus lane is painted red and there are bus lane markings painted on it.
The car also doesn't have a vision far ahead, although the front cameras are mounted higher than my eyes so it should be able to see better. In a city it often happens that a lane is blocked by a double-parked car or construction and all the car are trying to get out of that lane early on and get to a non-blocked lane. Not the FSD AI! It wants to switch to the lane which is blocked just a couple hundred feet ahead, just because it looks open right where the car is because everyone else already switched to another lane.
On packed streets with 4-5 one-way lanes when I'm not expecting any turns for a long time, the car should just stay in its lane and go with the flow. No! The AI constantly tries to make lane changes even in FSD Chill mode. Most of the changes are so sudden, unexpected and jarring that it doesn't only scare me, it must scare other drivers around me too.
It is also clear that the car either doesn't have a good map of lanes from which it could anticipate the correct lane, or it just doesn't use the lane map correctly. In order to have a smooth ride, I often rely on my knowledge of which lane I need to be in early on for a turn, otherwise it would be much more difficult to make the turn. The car tries to get into the wrong lane before turns more often than not.
Then there are the completely bonkers, seemingly unexplainable lane changes that just happen for no apparent reason. Like the car switching to a right lane immediately before wanting to turn left, or wanting to try to fit into a bike lane.
#2 Behavior in turns
Actually, once in the correct lane, the car tends to do pretty well in turns, but there are still some improvementes needed.
As many other testers mentioned, sometimes the car goes too close to curbs which makes me interfere. It doesn't happen very often though. Usually it keeps safe distance from curbs, other cars and obstacles.
The most severe issues are when the lane markers are not clearly visible during turns and there are no other cars around to box the AI into a lane. In that case it's pretty much anyone's guess which lane the car is going to end up in. It may actually change its mind 3 times during the turn and jerks the steering wheel left and right while it's hesitating.
# Too much speed
I often find the AI driving way too fast on packed city streets or special situations where it should go much slower.
Just 2 days ago I was driving on a packed 5 lane one way street in stop and go traffic. I needed to turn left and got the car into the left turning lane a couple blocks before the turn. Since it's not a turn a lot of cars take, this turning lane is typically open far ahead the turn because everyone knows that it's going to be a turning lane and they'll want to get out of it. So all other 4 lanes on the right are completely packed and the cars are stopped, and this skinny single left lane is open. Well, the AI saw that the lane is free and opened it up all the way to speed limit. I was inches from the other stopped cars on the right and inches from the curb on the left, driving some 35 mph until I braked to slow down. Scary.
Another example that happens the same way every time. There is a single lane one way street on a steep hill immediately followed by a monument around which the road curves. So when driving up the hill, I can't see the road around the monument. It's so steep and there are parking cars on the side of the road, that I also can't see if there are any pedestrians wanting to use the crosswalk on the top of the hill either. Anyway, when there is no other cars ahead of me and FSD is driving up, it sees that the road is open and accelerates to around 30-35 mph without seeing what's happening on the cross walk or if there are any other cars around the monument on the curvy road. I can't see it, I doubt the cameras can either, yet the car is driving very fast all the way to the top of the hill.
I got to drive some in small sleepy towns on the Jersey shore where most houses are vacation homes. The speed limit is 30-35 mph. The reasonable speed on small roads near the beaches where kids are running left and right all over the place between the parked cars? 15 mph. Yes, these towns should post lower speed limits on these roads, but reasonable drivers know that you have to drive way slower than the speed limit to be safe. The AI should also somehow know when it needs to drive slower.