City streets driving has a shorter window than other forms of driving to react, it's partly why we have lower speed limits. Have you never been using Autopilot and when approaching stationary traffic ahead, or a roundabout wondered if the car is going to stop and then take over because you fear it isn't? In a city streets situation there is a reduce window for this, the driver often needs to respond straight away which requires feet on the pedals, hands on the wheel, and any delay can result in the car travelling a few meters further with no time to recover that delay. The car could operate in shadow mode and step in, like it does when it thinks there is an parked car ahead, but this is in support rather than as the lead. The second issue is the assumption that drivers will apply the same level of concentration when using city streets as not, with many reporting a decrease in fatigue when using autopilot, there must be a degree of reduced cognitive processing for that to occur, if not, why bother?
I didn't say these things were easy, just that many of these edge cases, if solved for FSD/City Streets could be equally applied as passive safety systems
Our general thoughts are therefore that drivers are likely to have a reduced ability for timely intervention on city streets when using such a system, it's hard to see how it would be better, and it's whether the merits of such a system have a corresponding increase in performance to offset. However, you could have the best of both worlds where the system still surveys the surroundings and applies all the same thought processes as if it was driving, and provide additional alerts and safety intervention. The notion that this would result in false alarms and unwarranted intervention is a somewhat spurious one as the same false alarms and unwanted interventions would occur if the car was driving. They will no doubt happen, but they will happen in both cases. Maybe I have colleagues who have dealt with the implications of accidents in residential areas resulting in an overly cautious opinion, or maybe they're better positioned to judge than others.