Sounds like you've got a plan - how do you propose to implement it?
I'm not the OP but I would start first by measuring human driven behaviors on fleet vs FSD stack behaviors and then developing technology to lower that gap.
Optimize routing, signalling, speed profiles, etc within an overall outer boundary of "don't crash and obey vehicle code". Bring in location specific semantic tags.
This is a major long term research & development project. I think tesla's now good on the vision side (Karpathy's specialty) but not on the planning side. BTW, the Waymo/Cruise competitors who were sensor heavy bypassed the difficult vision interpretation parts with hardware and have worked on the planner for much longer I think.