Related, I was thinking about "full self-driving" today as I picked up my daughter at school. The traffic flows around her school are completely ill-conceived (to even say they were "conceived" is too polite, they just arose). Anyway, the only reason things work there at all is the aid of various volunteers directing people and local customs and mores regarding where and how you drive when doing pickup and dropoff. These are not documented in any statutes or ordinances (and sometimes violate same) and differ in at least some particulars from any other location's driving patterns. It's a bit of a stretch to imagine even a quite good self-driving vehicle would be able to figure out how to behave properly ("oh, obviously it's OK to straddle the yellow line here as long as the school safety patrol person is making eye contact"). More likely the car would say "whoa, all those drivers be cray-cray" and bail out -- drive safely and legally to some nearby place and call for instructions from its human.
I totally believe cars will be able to self-drive in well-behaved environments in the next few years. But many driving environments although manageable for humans are not well-behaved. It may be that we will ultimately resolve this by adapting the environment to the car and not vice-versa -- we've done it before, for example roads used to be shared equally by pedestrians (and wagons, donkeys, etc) but when cars came along we forced pedestrians off of them. But that will take some time; until there's a substantial fleet of self-driving cars out there people aren't going to rejigger their school's pickup procedures to cater to them.
(I pick up on foot so it's all moot to me anyway, thankfully.)