Fortunately we don't need to rely on Kurzweil for anything, AGI in a self-driving car would be counter-productive, I don't need my driving robot having an existential crisis while driving, or composing poetry, it just needs to stay in its lane and not hit stuff. The only ML you need is for vision and maybe a few other models for inferring parking lot and intersection topologies and predicting vehicle/pedestrian dynamics, like the cut-in detector. That's counted as "AI" I guess but it's really narrow. Once everything is in 3D then it can just be rule-based.
I have to kind of laugh not in disagreement, but I've been wondering if we should really be looking at General Intelligence to solve the problem of self-driving cars. As it currently stands we have total control over our autonomous machines, and we don't have to worry about them having an existential crisis.
I brought up AGI because it's required if we don't make significant efforts to harmonize rules/regulations of the road, infrastructure (like V2X communication), etc to simplify the requirements for self-driving if we're going to expect them to match/exceed ODD (Operating Design Domains) that professional human drivers are capable of.
I personally like the idea of growing white listed areas so that you don't need AGI. You simply use offline human intelligence to create rules for that area. Things like "Don't bother turning left here because it will take forever". Where part of growing white listed areas is creating a national map database for all autonomous cars. Where construction companies and/or road planners have the responsibility to keep this data up to date. That way an autonomous car had really good maps for visual redundancy, and route planning. Especially for things like reporting the exact position of pot holes where the autonomous cars help the road maintenance people.
Growing the white list also includes making sure the approved area has a connectivity. The number one thing an autonomous car needs to do is to call home for help. Because there will be situations that arise that it's going to need to be bailed out of. Like "help, it's Dec 24th and I'm in a parking lot and cars won't let me out because they keep coming". With that specific example it doesn't necessarily have to call home because with V2X it could possibly ask other vehicles for to help out.
There also has to be some ability to break rules. Humans have allowance for breaking rules, and that allowance is essential for driving in areas with imperfect infrastructure. Like for example when a left turn light doesn't change on consecutive cycles. Most human drivers like me will simply run it when its safe to do so without knowledge of what the rule is for that specific area. Breaking rules also comes in handy time to time when dealing with rude human drivers.
I think we can all agree that our road systems are a bit like the Wild West where there are rules, but so often they're not followed. Sure it's better than places like India, but worse than places like Germany where drivers are a lot more rules based.
Non-AGI is very much rules based so there is this incompatibility problem.
I see AGI in our lifetime as being a myth. Hopefully that's not true because I strongly believe humans have an obligation to either survive or to create life. Having AGI ether assures our survival (in helping us seed the universe with human life) or our destruction (by being self-aware and seeing humans as a threat). Either way the obligation of our existence is met.
In any case I agree that AGI isn't needed for FSD if we accept a very step by step gradual evolution in achieving it.
