VERY doubtful that any manufacture would sell L4 cars to consumers without controls. Since this would require them to have teams of remote operators at the ready 24/7. Also extremely unlikely manufactures would want the liability involved in remote driving personally owned cars much less the costs involved that would have to be passed onto the owner.
I agree that manufacturers will not do remote assistance or remote operators for personal cars. This will affect how autonomous personal cars are developed.
You need to ask yourself two questions: what is the L4 ODD and does the application require the car to leave its ODD. If the L4 car never needs to leave its ODD in order to fulfill its purpose, then it does not need a steering wheel or pedals. If the application does require the car to sometimes leave the L4 ODD, then the L4 car does need a steering wheel and pedals.
That is why I think autonomous personal cars will probably come in 3 types:
1) L5 with no steering wheel or pedals
This would work since L5 could go anywhere the driver would need to go. So there would be no need for the driver to take over, hence no need for steering wheel or pedals. But, I personally don't see L5 happening any time soon.
2) L4 with steering wheel and pedals
This would work since the owner could let the L4 drive when inside the ODD and the owner could take over when they needed to go outside the ODD or want to drive manually. I think this is the most likely type of personal AV we will see in the near future because automakers can simply build cars with steering wheel and pedals like they are doing now but equip them with L4 to drive in certain conditions. This would be the best of both worlds, L4 and manual driving depending on the situation. Automakers could even have a retractable steering wheel to give the owner a bit more space when in L4 mode. For example, I could envision L4 highway where the owner drives the car on city streets but once on the highway, they enable L4, the steering wheel retracts, and the owner can disengage mentally while the L4 drives. As they approach the off ramp, the steering wheel would come out again, and the L4 would signal to the driver to take over.
3) L4 with no steering wheel or pedal but only if the ODD were big enough.
Obviously, with no steering wheel or pedals, the car would be completely restricted to the ODD. The owner would never be able to take over to go outside the ODD. So the ODD would need to cover the owner's driving needs since owners would not accept a car that prevents them from going where they need to go. So it would only work if the ODD were big enough. So this type of personal AV will likely not happen until L4 is wide enough.
It should be noted that people don't actually drive everywhere. So it could be argued that people don't need L5, L4 with a big enough ODD would be good enough for most people. The question is how big does the ODD need to be in order for people to buy a L4 car with no controls. I think if the L4 covered say 99.999% of the owner's driving needs, that would probably be good enough. For example, I could maybe see dealerships in CA selling L4 cars with no controls that are geofenced to CA. People who live in CA and do all their driving in CA might be ok with a L4 car geofenced to CA.