Just a reminder:
Level 0: This one is pretty basic. The driver (human) controls it all: steering, brakes, throttle, power. It's what you've been doing all along.
Level 1: This driver-assistance level means that most functions are still controlled by the driver, but a specific function (like steering or accelerating) can be done automatically by the car.
Level 2: In level 2, at least one driver assistance system of "both steering and acceleration/ deceleration using information about the driving environment" is automated, like cruise control and lane-centering. It means that the "driver is disengaged from physically operating the vehicle by having his or her hands off the steering wheel AND foot off pedal at the same time," according to the SAE. The driver must still always be ready to take control of the vehicle, however.
Level 3: Drivers are still necessary in level 3 cars, but are able to completely shift "safety-critical functions" to the vehicle, under certain traffic or environmental conditions. It means that the driver is still present and will intervene if necessary, but is not required to monitor the situation in the same way it does for the previous levels. Jim McBride, autonomous vehicles expert at Ford, said this is "the biggest demarcation is between Levels 3 and 4." He's focused on getting Ford straight to Level 4, since Level 3, which involves transferring control from car to human, can often pose difficulties. "We're not going to ask the driver to instantaneously intervene—that's not a fair proposition," McBride said.
Level 4: This is what is meant by "fully autonomous." Level 4 vehicles are "designed to perform all safety-critical driving functions and monitor roadway conditions for an entire trip." However, it's important to note that this is limited to the "operational design domain (ODD)" of the vehicle—meaning it does not cover every driving scenario.
Level 5: This refers to a fully-autonomous system that expects the vehicle's performance to equal that of a human driver, in every driving scenario—including extreme environments like dirt roads that are unlikely to be navigated by driverless vehicles in the near future.
Please note that it doesn't mention steering wheel removal anywhere. That's an arbitrary option, albeit one that could only happen with L5 unless the car wouldn't be allowed to operate outside of regular road driving conditions.
Considering that Tesla is relying on vision for a generalised format of approaching autonomy, compared to others specific planned autonomy (GPS, Maps, Lidar etc) they appear to be the only company whose software has a chance at equalling a human driver in every driving scenario, including off-road. I believe Bjorn has unwittingly already been
training the Neural Net about offroading 
If they want to expedite that aspect, then I expect they could give a bunch of cars to green-laners and just run those drives through Shadow Mode and then the Dojo. The point is that every weird driving scenario is probably going to be done by an owner somewhere, and every time it is done, the NN becomes more able to "equal that of a human driver, in every driving scenario".
Do I think Elon's time-scales are a bit out? Yeah. I think we'll see L4 in on-road urban and highway USA environments by his date. L5 will take longer but only because of the time needed to gather the data for those other non-ordinary environments. This will be much sped up by the delivery of Model Y and the Pickup which are more likely to be used in alternative environments for soft-roading and off-roading.
But please remember linear vs exponential. Every car delivered, every Dojo super-speed simulation, every Shadow Mode test, cumulatively increases the ability, and I fully expect L5 to appear much sooner than the average person expects (which was 55% voted for 2023 according to a poll done by Hyperchange).
And the differences between L4 and L5? To the average driver who uses their car for 'social, domestic, commute and pleasure'? They're not going to see it.
