Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Training methodology for autonomous vehicles

This site may earn commission on affiliate links.
Hi all,

I am working on a problem, probably on the similar level of complexity as in autonomous cars but in a completely different industry. Nobody has tried to apply ML to this problem, although it seems that if people can apply to autonomous vehicles which have a lot more number of variables, it should be possible to do that in my industry as well.

Some questions:

  1. Is there any white paper that I can read about the high-level implementation of machine learning/process of training in autonomous vehicles? Specifically, I want to understand how and where you start building and training the model. My guess is that the ML model if any in Tesla cars are already trained, and they are being fine-tuned specifically for that car, because the input variables for a model car at Tesla, is probably different from any other car.
  2. This is my guess of what the training process would look like from the start:
    1. Step1: You will first start by training an ML model on a simulated car. By simulated car, I mean a car that is completely virtual, but one that has a model that predicts how fast it will move when the pedal is driven down a certain amount, will decelerate when the brakes are applied, be able to understand virtual images of road signs, etc.
    2. Step2: The next step would be to train the model in a controlled real-life experimental situation. So you would take a real-life Tesla car, put it in a road track perhaps in Tesla, or one of the rolling wheels at a stationary place, and test whether the car can drive itself autonomously in this controlled environment. Experiments and model simulations will never be similar, so you would have to further train the ML model, but this time on controlled real-life input parameters and real-life output parameters.
    3. Step3: The next step would be to train it in an uncontrolled environment, such as a road, with traffic, but perhaps with assistance from expert human drivers, so the car can learn from the human as to how it should behave when there is traffic.
    4. Step4: The next step would sell the car to a customer (who may or may not be a model human driver), and the car will learn from the mistakes and good driving habits of the human driver. Till now the car is only training a single model.
    5. Step5: The last step would be to learn from all drivers, so there would be a single all-encompassing model, somewhere in the cloud. As the car moves in the real-world, it feeds the data into the cloud and trains a master model. As new cars come into the real-life, this master model is first fed into the cars. The new cars then start learning by themselves, and as they learn, they feed certain parameters back into the master model in the cloud. The master model in the cloud will never be exactly what's needed to drive a real-life car, because each car will be slightly different from one another, and it's only if you train the model in the car, specifically with inputs from the car in the real-life, that it will work best. That is why its necessary to have a master model, and a fork of that master model for the individual cars.
Is this what the process like in training a model for Autonomous vehicles looks like?

Thanks for any pointers.
 
[Using arbitrary language for 'fleet' and 'unit' and assuming 'fleet' is a desired singleton]

From my experience with robotics ML, usually step 4 and 5 are in the opposite sequence, because you cannot for most models share much 'knowledge' from unit-based learning into fleet-based - especially in neural networks and/or other field- based or connectionist- based approaches that encode knowledge holographically. This also requires much more compute power in each unit. You do use the units for raw data capture of training data for the fleet, usually with interesting metadata from human interaction in case of supervised systems. And you use unit experience to feed back into adversarial models and to test fitness in 'shadow mode'

Another "step" (albeit many of these are done in parallel) is to create adversarial systems or consensus systems, where you have a different ML 'stack' (in my experience evolutionary) 'attack' the fleet-mind with recorded or simulated situations that may trip it (i.e. find where the conclusion is not the desired), and keeps the desired balance of specificity and sensitivity to event detection (avoiding 'overtrained' to either norm or exceptions n NN parlance).

Furthermore, given computing power you put this all in a loop that tunes it's own metaparameters, so you can compete whole fleet-minds against each other in simulation, recorded situations, or in shadow mode - assuming you can upload new ML models to units and download conclusions efficiently. Metaparameters may include high-level constructs related to self-driving cars like the 'clock tick' of the AI processing, or the 'steps ahead' the conclusions need to touch. Topology parameters include NN layer sizes & depths. Low level parameters include NN backpropagation levels/curves and compression ratios.

Not sure how Tesla approaches it, but it is also possible over time to look at having distinct vertical 'agents' compete / collaborate for conclusions and have a higher order agent decide which conclusion wins.

Training of the sort in 4 can typically used for localized knowledge and overrides. One thing I have not seen Tesla do is have people flag events to 'share them with autopilot'. I know it is not apple-esque in design to ask humans for feedback, but if it were me I'd try it and I think it would be super interesting metadata. Without this interesting feature/crutch tesla could prioritize upload of 'situations' by external parameters (geo-fence, weather,day/night), car status envelopes [speed, traction, lights, wipers, brakes slammed], or triggered by the AI subsystems itself (eg 'collect the 10s video clip when there were high chances of bike detected')