Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Autopilot is already improving.

This site may earn commission on affiliate links.
Double checked my car's Wifi data captures. Nothing new really. The car has periodically been downloading a 4 to 5MB data burst from Tesla every few days for months, which I've been guessing is the charger list and associated data.

Nothing new since the autopilot update.

Thanks. If you have an easy way to set up alerting (splunk or similar), would you mind letting us know if you see a bigger download at some point?

- - - Updated - - -

And this information is gathered in real time, and new model updates seem to be downloaded to the cars on a daily basis.

I certainly agree with your overall summary, but I can't yet reconcile how these model updates are being sent to the vehicles.
 
I wonder if we set up a mobile WiFi hotspot while driving, we could use that to see how much data is sent/received? I suppose Tesla could force this data over cellular, but the networking stack seems to use WiFi once it verifies it has a valid connection to the net.
 
I just got a notice that a new software version is available. My car is in the shop, so I can't install it! Currently I have the initial release of 7.0.

Has anyone else gotten a notice of a new version of 7.x? Or installed a new firmware after the initial 7.0?
 
I wonder if we set up a mobile WiFi hotspot while driving, we could use that to see how much data is sent/received? I suppose Tesla could force this data over cellular, but the networking stack seems to use WiFi once it verifies it has a valid connection to the net.

That's a pretty good idea, actually. If I have time I'll have to whip something up. Maybe like a Raspberry Pi 2 as a low power router. :p
 
After finally listening to the Autopilot press conference (really Auto Steer), I finally understand more about how it works.

Mobileye is the technology that is used to find lane boundaries, pedestrian and bicycle detection, car detection, and traffic sign decoding. At some point, it will also do stop sign detection and signal light decoding. Tesla then layers on top of this their own algorithms, and most importantly, high precision GPS maps.

It is the GPS maps that allows the system to get better on a daily basis. People have reported that initially their cars would try to follow the right hand freeway lane marker and exit the freeway prematurely. After a week of driving, the car doesn't do that anymore. What happened is that over the hundreds of trips made that week by the entire Tesla fleet (including non autopilot enabled cars, do note), they updated their model to say that in this lane on this part of the freeway, use the left lane marker, not the right as a guide. This system would be impervious to jokers trying to trick the system since such bad driving would be rare and ignored. This system seems to be smart enough to be used in areas that have either no lane markers, or contradictory lane markers (Elon gave an example of a horror show of an area on the 405 freeway).

So Tesla uses Mobileye lane detection, but augments it with their own high precision GPS maps which tells it what inputs to use in a particular location. And this information is gathered in real time, and new model updates seem to be downloaded to the cars on a daily basis.

This is very, very powerful. No other competitor, including Google, has anything approaching the ability for 80,000 cars (today) to be continually providing input to make the model smarter.

Indeed! My perception of the promise and power of the system went up enormously by listening to the press conference, and studying the accompanying slides. I am amazed (and disappointed) that they did not share all of this info, including the illustrations, when they published the blog and email blast to the owner base. Yet another example of suboptimal communication by Tesla.
 
This is very, very powerful. No other competitor, including Google, has anything approaching the ability for 80,000 cars (today) to be continually providing input to make the model smarter.

Agreed! I hope tesla come up with a cooler fully autonomous PRT(Personal rapid transit) POD/car, better than googles, for public transport.​ The Tesla P.
 
Thanks. If you have an easy way to set up alerting (splunk or similar), would you mind letting us know if you see a bigger download at some point?

- - - Updated - - -



I certainly agree with your overall summary, but I can't yet reconcile how these model updates are being sent to the vehicles.

Maybe they're only sending small updates for the road you're driving on rather than large updates that include all the road corrections for an entire region or the country. That might explain why you don't see a big uptick in data while parked on wifi. It probably only takes a few kbs of data for the road you're on for some significant number of miles. I suppose the counter to that proposal is that it doesn't cost Tesla anything to use your wifi so why not upload a huge update overnight while on wifi?
 
How does one see this, or estimate its size? Something in your router?
Yes. I happened to notice a sudden slowdown in the network so I looked at the router statistics. The timing, bandwidth and network segment used all resembled the 7.0 download last week, although this one didn't last as long.

I don't have wk057's sophisticated setup, so I don't know what it was. Also, it happened on Monday night/Tuesday morning, and if there was a repeat on Tuesday night/Wednesday morning, I didn't notice it. Based on what wk057 said, it could have just been a regular (pre-7.0) data download, although it seemed larger than the 4-5 MB he mentioned.
 
Maybe they're only sending small updates for the road you're driving on rather than large updates that include all the road corrections for an entire region or the country. That might explain why you don't see a big uptick in data while parked on wifi. It probably only takes a few kbs of data for the road you're on for some significant number of miles. I suppose the counter to that proposal is that it doesn't cost Tesla anything to use your wifi so why not upload a huge update overnight while on wifi?

You could be right. Obviously I don't have insight into the structure of their model. Initially I figured it would be like most problems, in that you'd make a model, train it on the massive data, and intend to have it generalize well to new conditions. It's definitely possible it's region specific and in a double layer, as postulated above. Having a geolocation-fractured model might make sense because the model wouldn't have to generalize as greatly and could better fit the conditions in the region.

Getting too specific (the idea of the road ahead for some miles) has the sound of hand coding to it, and I think we all agree this is some kind of machine learning algorithm. So I'm not too sure about that piece. I would hate to think that there are engineers hand coding specific highway exits as failsafes to a deep learning network.

To take the right-hand exit example, I'll give a very simplified and high level idea of how a machine learning system could work. The actions are based on features which have weights. There are many features but let's really pare it down for the purposes of this example.

Features:
- Left line location
- Right line location
- Center of vehicle ahead
- GPS longitude
- GPS latitude

Each of these features is given some weight and contributes some influence on the behavior of the model at any given time. Perhaps initially, for a given set of features, the weight of the right line location was higher than the left line location except in cases where the center of vehicle ahead was populated (existed). Then, AP might have a tendency to lean more towards following the right line. However, imagine that every time it does that at a right exit, the driver disengages it and keeps the car on the highway. Then, given some GPS coordinates and/or other features, the weight for the right line would decrease in the updated model. Every occurrence of a driver keeping the car on the highway would continue to lower the weight, until it reached some optimal value.

The chart for the right line location weight might get very high in locations where there is no exit, and then rapidly decrease every time you pass an exit. This is the amazingly cool thing about these predictive models - how they learn by creating a "cost" to doing a certain thing, and minimizing that cost.

Leaving GPS out of it, it's also possible that a model could just figure out that when the right line curves away when the left is going straight, it is probably an exit and could lower the weight. The upside to this is that it's generalizable to other locations that haven't been seen by the model.

The key piece of this is that these models require processing power and time to build. So I do not think they are continuously updated, unless they are treated as a kind of time-series problem, and I can't see how that would make too much sense.

Sorry for the long post. TL;DR - computers are cool.
 
I wonder if what's going on is that the car is constantly checking the built in map database against what the driver is doing and what the camera, and possibly other sensors, are seeing. If a simple interpolation of the data in the map at a location isn't accurate enough, the car reports the differences, possibly just by reporting some more control points based on the car's motion and things like the camera's lane sensing. If the map data is OK, nothing is sent.

That's all combined back at the mother ship and the high resolution corrections are generated so they can be pushed back out. Essentially all they are sending both ways are corrections to the existing map data. I'd think this would keep the data required to something reasonable, and just having this very good mapping data is probably enough to make a very large improvement in the car's behavior even without more complex algorithms. It's really a brute force approach, but memory is cheap and as the corrections build up, the amount of data traffic should greatly decrease.
 
(After all, autopilot's been in testing for a year, and beta testers have been using it for the last 2 months--why wouldn't it have learned some of this stuff during that time?)


I think as others have mentioned, the incredibly high detailed gps maps are the main reason behind this, as these maps are likely heavily limited to California during the beta test but are now being generated nationwide. The autopilot likely guesses which factor of the environment is most important to pay attention to without map knowledge, but as hundreds of Tesla trips increase confidence in some factor being better than another factor, it'll change which factor is best. Easiest example, that the left lane is more important than the right lane on all those exits.

I certainly agree with your overall summary, but I can't yet reconcile how these model updates are being sent to the vehicles.

I imagine this data is very small, small enough to be added into the GPS navigation system on 3g connections with ease.

It reminds me of when "replay" functionality came out for games. All of a sudden, an hour-long video of a game could be fit within a file less than 100 kb in size. This wasn't some sort of super-secret compression method, the game just simply saved a list of all commands, and those said commands could perfectly replicate a game for future viewers. Instead of saving a 500 mb video, all the replay file did was procedurely recreate exactly what happened--and I think autopilot is basically the exact same thing, procedurely recreating safe driving routes using metadata while a system still ensures safety against the unknown.

Autopilot similarly could probably very easily save metadata right on top of their maps. I imagine they could have a GPS value that'll tell autopilot what item on the road to prefer when navigating an area. Other lists could probably indicate times of day when sunlight will cause data to be spotty on the map visuals. Even curves could be handled in this way (a single data value could tell TACC to slow down to a specific speed, and the max speed is generated by people who previously tested but broke the AP on that area). I'm sure all Tesla has an extensive list of variables right on top of lanes/gps/roads/etc., self-generated by the Tesla fleet's autopilot successes and failures with their trainers' corrections. As long as nothing unexpected happens, the autopilot could trust these variables almost blindly without fault--but the system will still have the final say to compensate for the unexpected.

I think that's how autopilot's machine learning works, and also why they can send out these updates without consent but also without fear that the cars' performance will become dangerous. As others mentioned it probably requires a high confidence level (dozens of drivers/passes with the same behavior) to assume something is safe, and it could even carry a local set of data that is given higher preference when on your typical commutes. A Tesla doesn't need to know exactly how every commute goes exactly, they just need to know if the left lane is best, if it's a straight-shot intersection and what lines to follow after the intersection, if they should slow down on a curve, etc. and the autopilot handles everything else.
 
It is unlikely that they have super accurate maps down to inches that would be required to drive a car. GPS isn't that accurate.

Online maps are made of of segments from intersection to intersection, with infraction points defining the curves in the road. It is likely that they are tagging road segments or points between infraction points with added information to help the onboard system handle the road. For example, the technical debug data screen posted above had which line, right or left, to follow. They could tag practically anything that would help the system. Current speed, speed limit signs, user correction required, etc.

Waze works like this. The clients upload speed info per segment and/or info like hazards. Hazards are probably uploaded with the segment ID and an offset from the end.

No bulk upload/download. No major change to operation of the system. Just "helpers" that could eventually get very very helpful.
 
A week ago or so I created a thread where I speculated the on-board harddrive would be used for this, and that the car would process the data either on the fly or analyse when stationary and upload processed data (only the interesting stuff, cutting away all the non-interesting stuff). Particularly this would be when the driver has interfered/corrected or where AP was unsure and disengaged: what did the driver do at that point.

I got one reply: that the hard drive is likely not there anymore. Well, let's say it's just stored to the SD card then - same principle: lots of data flowing from sensors, car picks out interesting snippets and keep feeding it to the database that keeps learning (machine learning).
 
I think that's how autopilot's machine learning works

I think you've got a decent explanation, but what you're describing isn't machine learning, as it's not generalizable. You lay out a method for specific actions based on location, and while I think a machine learning algorithm may weight features differently in some GPS driven areas, I don't think that's exactly how you are spelling it out.

Maybe at TMC Connect next year someone can come and tell us everything. :smile:

- - - Updated - - -

A week ago or so I created a thread where I speculated the on-board harddrive would be used for this, and that the car would process the data either on the fly or analyse when stationary

I think the only thing the car could potentially do is clean the data before sending it up to the mothership. This is the same reason you need an Internet connection to use Siri on your iPhone. The phone can send the raw audio data to Apple, probably cleaned to some degree, and then Apple's powerful servers can actually interpret it with their deep net.
 
I think the only thing the car could potentially do is clean the data before sending it up to the mothership. This is the same reason you need an Internet connection to use Siri on your iPhone. The phone can send the raw audio data to Apple, probably cleaned to some degree, and then Apple's powerful servers can actually interpret it with their deep net.

You're likely right. I'm just thinking it would be very cool if Tesla somehow were able to use the entire fleet of Teslas as a kind of distributed computing network, kind of like the SETI network or something. The car is just sitting there charging at night right, why not fire up the CPU core and do some work? I know the computer is likely not extremely powerful, but a decent one. It's also not doing anything else, and is made for among other things this particular application. Now if you had several hundres of thousands of those with at least 50% at any given timen standing still, either plugged in or with enough battery, why not take advantage of that computing resource?