Autopilot: Learning Curve + Weather!

[tanner]

Okay, drove an AP car for the first time the other day... The advisor who accompanied me on the test drive told me he had specifically chosen that car because they've driven it enough that it was able to "learn" what to do in most situations. Does anyone have firsthand experience with the Model S' AP learning curve - e.g. was it horrible at first and is it better now? He even went as far as to tell me the new cars they get are HORRIBLE when it comes to AP and they tell you to take over all the time until they learn how to behave.

Next up, weather! How does your AP do in all sorts of weather conditions?!

 

[LuckyLuke]

I'm not sure it's so car specific, it's a cloud thing so the car actually learns from all other drivers/cars not just from the person driving that specific car?

 

[DDotJ]

AP holds up relatively well in rain. Was raining pretty hard in SoCal last week, and I tried the AP with both hands on the wheel. Worked great for the most part. It was able to "see" lanes even when I couldn't see them clearly.

But it did eventually run into problems from poor visibility and reflections of light on the ground. Just make sure you've got hands on the wheel because the car will hand over controls back to you at the blink of an eye.

And a disclaimer, AP isn't recommended in adverse conditions. I just tried it out for a few miles and resumed manual operation. It did better than I though it would, but I would still proceed cautiously. Road and weather conditions change rapidly and the vehicle may not be able to handle the conditions.

 

[mspohr]

Okay, drove an AP car for the first time the other day... The advisor who accompanied me on the test drive told me he had specifically chosen that car because they've driven it enough that it was able to "learn" what to do in most situations. Does anyone have firsthand experience with the Model S' AP learning curve - e.g. was it horrible at first and is it better now? He even went as far as to tell me the new cars they get are HORRIBLE when it comes to AP and they tell you to take over all the time until they learn how to behave.

Next up, weather! How does your AP do in all sorts of weather conditions?!

There's a lot of anecdotes in threads here but it's hard to come to any conclusions... The plural of anecdote is not data.
Night AP is good. Rain is OK but not recommend.

 

[ecarfan]

I'm not sure it's so car specific, it's a cloud thing so the car actually learns from all other drivers/cars not just from the person driving that specific car?

Yes, Elon has stated that there is "fleet learning", or something to that effect. So I am puzzled by the statement, posted just up thread, by a Tesla sales employee that the demo cars they get have poor AP performance until they have been driven on local roads for awhile.
We still have little specific information on how the Tesla AP "learning" process occurs and how what one car learns is disseminated to the whole fleet. Lots of speculation, no hard facts from Tesla. I don't expect Tesla to publicly share every detail of how AP learns, but would like to have an overview of this process in more depth than what we have now.

 

[Beryl]

Night AP is good. Rain is OK but not recommend.

My limited experience in the rain supports the "use autopilot with caution" guidance. Autosteer will turn off when the marking visibility is decreased due to rain. Cruise control remains on so I'd like to know how well TACC performs in precipitous weather. The car distance should probably be increased for safety but how much is advised?

 

[robert774]

Where I personally experienced the learning, more of a calibration really, is use of AP for parallel parking. Mine did not work initially but after a few weeks of use spots where it did not work previously work perfectly now. I found the auto steering to work well in most conditions other than early morning or late afternoon sun where there tens to be glare over the lane markings.

 

[tanner]

Yes, Elon has stated that there is "fleet learning", or something to that effect. So I am puzzled by the statement, posted just up thread, by a Tesla sales employee that the demo cars they get have poor AP performance until they have been driven on local roads for awhile.
We still have little specific information on how the Tesla AP "learning" process occurs and how what one car learns is disseminated to the whole fleet. Lots of speculation, no hard facts from Tesla. I don't expect Tesla to publicly share every detail of how AP learns, but would like to have an overview of this process in more depth than what we have now.

The manager at my service location (the advisor who I got the original info from) claims to know Elon and he informed me that the cars have yet to utilize "fleet learning". He basically said that for now, they're just learning from the driver of the specific car - however, in the next update or two (possibly 7.1), they will have analyzed the data from the fleet and push what cars have learned to one another.

- - - Updated - - -

Where I personally experienced the learning, more of a calibration really, is use of AP for parallel parking. Mine did not work initially but after a few weeks of use spots where it did not work previously work perfectly now. I found the auto steering to work well in most conditions other than early morning or late afternoon sun where there tens to be glare over the lane markings.

very interesting! Did you have your car after or before AP software was released in October.

- - - Updated - - -

My limited experience in the rain supports the "use autopilot with caution" guidance. Autosteer will turn off when the marking visibility is decreased due to rain. Cruise control remains on so I'd like to know how well TACC performs in precipitous weather. The car distance should probably be increased for safety but how much is advised?

Thankfully it doesn't rain much in SoCal (hardly ever), but that's good to know! Also, according to the DMV, you should ALWAYS maintain a distance of 1 car length per 10MPH increment - e.g. If you're going 60, about 6 car lengths... That's based on old ICE cars and the average time it takes for one to stop; granted Teslas stop much more quickly, I'd use that as a safe unit for distance measurement when it comes to autopilot (in case you have to over correct / need additional time).

- - - Updated - - -

AP holds up relatively well in rain. Was raining pretty hard in SoCal last week, and I tried the AP with both hands on the wheel. Worked great for the most part. It was able to "see" lanes even when I couldn't see them clearly.

But it did eventually run into problems from poor visibility and reflections of light on the ground. Just make sure you've got hands on the wheel because the car will hand over controls back to you at the blink of an eye.

And a disclaimer, AP isn't recommended in adverse conditions. I just tried it out for a few miles and resumed manual operation. It did better than I though it would, but I would still proceed cautiously. Road and weather conditions change rapidly and the vehicle may not be able to handle the conditions.

Yeah, seems more like a "let's test it to see if it works" kind of thing when it's raining than a "let's fully rely on it to get me to where I need to go in the rain" kind of thing. LOL! Thankfully we don't have to worry about it much here.

 

[ecarfan]

The manager at my service location (the advisor who I got the original info from) claims to know Elon and he informed me that the cars have yet to utilize "fleet learning". He basically said that for now, they're just learning from the driver of the specific car - however, in the next update or two (possibly 7.1), they will have analyzed the data from the fleet and push what cars have learned to one another.

Very interesting, thank you for posting. I see you are located in SoCal so I can imagine that your service center manager could have had direct contact with Elon and had this information passed on to him.

 

[tanner]

Very interesting, thank you for posting. I see you are located in SoCal so I can imagine that your service center manager could have had direct contact with Elon and had this information passed on to him.

No problem! And it's definitely possible, he's crazy knowledgable regarding all things Tesla - more so than anyone I've dealt with. Moreover, I've verified most of what he's told me, save this.

 

[S4WRXTTCS]

As far as I'm aware of the car has no significant self-learning functions, nor does it have any hardware that would assist with this task. I don't see where this would take place (processing/storage/etc).

Now I wouldn't be surprised if it doesn't mark areas where you had to take over since part of the fleet learning is marking those areas, and I imagine it has a mechanism for feeding corrections. So limited self-learning I could see being supported. But, I don't see how it would do the kind of self-learning that I wished it did.

So in summary I do think there is truth in what he is saying, but it's being exaggerated.

For me AP has been pretty consistent in it's behavior since it's release. I'm still on the first V7.0 release because apparently Tesla doesn't believe I'm worthy of any of the updates.

As to it's performance in the rain it's situational.

If it's a drizzle the Lane-Steering/TACC has no issues
If it's significant then the Lane-Steering doesn't see the lines, but TACC still functions fine. I'm really hoping fleet learning will significantly improve it's ability to show the lines in adverse conditions. Even if I didn't use AP in these conditions it would be nice to see the a virtual road to help me when the road becomes more water than road.

One thing I've seen a huge improvement in with V7.0 vs. V6.2 is the side monitoring behavior in the rain. With V6.2 it would disable all the time in the rain.

 

[tanner]

As far as I'm aware of the car has no significant self-learning functions, nor does it have any hardware that would assist with this task. I don't see where this would take place (processing/storage/etc).

Now I wouldn't be surprised if it doesn't mark areas where you had to take over since part of the fleet learning is marking those areas, and I imagine it has a mechanism for feeding corrections. So limited self-learning I could see being supported. But, I don't see how it would do the kind of self-learning that I wished it did.

So in summary I do think there is truth in what he is saying, but it's being exaggerated.

For me AP has been pretty consistent in it's behavior since it's release. I'm still on the first V7.0 release because apparently Tesla doesn't believe I'm worthy of any of the updates.

As to it's performance in the rain it's situational.

If it's a drizzle the Lane-Steering/TACC has no issues
If it's significant then the Lane-Steering doesn't see the lines, but TACC still functions fine. I'm really hoping fleet learning will significantly improve it's ability to show the lines in adverse conditions. Even if I didn't use AP in these conditions it would be nice to see the a virtual road to help me when the road becomes more water than road.

One thing I've seen a huge improvement in with V7.0 vs. V6.2 is the side monitoring behavior in the rain. With V6.2 it would disable all the time in the rain.

The car of course has onboard storage and processing capabilities... Data would be stored in the exact same place it would if they were learning from the whole fleet. Moreover, it's possible Tesla outsources processing of the data to their servers. It's honestly pure speculation at this point.

Furthermore, I was under the impression that v6.2 didn't have lane keeping / full autopilot?

 

[robert774]

very interesting! Did you have your car after or before AP software was released in

I got it in April, before the 7.0 software.

 

[tanner]

I got it in April, before the 7.0 software.

Apparently AP Tech-equipped cars were collecting data for autopilot capabilities the entire time.

 

[donv]

I've found TACC to work great in rain/low visibility-- probably better than I could do myself. I try to use it whenever I can in those conditions. Remember, it's using radar, which should be able to see through fog and low visibility better than you can.

Auto-steer works some of the time, and can be used with caution and attention. Frequently, it won't work at all, which is okay by me.

 

[tanner]

Auto-steer works some of the time, and can be used with caution and attention. Frequently, it won't work at all, which is okay by me.

For clarity, you're just referring to auto steer in crappy locations, right?

 

[donv]

Actually, I was referring to it in poor weather conditions like rain or low visibility.

For clarity, you're just referring to auto steer in crappy locations, right?

 

[tanner]

Actually, I was referring to it in poor weather conditions like rain or low visibility.

Right, I totally meant to say "conditions", not "locations"! Haha. It works fine in sunny/cloudy weather, right?

 

[S4WRXTTCS]

The car of course has onboard storage and processing capabilities... Data would be stored in the exact same place it would if they were learning from the whole fleet. Moreover, it's possible Tesla outsources processing of the data to their servers. It's honestly pure speculation at this point.

Furthermore, I was under the impression that v6.2 didn't have lane keeping / full autopilot?

It has multiple levels of onboard processing and storage. The question is what does what?

Speculation is the car stores the data to be uploaded for fleet learning on an SD card. I'm not sure where else it would store this data.

The Nvidia Tegra 3 is used for the Digital Cockpit, but it doesn't serve any AP function aside from UI/UX.

The Lane-Keeping/Auto-pilot is done via a Mobile Eye chip, but Mobile Eyes software runs there. I tend to see the MobileEye processor as something you load pre-trained DNN's in. A DNN being a trained Deep Neural network. Where the training of it took place offline (at Tesla or MobileEye). The DNN handles things like detecting lanes, cars (visually), speed limit signs, etc). It doesn't do any self-learning. I imagine most of the DNN's used for various functions (Object Detection, Lane Detection) is provided by MobileEye as libraries. Where Tesla can pick and choose what they want. Like the Tesla can do Speed Limit sign detection, but not Stop-Light detection.

The biggest hole I've seen in this communities understanding of Tesla's Autopilot system is how the system learns.

For learning to happen it has to have a feedback loop. Something that tells it what the "correct" way was.

If I take an exit one day, but then I don't take an exit the next day it's going to be awfully confused.

If I have to deviate because of construction (onto the shoulder) it's going to be confused.

Fleet learning is massively important because it brings so much data to the table. Not only does it bring the data to the table, but it allows humans to decide which is the "correct" way.

The only really informative technical article on the Tesla AutoPilot I've seen so far is this, but it doesn't go into the fleet learning that much.

Exclusive: The Tesla AutoPilot - An In-Depth Look At The Technology Behind the Engineering Marvel

As to the other question -
V6.2 had TACC and Blind Spot Monitoring (limited side spot monitoring).
V7.0 brought Lane-Keeping, and Side Spot Monitoring.

Technically the Side Spot Monitoring is not in the AP package, but I consider it to be within the realm of the Driver assist system.

 

[Max*]

AP works well in light rain, but when it's heavy rain AP either doesn't see the lane markings at all (wont activate) or if it does activate, ping pongs and jolts the car left/right (almost as if it's ping ponging from side-to-side) enough for me to turn it off and drive myself.