Wiki MASTER THREAD: Actual FSD Beta downloads and experiences

[thewishmaster]

I think its more like adaptive filtering and building more accurate estimates/probabilities.

Situation: Lets say you get your new SW build and you get phantom braking due to stuff it sees on the road like patchwork alphalt and concrete.
- NN(s) calculates 80% chance of road debris and brakes. Driver overrides with the gas pedal. NN(s) understand it's estimation was incorrect and adjusts to get better estimates.
- Next time on same stretch it calculates 30% chance of road debris and doesn't brake. Driver doesn't override with brake. NN(s) understands its calculation was correct and adjusts to get better estimates.

I don't think the NNs are reset every time you get in the car and start driving as if you power cycled your computer. I think it retains it's at least some states and continuously adjusts.

The NN weights are fixed between software updates; re-training or even augmenting existing weights with new data is a computationally expensive process and requires a good amount of verification to make sure other aspects are unaffected. Your overrides will feed into what snapshots are sent back and added to the training set for the next release. Hard to tell what proportion becomes new training data, but probably as much as they can handle with manual curation

 

[aronth5]

A little crazy Lane changing. Sorry for the LAZY iPhone video.

Crossing intersection onto a 2 lane + BUS lane. Beta hunting and weaving and hading for (correct) center lane and then goes into the BUS lane. Turns on left signal and moves over to center lane and turns signal off. Then heads over into the left turn lane with no signal as it is going through intersection. Need to make a left so OF COURSE Beta immediately moves back into the center lane (through another intersection) only to immediately move back into the left lane. If there is low traffic it will almost always repeat this same pattern.

Not how it works. Training happens only at Tesla - and in the car it is purely execution mode.

There has been some speculation that there is some configuration or something like that in the car that gets used. But it hasn't been confirmed ...

The question of whether your Tesla learns has been discussed many many times over the past few years. The answer always comes to training happens at Tesla and is reflected only when pushed out with a new update. Could that change in future? Who knows. FSD is treating every drive, every intersection, every everything for the first time with variations on how it will drive. Unfortunately that puts FSD at a significant disadvantage to human drivers since we adjust how we drive as we become familiar with the roads we drive over and over.

 

[EVNow]

The question of whether your Tesla learns has been discussed many many times over the past few years. The answer always comes to training happens at Tesla and is reflected only when pushed out with a new update. Could that change in future? Who knows. FSD is treating every drive, every intersection, every everything for the first time with variations on how it will drive. Unfortunately that puts FSD at a significant disadvantage to human drivers since we adjust how we drive as we become familiar with the roads we drive over and over.

See the ps I added to my post.

There is certainly some evidence that there is a car specific "memory" - possibly tied to the planner.

 

[aronth5]

See the ps I added to my post.

There is certainly some evidence that there is a car specific "memory" - possibly tied to the planner.

Thanks. That would fall into the "Could that change in future? Who knows." Maybe the future is here now but I'm not counting on it just yet.

 

[mark95476]

Completely wrong.

The NN weights are fixed between software updates; re-training or even augmenting existing weights with new data is a computationally expensive process and requires a good amount of verification to make sure other aspects are unaffected. Your overrides will feed into what snapshots are sent back and added to the training set for the next release. Hard to tell what proportion becomes new training data, but probably as much as they can handle with manual curation

 

[thewishmaster]

Completely wrong.

Which part, and if so what's your alternate explanation?

 

[Phlier]

Completely wrong.

What @thewishmaster said was (at least, IMO) a pretty accurate summation of what Tesla stated in their AI day presentation.

I'm not trying to be argumentative here, but "completely wrong" isn't a very helpful reply. Maybe expound on that a bit? What did he have wrong, sources, etc.

 

[mark95476]

I'm not describing training.

Adaptive filtering (or inference in this context) has been around for a looooong time. I doubt Tesla's NNs/inference is purely feedforward and doesn't have any feedback. I don't work in this space so I could be wrong.

Not how it works. Training happens only at Tesla - and in the car it is purely execution mode.

There has been some speculation that there is some configuration or something like that in the car that gets used. But it hasn't been confirmed ...

ps : To explain what I meant above better - we have all seen the car seems to perform better on the same routes after a while. Is that because we just get used to how it works - or is it because Tesla stores some secret configuration in each car that keeps track of some difficult situations and what worked, which can be used by the planner (not NN) later on ? There were reports on twitter a couple of years ago of someone testing this theory by using different cars with the same firmware on some roads. One car was "used to" the roads and performed better than a car that drove on those for the first time. I've to dig up those tweets - also I mentioned those tweets here.

 

[JulienW]

Guys kinda getting way off topic here. This thread is about your actual FSD Beta driving experiences. Please take AI day discussion to a proper thread.

Here is a curb that FSD Beta seems like it will never clear. I always take over but it seems to always be heading for severe curbing of the rear and maybe even the front. It shows it on the UI so it sees it. It is a sidewalk protector that does kind of stick out but of course not into the road. However Tesla needs to clear it.

 

[Mardak]

Most likely attributed to random variation and map enhancements which come outside of map version updates

I did notice this week that navigation shows an extra turn step and draws the line into the parking lot (so FSD Beta tries to turn) at a destination that last week would stop on the street instead. Map navigation version seems to be unchanged, but something is different. The parking lot exists on Google Maps, MapBox and OpenStreetMap but not TomTom, Bing Map nor HERE.

 

[old pilot]

Drove the same route today, chill mode dry roads no traffic. I decided to drive slower today and it did seem to help with the exception of one 90% left bend in the road. The car approached the curve a little fast but completed the turn without problem but after exiting the turn slowed dramatically then accelerated, then entered a 90% right curve at a reasonable speed crossing the yellow centerline. I feel more comfortable with the FSD, because I know what to expect, and that may give the impression it is preforming better which on reflection is not true.

 

[EVNow]

Adaptive filtering (or inference in this context) has been around for a looooong time. I doubt Tesla's NNs/inference is purely feedforward and doesn't have any feedback. I don't work in this space so I could be wrong.

From everything we know, Tesla is using "traditional" CNN. The weights are fixed during training and can only be changed by Tesla during training. Inference just uses the assigned weights.

 

[thewishmaster]

I did notice this week that navigation shows an extra turn step and draws the line into the parking lot (so FSD Beta tries to turn) at a destination that last week would stop on the street instead. Map navigation version seems to be unchanged, but something is different. The parking lot exists on Google Maps, MapBox and OpenStreetMap but not TomTom, Bing Map nor HERE.

My casual observation of a similar effect is this exit. The car has to take 2 forks correctly following the green line while avoiding the red branches, and it has always struggled with the 2nd fork. Previously it would start hesitating and wobbling halfway between the forks and choose the 2nd fork almost at random, requiring a disengagement most of the time. Last couple times however it was pretty smooth and almost seemed prepared. The nav voice has known the correct left/right directions from the start. Hope I didn't just jinx it to fail on the next attempt lol

 

[ikester]

FSD just curbed my passenger rear tire and rim on a right hand turn at a stoplight in relatively quiet evening in Chelan Wa. I thought it was getting close, but I wanted to give it a chance to correct. Big mistake! Took a chunk out of the tire’s sidewall.

Quite simple intersection, little traffic.

2018 M3P

 

[EVNow]

The parking lot exists on Google Maps, MapBox and OpenStreetMap but not TomTom, Bing Map nor HERE.

How do you see MpaBox maps ?

ps : In BBBike MapBox always comes up empty.

 

[Mardak]

How do you see MapBox maps

Various apps use their data, but simplest I've found is using the Android Demo App Mapbox Demo - Apps on Google Play
Pretty much any map view in the demo you can zoom out and then zoom in to the region you're interested in.

The menu includes a "Query map" -> "Query a map feature" where you can tap on to get some extra information about lines and areas. Unclear if they're only showing certain data and/or converting some values, e.g., OSM [oneway=yes] vs MapBox oneway - "true" or [highway=primary] vs type - "primary".

iOS doesn't have the Demo, but there's a "Studio Preview" which seems to require a login:

Mapbox Studio Preview

Mapbox Studio Preview lets you preview your Mapbox maps on your iPhone and iPad. Design beautiful custom maps using Mapbox Studio on your desktop (studio.mapbox.com), then take your maps with you with Studio Preview. Prototype maps and data visualizations in Studio on the web and get a live...

apps.apple.com

 

[LowlyOilBurner]

Yep

Last night I had a drive in pouring rain - quite different from the "drizzle" we normally get here.

For the first time - FSD just couldn't handle the drive. It kept getting confused about lanes, kept giving warning about poor weather conditions ... and I finally gave up and drove on my own. First time this has happened to me in over 2 years - I've always been able to use AP even in pouring rain at night.

This morning - again in heavy rain - the car gave waring of "Full Self Driving may be degraded" and actually turned into AP ! It stopped at the stop sign and wouldn't go until I pressed the accelerator. Till now I have been getting these weather warnings, but FSD has never "degraded" to AP.

Yep. I’ve had this same experience with some moderate snow falling. Roads were just wet, but all the same warnings/failures you mentioned.

 

[LowlyOilBurner]

See attached screenshot. This is consistent failure. When attempting the left from “Sheridan” to where the white car is on the bottom left, always fails. It’s only once made this turn correctly on 10.3.1 while it was raining. 10.4 doesn’t do it at all, no matter the weather.

 

[JeremyJ]

FSD just curbed my passenger rear tire and rim on a right hand turn at a stoplight in relatively quiet evening in Chelan Wa. I thought it was getting close, but I wanted to give it a chance to correct. Big mistake! Took a chunk out of the tire’s sidewall.

Quite simple intersection, little traffic.

2018 M3P

I clipped my mirror trying to go around a traffic barrier. Thought it was fine from the first one it went around but got too close. Had to replaced glass for 125. Rest of mirror was okay.

 

[omarsultan]

Unenrolled from 10.4 last Friday, yet the beta firmware remains on my Plaid. What does it take to replace this firmware with the current production build? An act of Congress? Divine intervention? Driving on FSD while not paying attention? What?!

You are probably stuck on the current level until the next release of the production build that will work with your config and the FW levels of the components in your car.