-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
The «Full» in Full Self-Driving Capability
- Thread starter lunitiks
- Start date
thegruf
Active Member
This is exactly Waymo's point.
Requiring the driver to remain alert for exceptional events is quite simply unreliable and therefore unwise.
Had Tesla been able to demonstrate all but flawless performance with the systems as specified to date, I think we could all be a lot more confident. But when the thing still struggles to even stay in lane sometimes ...
Yeah, Which description of FSD will eventually become available.. The original version promised 10/2018 when I bought it or the new changed watered down version?
strangecosmos
Non-Member
Possibly!
The analogy between DeepMind’s AlphaStar and Tesla’s Full Self-Driving
Almost everyone here is a layperson, and I don’t think you’re necessarily wrong about fleet learning. We’re all just trying to learn and speculate.
With fleet-scale machine learning, I surmise that there are two main kinds of data that could be collected: raw sensor data and state-action pairs.
Raw sensor data can, in theory, go through two pipelines:
- the images or video are uploaded and painstakingly labelled by a human annotator (it’s known for sure Tesla is doing this, e.g. Andrej Karpathy has given talks about it)
- the images or video are uploaded and only weakly labelled with some driving input, such as an image of a yellow light being weakly labelled with a human driver’s deceleration (it’s just my speculation that Tesla could be doing this)
Weakly labelled sensor data is something Tesla is currently in a unique position to collect (unless Mobileye or one of the car companies is doing stuff I don’t know about). A hand labelled image is much more valuable than a weakly labelled image, but Facebook has shown you can get pretty far with weakly labelled images. Caveat: Facebook used Instagram hashtags as the weak label, which might be more reliable than driver input like accelerating, steering, and braking. However, a lot of hashtags mislabel images.
State-action pairs are even cheaper to collect than raw sensor data, since they use less bandwidth and storage. State-action pairs come already labelled because the action is the label. A state-action pair is analogous to an image-label pair.
Outside of raw sensor data and state-action pairs, I suppose in theory data could be collected in the form of a training signal for reinforcement learning. For instance, every time a human takes over from Autopilot or FSD, that could deduct points from the reward. Points could be gained by miles between disengagements. Whereas we know for certain Tesla is collecting raw sensor data, and it’s been reported (but not confirmed by Tesla) that Tesla is collecting state-action pairs, there is no direct evidence I’m aware of that Tesla is doing fleet reinforcement learning.
Last edited:
strangecosmos
Non-Member
Training on 1 billion hashtag-labelled images, Facebook classified images with about 2 percentage points higher accuracy than others have accomplished training on 1 million hand labelled images.
I don’t know the specifics of how you would try to use both kinds of images. I guess you give weakly labelled images 1/1000th as much weight? And then see how you do on your test set.
thegruf
Active Member
Sorry if I missed any of this upthread, but wasn't there a clue abouit "shadow mode" when Elon said (paraphrased) "every time there is a disengagement we upload the data ..."
So basically they dont "shadow" anything (which I belived has bee nconfirmed elswhere) but just review disengagements; which is entirely valid methodology providing you accept it does not contain all relevant data for AP driving.
So if you have a regular place where AP messes up then disengage AP to force that snapshot to Tesla. Enough disengagements presumably force a fix for that location. I am thinkning his works much like the whitelisting of bridges etc for false braking.
I actually possibly have evidence of exactly this as there is one bit of road that AP regularly tries to drive you into the hedge and I regularly disengage (duh!) but in recent builds it is mysteriously fixed and it is really cool how well it works, but the fix is not explained by general AP improvements.
How this translates to "full" self driving i am not sure, but it suggests a strategy for managing edge cases that quite simply it is not possible to directly code for, at this stage of development
In turn though it also confirms the (obvious) that the software, AP3 hardware or not, is well short of holistically understanding the "full environment at this stage of development.
So basically they dont "shadow" anything (which I belived has bee nconfirmed elswhere) but just review disengagements; which is entirely valid methodology providing you accept it does not contain all relevant data for AP driving.
So if you have a regular place where AP messes up then disengage AP to force that snapshot to Tesla. Enough disengagements presumably force a fix for that location. I am thinkning his works much like the whitelisting of bridges etc for false braking.
I actually possibly have evidence of exactly this as there is one bit of road that AP regularly tries to drive you into the hedge and I regularly disengage (duh!) but in recent builds it is mysteriously fixed and it is really cool how well it works, but the fix is not explained by general AP improvements.
How this translates to "full" self driving i am not sure, but it suggests a strategy for managing edge cases that quite simply it is not possible to directly code for, at this stage of development
In turn though it also confirms the (obvious) that the software, AP3 hardware or not, is well short of holistically understanding the "full environment at this stage of development.
that was a lie, I covered it elsewhere (the said they upload enough data to see how to fix the disengagement which they don't, this before even getting into the whole not sall disengagements could be fixed at all)
My interpretation is that marketing (as in Elon) is glossing over the short term engineering objectives, and focusing on the art of selling. You sell by focusing on selling something tangible.
The reality of the situation though is they're going to have to make significant improvements to NoA, Auto-Lane Changes, etc in order to convince people to pay $8K for AP+FSD versus $3K for AP.
As it is right now everything under the AP package works pretty solidly, and everything under FSD either barely works or isn't there.
I'm eager for the update to NoA that does unconfirmed lane changes. Will know at that point where NoA stands in terms of usability. Right now it's really hampered by not having that. Its hard for me to tell what the road situation is like when I see a request from NoA. It is because I don't immediately see them and they're not in my visual path. By the time i've seen them the situation has changed.
It needs to have unconfirmed lane changes where it turns on the turn signal for a bit to allow me to double check and then allow it to continue. If it sucks I'll just turn off NoA, and wait for the next update to try again.
This won't happen, but I really wish Tesla had a development blog along with a bug tracker. If they're going to have the customer beta test then why not make it official with actual agile sprints?
thegruf
Active Member
strangecosmos
Non-Member
strangecosmos
Non-Member
What do you mean by the "mid-level"?
They used to upload the "location, heading, speed, type of disengagdment" kind of data, but that seems to have stopped now. Instead the aggregated statistics from autopilot-trip-log is the only thing uploaded now it seems.
strangecosmos
Non-Member
The mid-level representation is the predictions the perception neural network makes about what it sees. What's represented by bounding boxes, "green carpet", and text labels in your videos. I think you've called it metadata but the technical term in machine learning is mid-level representation.
oh, don't think they are not trying!
Usually none of that is captured, unless you have an active trigger that happened to match the conditions. But nowadays those triggers have 0.1% probability to capture anything even if the conditions DO match (I guess they don't want every one of those for whatever reasons).
So is most of their testing done on employee cars? Advanced Driver Assistance Systems (ADAS) Engineering Technician | Tesla
For the recent FSD development (I assume with HW3) they did a couple rounds (few hundred cars) of internal fleet sales/ signups.
I have little visibility into this. But it's pretty clear they have dedicated testing cars to collect detailed info about performance and whatnot.
I am just saying that he widely touted "billions of miles on AP2+ cars" actually don't mean as much when you consider the kind of data you get back from those.
This seems to happen a lot. The State-action-pairs of the Union has been post-poned.
Similar threads
