For instance, I watched a 2018.10.4 test video where the driver found that the car could handle any curve except on 1) especially sharp curves where it 2) didn't slow down ahead of the curve. Do we have any indication of HD map tiles for sharp curves that could be used to tag these curves and give the car an indication to slow down?
What's happening with stop sign detection? Any insight on that?
How is Autopilot now able to recognize lanes without markings, or improvised lanes made of pylons? Is this based on recognizing driveable road surface and non-road surface (e.g. grass, sidewalk) or undriveable road surface (e.g. the oncoming lane)?
Does the new neural network architecture mean we'll see exponential progress toward full self-driving? Or is that just Elon and other Tesla execs overhyping the tech?
These are just examples. I'm interested to hear what other people find interesting or are pondering.
I don't have any data beyond what can be gleaned from the shape of the vision networks - which is really limited.
But you rang, so here are a few thoughts:
AP2 2018.10.4 is a lot better on curves than previous AP2 versions. A *lot*.
But if you go fast enough, if the map tile isn't perfect, if the visual conditions are bad enough, if the route is weird enough or tight enough, if the surrounding cars aren't behaving, or if it is used in situations that it isn't suited to - you will find places where it doesn't stay inside your comfort zone. That's going to continue to be true for quite a while. The probable path forward for EAP is that the broken cases get less and less frequent. But they don't get so infrequent that nobody is going to be able to post video of something weird. On 2018.10.4 I'm getting maybe one broken situation every several hundred miles of driving (to be sure I think my situation is on the easy end and others have it worse). Maybe that'll decrease by a factor of 3 per year at which rate I'd see one broken situation per year in maybe 2020. That's a guess - it's just to illustrate the idea that errors are likely to decrease following a conventional industrial learning curve.
As you cross certain thresholds new things become reasonable to implement. For example for on-ramp-to-off-ramp you probably need a bit better error rate than the current system to enable the majority of interchanges but probably not much more. Of course you also need the ability to track and verify your lane on a multi-lane highway and side/rear visibility to verify open space for lane switching and passing maneuvers. The improved lane change capability in 2018.10.4 suggests to me that they might be able to do on-to-off this year if they prioritize it. But at the same time that error rate doesn't get you to generalized L3. Maybe it gets you to L3 if you whitelist and put in place real time road status tracking infrastructure.
There's a whole bucket of smallish features that should be possible now but what happens depends on what they prioritize, which is hard to guess. It's weird to me that easter eggs make the cut for feature additions. I guess I'm just no fun.
That's EAP. FSD has to include substantial stuff that we are not seeing in EAP. For example there's no simple set of extensions to what's in EAP that lets you do that coast-to-coast drive we've heard about. So FSD is a separate enough effort that what we see with EAP may not be a good gauge for FSD progress. So maybe FSD features suddenly show up. I can't rule that out but I'd be shocked (though delighted) to see even a limited whitelist-only FSD this year given the perception and planning limitations that we see in EAP so far.
Similar to FSD, it's hard to have any visibility on what they are doing with stop signs, but the limitation is probably not that the vision network can't see the signs. I've heard that determining the spot the vehicle needs to stop is a big challenge because of variation in intersections and the need to stop within a few feet of the proper location. Well maintained high volume intersections in the U.S. have paint markings showing the stop point, but that's probably less than half of stop sign controlled intersections. Overseas (where half of Tesla vehicles are) there's even more variation. Apparently there are various things you can do to support this but the solution is piecemeal (maps, heuristics, lots of special rules) until the NNs are good enough to generalize from the context of the intersection with high accuracy. That vision solution might be a ways off yet so the question becomes at what point do they have the parts in place to do it the piecemeal way. That's another thing that we can't tell by looking at the current software.
Incidentally - 2018.10.4 didn't change the vision network architecture. The inputs and outputs were mostly the same with some small changes, and there were some layer count changes. The biggest single change was a 50% increase in the number of kernels used for the main/narrow cameras. Those changes are not insignificant and they suggest that training methods might have changed and, at a minimum, a lot more data is probably being used. But the network *architecture* is seeing evolutionary changes, not revolutionary ones.
As for lane boundaries other than paint markings - NNs have been pretty good at this for a long while. Common CNN architectures (including GoogLeNet) integrate whole-frame information into their outputs. So a more heuristic approach (which is more like what was happening in AP1) looks for things that have previously been identified as the characteristics of lane markers and then locates the lane based on the distribution of lane markers in the frame. But an NN can notice lots of other things too - the location of other vehicles, barriers, trees, buildings, curbs, the shape of the terrain, overhead signs and the texture of the ground adjacent to the road.
My sense of development for AP1 and for AP2 has been that of steady progress in improving the vehicle's situational awareness, with new features being brought on board as they become enabled by sufficient accuracy in perception. I expect that to continue for EAP this year. And I don't know what to expect for FSD other than that I think it'll to be a surprise whenever it does show up.
