TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

What's next for Autopilot?

Discussion in 'Autopilot & Autonomous/FSD' started by strangecosmos, Mar 29, 2018.

  1. strangecosmos

    strangecosmos Non-Member

    Joined:
    May 10, 2017
    Messages:
    1,039
    Location:
    The Prime Material Plane
    The 2018.10.4 update is getting everybody excited about Autopilot again. I'm wondering if @jimmy_d, @verygreen, or anyone else who has a deep technical understanding of how Autopilot is evolving cares to speculate on what we'll see next.

    For instance, I watched a 2018.10.4 test video where the driver found that the car could handle any curve except on 1) especially sharp curves where it 2) didn't slow down ahead of the curve. Do we have any indication of HD map tiles for sharp curves that could be used to tag these curves and give the car an indication to slow down?

    What's happening with stop sign detection? Any insight on that?

    How is Autopilot now able to recognize lanes without markings, or improvised lanes made of pylons? Is this based on recognizing driveable road surface and non-road surface (e.g. grass, sidewalk) or undriveable road surface (e.g. the oncoming lane)?

    Does the new neural network architecture mean we'll see exponential progress toward full self-driving? Or is that just Elon and other Tesla execs overhyping the tech?

    These are just examples. I'm interested to hear what other people find interesting or are pondering.
     
    • Like x 3
  2. Tam

    Tam Well-Known Member

    Joined:
    Nov 25, 2012
    Messages:
    7,859
    Location:
    Visalia, CA
    It's doubtful that it can handle winding roads just yet.

    That includes s-curve below



    I am no expert but curves can be handled by camera. It's just a matter of when Tesla will be able to.

    None. No speed sign recognition either.

    Without lane markings, it was like suicidal. It could not go through intersections that don't have lane markings.

    Current version is pretty good with most laneless intersections except for a very few (may be because they are just too wide to cross.)

    Most likely. Improvement just takes time.
     
    • Informative x 2
  3. jimmy_d

    jimmy_d Deep Learning Dork

    Joined:
    Jan 26, 2016
    Messages:
    416
    Location:
    San Francisco, CA
    I don't have any data beyond what can be gleaned from the shape of the vision networks - which is really limited.

    But you rang, so here are a few thoughts:

    AP2 2018.10.4 is a lot better on curves than previous AP2 versions. A *lot*.

    But if you go fast enough, if the map tile isn't perfect, if the visual conditions are bad enough, if the route is weird enough or tight enough, if the surrounding cars aren't behaving, or if it is used in situations that it isn't suited to - you will find places where it doesn't stay inside your comfort zone. That's going to continue to be true for quite a while. The probable path forward for EAP is that the broken cases get less and less frequent. But they don't get so infrequent that nobody is going to be able to post video of something weird. On 2018.10.4 I'm getting maybe one broken situation every several hundred miles of driving (to be sure I think my situation is on the easy end and others have it worse). Maybe that'll decrease by a factor of 3 per year at which rate I'd see one broken situation per year in maybe 2020. That's a guess - it's just to illustrate the idea that errors are likely to decrease following a conventional industrial learning curve.

    As you cross certain thresholds new things become reasonable to implement. For example for on-ramp-to-off-ramp you probably need a bit better error rate than the current system to enable the majority of interchanges but probably not much more. Of course you also need the ability to track and verify your lane on a multi-lane highway and side/rear visibility to verify open space for lane switching and passing maneuvers. The improved lane change capability in 2018.10.4 suggests to me that they might be able to do on-to-off this year if they prioritize it. But at the same time that error rate doesn't get you to generalized L3. Maybe it gets you to L3 if you whitelist and put in place real time road status tracking infrastructure.

    There's a whole bucket of smallish features that should be possible now but what happens depends on what they prioritize, which is hard to guess. It's weird to me that easter eggs make the cut for feature additions. I guess I'm just no fun.

    That's EAP. FSD has to include substantial stuff that we are not seeing in EAP. For example there's no simple set of extensions to what's in EAP that lets you do that coast-to-coast drive we've heard about. So FSD is a separate enough effort that what we see with EAP may not be a good gauge for FSD progress. So maybe FSD features suddenly show up. I can't rule that out but I'd be shocked (though delighted) to see even a limited whitelist-only FSD this year given the perception and planning limitations that we see in EAP so far.

    Similar to FSD, it's hard to have any visibility on what they are doing with stop signs, but the limitation is probably not that the vision network can't see the signs. I've heard that determining the spot the vehicle needs to stop is a big challenge because of variation in intersections and the need to stop within a few feet of the proper location. Well maintained high volume intersections in the U.S. have paint markings showing the stop point, but that's probably less than half of stop sign controlled intersections. Overseas (where half of Tesla vehicles are) there's even more variation. Apparently there are various things you can do to support this but the solution is piecemeal (maps, heuristics, lots of special rules) until the NNs are good enough to generalize from the context of the intersection with high accuracy. That vision solution might be a ways off yet so the question becomes at what point do they have the parts in place to do it the piecemeal way. That's another thing that we can't tell by looking at the current software.

    Incidentally - 2018.10.4 didn't change the vision network architecture. The inputs and outputs were mostly the same with some small changes, and there were some layer count changes. The biggest single change was a 50% increase in the number of kernels used for the main/narrow cameras. Those changes are not insignificant and they suggest that training methods might have changed and, at a minimum, a lot more data is probably being used. But the network *architecture* is seeing evolutionary changes, not revolutionary ones.

    As for lane boundaries other than paint markings - NNs have been pretty good at this for a long while. Common CNN architectures (including GoogLeNet) integrate whole-frame information into their outputs. So a more heuristic approach (which is more like what was happening in AP1) looks for things that have previously been identified as the characteristics of lane markers and then locates the lane based on the distribution of lane markers in the frame. But an NN can notice lots of other things too - the location of other vehicles, barriers, trees, buildings, curbs, the shape of the terrain, overhead signs and the texture of the ground adjacent to the road.

    My sense of development for AP1 and for AP2 has been that of steady progress in improving the vehicle's situational awareness, with new features being brought on board as they become enabled by sufficient accuracy in perception. I expect that to continue for EAP this year. And I don't know what to expect for FSD other than that I think it'll to be a surprise whenever it does show up.
     
    • Informative x 8
    • Helpful x 2
  4. Tam

    Tam Well-Known Member

    Joined:
    Nov 25, 2012
    Messages:
    7,859
    Location:
    Visalia, CA
    Ideally, on winding roads or bad curves, the speed should be controlled by Autopilot and not human.

    Autopilot has cameras just like human have eyes.

    If human eyes can't see something beyond a curve because it's too sharp, most would slow down. So can cameras and there is no reason Autopilot cannot.

    In the beginning AP2 did not slow down on a curve and it could be deadly if I did not disengage Autopilot and slowed down manually. Now, gradually it does automatically on many curves.

    It's just a matter of time that Tesla will perfect the art.
     
  5. WannabeOwner

    WannabeOwner Well-Known Member

    Joined:
    Nov 2, 2015
    Messages:
    5,761
    Location:
    Suffolk, UK
    I read that AI progress is exponential. I know how to describe Exponential to someone who doesn't "get it", but nonetheless I still have great difficulty comprehending it.

    If the number of "broken cases" halves at each milestone, then in 12 milestones it will decrease to 0.0002. Maybe a Milestone is each release, and maybe there will be one of those a month ...

    I see humans judging that badly too. I don't suppose I see someone stopping short a whole car length, or very rarely, but would it matter? Biggest issue is probably with human drivers who think the stopped-short driver is an idiot, and then do something unexpected like trying to occupy that space, or runnnig into the back of them, which is a whole new set of problems that AP brings to regular drivers.

    So does it matter where the car stops? (Yeah, before it causes an accident in cross traffic, and not too far away, but maybe that is achievable?)

    There is a T-junction near me, house owner on one corner has a hedge that makes it very hard to see around, so you have to pull out a bit before deciding the road is clear. Cameras up on the nose would have an earlier view than me from the cockpit, but ...

    We have lines in all sorts of places (on rural roads) that we never did a decade or two ago. Not sure why that is (but thanks anyway!), quite possibly because in doubles as a "no stopping" indication on main (non highway) roads. But seems to me that the highways authority being required to do an excellent job on road markings is a fair ask, going forwards. A decent line on a junction will help humans too ... perhaps Safety is why I've seen an increase in road markings over the decades, it can't have been for AP 20 years ago, heck I doubt they have woken up to the need for that even today :)
     
  6. jimmy_d

    jimmy_d Deep Learning Dork

    Joined:
    Jan 26, 2016
    Messages:
    416
    Location:
    San Francisco, CA
    The use of 'exponential' as a descriptive is easy to misunderstand because it's a poor fit to the linear ways that people naturally think about change. From a linear perspective exponential growth seems to go from negligible to overwhelming very quickly, so 'exponential' becomes a synonym for 'surprisingly fast' in the popular imagination. But of course it's only surprising to people who aren't paying close attention. Bug attrition rates in code and the performance improvement of industrial processes from coal mining to IC fabrication to agriculture are all well described as exponential. Coal's exponent is about 1.02 so people see it as negligible. IC technology's exponent is about 1.5 (Moore's Law) and people see that as dramatic. But they are similar in terms of the mathematics of their growth. Similarly progress in NN's - both individual projects like AP and overall capabilities like speech recognition - are well described as exponential. My observation of AP's development over the last few years fits this model so of course I expect that to continue. But that doesn't mean that from a feature perspective it'll be more of the same. As it crosses useful thresholds new capabilities will 'suddenly' be possible.

    Speech recognition has been around in software applications for decades and that whole time it was seeing exponential improvement. But it seems very recent because it has crossed accuracy thresholds in the last couple of years. Elon has described his expectation of AP progress as 'terrible... bad... ok... good... incredibly good!' which is a fair description of how people experience exponential growth with high exponents. I think the exponent for AP is probably around 2, which is pretty high.

    Without info from the guys that are testing the current stuff it's hard to know much but I think it's probably being judged just as you say - by human standards - where if you're off by more than a couple of feet you're seen as either stopping before you get to the intersection or driving into it without stopping. 3 feet is a hard tolerance to guarantee if you rely on maps and GPS - it'll be within a foot 90% of the time but off by 30 feet 1% of the time. And if you can't rely on maps and GPS then you really need the car's situational awareness to be good. GPS will be off maybe 1% of the time and perception might be off 1% of the time - but then you have to pick one since they don't agree. Most of the time the car will know which is the right one by looking at the GPS error ellipse estimate and the NN's confidence level but those things are occasionally wrong so you could choose badly 1% of the time that they disagree. And then you have the cases where they are both wrong. Each of these bad scenarios is maybe a 1 in 10,000 thing - but a fleet of Tesla's probably sees 10,000 stop signs every hour. That would make me really careful about deploying a stop sign feature.

    Yeah the marks are really useful and getting more common. Growing up I never saw them in rural Texas but living in a city in CA it's pretty rare to *not* see them. Uncontrolled intersections are probably more common than poorly controlled ones here. Which raises a whole other set of interesting problems, I guess.
     
    • Helpful x 1
    • Like x 1
  7. Bladerskb

    Bladerskb Senior Software Engineer

    Joined:
    Oct 24, 2016
    Messages:
    1,902
    Location:
    Michigan
    This recent crash proves yet again to those who stubbornly refuse to accept the fact that advancements in lane keeping and adaptive cruise control in every new update has nothing to do with the progress of tesla's self driving software.
     
  8. BigD0g

    BigD0g Active Member

    Joined:
    Jan 12, 2017
    Messages:
    2,018
    Location:
    Somewhere
    Next “real” feature / not incremental improvement will be blind spot monitoring i suspect.
     
    • Like x 4
  9. jimmy_d

    jimmy_d Deep Learning Dork

    Joined:
    Jan 26, 2016
    Messages:
    416
    Location:
    San Francisco, CA
    2018.10.4 changed it's lane change behavior in a way that makes me wonder if they aren't using the repeaters to check the destination lane now.

    Any thoughts on that possibility?
     
  10. BigD0g

    BigD0g Active Member

    Joined:
    Jan 12, 2017
    Messages:
    2,018
    Location:
    Somewhere
    Oh sorry, I didn't see this, no sadly I think it's still ultrasonics based. The camera calibrations are still showing 0%, I apologize for the duplicate cross post.
     
  11. jimmy_d

    jimmy_d Deep Learning Dork

    Joined:
    Jan 26, 2016
    Messages:
    416
    Location:
    San Francisco, CA
    I think we simul-posted.

    I should try to cover a repeater with tape and see if it affects how the lane changing works.
     
    • Like x 1
  12. ChrML

    ChrML Member

    Joined:
    Feb 6, 2017
    Messages:
    685
    Location:
    Norway
    Impossible to tell for sure. But they should be focusing on making highway driving perfect first, so here are my guesses:

    1. General sign recognition. Start using it by acting on speed limit change signs. Seriously, this bothers me every day as the database they use is horrible. And this is really easy to implement.

    2. Blind spot / rear and reliable adjacent lane detection. To enable automatic lane changes when the car in front drives slower than your set speed.

    3. Reliable merging. Use this code and rear car detection for automatic on-ramp drive. Use navigation and adjacent lane detection for automatic off-ramp.

    All these are relatively easy to do.

    Then there's the harder task which probably will be mature much later with the FSD codebase:

    1. General vision mapping of drivable paths, obstacles, cars, bikes, peds. This is what can prevent disengagements where bad road markings lead you to a concrete wall. Or not braking for stand-still cars. Absolutely required for level 3 on highway.
     
    • Like x 1

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC