Autonomous Car Progress

[daniel]

Based on the definition of L4: Cruise has FSD that works in an entire city with the human driver rarely touching the controls.

The Cruise fleet did over 800,000 miles completely autonomously last year in SF with the safety driver only taking over about once every 10,000 miles. For example, Cruise cars do what you see in this video for thousands of miles with the safety driver never touching the steering wheel or pedals or stalk.

It's time compressed but what you see in this video is the car doing all the driving and making all the decisions for 1 hour:

This is not just a demo of a one off feature. This is what Cruise cars can do with NO driver intervention for thousands of miles.

Yes, I am sure that Tesla will release new features and that is awesome. I can't wait. But like you said, Tesla's FSD will require driver supervision at first, especially in tricky city driving. Cruise can already do FSD in very difficult city driving without driver supervision for thousands of miles. That's the difference. That is why they are ahead of Tesla.

Hopefully you see the difference between releasing a feature like traffic light response where the driver needs to tap the stalk to continue and the car being able to handle city driving for an hour with no driver supervision like in the Cruise video.

Now you might ask: if Cruise's FSD is so great, why don't they release it already? And the answer is because Cruise does not consider full self-driving for 10,000 miles without driver supervision to be good enough. They want better before they release it to the public. There are still some things that trip up Cruise and they need to improve that.

Does that Cruise L4 car allow the driver to sleep in the back? IMO it's not Level 4 if the driver needs to be in the driver's seat. And it's not even Level 3 if the driver needs to take over without being alerted by the car with enough advance warning for normal human reaction time. It might be a really really good Level 2, but it's only level 2 if the driver needs to remain alert to take over on her/his own discretion, even once a month.

That said, I don't know what that Cruise car is capable of. But if it's geofenced to somewhere that's not most of Maui including my area, then it's not useful to me.

 

[diplomat33]

CVPR 2020 Workshop is starting in a few minutes.

Here is the live stream:

Karpathy should be on around 6pm EST.

EDIT: You will need watch the video in youtube. The link won't play embedded in the post.

 

[Tikib2020]

That video was impressive and it's good evidence that Cruise are currently ahead of Tesla.

What it really shows in my opinion is that the naysayer's are wrong, it's going to be years not decades before autonomous vehicles are consistently better at driving than people.

 

[powertoold]

All the demo videos I've seen posted here are from well-lit days around 2-4pm. Are there any Cruise or Zoox videos from night driving?

 

[diplomat33]

All the demo videos I've seen posted here are from well-lit days around 2-4pm. Are there any Cruise or Zoox videos from night driving?

Yes. Here is a video from 2017 of Cruise driving autonomously at night for over an hour with zero disengagements.

 

[daniel]

CVPR 2020 Workshop is starting in a few minutes.

Here is the live stream:

Karpathy should be on around 6pm EST.

EDIT: You will need watch the video in youtube. The link won't play embedded in the post.

I don't have to patience to watch a long live stream, or the technical grounding to understand it. But thanks for posting. I'll look forward to somebody's summary later.

That video was impressive and it's good evidence that Cruise are currently ahead of Tesla.

What it really shows in my opinion is that the naysayer's are wrong, it's going to be years not decades before autonomous vehicles are consistently better at driving than people.

Yes! This!

Years, not decades and not months.

And in the meantime, in a Tesla, we can enjoy the most advanced driver-assist system available to consumers today.

 

[diplomat33]

I don't have to patience to watch a long live stream, or the technical grounding to understand it. But thanks for posting. I'll look forward to somebody's summary later.

I plan to do a summary of the Karpathy presentation when it happens. I started a separate thread for it here: CVPR 2020 Workshop on "Scalability in Autonomous Driving" LIVE DISCUSSION

 

[powertoold]

And in the meantime, in a Tesla, we can enjoy the most advanced driver-assist system available to consumers today.

Definitely begs the question, if these other companies are so close to achieving FSD, why is Tesla's driver assistance system still the best available (with no real competition in sight for next year or two).

 

[diplomat33]

Definitely begs the question, if these other companies are so close to achieving FSD, why is Tesla's driver assistance system still the best available (with no real competition in sight for next year or two).

There are some automakers that are trying to do limited L3 highway. But the only companies that are close to achieving actual full autonomy are Waymo and Cruise and they are not automakers.

 

[S4WRXTTCS]

Definitely begs the question, if these other companies are so close to achieving FSD, why is Tesla's driver assistance system still the best available (with no real competition in sight for next year or two).

But, what makes it the best?

If you had 10 different independent test of NoA of more than 100 miles you'd likely have at least a 50% failure rate. Where it failed badly enough for the human intervene.

I personally find NoA a complete embarrassment. These days I only use it to test it, and not to enjoy it.

It certainly has more theoretical capability than any other driver assistance system available today, but a lot of it simply doesn't work. Its often frustratingly inconsistent. Even simple things like Adaptive Cruise control doesn't work without occasional false braking. It's pretty sad that the Adaptive Cruise control in my Jeep Wrangler is smoother, and is way less prone to false braking than my FSD enabled Tesla.

 

[powertoold]

It's pretty sad that the Adaptive Cruise control in my Jeep Wrangler is smoother, and is way less prone to false braking than my FSD enabled Tesla.

Biggest difference between AP and other TACC is dealing with merging cars. Sure, you might get a smoother TACC with other cars, but that's because they aren't paying attention to as many "corner" cases.

 

[daniel]

Definitely begs the question, if these other companies are so close to achieving FSD, why is Tesla's driver assistance system still the best available (with no real competition in sight for next year or two).

Because There are two separate things going on: There are companies putting driver assist systems in cars; and there are companies working on developing cars that would drive themselves. In the first category, Tesla is (IMO) the leader. In the second category Tesla seems to be lagging behind.

But, what makes it the best?

If you had 10 different independent test of NoA of more than 100 miles you'd likely have at least a 50% failure rate. Where it failed badly enough for the human intervene.

I personally find NoA a complete embarrassment. These days I only use it to test it, and not to enjoy it.

It certainly has more theoretical capability than any other driver assistance system available today, but a lot of it simply doesn't work. Its often frustratingly inconsistent. Even simple things like Adaptive Cruise control doesn't work without occasional false braking. It's pretty sad that the Adaptive Cruise control in my Jeep Wrangler is smoother, and is way less prone to false braking than my FSD enabled Tesla.

There are no freeways in Maui, so I have no opportunities to try NoA. Phantom braking has getting much better for me (seldom happens anymore, and when it does it's gentler and momentary). Though I have EAP the only features I use are AP and TACC, and, occasionally, lane change. But those three things work beautifully for me. And they work almost everywhere I go. Including some roads where I don't use them because I feel it would be pushing it. The biggest problem for me is that our speed limits are set ridiculously low so that everybody drives 15 mph over the limit, so if there's a lot of traffic I can't use autosteer without being a nuisance. When the traffic load is light and people can easily pass me, I use it.

I don't think there's another car maker selling a car that will allow autosteer and TACC on Maui's roads. I think there's one that allows autosteer in creep-and-stop traffic on the freeway? Again, no freeways here.

I'm really interested to see if GM offers for sale to the public a Bolt with Level 3 or 4 autonomy that functions on all regular roads on Maui, city and highway. If they do, I might have to buy it. But I'd much rather that Tesla drop its in-house FSD development and lease the technology from whoever gets there first.

 

[diplomat33]

Karpathy is "senior director of AI". That sounds like a title promotion. I don't think he had the senior in the title before.

 

[diplomat33]

I don't have to patience to watch a long live stream, or the technical grounding to understand it. But thanks for posting. I'll look forward to somebody's summary later.

Karpathy's presentation was pretty much the exact same presentation he did before. But here is a quick summary as promised.

1) He confirmed that Tesla is working on speed limit signs and showed some examples from around the world, including from China and Japan. So Tesla is training their NN on speed limit signs from all over the world, not just the US.
2) He did mention that Tesla is now using more than the 48 NN mentioned in his previous presentation. So Tesla has added more NN.
3) He said that Tesla is using more NN to replace previous "software 1.0", including lane keeping, cut-ins, parked cars and intersections.
4) Tesla is building NN to predict the lanes of an intersection based on camera vision. The idea being that the car will be able to correctly predict in advance what an intersection will look and won't need to rely on HD maps to know what an intersection looks like.

Here is on example of Tesla's prediction NN. Top view is the camera vision. The bottom view is the NN predictions of where the lanes and dividers are:

But he admitted that this approach is very difficult because there are so many complex intersections that the NN has to figure out correctly. He gave an example of a challenging intersection for Tesla.

5) He also talked a bit about how Tesla does use standard maps with positions of traffic lights but not HD maps with cm level accuracy. He explained that Tesla does not use HD maps because they are costly to maintain, are not scalable and don't work if something changes. So he prefers the Tesla camera based approach because it is cheaper, scalable and the car will be able to navigate any intersection the first time it sees it. But he admitted the Tesla approach is a lot harder because training camera vision to understand all intersections reliably is super hard.
6) He also said that Tesla has a "needle in the haystack" problem. Basically Tesla gets this huge haystack of fleet data and most of the fleet data is not useful but they have to find the needle that is a useful edge case. He gave the example of a chair falling off the back of a truck as a rare edge case that is hard to find in all the fleet data.

You can find more stuff in the other thread.

 

[daniel]

Karpathy's presentation was pretty much the exact same presentation he did before. But here is a quick summary as promised. <...snip...>

Thanks for posting this.

 

[S4WRXTTCS]

Biggest difference between AP and other TACC is dealing with merging cars. Sure, you might get a smoother TACC with other cars, but that's because they aren't paying attention to as many "corner" cases.

In terms of false braking I agree with you in that it's really the result of Tesla trying to do more with it. One example of this is if you're about to pass a semi in the right lane it might false brake because it thinks the Semi is going to cross over even if the Semi is making no movement.

Ultimately what I compare it to is the smoothness that I experienced back in 2015/2016 with a Model S with AP1. Back then TACC wasn't nearly as prone to false brake, and was way smoother.

The difference is it's not just their own hardware/software not, but they've prioritized skittishness in the name of safety over usability. I'm sure they've cut down on the percentage of times it slams into stopped cars. But, they've also made it more difficult for those of us that pay attention. Where we have to be more on-guard to hit the accelerator when we feel the tell-tale sign of it false braking (which is typically fairly moderate braking).

With AP1 my biggest worries tended to be maintaining situational awareness, and being ready to take over in case of cut ins.

With AP2 the biggest worry tends to be keeping myself from embarrassment. The last time I used it NoA decided to turn on the right blinkers while in the right most lane when the lane widened due to a merging on-ramp from the right. It's never allowed me to get comfortable enough to worry about losing situational awareness.

Now keep in mind I know everything is in flux. Who knows what will be retained, and what will be thrown away. If anything is clear from the development so far neither does Tesla.

 

[diplomat33]

Interesting podcast interview with Waymo's Head of User experience, Ryan Powell:

Presentable #89: The Human Experience of Self-Driving Cars - Relay FM

He confirms that Waymo does have a ride hailing robotaxi service called Waymo One operating right now in Phoenix that services thousands of people and is mixed, ie some cars are driverless and some cars have safety drivers. It is a bit on pause because of the coronavirus pandemic but yes, the general public can hail a driverless Waymo robotaxi in Phoenix.

 

[daniel]

Interesting podcast interview with Waymo's Head of User experience, Ryan Powell:

Presentable #89: The Human Experience of Self-Driving Cars - Relay FM

He confirms that Waymo does have a ride hailing robotaxi service called Waymo One operating right now in Phoenix that services thousands of people and is mixed, ie some cars are driverless and some cars have safety drivers. It is a bit on pause because of the coronavirus pandemic but yes, the general public can hail a driverless Waymo robotaxi in Phoenix.

I visited Phoenix once. It's a huge, horrid, smoggy city. But it would almost be worth visiting just to ride in a driverless taxi.

 

[diplomat33]

Cruise just posted this 17 mn video from one of their presentations that discusses how Cruise uses machine learning to make path predictions. It's very informative:

 

[powertoold]

Cruise just posted this 17 mn video from one of their presentations that discusses how Cruise uses machine learning to make path predictions. It's very informative:

Just my intuition, but it seems a lot of these "small" scale FSD approaches are vulnerable to overfitting their models and driving policies.