Andrej Karpathy - AI for Full-Self Driving (2020)

[mongo]

No. Trains are constricted to only stay on a single path and can't maneuver on their own. Waymo cars can maneuver independently on their own. Waymo cars are not constricted to just stay on a single path every time. The path on the HD map is simply to help the car make the maneuver more precisely. So for example, when the Waymo car approaches an intersection, it does not have to "guess" with vision alone how to make the turn but has map data to know how to make the turn accurately.

This response is saying it is not the same every time, which is what I would expect from a usable system. Your previous response indicated it was the same every time which I would call train like.

Either it is using a premapped percise predetermined path, or it is adjusting for the environment around a desired destination (the new lane) and preferred (not exact) route. The map cannot tell you how to make the turn if the scene has changed.

Example: I turn left from the inner of two left turn lanes onto a 6 lane road section (right, two straight, left, two in my direction). The right lane ends and people stop long in the oncoming left lane. There is no constant ideal path to take.
And that is without accidents.

Well they could have used active sonar to get distance to the undersea cliff. But active sonar works like lidar. Passive sonar (the equivalent of cameras) would not have worked in this scenarios.

Yeah, this is so off topic, but the underwater camera did a great job showing the relationship of Red October to the Neptune Massif.

 

[diplomat33]

This response is saying it is not the same every time, which is what I would expect from a usable system. Your previous response indicated it was the same every time which I would call train like.

Either it is using a premapped percise predetermined path, or it is adjusting for the environment around a desired destination (the new lane) and preferred (not exact) route. The map cannot tell you how to make the turn if the scene has changed.

Example: I turn left from the inner of two left turn lanes onto a 6 lane road section (right, two straight, left, two in my direction). The right lane ends and people stop long in the oncoming left lane. There is no constant ideal path to take.
And that is without accidents.

I think there are might be some miscommunication.

This is what I am talking about. This is a HD map that Waymo uses.

You will note that the HD map has road edges, lane lines, cross walks, traffic lights and paths traced out to help the car navigate this complex intersection. My point is that the car can use these paths on the HD map to help it make the turns successfully.

But the cars are not stuck on a predetermined path no matter what. They are are able to determine a new path to go around obstacles like construction or cyclists. Here is a video of Google self-driving cars able to navigate around obstacles like construction and cyclists. You will see that the cars are able to make a new path to go around obstacles. Keep in mind that this is from 2014. So this what Google self-driving cars were already able to do back then.

 

[willow_hiller]

But the cars are not stuck on a predetermined path no matter what. They are are able to determine a new path to go around obstacles like construction or cyclists.

My hypothetical question would be: what would a Waymo vehicle do if recent construction added new drivable space (unmapped), meanwhile the mapped drivable space was blocked. It's definitely able to deviate from the center of a lane to avoid obstacles within premapped drivable space, but would it ever drive on unmapped drivable space?

 

[diplomat33]

My hypothetical question would be: what would a Waymo vehicle do if recent construction added new drivable space (unmapped), meanwhile the mapped drivable space was blocked. It's definitely able to deviate from the center of a lane to avoid obstacles within premapped drivable space, but would it ever drive on unmapped drivable space?

I would imagine yes. Waymo must have camera vision to detect drivable space in unmapped areas. Wayne just uses mapped areas for maximum reliability because while camera vision can determine drivable space, having a mapped area in addition to the camera vision will make your driveable space detection even better.

 

[willow_hiller]

I would imagine yes. Waymo must have camera vision to detect drivable space in unmapped areas. Wayne just uses mapped areas for maximum reliability because while camera vision can determine drivable space, having a mapped area in addition to the camera vision will make your driveable space detection even better.

It's really hard to find evidence for this, but after some digging I think you are correct. From this Waymo patent US10203696B2 - Determining drivability of objects for autonomous vehicles - Google Patents

"In one example, the set of characteristics includes a location of the object, and the method further comprises prior to classifying, determining that the object was not included in pre-stored map information describing a driving environment of the vehicle at the location. In another example, the receiving of the sensor information occurs when the vehicle is approaching the object such that the classification and maneuvering are performed in real time. In another example, when the classification is not drivable, maneuvering the vehicle includes altering the expected future path of the vehicle to avoid driving over the object. In another example, the method also includes when an object is classified as not drivable, further classifying the object as not drivable but likely to move out of the way (or rather, out of the way of an expected future path of the vehicle)."

Talks about detecting drivable objects, not necessarily drivable space, but it's pretty close.

 

[Bladerskb]

It serves to show that your beloved project is a failure, for past few years that they declared it would work. Hopefully they can turn it around, but I'm dubious. A boondoggle of a project so far.

and your alternative is a company that proclaims they will have Level 5 this year, every year since 2015?

 

[willow_hiller]

Haha, okay, with a small amount of irony that same Waymo patent does actually seem to use the term "rails" for the path the car can follow:

"FIG. 2 is an example of map information 200 for a section of roadway including intersections 202 and 204. In this example, the map information 200 includes information identifying the shape, location, and other characteristics of lane lines 210, 212, 214, traffic signal lights 220, 222, crosswalks 230, sidewalks 240, stop signs 250, 252, and yield sign 260. Areas where the vehicle can drive may be associated with one or more rails 270, 272, and 274 which indicate the location and direction in which a vehicle should generally travel at various locations in the map information. For example, a vehicle may follow rail 270 when driving in the lane between lane lines 210 and 212, and may transition to rail 272 in order to make a right turn at intersection 204. Thereafter the vehicle may follow rail 274. Of course, given the number and nature of the rails only a few are depicted in map information 200 for simplicity and ease of understanding."

 

[diplomat33]

It serves to show that your beloved project is a failure, for past few years that they declared it would work. Hopefully they can turn it around, but I'm dubious. A boondoggle of a project so far.

This is laughable. Waymo actually achieved real L4 autonomy. There is nothing about Waymo that is a failure.

 

[diplomat33]

Haha, okay, with a small amount of irony that same Waymo patent does actually seem to use the term "rails" for the path the car can follow:

"FIG. 2 is an example of map information 200 for a section of roadway including intersections 202 and 204. In this example, the map information 200 includes information identifying the shape, location, and other characteristics of lane lines 210, 212, 214, traffic signal lights 220, 222, crosswalks 230, sidewalks 240, stop signs 250, 252, and yield sign 260. Areas where the vehicle can drive may be associated with one or more rails 270, 272, and 274 which indicate the location and direction in which a vehicle should generally travel at various locations in the map information. For example, a vehicle may follow rail 270 when driving in the lane between lane lines 210 and 212, and may transition to rail 272 in order to make a right turn at intersection 204. Thereafter the vehicle may follow rail 274. Of course, given the number and nature of the rails only a few are depicted in map information 200 for simplicity and ease of understanding."

The "rails" are just a reference to the paths on the HD maps that I referenced earlier.

Don't take it too literally. It's a metaphor. They are just virtual paths to help the car. As I indicated before, the cars can deviate from these virtual "rails" when they need to get around an obstacle.

 

[willow_hiller]

The "rails" are just a reference to the paths on the HD maps that I referenced earlier.

Don't take it too literally. They are NOT rails like what trains use. They are just paths to help the car.

As with most things in life, I think reality does lie somewhere in-between our two original opinions. We can think of them safety rails, if you prefer, instead of train tracks; because Waymo is implementing them to give their system more reliability instead of relying on sensor data alone. They are following rails, but they are also equipped to deviate from the rails when obstructions occur.

 

[Maitri982]

2022 at earliest for full self driving at earliest IMO. Elon completely missed the boat on how hard this was, and frankly has been full of it with his comments like summon being mind blowing, etc. Not only can't the cars full self drive anywhere, they cannot do it even on relatively easy highway conditions either. I need to watch my Model X very closely or I could end up pushing daisies.

First they need to get their software stack fleshed out completely. So letting NN "eat" as much as they think it should of the FSD problem. This will take probably at least remainder of this year. They then need to get dojo in place, which will take well into 2021. Then its more a rote exercise of lather rinse repeat training exercise with fleet data + dojo to train. In that latter part there will be releases to the fleet with lots of shadowing and more control given to the computer to see how it performs.

I also believe the ultimate neural network will be HUGE. Maybe 100 billion+ parms to do the job. So this explains why they are working on FSD computer 2 already...they may need it to handle the load. Also, they may find that other vehicle functions can use NN's too and the computer could handle those as well. We already have one case with Deep Rain, which also still needs to be better.

 

[Ludalicious]

I can't be the only one who think's Tesla can't pull this off without installing different cameras, Sensors or mapping system, computer ect.. They really don't sound too confident that it will work. I am a strong believer that it will not infact work. As someone who purchased FSD and have owned their vehicles for years. I really hope they can pull it off but realistically they are not even close. Let's not forget that Tesla claimed that my 2017 HW2.0 was capable of FSD. Look how far off they where then.

JM2C

 

[diplomat33]

I can't be the only one who think's Tesla can't pull this off without installing different cameras, Sensors or mapping system, computer ect..

You are not alone. I think Tesla needs better/more sensors and mapping to properly do safe, reliable L5 autonomous driving but I get voted down by the more vocal Tesla fans.

 

[Ludalicious]

You are not alone. I think Tesla needs better/more sensors and mapping to properly do safe, reliable L5 autonomous driving but I get voted down by the more vocal Tesla fans.

I am still a strong believer the vehicles will need to have communication with local traffic control systems. Add in a nice mapping system and you would have one heck of an operating system. I believe Tesla has recognized their flaws and thats why Space-X started deploying satellites.

 

[Maitri982]

You are not alone. I think Tesla needs better/more sensors and mapping to properly do safe, reliable L5 autonomous driving but I get voted down by the more vocal Tesla fans.

Don't sweat the downvotes. for some people any negative comment on Elon evokes what is akin to religious zealots defending their deity of choice. Elon is once in a generation guy...amazing really what he has done. I wish there were more of him in the world.

But...like all of us he makes mistakes in judgement, is flat out wrong about some things, and is often hyperbolic and he knows it too. But guys like him tend to filter out the noise (critics) to some extent and almost have superhuman belief in their own inner dialog over all else. This frankly is the only way sometimes you can do great things otherwise you would give up or wilt with all the critics out there. But this also instills legions of thoughtless followers who will go to war for him on literally anything they think slights him. don't mind such people.

Your comments have (lately) been mostly spot on about FSD. I frankly don't know that we need other sensors yet. HD maps for sure would be useful adjunct to current sensor suite. They are still at the "let's figure out our architecture and what will do what stage" of the project. But what I know and have seen about Deep NN's is that given proper data and enormous sized networks they can do amazing things.

Waymo is ahead of Tesla. End of story. They started earlier and have smart AI people. BUT...once the Tesla technical architecture is in place and can fully leverage all the training data we could see a vertical leap for Tesla instead of these minor increments we are seeing now. I look forward to that day.

 

[Bladerskb]

It is not a strength, it is hard coding around existing system weaknesses. If the car can't do it on its own, it is not extra reliability, it is the only 'reliability'.

Gosh you would hate airplanes. So much weaknesses disguised as strength and reliability.

 

[DanCar]

This is laughable. Waymo actually achieved real L4 autonomy. There is nothing about Waymo that is a failure.

Waymo said three years ago they would have driverless service the following year. They didn't, they failed. They said the same two years ago, they didn't they failed. I know Waymo knows they have failed, but they are optimistic they can turn it around. And in case you missed it, they said again last year they will have driverless service this year, but I'm willing to bet it won't be what we call success. Success to me means public debut without an employee in the car.

 

[DanCar]

and your alternative is a company that proclaims they will have Level 5 this year, every year since 2015?

I don't know from what perspective it is an alternative. I would say there is massive failure in the industry and there will continue to be so for next several years. I'm surprised Tesla hasn't been sued, for not delivering.

 

[diplomat33]

Waymo said three years ago they would have driverless people moving the following year. They didn't, they failed. They said the same two years ago, they didn't they failed. I know Waymo knows they have failed, but they are optimistic they can turn it around. And in case you missed it, they said again last year they will have driverless service this year, but I'm willing to bet it won't be what we call success. Success to me means public debut without an employee in the car.

What are you talking about? Waymo has driverless taxi service in Chandler, AZ.

 

[DanCar]

What are you talking about? Waymo has driverless taxi service in Chandler, AZ.

Have you succumbed to their marketing stunts? Can you go to Chandler and get driverless taxi service? No. If you are on their beta list, you get a $200K minivan with a safety driver. Very few times if you are one of the few beta customers, you get a driverless vehicle with an employee in the car. So no, they don't have driverless taxi service in Chandler available to the public without an employee in the car and my bet is they won't for several years. Yes, Waymo has had a few marketing video's of driverless taxi service in Chandler. I can do the same in my car, have a video without me driving, have the car do lane centering around a turn. That doesn't meet most people's definition of driverless.

From a technical standpoint, I would agree they have driverless, but from a business perspective, weighing in the risk of an accident, they don't.