Level 5 nightmares

[Gregkeys]

What happens if during the AP AI training it learns a bad behavior.

e.g. let's say a certain road condition causes drivers to swerve recklessly and all teslas going through that road condition swerve so that the AI thinks that is normal and spreads the learning to all other teslas.

Also is the AI limited to by learning from people or does it have the capability to extrapolate and improve its techniques so that it realizes swerving is the wrong behavior and it should slow down instead.

 

[mblakele]

I don't think that will be a problem, because training won't be the only system at work. To see what I mean, watch this 2015 talk by Sterling Anderson. Anderson runs the AP program. Here he explains a safety system that calculates a "safe tube" through an environment. It intervenes whenever the vehicle is at risk of getting into trouble. This system was developed for human drivers, but I think it's likely that AP2 will include something like this as a safety check on its own driving. Maybe we'll also be able to enable it when we want to drive ourselves.

 

[Dithermaster]

The AP "learning" isn't about specific cases. All the data blends together in the neural network. So don't expect that because drivers swerve at particular piece of road that AP will learn that and mimic. Unless most driver would also swerve on other roads that look quite similar. Then it might!

 

[Gregkeys]

another interesting scenario, level 5 will make for easier repossession of the vehicle, imagine missing one payment and the car opens the garage and drives away.. ouch

 

[Gregkeys]

I wonder if FSDC will come with unlimited map updates, otherwise what happens if the car routes over a bridge that no longer exists?

 

[msnow]

I wonder if FSDC will come with unlimited map updates, otherwise what happens if the car routes over a bridge that no longer exists?

You're sunk.

 

[Adrien]

Greg,

I hope I don't come off condescending, because that's not my intention, but there is a lot you are missing here and some things that are not the way you envision.

1.) As someone mentioned single cases don't "teach" the neural net. Just like neural net refers to our brains and the capacity to increase connections based on reinforcement the same is going on here. The cars will still be hard wired to not break laws and such which actually could still cause issues. Example: going over yellow lines for a stopped delivery truck is an acceptable societal norm given you give right of way.

2.) Repo: Who cares...make your care payments

3.) High accuracy maps are important, but visual cues will override. More to the point, people here worried about potholes and said autonomy would destroy a car quickly in certain parts of the country. I think, they too, do not realize that even mobileye's system has centimeter level accuracy and already has the capacity to avoid such things.

I think it's best to understand from the experts and I hope these videos can help understand just a fraction of how these systems operate.

Great insight and nice visuals at the 28min mark:

Extremely good video from Google:

It's definitely an interesting topic, but don't think for a second me or anyone else on here that doesn't do this as a profession knows what they are talking about. I've been seeing on the forum recently the certainty of situations or functions that people believe will be an issue. It's kind of like that person that tries to disprove something that scientists have studied for decades with one sentence/paragraph. Hopefully these videos drive more questions, but all I'm saying is don't fall for assumptions....many people that are many times more brilliant are working on this 10 hours a day.

 

[WhiteCap]

another interesting scenario, level 5 will make for easier repossession of the vehicle, imagine missing one payment and the car opens the garage and drives away.. ouch

Not just that, Level 5 has lot's of other implications:

What if local LEO's issue a warrant for your arrest. Might Tesla respond to a summons (or equivalent but 'less constitutional' govt order) to have your car recognize when you enter it and program it to lock the door, take you into custody, and deliver you to the desired arrest location?

If the car is operating unattended, and someone else causes an accident, what will the protocols be for dealing with it when you are unavailable? Can/should LEO have the authority to commandeer an unattended vehicle for "evidentiary" purposes?

Given the state of property forfeiture laws in most states, do you want a local sheriff having the ability to order your unattended autonomous vehicle to pull over where he can then mysteriously find illegal substances, and in doing do so it becomes the property of his department?

What if the car learns to take evasive actions that include diverting off the roadway in order to avoid impact with pedestrians. What's to prevent people from shooting YouTube videos of them running Autonomous cars off the road?

Society is sufficiently immature that it's hard to be confident that this technology won't become a quagmire of issues. When Level-5 is ready, I suspect that I will probably be tinting my glass illegally dark just so observers can't tell if the car is occupied or not to help minimize the temptation to screw around with it.