Full Self-Driving Capability

[DCGOO]

Once Tesla's software is able to detect objects in front and around the car, the biggest challenge will remain - interpreting that data to operate the car safer than a human driver.

The hard part is handling all of the unusual cases that come up during driving - complicated by the lack of standardization on the design of roads and signs, along with interactions with humans such as police, pedestrians, ... [Still concerned about how FSD will detect non-visual indicators, such as train horns or emergency vehicle sirens - for objects that are approaching but not yet visible...]

I don’t think covering this situation will be possible until ALL cars dynamically communicate with each other on the roadway. In this case, emergency vehicles can “clear a path” dynamically, far ahead of their current location. That is a long way off, I am afraid. Tesla’s efforts are baby steps in the grand scheme of things.

 

[ChrML]

Sirenes can be picked up visually by the rear camera. Emergency vehicles rarely use sirenes anyway due to respect for the people living nearby, mainly just used inside tunnels with limited visibility and if some tard in front never looks in the rear view mirror.

When sirenes detected, slow down and keep to furthest right of drivable path within the road at the first opportunity with wide enough road that it can pass. Not really hard logic.

Like I've said earlier, the main challenge reliable vision and classification of objects.

 

[Peteski]

I don't think decisions is the hard part once you have all the data about your environment. Perhaps the most complex decisions to be made is where to drive in complex intersections that may have bad signs and road marking. Most driving decisions require no AI at all.

The rest is a matter of priority of where to drive given the data you have. Eg avoiding death > avoiding damage to car/rims/property (eg pothole) > keeping the car on a legal path > keeping the car on a correct path > keeping the car on a comfortable path.

What is a major challenge is vision and classifying the drivable areas into the metadata needed to make good decisions. Eg how do you tell a windshield bug from a pothole in the road?

I think there are a lot of nuances to driving that would be hard to capture. Things that experienced drivers take for granted, like identifying other road users around them and classifying them accordingly e.g. the aggressive teenager in a blinged up hatch, the impatient exec in the Audi A5, the doddering old granny who can barely see over the steering wheel. With experience you can pretty much predict what's going to happen next in every scenario and take the appropriate action. Obviously this will be much less of an issue when all cars are automated, but right now any automated car will be in a vast sea of unpredictable human drivers. Imagine an automated car trying to navigate through Paris rush hour or any Italian city! It's just not going to happen any time soon.

 

[chillaban]

I think there are a lot of nuances to driving that would be hard to capture. Things that experienced drivers take for granted, like identifying other road users around them and classifying them accordingly e.g. the aggressive teenager in a blinged up hatch, the impatient exec in the Audi A5, the doddering old granny who can barely see over the steering wheel. With experience you can pretty much predict what's going to happen next in every scenario and take the appropriate action. Obviously this will be much less of an issue when all cars are automated, but right now any automated car will be in a vast sea of unpredictable human drivers. Imagine an automated car trying to navigate through Paris rush hour or any Italian city! It's just not going to happen any time soon.

Human prediction can also just be a necessary evil due to our inability to monitor everything at once. We can’t even check for a safe lane change without taking our attention off forward distance regulation for around a second. So many excuses for accidents boil down to “I did not expect the other person to _____”

There is definitely some software complexity needed to predict an immediate travel path for cars of interest but I think human intuition for aggressive driving is overrated. The far bigger problem I think is dealing with the navigation and lane/traffic signal localization issues.

 

[ChrML]

I think there are a lot of nuances to driving that would be hard to capture. Things that experienced drivers take for granted, like identifying other road users around them and classifying them accordingly e.g. the aggressive teenager in a blinged up hatch, the impatient exec in the Audi A5, the doddering old granny who can barely see over the steering wheel. With experience you can pretty much predict what's going to happen next in every scenario and take the appropriate action. Obviously this will be much less of an issue when all cars are automated, but right now any automated car will be in a vast sea of unpredictable human drivers. Imagine an automated car trying to navigate through Paris rush hour or any Italian city! It's just not going to happen any time soon.

I don't think the first FSD vehicles are going to be perfect neither. But it should be able to predict the possible paths of moving vehicles around, and keep decent speed/distance to them in order to be able to act on unexpected behaviour with low probability of car damage and zero probability of person injury. It would be the other vehicle's fault.

In other words, it will treat all other cars as A5 teenagers, then there's a matter of balancing a decent amount of carefulness, and not being overly careful trying to account for every situation physically possible (meaning you pretty much cannot drive).

You can only start being overly careful to manual human drivers once a majority 95%+ of the vehicles are autonomous. Only then near zero accidents is possible. Obviously this is many many decades to, but you gotta start somewhere to get there eventually.

 

[Carl]

I think there are a lot of nuances to driving that would be hard to capture. Things that experienced drivers take for granted, like identifying other road users around them and classifying them accordingly e.g. the aggressive teenager in a blinged up hatch, the impatient exec in the Audi A5, the doddering old granny who can barely see over the steering wheel. With experience you can pretty much predict what's going to happen next in every scenario and take the appropriate action. Obviously this will be much less of an issue when all cars are automated, but right now any automated car will be in a vast sea of unpredictable human drivers. Imagine an automated car trying to navigate through Paris rush hour or any Italian city! It's just not going to happen any time soon.

That is so true. It's about seeing that the driver at an intersection is obviously fighting with someone over his hand-held phone and will never think of granting you your priority because he's into something else (happened to me yesterday), plus all other examples you mentioned!

And BTW even at a much less 'psychological' level, it's still about differentiating, in the middle of your lane in front of you, a plastic bag from a football, from a dead cat, from a living cat, from a newborn baby, etc., and reacting accordingly. AP not there yet and that's obviously the first big hurdle to take.

 

[Peteski]

There is definitely some software complexity needed to predict an immediate travel path for cars of interest but I think human intuition for aggressive driving is overrated. The far bigger problem I think is dealing with the navigation and lane/traffic signal localization issues.

I'm not convinced. While I appreciate AP as a driving aid, it's pretty dumb when it comes to anticipating other vehicle movement. Pretty much every time I can react well before AP does. So if we're going from current AP to FSD, it's going to have to get a LOT better in its awareness of other vehicles or it will be carnage on the roads. True FSD still seems like a decade or more away to me except in carefully controlled environments.

 

[chillaban]

I'm not convinced. While I appreciate AP as a driving aid, it's pretty dumb when it comes to anticipating other vehicle movement. Pretty much every time I can react well before AP does. So if we're going from current AP to FSD, it's going to have to get a LOT better in its awareness of other vehicles or it will be carnage on the roads. True FSD still seems like a decade or more away to me except in carefully controlled environments.

I’m absolutely not saying that AP in its current state is in any way ready for challenging urban driving. I'm just saying that human intuition is not as important for performing well in this environment. It's a lot more about utilizing and parsing the cameras correctly to understand drivable path as well as unsafe nearby driver behavior…. and that's something the current Autosteer/TACC are not very capable of doing currently.

 

[Peteski]

I’m absolutely not saying that AP in its current state is in any way ready for challenging urban driving. I'm just saying that human intuition is not as important for performing well in this environment. It's a lot more about utilizing and parsing the cameras correctly to understand drivable path as well as unsafe nearby driver behavior…. and that's something the current Autosteer/TACC are not very capable of doing currently.

Having given this a bit of casual thought lately, I'm not so sure. I'm sure a lot of the reason why I've managed to drive many hundreds of thousands of miles over the last 3 decades without a scratch is because of my experience and intuition. I had one small accident in the first year after I passed my test, but I learnt from that mistake and didn't do it again. Now maybe some of my experience can be effectively built into the AI, but it's going to be a monumental task.

For now I'm happy with semi-autonomous systems that compensate for human traits like tiredness and distraction. The current version of AP2 I think is excellent for long motorway trips and makes driving safer. But we can agree it's nothing like FSD capable yet. I don't think we are too far away from automation in a simple motorway setting, but the urban challenge in a busy European city is in a different league entirely!

 

[RDoc]

I'm pretty sure that human drivers make a model of what's going on around them with "annotations" such as flaky drivers, pedestrians that look like they are approaching a cross walk, a car that may be slowing down to make a left turn etc. Without that kind of model everything would be reliant on reacting to things that have already happened, which sounds very problematic to me.

While I firmly believe that software could be built to do something analogous, I'm extremely doubtful it will happen anytime soon.

 

[chillaban]

I can't see that programming these stereotypes into the system would make it any better. Driving extra precautios around these implies that you take the necessary precautions here, but drive extra careless around drivers you assume to be safe. It's better to drive within acceptable risk around all drivers, eg Risk * Consequence lower than a certain threshold all over. Overly careful implies no driving at all, as with manual drivers around there is always a risk.

I agree, driving is NOT rocket science even though some humans try to turn it into an elaborate form of behavioral / gender / race / car type profiling. I think humans need intuitions like that because of our single-tasking attention span and single set of eyes with the ability to focus basically on one thing in our central vision at once. However, automation doesn't have that problem.

Safe driving is a matter of consistent safe distance regulation and moving to where you need to be.

 

[dgatwood]

There has been discussion about 2 independent software stacks: one for EAP, and another for FSD. I've never been convinced of that. I think that with all the effort in EAP, it would be a shame to throw that out and start fresh with FSD. It would make much more sense to take the best of EAP and add enhancements and features while progressing towards FSD.

They already dumped the old EAP stack. That was the 2018.10 update, IIRC.

But I have to say... one feature that would significantly "enhance" EAP, and would be absolutely essential for FSD, would be the ability to avoid slamming into things like stationary fire trucks. If FSD was really making great strides, why wouldn't they break off this little gem and add it to EAP, to save Tesla's public image, and maybe a life?

You're assuming adequate hardware performance. Being able to run the self-driving models in a simulation doesn't necessarily translate to running them on actual hardware at highway speeds.

 

[pers1]

I`ve opted for FSD in the Model X that I ordered this week, based on what Elon said on the Q1 earnings call this year.
That the FSD computer would in all posebilities have to be upgraded and for us FSD that would be free of charge - my thought was then I would have an even faster better computer also for just the EAP as well.

look at this article:
Tesla could have to offer computer retrofits to all Autopilot 2.0 and 2.5 cars by the end of next year
Tesla could have to offer computer retrofits to all Autopilot 2.0 and 2.5 cars by the end of next year

 

[pers1]

Tesla (TSLA) Q1 2018 Results - Earnings Call Transcript
May. 3.18 | About: Tesla, Inc. (TSLA)
Tesla (TSLA) Q1 2018 Results - Earnings Call Transcript | Seeking Alpha

 

[Peteski]

I agree, driving is NOT rocket science even though some humans try to turn it into an elaborate form of behavioral / gender / race / car type profiling. I think humans need intuitions like that because of our single-tasking attention span and single set of eyes with the ability to focus basically on one thing in our central vision at once. However, automation doesn't have that problem.

Safe driving is a matter of consistent safe distance regulation and moving to where you need to be.

Automation will have that problem while there are still human drivers around. We've seen how AP can't even cope with other cars suddenly cutting in front of it. If it was easy, automated driving would be a real thing by now.

 

[_jmk]

Tesla (TSLA) Q1 2018 Results - Earnings Call Transcript

Elon is a dreamer, overly optimistic in this regard. Things will eventually work but not anywhere near this fast.

 

[Barklikeadog]

I`ve opted for FSD in the Model X that I ordered this week, based on what Elon said on the Q1 earnings call this year.
That the FSD computer would in all posebilities have to be upgraded and for us FSD that would be free of charge - my thought was then I would have an even faster better computer also for just the EAP as well.

look at this article:
Tesla could have to offer computer retrofits to all Autopilot 2.0 and 2.5 cars by the end of next year
Tesla could have to offer computer retrofits to all Autopilot 2.0 and 2.5 cars by the end of next year

"based on what Elon said" should never be used to make a responsible financial decision.

He also repeatedly said that the hardware is already in the vehicles ready to go for FSD...

 

[bob_p]

I agree, driving is NOT rocket science even though some humans try to turn it into an elaborate form of behavioral / gender / race / car type profiling. I think humans need intuitions like that because of our single-tasking attention span and single set of eyes with the ability to focus basically on one thing in our central vision at once. However, automation doesn't have that problem.

Safe driving is a matter of consistent safe distance regulation and moving to where you need to be.

The biggest challenges with FSD will not be with lane keeping and maintaining spacing relative to the vehicles around the car.

The biggest challenges will be in handling all of the unusual circumstances that come up during normal driving. There have been recent accidents involving cars that may have been operating under AutoSteer and AP did not prevent the car from hitting a stationary emergency vehicle.

Not only didn't the system detect the stationary vehicle, there probably isn't any detection yet of emergency vehicles. Many states have laws requiring you to move out of an adjacent lane or slow down when passing emergency vehicles parked along the road.

What about detection of smaller stationary objects - like railroad crossing bars or small objects (like a child or animal) in the street?

EAP and AP1 were primarily designed for limited access highways - which have a much simpler environment - and is primarily lane keeping.

Driving on other streets are much more difficult - and it's all the special cases that easily confuse new drivers (why young drivers pay more for insurance) and why some of the "full self driving" cars on the road today will only drive slowly and stop frequently whenever they detect anything unusual ahead.

That said - still included FSD in the Model X we hope to pick up in a few weeks...

 

[chillaban]

The biggest challenges with FSD will not be with lane keeping and maintaining spacing relative to the vehicles around the car.

The biggest challenges will be in handling all of the unusual circumstances that come up during normal driving. There have been recent accidents involving cars that may have been operating under AutoSteer and AP did not prevent the car from hitting a stationary emergency vehicle.

Not only didn't the system detect the stationary vehicle, there probably isn't any detection yet of emergency vehicles. Many states have laws requiring you to move out of an adjacent lane or slow down when passing emergency vehicles parked along the road.

What about detection of smaller stationary objects - like railroad crossing bars or small objects (like a child or animal) in the street?

EAP and AP1 were primarily designed for limited access highways - which have a much simpler environment - and is primarily lane keeping.

Driving on other streets are much more difficult - and it's all the special cases that easily confuse new drivers (why young drivers pay more for insurance) and why some of the "full self driving" cars on the road today will only drive slowly and stop frequently whenever they detect anything unusual ahead.

That said - still included FSD in the Model X we hope to pick up in a few weeks...

I agree with everything you said.

Regarding the system not detecting something right now, I wouldn't necessarily read into that a pessimistic way. Autosteer and TACC are both currently designed to be narrow-domain driver assist systems. Just because it's not within the scope of AP's current functionality doesn't necessarily mean there's anything particularly difficult about implementing it in the future.

There's definitely a lot of work to be done, but just a few months ago, the AP2 camp was whining about the car's inability to maintain center around mild curves, or suddenly jerk in some direction when hitting a bump. But an update out of the blue magically fixed most of that, and now we're off complaining about a different set of problems.


I also think the initial forms of L3/L4/L5 autonomy involve a lot more paranoid driving than most humans would consider tolerable. Heck, some of the press impressions about the Cruise Automation self-driving Bolts in San Francisco complain too about how the vehicle unnecessarily stops for nonthreatening things, sometimes upwards of 1 minute before a human takes over.

But right now, I think most of us are speculating. It's unlikely people have truly sat in many L4/L5 prototypes at this point to have experience to draw from, other than what another company's curated videos or marketing or carefully maintained DMV reports are trying to tell.

 

[bob_p]

It's likely Tesla's development group did some proof-of-concept tests verifying the sensors (with the necessary software) should be capable of detecting all of the objects required to implement full self-driving. The FSD video shows the objects being tracked by the software.

So what we are seeing now should be more likely a limitation in the software - they haven't gotten around to detecting stationary objects or emergency vehicles yet (which is more difficult than lane tracking).

We're not expecting to see FSD quickly, because handling all of the unusual cases is hard - and will take time. But we are expecting the software to start using all 8 cameras sooner, and even in driver assist mode the software should be better than AP1 and AP2 today.