Q3 Earnings Call: Questions about FSD

[Daniel in SD]

which actually makes it Level 2 no different from the current high-speed AutoPilot.

This is incorrect. SAE does not allow you to classify a Level 5 system as Level 2 simply because it requires a safety driver.
”The level of driving automation system feature corresponds to the feature’s production design intent ... As such, it is incorrect to classify a level 4 design-intended ADS feature equipped on a test vehicle as level 2 simply because on-road testing requires a test driver to supervise the feature while engaged, and to intervene if necessary to maintain safe operation.” - SAE J3016

My [f]ear is that “Driver Supervision” means hands on the wheel with nags and NOT just an alert when it needs help or sees a problem. The nagging on long commutes (me everyday) is quite annoying. I’ll take Level 3 on highways for a long time before anyone else if they would just get that part working.

A Level 3 system has never been in Tesla's plans. "Driver Supervision" will always mean that the driver is responsible. I do agree that a level 3 system for highways would be awesome.

Your interpretation is a perfectly reasonable way of saying Tesla lied on Autonomy Investor Day when they claimed they were aiming for Level 5 no geofence feature complete at the end of 2019.

All I am saying is that we have not yet seen confirmation of that. I agree it is possible and likely.

It's not a lie if you believe it! You're never going to be able to prove the Elon Musk didn't believe that it would be "level 5 no geofence" by the end of the year. If you believe in the magical power of neural nets then anything is possible. Now, if he said he thought Tesla could implement self driving the way Waymo and Cruise do it by the end of the year then that would be a lie.

 

[ZeApelido]

You are barking at the wrong tree. I am the one guy here who leaves open the possibility Tesla told the truth when they said on Autonomy Investor Day that they are aiming at Level 5 no geofence feature complete at end of 2019.

@diplomat33, @J1mbo etc seem to have already made up their mind that Tesla were not aiming at Level 5 no geofence feature complete at end of 2019, even though they said so to the investors and the public.

Wow. Is English your first language?

No offense but you need to get your reading and listening comprehension out of the toilet.

 

[electronblue]

Wow. Is English your first language?

No offense but you need to get your reading and listening comprehension out of the toilet.

Or maybe you need to develop a sense of irony?

 

[electronblue]

It's not a lie if you believe it! You're never going to be able to prove the Elon Musk didn't believe that it would be "level 5 no geofence" by the end of the year. If you believe in the magical power of neural nets then anything is possible. Now, if he said he thought Tesla could implement self driving the way Waymo and Cruise do it by the end of the year then that would be a lie.

I fully expect Tesla/Elon to continue soldiering on unimpeded on this. The discovery would certainly be interesting if it ever got to that — what was known internally vs. what was spoken externally. But no such discovery is likely to happen, so all we can judge is their actual progress.

But no amount of whitewashing changes the fact that Elon was asked about what he means by feature complete at the end of 2019 and he chose to confirm, at an investor event no less, that it means Level 5 no geofence. If he instead meant urban Level 2 with some limited traffic sign automation, he should have said that, not confirmed Level 5 no geofence as the feature completion benchmark.

 

[Mardak]

feature complete have widely accepted industry definitions

I'm not sure what industry you're in, but at least in software, a simple "provide your address" feature could be considered "complete" with an input box that accepts anything even without street numbers or city, etc.; so it can be improved with data verification but that gets complicated if supporting any address worldwide, which could get broken down into "support US addresses with numeric zip codes" then Canadian provinces and equivalent zip code then countries in Europe and Asia depending on the actual product requirements and urgency.

If we say this "address" feature is part of a "get customer information" project, technically it's feature complete if there's no additional input boxes to implement, but there can be improvements to the "phone number" feature for better verification and "birthday" feature for providing a calendar, etc.

So Elon Musk saying FSD is feature complete with traffic light and stop sign means everything else to reach "no supervision" is just improving things, e.g., lane keeping, pathing, object detection, signal accuracy.

 

[S4WRXTTCS]

If Tesla claimed at Autonomy Investor Day aiming to be ”Level 5 no geofence feature complete” by end of 2019 (which they did) when in actual fact they were aiming simply for ”Level 2 urban lane keeping with traffic light and stop sign stopping feature complete” (which some of you now assume, probably rightly so), that would have been a lie, as no reasonable observer would have assumed ”Level 5 feature complete” means actually a feature set that much smaller than required for even a prototype (non-reliable) Level 5... and it is not reasonable to assume Elon did not know what he was talking about.

I'm a little confused because I don't know anyone that was interpreting that we'd have unsupervised FSD by the end of 2019.

We simply expected NoA on city streets, and stop light/sign stopping. That was what was so scary about it, and hence everyone who was concerned about still requiring supervision and hands on the steering wheel. Where they felt like it was a terrible idea.

My expectation was that I'd be able to start it at my house, and supervise it all the way to my work.

As to SAE levels I interpreted as an L4 system being limited to an L2 system due to lack of regulatory approval. My understanding of L5 is that it can't be geofenced, and it can't be restricted by weather. Both of those pose huge issues with Tesla. I also don't see humans as being really capable of L5 either. I'm not an L5 driver, and I don't care to be.

 

[S4WRXTTCS]

I'm not sure what industry you're in, but at least in software, a simple "provide your address" feature could be considered "complete" with an input box that accepts anything even without street numbers or city, etc.; so it can be improved with data verification but that gets complicated if supporting any address worldwide, which could get broken down into "support US addresses with numeric zip codes" then Canadian provinces and equivalent zip code then countries in Europe and Asia depending on the actual product requirements and urgency.

If we say this "address" feature is part of a "get customer information" project, technically it's feature complete if there's no additional input boxes to implement, but there can be improvements to the "phone number" feature for better verification and "birthday" feature for providing a calendar, etc.

So Elon Musk saying FSD is feature complete with traffic light and stop sign means everything else to reach "no supervision" is just improving things, e.g., lane keeping, pathing, object detection, signal accuracy.

When it comes to Tesla "feature complete" is whatever the accountants will let them get away with.

 

[electronblue]

I'm not sure what industry you're in, but at least in software, a simple "provide your address" feature could be considered "complete" with an input box that accepts anything even without street numbers or city, etc.; so it can be improved with data verification but that gets complicated if supporting any address worldwide, which could get broken down into "support US addresses with numeric zip codes" then Canadian provinces and equivalent zip code then countries in Europe and Asia depending on the actual product requirements and urgency.

If we say this "address" feature is part of a "get customer information" project, technically it's feature complete if there's no additional input boxes to implement, but there can be improvements to the "phone number" feature for better verification and "birthday" feature for providing a calendar, etc.

So Elon Musk saying FSD is feature complete with traffic light and stop sign means everything else to reach "no supervision" is just improving things, e.g., lane keeping, pathing, object detection, signal accuracy.

At Autonomy Investor Day Tesla/Elon confirmed Level 5 feature complete was the aim for end of 2019. In the context that can only mean SAE Level 5 (full robotaxi basically) and feature complete of course has the usual software development definition.

You can not be Level 5 feature complete without a significantly larger array of features than simply urban lane keeping, stop sign recogition and traffic lights. That would not be Level 5 feature complete by any reasonable definition.

You can of course be Level 5 feature complete with unreliable features, but they can not be missing features.

 

[electronblue]

I'm a little confused because I don't know anyone that was interpreting that we'd have unsupervised FSD by the end of 2019.

We simply expected NoA on city streets, and stop light/sign stopping. That was what was so scary about it, and hence everyone who was concerned about still requiring supervision and hands on the steering wheel. Where they felt like it was a terrible idea.

My expectation was that I'd be able to start it at my house, and supervise it all the way to my work.

As to SAE levels I interpreted as an L4 system being limited to an L2 system due to lack of regulatory approval. My understanding of L5 is that it can't be geofenced, and it can't be restricted by weather. Both of those pose huge issues with Tesla. I also don't see humans as being really capable of L5 either. I'm not an L5 driver, and I don't care to be.

Your message does not seem to have anything to do with my message. I never talked about unsupervised at all, nor did I expect it (or expect it for years).

I talked about the thing Tesla said they were aiming for at the end of 2019: Level 5 no geofence feature complete. By definition feature complete can mean unreliable, so no supervision is of course out of the picture. But it can not mean missing features for Level 5 level of autonomy...

 

[S4WRXTTCS]

At Autonomy Investor Day Tesla/Elon confirmed Level 5 feature complete was the aim for end of 2019. In the context that can only mean SAE Level 5 (full robotaxi basically) and feature complete of course has the usual software development definition.

You can not be Level 5 feature complete without a significantly larger array of features than simply urban lane keeping, stop sign recogition and traffic lights. That would not be Level 5 feature complete by any reasonable definition.

You can of course be Level 5 feature complete with unreliable features, but they can not be missing features.

Did he actually say Level 5?

I'm asking because in all his tweets that I see it seems like he tries to avoid using the SAE definitions. Not only that, but the entire way he's rolling out FSD is to constantly add features to an L2 system which wasn't intended by the SAE definitions.

 

[electronblue]

Did he actually say Level 5?

I'm asking because in all his tweets that I see it seems like he tries to avoid using the SAE definitions. Not only that, but the entire way he's rolling out FSD is to constantly add features to an L2 system which wasn't intended by the SAE definitions.

He confirmed in an answer to a question:

Youtube has transcripts.
3:31:45
Colin Lang and UBS: "Just so we understand the definitions when you refer to feature complete self-driving it sounds like you're talking level 5, no geofence. Is that what's expected by the end of year?"
Elon Musk: "Yes."

I guess we should really compile a list of these ridiculous claims.

If it was a mistake, he could have offered a retraction any time...

 

[S4WRXTTCS]

He confirmed in an answer to a question:
If it was a mistake, he could have offered a retraction any time...

and, there you have your answer.

You know the sensor hardware that HW3 has is not capable of operating in adverse weather conditions. So by definition the intention can't be L5. In a lot of ways the SAE definition of L5 is a lie. Where it's an attempt at having a robot drive on a roads that include places/times where even humans struggle. The very fact that there is an L5 likely means were not working on infrastructure improvements like we should be.

Now sure Elon could have retracted, and better defined exactly what they mean by FSD. What we can reasonably expected from HW3.

But, they keep exaggerating and refuse to be realistic about it.

So they're pretty much always lying where they simply expect everyone to know that they're exaggerating.

Another way of looking at the issue is that they're lying, but it's only half the lie it was last time.

Maybe their way out is to lie only as half as much as the time before until the time which the lie and the truth aren't that far away.

It's only half the lie it was because on autonomy day it was their SW running. The previous time it was largely SW provided by NVidia.

 

[electronblue]

You know the sensor hardware that HW3 has is not capable of operating in adverse weather conditions. So by definition the intention can't be L5.

I don’t know that.

Back in 2016 Tesla said AP2 hardware (with just a possible computer swap) is Level 5 capable hardware. HW3 is that computer swap. If anyone doubts this, the conference call is on YouTube where they make this claim. Press also circulated the claim widely at the time.

In 2019 at the Autonomy Investor Day Tesla guided Level 5 no geofence feature complete on HW3 in 2019 — again expressing progress on the Level 5 claims.

Level 5 has consistently been Tesla’s story. No other SAE Levels, just Level 5. Summon from coast to coast, sleep in your car...

Now, I can of course wager that many (all?) of these were lies, but I don’t know that.

 

[croman]

Transcript: Elon Musk's Autopilot 2.0 Conference Call

"Basic news is that all cars exiting the factory have hardware necessary for Level 5 Autonomy so that’s in terms of Cameras, Compute Power, it’s in every car we make on the order 2,000 cars a week are shipping now with Level 5 literally meaning hardware capable of full self-driving for driver-less capability."

I spent a couple weeks gathering evidence so I've got it handy but I keep being too busy with my actual work to embark on a mostly fruitless quest to get Elon to stop lying. Product is good enough without all the ****ing sleaze.

 

[emmz0r]

Is it though?

It gets it's speed limits from OpenStreetMaps.

Source for this claim?

If you find a speed limit that isn't correct for a given road you simply update OpenStreetMaps. Then wait for it to eventually make it onto your car.

Doesn't work, tried months ago.

Now sure ideally Tesla would have sign reading, and would have some internal mechanism to update OSM maps themselves so the map data was constantly being updated. But, I wouldn't hold my breath on that happening.

I would even suggest using updating OSM for all the places you park at in order to prepare for smart summon. For when/if it ever arrives in Europe.

It's fundamentally unreliable because of tunnels, overpasses with different speed limits, roads under bridges, inaccurate GPS signals etc.

 

[Mardak]

You can of course be Level 5 feature complete with unreliable features, but they can not be missing features.

That's exactly what Elon Musk is suggesting as the target for end of this year feature complete. What feature do you think is missing? I would guess Tesla would consider anything you come up as a missing feature is just an improvement of something AutoPilot already handles (potentially unreliability).

For example passing double parked vehicles is better pathing. Stopping for school bus with flashing lights is better stop detection. Avoiding debris or ignoring plastic bags is better object detection. Yielding for pedestrians and bikes is better prediction.

 

[diplomat33]

For reference, here is the verbatim quote of what Elon said in the earnings call on FSD:

Feature complete:
"Yeah, feature-complete, I mean, it’s the car able to drive from one’s house to work, most likely without interventions. So it will still be supervised, but it will be able to drive — it will fill in the gap from low-speed autonomy with Summon. You’ve got high-speed autonomy on the highway, and intermediate speed autonomy, which really just means traffic lights and stop signs.

So feature-complete means it’s most likely able to do that without intervention, without human intervention, but it would still be supervised. And I’ve gone through this timeline before several times, but it is often misconstrued that there’s three major levels to autonomy. There’s the car being able to be autonomous, but requiring supervision and intervention at times. That’s feature complete. Then it doesn’t mean like every scenario, everywhere on earth, including ever corner case, it just means most of the time. And then, there’s another level which from a Tesla standpoint, we think the car is safe enough to be driven without supervision. Then the third level would be that regulators are also convinced that the car can be driven autonomously without supervision. Those are three different levels."

FSD timeline:
"While it’s going to be tight, it still does appear that we will be at least in limited early access release of a feature-complete Full Self-Driving feature this year. So, it’s not for sure, but it appears to be on track for at least an early access release of a fully functional Full Self-Driving by the end of this year."
https://electrek.co/2019/10/24/tesla-full-self-driving-timeline

 

[S4WRXTTCS]

Source for this claim?

Other people came before me, and did it.
OpenStreetMaps and Smart Summon

I'm still in the process of testing it for myself. I'll report back my results in that thread. It's probably going to take awhile though.

Now keep in mind there might be regional differences between the States, and Norway in terms of navigational mapping.

 

[electronblue]

At Autonomy Investor Day Tesla/Elon confirmed Level 5 feature complete was the aim for end of 2019. In the context that can only mean SAE Level 5 (full robotaxi basically) and feature complete of course has the usual software development definition.
...
You can of course be Level 5 feature complete with unreliable features, but they can not be missing features.

That's exactly what Elon Musk is suggesting as the target for end of this year feature complete. What feature do you think is missing? I would guess Tesla would consider anything you come up as a missing feature is just an improvement of something AutoPilot already handles (potentially unreliability).

For example passing double parked vehicles is better pathing. Stopping for school bus with flashing lights is better stop detection. Avoiding debris or ignoring plastic bags is better object detection. Yielding for pedestrians and bikes is better prediction.

If it were simply edge cases of driving policy you list, I might even be inclined to agree, but a Level 5 feature complete system must include for example wide traffic sign recognition. So far they only talked about stop signs and traffic lights. No way can you call something a Level 5 feature complete system if it can’t understand traffic signs.

One list to help get your mind around this that has been floating around is this. Now mind you Waymo is only aiming at Level 4 here, so Level 5 requirements would be even higher.

... from Waymo Safety Report 2018, Appendix A on page 36-37 ...

Set of Behavioral Competencies Recommended by NHTSA
1 Detect and Respond to Speed Limit Changes and Speed Advisories
2 Perform High-Speed Merge (e.g., Freeway)
3 Perform Low-Speed Merge
4 Move Out of the Travel Lane and Park (e.g., to the Shoulder for Minimal Risk)
5 Detect and Respond to Encroaching Oncoming Vehicles
6 Detect Passing and No Passing Zones and Perform Passing Maneuvers
7 Perform Car Following (Including Stop and Go)
8 Detect and Respond to Stopped Vehicles
9 Detect and Respond to Lane Changes
10 Detect and Respond to Static Obstacles in the Path of the Vehicle
11 Detect Traffic Signals and Stop/Yield Signs
12 Respond to Traffic Signals and Stop/Yield Signs
13 Navigate Intersections and Perform Turns
14 Navigate Roundabouts
15 Navigate a Parking Lot and Locate Spaces
16 Detect and Respond to Access Restrictions (One-Way, No Turn, Ramps, etc.)
17 Detect and Respond to Work Zones and People Directing Traffic in Unplanned or Planned Events
18 Make Appropriate Right-of-Way Decisions
19 Follow Local and State Driving Laws
20 Follow Police/First Responder Controlling Traffic (Overriding or Acting as Traffic Control Device)
21 Follow Construction Zone Workers Controlling Traffic Patterns (Slow/Stop Sign Holders)
22 Respond to Citizens Directing Traffic After a Crash
23 Detect and Respond to Temporary Traffic Control Devices
24 Detect and Respond to Emergency Vehicles
25 Yield for Law Enforcement, EMT, Fire, and Other Emergency Vehicles at Intersections, Junctions, and Other Traffic Controlled Situations
26 Yield to Pedestrians and Bicyclists at Intersections and Crosswalks
27 Provide Safe Distance From Vehicles, Pedestrians, Bicyclists on Side of the Road
28 Detect/Respond to Detours and/or Other Temporary Changes in Traffic Patterns

Examples of Additional Behavioral Competencies Tested by Waymo
29 Moving to a Minimum Risk Condition When Exiting the Travel Lane is Not Possible
30 Perform Lane Changes
31 Detect and Respond to Lead Vehicle
32 Detect and Respond to a Merging Vehicle
33 Detect and Respond to Pedestrians in Road (Not Walking Through Intersection or Crosswalk)
34 Provide Safe Distance from Bicyclists Traveling on Road (With or Without Bike Lane)
35 Detect and Respond to Animals
36 Detect and Respond to Motorcyclists
37 Detect and Respond to School Buses
38 Navigate Around Unexpected Road Closures (e.g. Lane, Intersection, etc.) 39 Navigate Railroad Crossings
40 Make Appropriate Reversing Maneuvers
41 Detect and Respond to Vehicle Control Loss (e.g. reduced road friction)
42 Detect and Respond to Conditions Involving Vehicle, System, or Component-Level Failures or Faults (e.g. power failure, sensing failure, sensing obstruction, computing failure, fault handling or response)
43 Detect and Respond to Unanticipated Weather or Lighting Conditions Outside of Vehicle’s Capability (e.g. rainstorm)
44 Detect and Respond to Unanticipated Lighting Conditions (e.g. power outages)
45 Detect and Respond to Non-Collision Safety Situations (e.g. vehicle doors ajar)
46 Detect and Respond to Faded or Missing Roadway Markings or Signage 47 Detect and Respond to Vehicles Parking in the Roadway

 

[electronblue]

For reference, here is the verbatim quote of what Elon said in the earnings call on FSD:

Feature complete:
"Yeah, feature-complete, I mean, it’s the car able to drive from one’s house to work, most likely without interventions. So it will still be supervised, but it will be able to drive — it will fill in the gap from low-speed autonomy with Summon. You’ve got high-speed autonomy on the highway, and intermediate speed autonomy, which really just means traffic lights and stop signs.

So feature-complete means it’s most likely able to do that without intervention, without human intervention, but it would still be supervised. And I’ve gone through this timeline before several times, but it is often misconstrued that there’s three major levels to autonomy. There’s the car being able to be autonomous, but requiring supervision and intervention at times. That’s feature complete. Then it doesn’t mean like every scenario, everywhere on earth, including ever corner case, it just means most of the time. And then, there’s another level which from a Tesla standpoint, we think the car is safe enough to be driven without supervision. Then the third level would be that regulators are also convinced that the car can be driven autonomously without supervision. Those are three different levels."

FSD timeline:
"While it’s going to be tight, it still does appear that we will be at least in limited early access release of a feature-complete Full Self-Driving feature this year. So, it’s not for sure, but it appears to be on track for at least an early access release of a fully functional Full Self-Driving by the end of this year."
Elon Musk: Tesla Full Self-Driving in early access this year, without supervision next year - Electrek

The more I read this... the more it sounds like all the talk from 2016 until a few days ago about Level 5, sleeping in your car, summoning from coast to coast, robotaxis somewhere next year... just can not have been true, not even aspirationally. I mean if this is how Musk really sees autonomy, he has no clue and Tesla is nowhere near it.

Therefore let’s hope the experssion on the latest earnings call was not actually representative of Tesla’s progress but that they are actually better in the details than Musk made it sound like.

Because Level 5 no geofence feature complete sure as heck ain’t NoA + Smart Summon + ”intermediate speed autonomy, which really just means traffic lights and stop signs”.