Elon: FSD Beta tweets

[Knightshade]

This does not seem to be the case to me. If a car can drive 10 billion miles between unsafe disengagements, I would consider it autonomous. If it can only drive 2 miles, probably not. So, at some point between 2 and 10 billion miles per disengagement, it becomes autonomous. What that number is, I'm not sure, but I do know that it exists.

It doesn't though.

I'd suggest you go read SAE J3016, and also any relevant state laws that do so (there's at least half a dozen US states that already permit self driving cars and contain such language).

L3 and above are consider Automated Driving Systems.

L0, L1, and L2 are not- at those levels a given PART of the driving task is automated, but the entire driving task is not.

The vehicle does not ever "drive autonomously" below L3, even though specific PARTS of driving might be automated.


At no point is "disengagement rate" a criteria for anything.

An L3, L4, or L5 system must be able to perform the entire dynamic driving task on a sustained basis. That is, it must be able to do a list of specific things without a human involved.

See attached for SAEs schematic of the dynamic driving task- specifically the DDT is the part in the shaded DTT area (so excludes destination and waypoint planning from that diagram)

L0 is just manual driving.

L1 systems do PART of the innermost loop- either lateral vehicle control or longitudinal vehicle control and a limited OEDR for that axis of motion control. (So JUST autosteer, or JUST TACC as examples. The human still needs to be there to do the rest of the driving task, including to constantly monitor the environment, and immediately take control if they see a situation the vehicle can not, or is not, correctly detecting, recognizing, or responding to.


L2 systems do all of the innermost loop... this is Teslas autopilot (and supercruise from GM and other less capable systems that combine autoteer and TACC type functions). They still have a limited OEDR associated with such motion control. The human still needs to be there to constantly monitor the environment, and immediately take control if they see a situation the vehicle can not, or is not, correctly detecting, recognizing, or responding to. Even if it's very very rare such situations come up.

L3-L5 systems require all of the L2 stuff and a complete OEDR. That is the car can do the entire task of monitoring the driving environment, and responding appropriately.


The lack of a complete OEDR (and the fact Tesla has explicitly said they have no intention of having a complete one when FSDBeta goes to full production release) is what tells you the system can not be L3 or higher.



SAE makes a point of the fact that it is human is not performing any of the OEDR task as the single cutoff distinction between being an automated driving system or not.

See chart below.

If it does not have a complete ODER is it not an automated driving system. By definition.

Tesla themselves is explicitly telling you (and anyone actually using the beta can confirm) FSDBeta lacks (and will even when fully released lack) a complete OEDR.

It lacks the ability to know what it doesn't know and handle that appropriately and safely.

Thus the human is still required to be on the lookout, at all times, for such things and immediately intervene.

And thus is it not, and never will be, a ADS.


Some future system, or at least this is Teslas stated intent- which is not FSDBeta, and won't ever be part of it- will offer a complete OEDR, and no longer require constant human monitoring.

 

[Knightshade]

Im not however convinced that the camera placement and count is wrong. An interesting experiment would be having an 8 monitor setup with all camera feeds and seeing if if was possible to drive the car from that (Not that anyone is going to do this!). I reckon it would be possible, although more cameras in the future could further improve performance.

Some beta folks have already done video on B-pillar limitations-

Here's one from Chuck Cook for example:

He shows you the B-pillar camera location view, and what the view from the headlight area would be by mounting another camera there (where I suggest it could use one).

Visibility difference for obstructed views are pretty significant.

First good example at 1:17 traffic oncoming from the left is visible on the headlight camera... it doesn't appear in the B-pillar one until it's MUCH closer to the intersection, 3 seconds later (which assuming he's doing only the speed limit there is about 200 feet!)-- in fact the car had already just begun creeping into the intersection because the B-pillar couldn't see well enough just before this car enters the B-pillar view.

At 1:21 it's still creeping- and the headlight cam already sees another car coming from the left... that the B-pillar can't see and does not see for another 5 seconds.


Jump to 3:57 for the next good example... there's a tree and branches blocking the B-pillar cam pretty badly compared to the headlight one... the headlight cam shows an oncoming car--- the B-pillar cam can't see it at all yet... it doesn't appear on the B-pillar cam for again another 3 seconds, or ~200 feet (if the guy was not speeding).

Jump to 6:55 for another good solid-fence example... the headlight cam can already see an oncoming truck just entering frame... the B-pillar doesn't see it until 6:58.

There's a few more examples of partial-obstruction examples with trees and railed fences--- those MIGHT be fixable to some degree with NN work... solid blockage not so much though.



My own personal experience on the beta is it has creeped unsafely far into intersections trying to gain visibility before deciding to go (though I was monitoring of course, and since my head was well in front of the B-pillar cam I could tell nobody was coming... or in cases where someone was I'd disengage and hit the brake before it crept unsafely far).

A side-facing cam mounted further forward would fix that.


And same issue backing 90-degrees into traffic of course, cross traffic in parking lots and folks who have to back out of a driveway onto an active road being real-life examples where that could be useful..... though this obviously comes up a lot less often than forward-facing intersections with obstructed side views.[/QUOTE]

 

[S4WRXTTCS]

I don't care about stupid levels.

"Semi-autonomous" is still "autonomous".

You can't possibility believe that as historically speaking we've both been opposed to levels.

You either have autonomous driving where the car takes responsibility or you don't. Having partial autonomy gives you freedom from requirements which total autonomy would have. Partial also frees you from the level of liability that you would otherwise have.

Tesla does not have autonomous driving therefor $0 of FSD revenue should count when being compared to revenue of companies with autonomous driving.

 

[S4WRXTTCS]

Hmmmm.... So why are people complaining about the Beta doing the worst thing at the wrong time if that's what we're supposed to expect?

Apologies, my sarcasm doesn't always translate well on text. I read people complaining that their car does something terrible on Beta and saying that FSD is terrible or not what was advertised. Since there's nothing you can do about it, except report it to Tesla, why all the negatives?

If I'm beta testing a video game, and my character runs off a cliff when I told it to go into the vendor's hut, I write up a bug report and submit it to the game developers. What would the point be of complaining on a forum and ranting about how terrible the game is? It's in beta...

On a whole I think the biggest problem beta testers have is the lack of customer engagement.

Tesla has no good system for bug reports and instead they only have the FSD email address where you have no idea how many people have reported identical bugs.

Tesla doesn't respond in any way that gives the tester a feeling of any importance of their input so lots of us including me don't report issues.

This isn't going so well for Tesla as the public is starting to notice the sheer amount of issues all the beta SW in a Tesla has. The lack of engagement seems to have driven customers to complain to the NHTSA about PB issues, and other customers have been abusing the FSD Beta so much that they should be kicked out. If Tesla had better customer engagement, and oversight they'd be able to get things straightened out a lot more quickly.

 

[S4WRXTTCS]

Some beta folks have already done video on B-pillar limitations-

Here's one from Chuck Cook for example:

He shows you the B-pillar camera location view, and what the view from the headlight area would be by mounting another camera there (where I suggest it could use one).

Visibility difference for obstructed views are pretty significant.

First good example at 1:17 traffic oncoming from the left is visible on the headlight camera... it doesn't appear in the B-pillar one until it's MUCH closer to the intersection, 3 seconds later (which assuming he's doing only the speed limit there is about 200 feet!)-- in fact the car had already just begun creeping into the intersection because the B-pillar couldn't see well enough just before this car enters the B-pillar view.

At 1:21 it's still creeping- and the headlight cam already sees another car coming from the left... that the B-pillar can't see and does not see for another 5 seconds.


Jump to 3:57 for the next good example... there's a tree and branches blocking the B-pillar cam pretty badly compared to the headlight one... the headlight cam shows an oncoming car--- the B-pillar cam can't see it at all yet... it doesn't appear on the B-pillar cam for again another 3 seconds, or ~200 feet (if the guy was not speeding).

Jump to 6:55 for another good solid-fence example... the headlight cam can already see an oncoming truck just entering frame... the B-pillar doesn't see it until 6:58.

There's a few more examples of partial-obstruction examples with trees and railed fences--- those MIGHT be fixable to some degree with NN work... solid blockage not so much though.



My own personal experience on the beta is it has creeped unsafely far into intersections trying to gain visibility before deciding to go (though I was monitoring of course, and since my head was well in front of the B-pillar cam I could tell nobody was coming... or in cases where someone was I'd disengage and hit the brake before it crept unsafely far).

A side-facing cam mounted further forward would fix that.


And same issue backing 90-degrees into traffic of course, cross traffic in parking lots and folks who have to back out of a driveway onto an active road being real-life examples where that could be useful..... though this obviously comes up a lot less often than forward-facing intersections with obstructed side views.

[/QUOTE]

I fail to see how anyone with experience using the beta wouldn't come to the same conclusion that the Sensor Suite simply isn't there yet. That you need to have a 360 degree vision, and you need to be able to see moving obstacles that are close to the car (kids, animals, personal injury lawyers, etc).

There is also the weather issue. Anyone that drives in the rain should look at the rear facing side camera view to see if they could see anything out of them. Even in every day rain in the Seattle area the water drops that form over the sensor causes the image to bounce around so much that it can't be reliably used for lane changes. In some cases the SW is smart enough to know this, but in other cases it happily lets you use the auto-lane change feature.

I don't think anyone is too worried about compute capabilities as Tesla already has a track record of upgrading them. I think its a safe bet to say Tesla will upgrade HW3 computers to HW4. Some people like myself might take issue with the environmental waste of doing so when the car is so old.

 

[jabloomf1230]

If Tesla had better customer engagement, and oversight they'd be able to get things straightened out a lot more quickly.

I can't disagree with that, but keep in mind that Tesla supposedly has several thousand employees who also have FSD beta and they are probably covered under an NDA. It makes sense that Tesla might be worried about trade secrets and as a result, probably only the drivers testing under the NDA would have a better line of communication. I suspect that those drivers generate the primary bug list and then the reports from the rest of us are just used for confirmation. If the overall beta fleet reports something that the internal testers don't report then the significance of that bug is probably assigned by the prevalence of the report.

 

[S4WRXTTCS]

I can't disagree with that, but keep in mind that Tesla supposedly has several thousand employees who also have FSD beta and they are probably covered under an NDA. It makes sense that Tesla might be worried about trade secrets and as a result, probably only the drivers testing under the NDA would have a better line of communication. I suspect that those drivers generate the primary bug list and then the reports from the rest of us are just used for confirmation. If the overall beta fleet reports something that the internal testers don't report then the significance of that bug is probably assigned by the prevalence of the report.

They should route my email to a local employee who could do a quick reply of "I've added this to the list of issues I've been reporting"

If they have 50,000 FSD Beta testers, and 1K employees then the ratio would be 50:1 which wouldn't be too bad.

 

[S4WRXTTCS]

I can't disagree with that, but keep in mind that Tesla supposedly has several thousand employees who also have FSD beta and they are probably covered under an NDA. It makes sense that Tesla might be worried about trade secrets and as a result, probably only the drivers testing under the NDA would have a better line of communication. I suspect that those drivers generate the primary bug list and then the reports from the rest of us are just used for confirmation. If the overall beta fleet reports something that the internal testers don't report then the significance of that bug is probably assigned by the prevalence of the report.

I agree that this is likely how they do it.

But, they're missing some massive opportunities to fix things. A lot of bugs that impact my ownership experience aren't FSD Beta bugs themselves, but are maps/navigation related.

Having a proper bug reporting system or emails with regional representatives would go a long way to fixing problems before the media catches wind of them. It would also retain customers who bought a Tesla for a more direct line of communication with the manufacture. That there is a reason we wanted the Tesla eco-system.

We wanted something that would change with time, and that we could be a part of. That it's a computer on wheels, and SW will have bugs. The important aspect is those bugs get fixed quickly so they don't impact our ownership experience.

Tesla set my customer expectations back in 2015 with my first Tesla, and their failure to uphold those expectations with my second means that I'm moving to Rivian. I'm not moving quickly becomes Rivian has a long waiting list.

 

[Yelobird]

They should route my email to a local employee who could do a quick reply of "I've added this to the list of issues I've been reporting"

If they have 50,000 FSD Beta testers, and 1K employees then the ratio would be 50:1 which wouldn't be too bad.

So I understand, with 50k users likely averaging 5-10 button presses per Day you think the engineers should be replying to 500k emails per day or 3.5Million per week with “yeah, got that on the list thanks pal!” Lol. That reply would make you feel heard? And you feel That use of resources would move this along faster?? Totally don’t get it. That solution Might actually work with Rivian as they only have 200 cars out there. Never mind, those were all recalled. Lol

 

[S4WRXTTCS]

So I understand, with 50k users likely averaging 5-10 button presses per Day you think the engineers should be replying to 500k emails per day or 3.5Million per week with “yeah, got that on the list thanks pal!” Lol. That reply would make you feel heard? And you feel That use of resources would move this along faster?? Totally don’t get it. That solution Might actually work with Rivian as they only have 200 cars out there. Never mind, those were all recalled. Lol

The button presses don't generate an email. An email is generated by the owner of the vehicle, and the majority of owners aren't going to be emailing 5-10 times a day. That would be just nutty.

Secondly an engineer wouldn't be doing it, but I could certainly it being a nice work from home position where regional representatives could sort through all the emails where they did quick classifications of the issues types.

I'm not a strong supporter of bug reports though email as it's not an optimal way of doing it. They need a bug type tracker to help owners know what issues have already been reported where they can simply confirm it. You really want the users themselves to do most of the work in getting common issues, and complaints to the top.

I can report a map issue to Apple and they fix it within a couple of days, but Tesla has no mechanism for doing this. This section on TMC has a lot of smart people, and not a single one of them has figured out how to swiftly fix a Tesla map issue. They're trying to figure how to this out of sheer desperation because anyone who uses FSD Beta knows the map issues will ruin it.

The >50K beta tester is also a bit of a mistake because that puts too much burden on employees overseeing how its being used. It would have been better to limit it fewer users who contributed more signal than noise.

Long term they're going to need to implement much better customer engagement because all of the reoccurring revenue from FSD will be monthly subscribers. They're not going to want to continue to subscribe to something unless issues they encounter can get quickly fixed.

Tesla is the one that set my customer engagement expectation. I'm not going to get a Jeep because I don't feel like there is mechanism to fix the bugs, and other issues my Jeep has. With Tesla I used to be able to report things, and get feedback on its resolution.

 

[Knightshade]

FWIW, they don't need humans to sort through emails to find "common" issues at all if people use the report button consistently.

There's one specific T-intersection FSDBeta has always screwed up, in every version so far (though how it screws it up has changed)

I hit the report button every time.

If they sort reports by "repeatedly reported in same spot" they'd have common issues float right to the top without needing 1000 paid staff handling email.

And the issue itself (or issues since it's done the wrong thing several different ways over time) are obvious from looking at the reports auto-submitted with the button.


Individual emails should be for uncommon weird things that need a verbose explanation beyond the button report.

 

[2101Guy]

Didnt elon tweet that humans dont manually "fix" these? That the Dojo/Supercomputer/Gigacomputer would automatically process these reports using AI and ML and other hi tech means that sound great?

 

[Knightshade]

Didnt elon tweet that humans dont manually "fix" these? That the Dojo/Supercomputer/Gigacomputer would automatically process these reports using AI and ML and other hi tech means that sound great?

No, he did not tweet that.

 

[Dewg]

Tesla has simulators they run the AI through. Perhaps when you report the telemetry is sent to the simulation to see how it reacts, and if it reacts badly it's flagged for a human to review.

 

[Gilliland]

By the end of this year...

I'm sure the answer is "two weeks".

 

[EVNow]

It's really not though.

Nothing Tesla currently offers is capable of performing the entire dynamic driving task on its own. It always requires active human overnight.

So does "auto"pilot.

As I said earlier Waymo didn't start off as a "non autonomous" company and one fine day after they hit a magical disengagement rate become "autonomous" driving company.

Its a gray scale - not white & black.

In any case as I have said before - let us agree to disagree and not waste time navel gazing.

 

[Knightshade]

So does "auto"pilot.

Which also is not autonomous.

As I said earlier Waymo didn't start off as a "non autonomous" company and one fine day after they hit a magical disengagement rate become "autonomous" driving company.

No, they did not.

This is a nonsense claim you haven't offered any evidence of. The words you write don't even make sense in that order.

You appear to not understand what the difference is between a system that qualifies as an ADS and what does not

(Spoiler: Disengagement rate has nothing, at all, to do with it)

 

[Bouba]

To me it’s all about liability....when my car is responsible for all the mayhem it causes and not me...then it’s fully autonomous

 

[JulienW]

To me it’s all about liability....when my car is responsible for all the mayhem it causes and not me...then it’s fully autonomous

While the car may be seen as responsible you as the owner will still have to have the same insurance to cover any "mayhem". The good news is that it will be less expensive to cover as the accident rates go down.

 

[Daniel in SD]

While the car may be seen as responsible you as the owner will still have to have the same insurance to cover any "mayhem". The good news is that it will be less expensive to cover as the accident rates go down.

I don't understand how this would work. Presumably any "mayhem" the car causes would be characterized as a design defect and the manufacturer would be liable. Accident rates would go down but payouts would no longer be limited by the liability limits (plus the owners assets).