FSD Beta 10.69

[EVNow]

I wonder if some of this is self selecting bias. I think anyone rational would have tried my test route once and just turned off FSDb entirely. It’s hilariously awful.

I bet if Omar does your route he will have fewer disengagements & interventions.

Unless your area map is really screwed up (unlikely given its LA), I doubt its going to be that bad if someone else does that route and pushes the limits a bit more. Not saying you should - don't do anything you are not comfortable and feel safe about.

 

[AlanSubie4Life]

Solving a harder problem (I assume you mean driver assist that people will pay $15k+ for) instead of solving a problem that will make FSD worth $100k. Makes sense.

This is a tough thing to tease out. If they make FSDb usable there would be too many accidents, and then it may get shut down, which would limit revenue in short term. They needed to release something so they could recognize revenue, though. So release something unusable and claim the revenue! ( 1. release FSD 2. profit )
In the meantime, they can continue to develop the system and perhaps improve safety while maintaining low mileage use.
So the design intent is to produce a system that allows recognition of revenue while minimizing risk to future strategies (and allows them to continue to sell the product!).

It’s not clear what the long-term strategy for maximizing revenue is, using existing hardware. They presumably know the robotaxi is unlikely with existing hardware.

That strategy is more nebulous to me and I have a hard time figuring out what path they might take. To me it seems that winning the race is unlikely with their current strategy. But if it is not possible for anyone to win the race in the next 5-10 years, then their current path may be optimal for maximizing short- and long-term revenue (and profit). They would be developing significant experience in delivering safe driver assist products over the long term.

If robotaxis are not feasible, the ideal case is to continue to sell $15k+ software packages which receive very little use, but look good in free marketing videos.

 

[EVNow]

So the design intent is to produce a system that allows recognition of revenue while minimizing risk to future strategies (and allows them to continue to sell the product!).

From when do you deal with conspiracy theories ?

 

[AlanSubie4Life]

From when do you deal with conspiracy theories ?

This to me does not seem conspiratorial! I definitely think Tesla wants to maximize revenue and profit.

They also want to make sure risks to the business are low.

These things are good for maximizing shareholder value.

Of course there are differences of opinion in how this is best accomplished.

But we have to look at the existing release and try to determine the reasons for there existing a product which satisfies the product description (hence revenue recognized), yet most people don’t use. Is this optimal? It may be!

 

[mtndrew1]

I bet if Omar does your route he will have fewer disengagements & interventions.

Unless your area map is really screwed up (unlikely given its LA), I doubt its going to be that bad if someone else does that route and pushes the limits a bit more. Not saying you should - don't do anything you are not comfortable and feel safe about.

Anyone is welcome to try the route; the Google maps link is in a tab in the tracking spreadsheet linked above.

Within one mile of my test origination point the car tries to drive through a block of clearly defined parking spaces (if cars are in them FSDb will sometimes go around but it usually freezes in the middle of the street indefinitely) then a few hundred yards later it comes to a complete stop in the middle of 1st street, not near any intersections, blocking traffic.

To be frank I allow FSDb to try and work things out, probably more than most people would. I disengaged during my route the other day when it tried to cross four lanes of traffic while a box van was occluding its view. I was nearly T-boned by a pickup.

FSDb is rough and the progress over the last 18 months has been very very limited. You can see it in my logs. The system is just very poor.

 

[AlanSubie4Life]

FSDb is rough and the progress over the last 18 months has been very very limited.

The progress in revenue recognition has been extremely substantial though. That pressure is substantially reduced now so further improvements in the feature-complete system will likely be driven by competition.

 

[mtndrew1]

The progress in revenue recognition has been extremely substantial though. That pressure is substantially reduced now so further improvements in the feature-complete system will likely be driven by competition.

After five years of waiting for FSD to materialize and 18 months of watching it stumble around city streets I’ve determined that what I truly want is something like supercruise. I want to zone out in numbing traffic for two hours per day.

Ever since Tesla focused on FSD, highway autopilot development has slowed to a crawl. At least for me the system has gone backwards on the highway. When radar disappeared autopilot’s performance in stop and go traffic became very uncomfortable. Accel-brake-ACCEL-BRAKE over and over.

Maybe single stack will be a leap forward. Maybe it will be another regression. But I can say one thing, 18 months of FSDb has given me no confidence in Tesla’s solution for autonomy.

I really bought into the hype (literally!) and that was foolish of me.

 

[EVNow]

progress over the last 18 months has been very very limited.

I completely disagree with this. Just FYI, I record and log all my drives and I've shared the improvements here. See the way my % of trips with zero disengagements has gone up over the releases.

Not saying your observations are not true for you - but let us not generalize.

 

[mtndrew1]

I completely disagree with this. Just FYI, I record and log all my drives and I've shared the improvements here. See the way my % of trips with zero disengagements has gone up over the releases.

Not saying your observations are not true for you - but let us not generalize.

View attachment 902969

Looks like you’re averaging almost 6 miles per disengagement! Huzzah.

We’re supposed to be chasing nines by now. We should be in the thousands of miles per disengagement neighborhood if FSD is to ever amount to anything resembling the original breathless promises.

At this rate of improvement you’re tracking to be at 30 miles per disengagement in another year and a half. I wonder if that will result in “the greatest increase in asset value in history.”

Do you have a Google link to a common sample route of yours? I’m curious to see what it takes to get six continuous miles of FSDb performance without intervention.

 

[KLam]

Looks like you’re averaging almost 6 miles per disengagement! Huzzah.

We’re supposed to be chasing nines by now. We should be in the thousands of miles per disengagement neighborhood if FSD is to ever amount to anything resembling the original breathless promises.

At this rate of improvement you’re tracking to be at 30 miles per disengagement in another year and a half. I wonder if that will result in “the greatest increase in asset value in history.”

Do you have a Google link to a common sample route of yours? I’m curious to see what it takes to get six continuous miles of FSDb performance without intervention.

I took a look at your test route. A great one to highlight all the issues of FSDbeta.

However, no one with the right mind will take approximately 30 turns (almost all alternating Left, Right, Left turns) to go from one place to another with that distance. May be a very good robot! But not FSDbeta.

 

[mtndrew1]

I took a look at your test route. A great one to highlight all the issues of FSDbeta.

However, no one with the right mind will take approximately 30 turns (almost all alternating Left, Right, Left turns) to go from one place to another with that distance. May be a very good robot! But not FSDbeta.

The test route is made up of seven distinct segments (see: “Segments” tab at this link) and they are totally typical trips through Los Angeles. There are no “gotcha” or unusual routes; they are very representative of day to day driving in the city. Each segment is recorded independently.

FSD, for the most part, has never been able to complete most of those individual segments and it has not meaningfully improved over the last 18 months. In fact FSDb is now regressing, as you can see in my logs.

Edit: I originally created these specific segments because they seemed like common hypothetical robotaxi ride requests. They’re all trips to various restaurants (chicken, in this case) around the city. I felt it would be common for someone to request an autonomous car, like they would an Uber, to head out for a meal.

 

[aronth5]

I bet if Omar does your route he will have fewer disengagements & interventions.

Unless your area map is really screwed up (unlikely given its LA), I doubt its going to be that bad if someone else does that route and pushes the limits a bit more. Not saying you should - don't do anything you are not comfortable and feel safe about.

I agree with this. The more I use FSD the more likely I will simply let FSD continue on it's own even though it may not be comfortable. It's always a decision based on how FSD will affect cars around me.

 

[AlanSubie4Life]

The test route is made up of seven distinct segments (see: “Segments” tab at this link) and they are totally typical trips through Los Angeles. There are no “gotcha” or unusual routes; they are very representative of day to day driving in the city. Each segment is recorded independently.

FSD, for the most part, has never been able to complete most of those individual segments and it has not meaningfully improved over the last 18 months. In fact FSDb is now regressing, as you can see in my logs.

Better application for FSD beta is wide well-marked suburban streets (parkway-like roads) with dedicated turn lanes and no changes in lane numbers (no merges or forks). Ideally travel in just a straight line with no turns and no hills. Light traffic with all green lights hopefully. This will result in fairly good performance, though you may have to use signals to cancel unnecessary lane changes.

The key is to avoid turning, stopping, and starting. And other vehicles. I have found the performance to be quite good in this domain.

 

[aronth5]

After five years of waiting for FSD to materialize and 18 months of watching it stumble around city streets I’ve determined that what I truly want is something like supercruise. I want to zone out in numbing traffic for two hours per day.

Ever since Tesla focused on FSD, highway autopilot development has slowed to a crawl. At least for me the system has gone backwards on the highway. When radar disappeared autopilot’s performance in stop and go traffic became very uncomfortable. Accel-brake-ACCEL-BRAKE over and over.

Interesting. For me highway traffic is never "Accel-brake-ACCEL-BRAKE over and over". Very smooth. I rarely disengage and with lane changing disabled I now rarely do any interventions except to manually initiate a lane change Only significant problem is merging onto the highway during rush hour. I wonder why our experiences are so different. I do have forward collision warning disabled but wouldn't think that would have any affect on how NoA drives.

 

[mtndrew1]

Better application for FSD beta is wide well-marked suburban streets (parkway-like roads) with dedicated turn lanes. Ideally travel in just a straight line with no turns and no hills. Light traffic with all green lights hopefully. This will result in fairly good performance, though you may have to use signals to cancel unnecessary lane changes.

The key is to avoid turning, stopping, and starting. And other vehicles.

My “surely FSD can’t screw this up” test is a straight shot for about 3.5 miles down Figueroa Blvd from roughly 39th street to 2nd street.

This route has no turns or merges. FSD needs to pick one lane and proceed the entire way. I engage the system in the correct lane. The car has never successfully completed this very basic drive in 18 months.

In fact I tried it today on the newest 10.69 (I have to add a few minutes to my commute to do so) and it came to a dead stop in the middle of the 3rd St/Figueroa intersection with a green light.

A million robotaxis any day now…

 

[mtndrew1]

I agree with this. The more I use FSD the more likely I will simply let FSD continue on its own even though it may not be comfortable. It's always a decision based on how FSD will affect cars around me.

I can assure you that I give FSDb a very, very long leash to figure things out.

 

[swedge]

So the design intent is to produce a system that allows recognition of revenue...

Recognizing "revenue" would require a whole new AI stack, trained on IRS and FASB GAAP rules.

I'll be thrilled when FSB starts recognizing and paying attention to street signs like School Xing, Oncomming Traffic Does Not Stop, etc, etc, etc.

While it is an interesting thought, Tesla got our FSB cash already, and moving it around on the balance sheet would be a tactical decision of magnitude and impact unknown to those of us not trained on IRS and FASB rules. Tesla recognizes FSB revenue as features are released, which means they can tweak their revenue numbers by adjusting the timing and estimation of the value of update releases. On the other hand, in Q4 of 2022, they recognized $324M (10K, pg 37), which was only 0.4% of their total annual revenue. I doubt it moved the stock price very much, what with all sorts of other factors in play, e.g. the Fed, twitter, china, Covid, NTSB, robotaxi uncertainty, etc, etc, etc.

 

[Tronguy]

Interesting. For me highway traffic is never "Accel-brake-ACCEL-BRAKE over and over". Very smooth. I rarely disengage and with lane changing disabled I now rarely do any interventions except to manually initiate a lane change Only significant problem is merging onto the highway during rush hour. I wonder why our experiences are so different. I do have forward collision warning disabled but wouldn't think that would have any affect on how NoA drives.

Funny bit. @AlanSubie4Life made that comment about accel/brake/accel/brake. So, on the next major drive at speed (55 mph with the occasional stoplight and traffic) I went looking for it.

What I saw: brake/harder brake/brake/harder brake/etc. It... wasn't that noticeable, but it was there. A human would have been more consistent.

And, being the control theory guy that I am, I now make a half-A** guess that the observed behavior could be because the car has a less direct way of figuring the distance/relative speed of the car in front with vision vs. radar. Radar would be more accurate; vision has to gather an image and then, over multiple frames, work out how big/how small the image is getting over time; given the uncertainty, it would be no surprise that uncertainty would bleed over to the deceleration. Humans do a bit better job at this kind of thing (think: quarterbacks throwing footballs at wide receivers. But they miss fairly often.).

But was it a neck-snapper? Nope. Was it noticeable? Only when pointed out. Will it ever get fixed? (spoken in a Dorothy & Toto voice). Um. Probably yeah, after the fixers at Tesla get the more obvious bugs, like swing right on left turns, picking the wrong lanes, and all the other complaints.

Just so we're clear: I've driven this car, in FSD-b mode, literally thousands of miles across the landscape, including (but not limited to) a back-and-forth between NJ and FL. I'm pretty sure that that last was with the vision system. And I hadn't noticed the deceleration madness. Other stuff, sure, but not that.

Now, if this accel/decel/accel/decel stuff is more than the minor stuff I'm describing, then I'm wondering if there's something else at play. Tesla playing with different parameters on different cars? Camera calibration out of whack? Something broken somewhere? Dunno, but gotta wonder.

 

[AlanSubie4Life]

And I hadn't noticed the deceleration madness. Other stuff, sure, but not that.

That is…surprising. I guess it depends on what one is used to?

Will it ever get fixed? (spoken in a Dorothy & Toto voice). Um. Probably yeah, after the fixers at Tesla get the more obvious bugs, like swing right on left turns, picking the wrong lanes, and all the other complaints.

I think this minimizes the possibility that it identifies an underlying issue with the perception - is it too noisy and inaccurate? Is that the reason for the behavior?

If so it is kind of a critical issue.

I’m not sure that that is the reason though. For the most part the issue when it occurs (which is not on every stop - some are smooth and reasonably normal) is way too strong initial braking, followed by insufficient braking. I don’t think that fits the pattern of noisy perception. I also don’t think it quite fits the pattern of a reaction to latency either (see above observations posted here a week ago or so - specifically acceleration after an initial hard brake). Though that is harder to say. Latency leading to unstable control loops might have consequences that are quite hard to predict.

Anyway, latency or perception or just poor programming - I think it leaches into all aspects of the vehicle behavior.

 

[kabin]

The key is to avoid turning, stopping, and starting. And other vehicles. I have found the performance to be quite good in this domain.

It's a short list for sure. Maybe also include singles lane roads to avoid random turn signal/lane changes. When the dust settles adaptive cruise technology from 2016 about matches performance at a fraction of the cost.