I bet it is being and has been simulated 100,000 of times. Probably ran several simulations while I typed.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
FSD Beta 10.69
- Thread starter Buckminster
- Start date
Not being privy to Tesla's software design and planning priorities, I would hardly critique their decisions. But, let's play your game for a moment. I'll assume that 'broad spectrum' means a quantitative count as opposed to some subjective measure of importance. If you do mean it as a measure of importance, then there's no need to discuss further as your estimate will be different than everyone else's.
Take school zones. In my estimation, there are far more ULTs that school zones. So, I would say that ULTs are broader spectrum than school zones. In my city, there are only a few schools, but many ULTs. I'm referring to ULTs with cross traffic speed limits 45 mph or greater, not residential intersection. I also encounter ULTs far more often than school buses. You may have a different experience, depending on the time of day and routes you drive. But, my experience ULTs are a broader spectrum issue than school busses. That's not to say that school zones and school buses are not important. They are.
I believe that there are more ULTs than railroad crossings as well. But that is highly dependent on where you live. So, I believe that ULTs are broader spectrum issue than railroad crossings. BTW, not all railroad crossings require slowing to speed bump speeds. Many of them are well maintained and can be driven at the speed limit. FSD ultimately needs to be able to see which ones are smooth vs. rough. But they may need to be lumped together with a general obstacle identification capability.
BTW, all your pet peeve issues above are on my list of issues except for the railroad crossings, as few of these in my area are an issue. But ULTs are also on my list as well as occluded URTs and incorrect lane selection. All these fall into the 'kill you first' category.
Daniel in SD
(supervised)
Exactly, it's seems like you should be able to get to 99.9% on Chuck's turn with simulation alone. I haven't watched them all so maybe he has seen some edge case but every case I've seen should have been covered by simulating random combinations of vehicle positions and speeds. There's no way you can get that kind of test coverage with real life testing. In simulation you can do millions of permutations.
Also, without good simulation tools how are they going to make sure they don't suffer regressions on future versions?
I spent many years developing systems that responded to real world uncontrolled inputs. Simulations never fully replicate real world conditions.
Regardless, I find it curious that you complain about Tesla doing real world testing. Do you also fault Boeing for doing the same? Or all the other auto makers? How about Waymo, who boasts great simulation capability, yet has driven hundreds of thousands to test miles in the Phoenix area alone?
No doubt NASA's wasting an entire Artemis spacecraft by planning to send it unmanned to the moon must slightly ruffle your feathers.
AlanSubie4Life
Efficiency Obsessed Member
In the last few posts, that is not what he has been doing. I don’t see a complaint about real-world testing (even if his larger opinion is perhaps doubtful about the safety of beta testing, that wasn’t being expressed here). Maybe I missed it.
Post #568 expressed a believe that simulations should have solved the ULT. He did not seem to understand that simulations are not a replacement for testing and that testing will likely find issues that are not found in simulation.
As an aside, in all fairness, the FSD team driving tests that Chuck Cook posted showed unknown software. We have no idea whether the team was testing daily release candidates or were switching out different NN parameters between test runs. It was also indeterminate which tests had interventions. Given all these ambiguities, it's not possible to logically conclude how well ULTs were solved by simulation vs driving. Maybe there were multiple NN parameter sets that performed similarly in simulation and needed real world testing to nail down. We just don't know.
Daniel in SD
(supervised)
The problem isn't the real world testing, the problem is that the problems found in real world testing should have shown up in simulation.
And Boeing should have done more simulation of the 737 MAX!
Last one, then I'm done with this issue in the thread.
I'm all for real world testing. I'm a P.Eng after all. It's a critical phase of the product cycle. Tesla can test all it wants.
As Alan alluded to, yes I have concerns about the overall beta testing program and I'm not really thrilled with allowing real world testing on public roads (unaware drivers, running children...) using minimally "trained" randoms, the lack of a real two-way safety communication, and a bunch of other stuff, it's not like Tesla is discouraging the insane running children "tests". Plus I doubt the viability of the whole FSD hardware suite. If Tesla tested FSD Beta exclusively using trained company testers in controlled environments fine. If FSD Beta was actually a Beta and worthy of imminent release then I'd probably be ok with the public testing it in a well-managed safety cycle - however FSD Beta is a research product, not really even an Alpha, so I don't approve of it being done this way.
That's my opinion, you sort of asked. I still like to follow what's going on though. I'm hopeful it works out for the best too.
Last edited:
What kind of rail crossings do you encounter? I can't off the top of my head think of a crossing that I slow down for.
AlanSubie4Life
Efficiency Obsessed Member
Sure. There is no complaint about real-world testing.
The argument (I think) would be that the failures we have seen:
1) Going too slowly and leaving too little margin to an oncoming vehicle.
2) Going and then stopping halfway into a traffic lane.
3) (Arguable) Missing sufficiently large gaps (note: the ego vehicle seemed to go on smaller gaps than the one that was missed).
4) (Very Arguable - I think ok) Asserting right of way and forcing van to change lanes.
…that all these failures should show up in simulation. Shouldn’t they? They’re very basic and have nothing really (apparently) to do with capability or corner cases.
It may be that they internally simulate and dial up the allowed jerk, etc. to pass all these cases easily, and then dial it down somehow for release…but that actually seems dangerous since you aren’t releasing what you simulated.
I just cannot figure it out, it is too mysterious for me.
I definitely think real-world testing will highlight and find gaps in the simulation. But hopefully not this sort of gap.
There are at least four near me in the Philly burbs, in different directions. Two are for a commuter rail line, one might only be used during the holidays any more for a Santa Train kind of thing, one I think is just unused. But three of the four are raised maybe four to six inches off the roadway with short ramps on either side — certainly enough to make a very unpleasant bump at 40-50mph. Fortunately the flat one is the one I most often use FSD through.
There are two on my commute on the curvy two-lane roads I reference. These are both sidings that are rarely if ever used - one at a brick factory and one at a chemical plant - so they are not maintained by the railroad company but the individual company served. They require you slow down to basically speed bump speeds to avoid damage. They are well marked as RR crossings, both on the pavement and by signage, and the Tesla shows the RxR on the screen. But FSD doesn't slow down even a little despite the crossings being raised and extremely bumpy. Any driver with eyes would slow down for these crossings.
Uh oh! FSD Beta 10.69 has a real life trolley problem!
Seems like occupancy network currently has restrictions on how far it visualizes and/or conditions on when it can be shown, e.g., don't show gray areas if it intersects with "known" moving objects like predicted vehicles or pedestrians.
FSD Beta 10.69 has a real life trolley problem!
Seems like occupancy network currently has restrictions on how far it visualizes and/or conditions on when it can be shown, e.g., don't show gray areas if it intersects with "known" moving objects like predicted vehicles or pedestrians.
Regarding rail crossings, I'm astonished that some of you people live in places where they are routinely so smooth you expect NOT to slow down. I've slowed for them ever since growing up in western Ohio.
About ten years ago my wife and I drove across Texas using the non-Interstate, but major, highway that goes through Lubbock on the way to San Antonio. We were astonished that there was no signage suggesting that we treat rail crossings with respect, but gradually worked our way up to taking them at full speed. I'd never in my life crossed a rail crossing that smooth. And I've lived a while--there were still some steam freight locomotives in western Ohio when I was growing up.
About ten years ago my wife and I drove across Texas using the non-Interstate, but major, highway that goes through Lubbock on the way to San Antonio. We were astonished that there was no signage suggesting that we treat rail crossings with respect, but gradually worked our way up to taking them at full speed. I'd never in my life crossed a rail crossing that smooth. And I've lived a while--there were still some steam freight locomotives in western Ohio when I was growing up.
I agree that many priorities would depend on your geography, but you seem to live in an area with A LOT of unprotected lefts over multi-lane highways. Maybe your state/locale doesn't believe in stop lights? Here in my area just about everything that's not a small neighborhood exiting onto a two lane road is a controlled intersection. If a neighborhood of shopping center exits onto a multi-lane highway, it's usually a right-turn only with an acceleration lane, and folks wanting to go the other way have to U-turn. There are older areas that have multi-lane roads with center turn lanes that people exit out of shopping centers and such going left, but those intersections are not included (and I don't really drive in these areas much - none not on my commute).
Interesting that it shows the people hanging onto the trolley. Functionally, that should provide for a good outcome since the car gives wide berth to pedestrians. But, had there not been standing people, I wonder what would have been displayed.Uh oh!
Seems like occupancy network currently has restrictions on how far it visualizes and/or conditions on when it can be shown, e.g., don't show gray areas if it intersects with "known" moving objects like predicted vehicles or pedestrians.
sleepydoc
Well-Known Member
Simulation is only as good as out currrnt understanding of the problem and the conditions and simulations are often developed using real world testing.
EVNow
Well-Known Member
In theory yes, but in practice no
That’s why you do real world testing. When you find an important missing scenario you add it to simulation. That’s real world engineering ….
Daniel in SD
(supervised)
But why couldn't they have used to all of Chuck's data to create the simulation? Or use their test vehicles to capture data? It looked much more like the were iterating by doing real world testing. When you look at the failures it's hard to imagine that they couldn't be covered by simulation.In theory yes, but in practice no
That’s why you do real world testing. When you find an important missing scenario you add it to simulation. That’s real world engineering ….
Similar threads
- Article
