Elon: FSD Beta tweets

[Knightshade]

Tesla fans are the only group of people who discuss technological superiority but exclude every other country because the argument doesn't favor them. Tesla is the best in the world but then exclude the world in their comparison.

Again you're strawmanning... do you EVER have a reasonable argument?

Nobody is excluding it because it "doesn't favor" the argument. They're excluding it because they can't actually get the product

A product I can't buy, and nobody else outside one country on the other side of the world, can buy, and which wouldn't work here even if I somehow imported one, is essentially an imaginary product for anyone not living in that one country.


You asked why I said "western" in my remark. That's why.

You then built yet another Insane Troll Logic strawman around it.

Every car maker having a lane keeping system doesn't reduce the requirements and challenges an autonomous system on the highway has to meet.

Autonomous Driving on Highway is NOT lane keeping..

And FSDb is not "lane keeping" it's an L2 ADAS for city driving.

Nobody other than Tesla offers one in western markets.

EVERYBODY offers one on highways though.

Suggesting highways just might be easier than city driving.

(and for fairly obvious reasons... all traffic can be assumed to be going the same direction, there's no cross traffic ever, there's no intersections or crosswalks, there's no traffic controls beyond speed limit signs, there's no pedestrians or bicycles, entry/exit to roadway is controlled access, etc...)

 

[diplomat33]

They have every incentive to make custom versions of the Waymo driver firmware for every city. Why wouldn't they?

No, they do not have every incentive to make custom versions of the Waymo Driver because that approach would not be scalable. Waymo actually has every incentive not to customize and to do generalized autonomous driving because that would be more scalable and help them achieve their business goals faster of a profitable robotaxi service. And at the end of the day, Waymo wants to be able to expand a profitable robotaxi service as quickly as possible to as many cities as possible, and generalized stack is the best way to do that.

And it's consistent with the amount of time it takes them to adapt the Waymo driver to new cities. If it was solely a matter of making a new map, they would have services in many more cities by now.

It is not just a matter of making a new map though. There are many other factors that explain the time it takes to add new cities, like safety validation, adding operations and logistics, adding more vehicles to the fleet, updating the app, getting permits, etc... And these factors have nothing to do with the software itself. My point is that Waymo could be generalized and it could still take time to add new cities because of other factors. The fact that it takes Waymo 2-3 months to add a new city does not necessarily mean the software is customized, it could just be the time to make the map, validate the software, set up operations and logistics, get permits, update the app, etc...

 

[DanCar]

OTA upgrades, seemingly revolutionary at first, has really just meant you can now sell a product that doesn't exist. Then when it does, you can pay $2,000 for firmware acceleration, that is just a bug fix in the original acceleration algorithm.

I find it odd that my mid range rear drive model 3 is faster than my long range dual motor model Y. I suppose they decided to hold back performance and sell it as an upgrade.

 

[Knightshade]

I find it odd that my mid range rear drive model 3 is faster than my long range dual motor model Y. I suppose they decided to hold back performance and sell it as an upgrade.

Eh? LR DM Model Y lists at Tesla.com at 4.8 0-60....which last I knew was quicker than an MR Model 3, despite the Y being ~600 lbs heaver.

 

[t3sl4drvr]

Unless Waymo has shown one driving from one city to another, how do we know?

Here is a Waymo* car driving from San Francisco to Oakland, without a driver in 2008.
(* Google acquired 510 Systems and it was eventually renamed Waymo)

Yes, this was 15 years ago. They have been improving the tech since.

On a more serious note, I often see Waymo cars on highways driving between the cities in SF Bay Area. Looks like highway-driving is relatively easy for most self-driving car companies nowadays: Even Tesla is pretty good at it (yet choosing to remain L2).

 

[Bitdepth]

If this is true, then why aren't these companies testing their generalized solutions in more locations at once? What benefit do they have to assigning a fleet of 50 to a single city, when they could deploy one vehicle to 50 of the largest cities in the US to demonstrate their technology?

So, something like this?

Waymo
Jan 22 2018

With Atlanta, Waymo has now officially tested in 25 cities across the U.S. That includes complex places such as the foggy hills of SF, the snowy streets of Michigan and the rainy roads of Kirkland, WA.

Mobileye 2022

These latest locations join our AV testing programs previously launched in Detroit, New York, Jerusalem, Tel Aviv, Munich, Paris, Tokyo, and Shanghai. With the addition of Miami and Stuttgart, we have had testing programs in ten cities in six countries across three continents around the world.

Why test in these diverse locals?

Waymo

We’re building our technology to drive anywhere—any city, one driver. So the lessons we learn from Austin will help us improve the Waymo Driver everywhere we operate, both today and in the future.

Mobileye

Scalable by Design

As with each city our autonomous vehicles have reached, the unique parameters presented by these latest locations pose a novel set of challenges and conditions for our autonomous vehicles to adapt to. And they’ve adapted rather quickly: in fact, our AVs were cruising the streets of Miami within just two weeks of arrival.

Literally everyone is working on a scalable autonomous driving technology. Nobody is designing with intent to deploy in only one city. Why this misinformation keep persisting is beyond me. What do you gain by deploying 50 vehicles to 50 cities simultaneously? You are spreading your finite resources unnecessarily to prove what exactly?

Just so happened to be looking at how long it's been taking Waymo to begin a new service in NYC, and found a familiar face asking a question:

They began that mapping work in 2021. It's 2023 now, and we still don't have insight on when they'll begin service. Surely a generalized solution doesn't take that long to expand.

They were in NY to Map and collect data to use to improve their software. NY was never announced as Waymo One autonomous taxi test city.

The Waymo Driver has tested in dozens of cities spanning a diverse range of climates and topographies. Starting tomorrow, we’ll build on these learnings as we begin to map in New York City for the first time.

New York City is the most densely populated city in the country, with bustling avenues, unusual road geometries, complex intersections, and constantly evolving layouts, and we’ve designed the Waymo Driver to handle these types of complex and dynamic activities that define city driving. Our vehicles will be manually operated by autonomous specialists at all times, to help us scale and advance our technology in support of our mission to make roads safer.

The weather also gives us unique opportunities to learn. Experiencing icy, snowy conditions will allow continued improvement of the Waymo Driver in the real world, and we will apply those learnings across our entire fleet.

We’ll be manually driving with five hybrid Chrysler Pacificas on the street during daylight. Later, we’ll manually drive several of our zero-emission Jaguar I-PACEs equipped with our latest technology on the same streets in Manhattan, as we continue learning from NYC’s busy traffic and unique geometric features. The insights we’ll gain will help the Waymo Driver improve its ability to perceive and predict the actions of other road users in dense urban areas.

You are already seeing the improvements from data gotten from NY in how well the Waymo drives in San Francisco. It is one software designed to drive anywhere.

Edit: this is what happens when you start comenting and leave for dinner then come back and hit post, many people already addressed it above. Lol

 

[Bitdepth]

First, no one really knows all the details of Tesla, Waymo or GM's approaches.

No one outside the people that work there knows all the details. Thats just common sense.

We cam make inferences based on observations, tweets (which can't actually be verified) and other available data but there's still a lot we don't know. Because it's an available consumer product, people like verygreen can hack the system and see more data to give us a better idea but there's still a lot we don't know.

We do have a high-level view on how their autonomous architecture and stack work thanks to those who publish their research papers and regularly discuss their works at ML and CV conferences. We know a lot, at least people who bother reading research papers and follow technical discussions.

Broadly there seem to be two approaches - getting hyper-accurate 'HD' mapping data so the car can navigate without processing anything and developing a human-capable system so the car doesn't need any data and can figure everything out locally. Clearly, all 3 companies are doing a mixture of the two but it appears that Waymo and GM are tilting more towards the former while Tesla is more on the other side.

This is an entirely made-up approach to autonomy by you.

1. Everyone processes everything locally in real-time.
2. Map is a supplemental (prior) data not a replacement for real-time scene segmentation from sensor data.

Let's take a look at a broad overview of the modular architecture which about 95% of all AV companies use including Tesla, with variations.

For example, Waymo

The software takes Maps and Sensor data (Cameras, Lidar, Radars, IMU, GPS, Microphones, etc) and feed both into modules tuned for specific tasks.

Localization: The Map has a localization layer which are prior landmarks scan by Lidar and or Camera, the software compares Sensor data to Map data and then localizes within the map and real world. That is processed in real-time several times per second, so the vehicle knows its position relative to the surrounding using Lidar or Camera data processed in real-time.

Zoox for example

Now that the vehicle has the map of its environment, it's time to work out where on that map it is.
 This This process is known as localizing and is done by matching real-time sensor data to the map.

Perception: The vehicle has prior information on where things "should" be. But it also uses the sensors to perceive the world also known as scene understanding. Given the data from the camera, lidar or radar, what is a drivable space, where are all the lanes, traffic lights, cones, pedestrians, cars, etc. Also known as segmentation. That is processed in real-time, a Map does not replace real-time processing and scene segmentation of world around the car, it is an additional source of information for the software to use to make safe driving decision even when sensors are occluded.

Zoox for example from computer vision alone

2D Bounding Box

Given the camera feeds, we compute several important outputs on each image using neural networks. That includes computing for each object of interest in the scene, a 2D box that encloses it.

3D Bounding Box

In addition to this, we predict the 3D bounding box location for each dynamic object of interest in the scene just from cameras, for example, for the vehicle seen here. If you've been watching our autonomous driving videos for a while, you already know there are many objects that we see in the real world that our vehicles are able to detect and classify. To name just a few, pedestrians, pedestrians with different appearances and poses, pedestrians on scooters, dogs, pedicabs, motorcycles, trams, fire trucks and so on.

 

[Bitdepth]

Segmentation and Depth estimation.

with a mask indicating which pixel belongs to this object, a task that is called instant segmentation. We also compute what semantic class each pixel belongs to. For example, is it a pedestrian, a car, a sidewalk, road, vegetation? And we can also predict for each pixel what is its depth in the world, even though cameras can't measure distances directly.

Pedestrian and Vehicle Detection

Final signal from vision that we cover for pedestrians is incredibly powerful, pedestrian gestures. Many times we have to infer pedestrian intentions from a variety of input signals, including position, heading, velocity, the semantics of the scene, and some attributes that I described before. But other times, a gesture can tell us exactly what the pedestrian wants. Do they want us to stop or do they want us to continue driving? We've found that gestures are actually very strongly correlated with what people end up doing in the future. Here are a couple of examples where we detect whether people are indicating they want us to stop because they want to cross the road, usually by holding up their arm or their hand. And here are some more examples where people wave us on, indicating they want us to continue. Let's switch gears now and talk a bit about the various vehicle attributes our system is able to classify. There are several very important attributes we compute for vehicles, starting with vehicle lights. Vehicle lights such as indicator lights, reverse lights, brake and hazard lights are very important indicators of the intentions of the drivers around us. And using these signals enables us to build much better prediction of what people are going to do.

In addition to this classification, we also compute several attributes for our detected classes that are useful for the rest of the stack. I'm going to talk about some of the attributes for just two of our classes today, pedestrians and vehicles. First up, we have skeleton detection. For each pedestrian, we detect the positions of all the key points on their skeleton, like their hands, their elbows, torso, knees, and so on. This is a video showing these key points for people encountered in several different drives. These key points can then be used for higher level tasks, such as tracking, gesture detection, prediction of intent. Speaking of higher level signals, one of the most important things we can infer about a pedestrian is whether they are standing or walking. This is a cue that people can often pick up on given just an image or two, and we can use this cue in order to be able to quickly say whether, for example, a person who was stopped suddenly decides to cross the road. In our software, we have a network that is able to produce this signal, which we are visualizing in this video. Semantics of scenes are very important. For example, if the person standing in our path is a construction worker, we might need to slow down or stop, or understand that the rules of the road might somehow have changed. In these videos, we show our networks classifying whether each person detected is a construction worker, as indicated by the construction person icon. We also detect whether a construction person is holding a sign, as shown in this video.

Then comes.

Behavioral Prediction: Given the known information about the world around the car, what are the dynamic objects doing which means tracking every moving object in the scene which involves 3D bounding boxes around the cars, and pedestrians, segment them from the environment, estimate the depth, track pose and predict what they are likely to do in the future given the current information. This is processed in real-time; I have to repeat again that a Map is just a prior information about the state of the static world, it is not a replacement of real-time processing of sensor data.

Zoox for example

The signal we use as humans to predict what people will do in the future and how to behave around them is how distracted they might be. Generally, an alert driver will try to drive more cautiously around people who are distracted and are therefore less likely to be paying attention. Here we show our network predicting one of these key distractions for pedestrians, looking at their phones. Fun fact, we label more than 30 different attributes for people including whether they are pushing strollers, riding scooters, exiting cars and staring at phones is one of the attributes we see the most often in real life.

The problem with the HD mapping approach is that heavily relying on mapping data to lessen the required processing abilities commits you to perpetually spending a large amount of resources to keep the mapping data up to date. Even with such efforts there will necessarily be a lag between any street level change and the mapping data, leading to issues.

This is false. A Tesla creates a map as it drives just look at FSDb visualization, every self-driving car creates maps as they drive. Tesla has spent a lot of money on compute so they can reconstruct the real world from driving data. Multi-Trip reconstruction. A lot of this task is automated these days.

Zoox for example

Our vehicles use their perception system to automatically compare features like the lane lines against the map. When a vehicle detects changes in the map or degradation of its sensor calibration, it can send reports back to our CLAMs and ZRN teams.

I agree with @willow_hiller - the approach Tesla appears to be taking seems to be more generalizable. It may well be that all the companies' approaches wind up coalescing somewhere in the middle. If you think about a human driver, we are able to drive in a new area but are better drivers in familiar areas because we have a 'map' in our memory and know what to expect. The same applies to a computer.

Everyone is creating a generalizable autonomous driving stack.

 

[JHCCAZ]

...
L4 or L5 automation in consumer vehicles is practically useless, IMO. Give me a solid L3 on highways and interstates - that’s a useful feature.

My own personal L4 car is exactly what I want. I'm experiencing slowly deteriorating eyesight and I can already predict that I will voluntarily give up driving when I should, rather than fight it. There are many others, presenting a significant market of young and old, and in-between with reasons to need the automated driver.

I suspect that most of us have known relatives or friends who should have stopped driving but were extremely reluctant to give up that Independence. And we know others, if not ourselves, who take risks trying to multitask driving with phone calls, work meetings and an assortment of other activities because of time pressures, boredom or impairment.

Most of us can't afford chauffeurs for children who could benefit from Safe transportation to a wide variety of activities - I note that our society now seems to frown on allowing kids to walk long distances, ride bikes or even saunter down to the bus stop without protective supervision.

Robotaxi is not the desirable solution for many reasons, though it will probably be a viable service in dense cities. But there's no good substitute for your own personal vehicle, in which you can leave some belongings permanently or just between stops during your day. I've tried taxis and rideshare; not a good substitute for having one's own vehicle in a suburban to rural environment.

The use cases above don't even include the mitigation of >30,000 traffic deaths and >1M injuries in the USA alone. I think that's one of the main takeaways people will mention in 20 to 30 years, when they're explaining to their kids and grandkids how things used to be.

I'm far from a safe-space or nanny-state advocate and I believe in personal freedom at many levels. It''s clear that this developing technology can and will enable us to make personal transportation more widespread, cleaner and safer without taking away the "personal" part.

I do understand the common refrain that Highway L3 is good enough. That comes from a large population of frustrated commuters. But as important as that demographic is, it's far from the only customer base for competent autonomous vehicles.

 

[diplomat33]

L4 or L5 automation in consumer vehicles is practically useless, IMO. Give me a solid L3 on highways and interstates - that’s a useful feature.

Why not L4 highways? That would be more useful than L3 highway. L4 highways would mean you never have to take over from on ramp to off ramp and could completely take your eyes off the road on the highway and interstates. With L3 highway you could take your eyes off the road but only for relatively short times because you might be asked to take over. So I would think L4 highway would be more useful to you. Wouldn't you prefer L4 highway where you never have to take over on the highway rather than L3 highway where you would still need to take over sometimes on the highway?

 

[Twiglett]

I got excited when I read that Waymo were coming back to Austin, maybe that I'd be able to try them out. Except they aren't really, it's just more testing, without picking up passengers.
According to the Waymo website there are only three cities you can actually use them to get a ride, Phoenix, LA and San Francisco.
Waymo sounds really cool and they're doing lots and lots of testing, which is nice - but their delivered service still only consists of small sections of three cities.
Hearing that makes it about as relevant as Musk saying FSD will be L4/5/20 by the end of the year.
I can use neither of them.

 

[JulienW]

Why not L4 highways? That would be more useful than L3 highway. L4 highways would mean you never have to take over from on ramp to off ramp and could completely take your eyes off the road on the highway and interstates.....

Instead of chasing 0.99999 for L4 why not a SOLID L3 that works >95% of the time from on to off ramp and then back to L2 (FSD Beta) would seem very useful and more doable soon.

 

[spacecoin]

Instead of chasing 0.99999 for L4 why not a SOLID L3 that works >95% of the time from on to off ramp and then back to L2 (FSD Beta) would seem very useful and more doable soon.

Are you talking about ODD size here or reliability within the ODD? An L3 need to be as reliable as L4 within the ODD, since the human in not driving.

 

[JulienW]

Are you talking about ODD size here or reliability within the ODD? An L3 need to be as reliable as L4 within the ODD, since the human in not driving.

No, I talking about ability of the ODD. For example if it starts raining hard or heavy fog a L3 can ask you to take over driving. This is an example of the <5% of the time it might not work and you would need to drive.

EDIT: Also by "works 95% of the time" I mean more or less that 95% of your drivers that a solid L3 system would not request any takeovers.

 

[diplomat33]

I got excited when I read that Waymo were coming back to Austin, maybe that I'd be able to try them out. Except they aren't really, it's just more testing, without picking up passengers.

Just speculation on my part but I have a good feeling Austin will be Waymo's 4th city with a ride-hailing service. If that happens, you will get to try them.

According to the Waymo website there are only three cities you can actually use them to get a ride, Phoenix, LA and San Francisco.

Waymo sounds really cool and they're doing lots and lots of testing, which is nice - but their delivered service still only consists of small sections of three cities.

Waymo will add more cities. It won't be just PH, LA and SF.

Part of the reason for all the testing is to make the Waymo Driver even better so that they can offer a safe, reliable and confortable driverless ride-hailing in more diverse places.

 

[Bladerskb]

Again you're strawmanning... do you EVER have a reasonable argument?

I never strawman, this exact argument is what I see daily from Tesla fans. Heck @willow_hiller used this exact argument acouple pages ago.

You debate technological supremacy but exclude every other country other than your favorite companies home country and use excuses like "Well i can't buy it so it doesn't matter".

Nobody is excluding it because it "doesn't favor" the argument. They're excluding it because they can't actually get the product

A product I can't buy, and nobody else outside one country on the other side of the world, can buy, and which wouldn't work here even if I somehow imported one, is essentially an imaginary product for anyone not living in that one country.

This is the most illogical thing I have ever read. The ability to purchase a product has nothing to do with its technological supremacy.
Stay away from technology comparison debates then.

And FSDb is not "lane keeping" it's an L2 ADAS for city driving.

I never said FSD Beta is Lane Keeping.
My statement is in direct response to you saying "But again, even in china far more companies offer L2 on highways than in cities".
Again, Lane Keeping is not Autonomous Driving on Highways.

Nobody other than Tesla offers one in western markets.

Bringing local restrictions in a who has the best tech in the world debate, gotta love Tesla fans. immaculate logic and reason there!
And note that western markets only include US & Canada as its not legal anywhere else in the west.

EVERYBODY offers one on highways though.

Suggesting highways just might be easier than city driving.

(and for fairly obvious reasons... all traffic can be assumed to be going the same direction, there's no cross traffic ever, there's no intersections or crosswalks, there's no traffic controls beyond speed limit signs, there's no pedestrians or bicycles, entry/exit to roadway is controlled access, etc...)

You keep thinking in lines of ADAS while the Zoox CEO and everyone else is thinking in lines of driverless AV. Its not that highway isn't easier than city driving, its that the risks are higher on highway, sensor requirements are higher and room for error/recovery are lesser. Which makes it harder in a different sense.

The very fact that Waymo is driverless in 3 distinct cities and has been since 2020 but still use safety drivers on highways completely debunks your entire argument. They clearly too believe that the catastrophic risk on highway is greater and that there is less wiggle room for error due to speed. You slow down on the highway and you might cause a 20 car pileup.

But I'm not surprised that you don't get it. These people are actually creating the technology, you on the other hand...

Also there are multiple companies that offer door to door L2 on city streets in their home countries (china).

 

[Goose66]

Why not L4 highways? That would be more useful than L3 highway. L4 highways would mean you never have to take over from on ramp to off ramp and could completely take your eyes off the road on the highway and interstates. With L3 highway you could take your eyes off the road but only for relatively short times because you might be asked to take over. So I would think L4 highway would be more useful to you. Wouldn't you prefer L4 highway where you never have to take over on the highway rather than L3 highway where you would still need to take over sometimes on the highway?

As many (including you, I believe) have previously asserted, there is a big jump in capabilities between L3 and L4 - true driverless, fallback routines, possible remote intervention, etc. That takes developmental time, infrastructure, approvals, more sensors (?), etc. And for what? So I can sit in the passenger seat instead of the driver seat? I don’t mind having to take over for an L3 every once in a while (construction, accident, road closure, etc.). And for a good L3 it would be rarer still. Plus, I’m still convinced that if Tesla made it a priority I could have L3 in my car by the “end of the year,” while L4 is not in the cards for HW3. If I want to be in the passenger seat, I’ll take a bus.

 

[diplomat33]

Elon Musk on FSD.

 

[texas_star_TM3]

Elon Musk on FSD.

yup. I still remember driverless robotaxis Model 3s ... lol... Level 5 in a vehicle just equipped with a bunch of low res cameras

 

[kabin]

Elon Musk on FSD.

So true. And then Elon boldly says Tesla's future profits rely on it. Rut roh.