Blog Musk Touts ‘Quantum Leap” in Full Self-Driving Performance

A “quantum leap” improvement is coming to Tesla’s Autopilot software in six to 10 weeks, Chief Executive Elon Musk said a tweet.

Musk called the new software a “fundamental architectural rewrite, not an incremental tweak.”

https://twitter.com/x/status/1294374864657162240

Musk said his personal car is running a “bleeding edge alpha build” of the software, which he also mentioned during Tesla’s Q2 earnings. 

“So it’s almost getting to the point where I can go from my house to work with no interventions, despite going through construction and widely varying situations,” Musk said on the earnings call. “So this is why I am very confident about full self-driving functionality being complete by the end of this year, is because I’m literally driving it.”

Tesla’s Full Self-Driving software has been slow to roll out against the company’s promises. Musk previously said a Tesla would drive from Los Angeles to New York using the Full Self Driving feature by the end of 2019. The company didn’t meet that goal. So, it will be interesting to see the state of Autopilot at the end of 2020.

 

[Tikib2020]

Ugh I can't believe people on this website still see lidar as essential for L5 autonomy. It simply isn't. We have been driving cars for decades without lidar on our heads. And animals have been navigating 3D spaces for millions of years without evolving lidar.

It's simply not neccesssry.

 

[diplomat33]

Ugh I can't believe people on this website still see lidar as essential for L5 autonomy. It simply isn't. We have been driving cars for decades without lidar on our heads. And animals have been navigating 3D spaces for millions of years without evolving lidar.

It's simply not neccesssry.

Not this stupid "humans can drive with just eyes so autonomous cars don't need lidar" argument again! ARGH!!!

Autonomous cars don't have camera vision as reliable as human vision and they don't have a computer as advanced as the human brain! So it is not a fair analogy. Computer vision is much less sophisticated than human vision. And our best NN are much less sophisticated than the human brain which has billions of connections. It is really as simple as that. Remember that humans start with a supercomputer with billions of neurons and train their vision NN for 15 years BEFORE they even start learning how to drive.

Here we see just one example of Tesla Vision seeing a pedestrian where there is no pedestrian. You cannot do L5 with camera vision like that! And this is after Tesla has trained their camera vision for 4 years, it sees phantom pedestrians!

Will camera vision get better? Yes. Will our computers and NN get more sophisticated? Yes. So maybe in the future, we will have camera vision good enough for L5 and lidar won't be needed anymore. That is entirely possible. But we are not there yet. We are still a long way from that. So, right now, computer vision is not good enough for L5 so autonomous cars need lidar! It is as simple as that.

 

[diplomat33]

Birds have evolved over millions of years to fly without radar. So why do planes use radar? Just because humans have evolved without lidar does not mean that we have to do autonomous cars without lidar. It's silly to arbitrarily constrict your design to how humans happen to work. Autonomous cars don't have to follow the same path that evolution picked for humans. We can design autonomous cars anyway we want. The fact is that lidar has a lot of advantages over vision. So it just makes sense to add lidar on autonomous cars. In fact, the whole point of autonomous cars like Waymo having cameras, radar and lidar is because they want to take advantage of the benefits that those 3 sensors offer. To say we are not going to take advantage of the benefits of lidar because humans don't use lidar is absurd and bad engineering. Humans don't have radar or ultrasonics either but Teslas have a front radar and ultrasonics. So Tesla is willing to use sensors that humans don't have. So why when it comes to lidar, suddenly argue that we don't need it because humans don't need it? Why one but not the other? It's not an intellectually honest argument because you are willing to use some sensors that humans don't use. It's just lidar that you are against because the boss is against it.

 

[nukequazar]

Ugh I can't believe people on this website still see lidar as essential for L5 autonomy. It simply isn't. We have been driving cars for decades without lidar on our heads. And animals have been navigating 3D spaces for millions of years without evolving lidar.

It's simply not neccesssry.

This is a joke, right? Sarcasm, I assume, I hope.

 

[Tikib2020]

I agree with most of what you say. But I think you significantly underestimate what can be done with computer vision alone.

Lidar doesn't provide a lot of information that can't be gained from cameras since they use the same wavelenths etc. It's just the data from the cameras is harder to analyze. However the ability to analyze this data effectively is increasing very rapidly.

If you actually listen to what Tesla said they actually expected to be behind in the race to full autonomy initially because of the difficulty with computer vision however they predicted that the computer vision problem would be solved relatively quickly.

Your making the assumption that computer vision will not be solved any time soon without any evidence for doing so.

I really do think your going to end up with egg on your face on this issue Diplomat.

 

[DanCar]

... So, right now, computer vision is not good enough for L5 so autonomous cars need lidar! It is as simple as that.

Lol. You could say the same about any sensor. Computer vision is not good enough for L5 so cars need a temperature sensor. so cars need a humidity sensor, so cars need military grade radar, so cars need high powered sonar, etc... Why don't we just put every sensor imaginable, make it very expensive, and require a few super computer to process it? You won't see L5 anywhere for 10+ years, so mentioning L5 in any argument is humor. If you want to step back to L4 then we will have to see when there is a winner with vehicle add on equipment that costs a reasonable amount. There won't be a winner in the next few years. Winner = something we can use, rather than the gullible slopping up company marketing stunts.

 

[diplomat33]

I agree with most of what you say. But I think you significantly underestimate what can be done with computer vision alone.

I don't think so. I understand that camera vision is improving. I know camera vision can do pseudo-lidar and detect distances and create accurate HD maps. I know what camera vision is capable of. I am a big proponent of camera vision. But so far, if you look at every single company doing FSD, nobody has camera vision that can do L5 alone. Look at Mobileye which has camera vision that can create accurate HD maps on the fly and can drive autonomously on busy city streets in impressive demos. They still plan to include lidar to do L5.

if anything, I think Elon overestimated when camera vision would be solved. He thought it would happen relatively soon. That's why he said FSD was solved in 2015 and why he sold FSD in 2016. But we see that 4 years later, Tesla has not achieved FSD yet and keeps doing rewrites and working on vision. So it does appear to be taking longer than Elon thought.

Lidar doesn't provide a lot of information that can't be gained from cameras since they use the same wavelenths etc. It's just the data from the cameras is harder to analyze. However the ability to analyze this data effectively is increasing very rapidly.

This is objectively false. Lidar does not use the same wavelengths as cameras. Lidar that is used in autonomous cars operates at wavelengths of 905 nm, 940 nm and 1550 nm. Cameras of course operate in the visible spectrum which goes from around 400 nm to 700 nm.

Lidar is also an active sensor whereas cameras are passive sensors. So lidar generates its own EM radiation to detect objects and therefore can operate in zero ambient light whereas cameras can only operate if there is some external or ambient light source. So they operate very differently.

If you actually listen to what Tesla said they actually expected to be behind in the race to full autonomy initially because of the difficulty with computer vision however they predicted that the computer vision problem would be solved relatively quickly.

I agree with you that Elon believes that camera vision will be solved relatively quickly. It's why he rejects lidar. Why bother adding lidar if you won't need it in a couple years. But so far, Elon has been wrong on his timelines of when camera vision would be solved. it is not happening as quickly as he seemed to think.

Your making the assumption that computer vision will not be solved any time soon without any evidence for doing so.

It's an assumption based on observation so far. If camera vison could be solved quickly, Waymo, Cruise, Mobileye are in the best position to do it. if it was going to happen quickly, they would have either done it by now or at least, they would have ditched lidar because camera vision was going to be solved soon. But they have not done that. Frankly, it is a bit arrogant to suggest that Waymo, Cruise, Mobileye and others somehow can't solve vision but Tesla alone will solve it and do it relatively quickly.

I really do think your going to end up with egg on your face on this issue Diplomat.

We shall see. So far, I've been proven right because we have not seen camera vision that can do L5 yet.

But again, I am not saying it will never ever happen. I am just saying that it has not happened yet.

 

[nukequazar]

Ugh I can't believe people on this website still see lidar as essential for L5 autonomy. It simply isn't. We have been driving cars for decades without lidar on our heads. And animals have been navigating 3D spaces for millions of years without evolving lidar.

It's simply not neccesssry.

We also drive with our RADAR. Seriously, this is insane.

 

[Huskyf]

Not this stupid "humans can drive with just eyes so autonomous cars don't need lidar" argument again! ARGH!!!

Autonomous cars don't have camera vision as reliable as human vision and they don't have a computer as advanced as the human brain! So it is not a fair analogy. Computer vision is much less sophisticated than human vision. And our best NN are much less sophisticated than the human brain which has billions of connections. It is really as simple as that. Remember that humans start with a supercomputer with billions of neurons and train their vision NN for 15 years BEFORE they even start learning how to drive.

Here we see just one example of Tesla Vision seeing a pedestrian where there is no pedestrian. You cannot do L5 with camera vision like that! And this is after Tesla has trained their camera vision for 4 years, it sees phantom pedestrians!

Will camera vision get better? Yes. Will our computers and NN get more sophisticated? Yes. So maybe in the future, we will have camera vision good enough for L5 and lidar won't be needed anymore. That is entirely possible. But we are not there yet. We are still a long way from that. So, right now, computer vision is not good enough for L5 so autonomous cars need lidar! It is as simple as that.

Sorry but how lidar are better in this situation ? Laser not pass objects like this gril or bounce on glasse - ice water drop or reflected surfaces not working on fog.

 

[Huskyf]

Lidar is not good for animals !

 

[diplomat33]

Sorry but how lidar are better in this situation ? Laser not pass objects like this gril or bounce on glasse - ice water drop or reflected surfaces not working on fog.

Lidar can see through the grill.

905 nm lidar cannot see through fog but 1550 nm lidar can see through fog as we see in this demo. We see that the 1550 nm lidar is able to stop for the pedestrian in the fog.

We also see that software can be used to filter out the false point clouds generated from the fog. This works similar with rain too. So lidar can work in fog and rain.

And here we see Waymo lidar and camera data from driving in rain. We see that lidar does pick up the water being kicked up by vehicles behind them but lidar does still work in rain. Lidar still detects the lanes and the other vehicles.

But like I've said a million times, there is no one sensor that can do L5 presently. So you need multiple sensors. Nobody is suggesting replacing cameras with lidar. You use cameras, radar and lidar together since they all work better in different conditions. That way you cover all your bases. By having cameras and lidar you get the best of both worlds.

 

[gluu]

L5 does not currently exist, so it is false to say that LIDAR is a requirement. If anything, LIDAR suffers from a chicken and egg problem. Currently there are no major applications that I can see of for LIDAR that will drive its costs down the learning curve. In order for the price to be driven down, one must already have L5 working to get appropriate volumes, but in order to get L5 working, one likely needs LIDAR to have already been driven down the cost curve and made available in a large scale testing fleet. How will Waymo ever show regulators that their vehicles are 10x safer than human? By showing simulations? If I were a regulator I would scoff, simulations are always doomed to succeed. Tesla has an obvious method, it simply needs to deploy the software and incur many miles and show that interventions/accidents are less than x%. Elon estimates this to be 10 billion miles. Do some quick math about how much money this would cost, to outfit vehicles with LIDAR and hire enough drivers to do this validation. By my estimations, this would cost in the 10's of billions of dollars to achieve. Not only that, once it is achieved, Tesla could still underprice Waymo if they are successful in their strategy.

Tesla's strategy works because it allows the Autopilot team solve issues incrementally. It is a harder path than using a LIDAR based system no doubt, but at least success is one of the possible options. I see no such ability for LIDAR based systems to achieve commercial success.

 

[diplomat33]

@Tikib2020 @Huskyf By the way, it's not me making up that camera-only is not good enough for L5. I am not just making assumptions or underestimating how good camera vision is. The consensus of the Industry that works on autonomous driving is that, as of today, camera-only is not good enough for L5. This is what Safety First for Automated Driving, written by the top engineers in dozens of companies working on autonomous driving, says on page 47-48:

"As of today, a single sensor is not capable of simultaneously providing reliable and precise detection, classifications, measurements, and robustness to adverse conditions. Therefore, a multimodal approach is required to cover the detectability of relevant entities.

In more detail, a combination of the following technologies shall provide suitable coverage for the given specific product:

CAMERA: Sensor with the highest extractable information content as sensor captures visible cues similar to human perception. Main sensor for object/feature type classification. Limited precision in range determination, high sensitivity to weather conditions.

LIDAR: High-precision measurement of structured and unstructured elements. Medium sensitivity to environment conditions.

RADAR: High-precision detection and measurement of moving objects with appropriate reflectivity in radar operation range, high robustness against weather conditions.

ULTRASONIC: Well-established near-field sensor capable of detecting closest distances to reflecting entities.

MICROPHONES: Public traffic uses acoustic signals to prevent crashes and regulate traffic, e.g. on railway intersections. Thus, devices capturing acoustic signals are required for automation levels where the systems need to react to these.

HD MAP AS A RELIABLE SENSOR
An in-vehicle map has never played a safety-related role as it could do in automated driving. For a relatively long period of time, the capabilities of onboard sensors alone will be insufficient to meet the high reliability, availability and safety requirements of the automated vehicle system in certain situations. A HD map is therefore necessary as a reliable off-board sensor containing carefully processed a-priori information to “detect” features that are not easily detectable by on-board sensors or to provide a redundant source of information for on-board sensors, including location-based ODD determination, environment modeling in adverse conditions and precise semantic understandings in complex driving situations. In situations where on-board sensors cannot reliably detect features, the HD map can be utilized as a more reliable redundant source of information."

 

[diplomat33]

How will Waymo ever show regulators that their vehicles are 10x safer than human? By showing simulations? If I were a regulator I would scoff, simulations are always doomed to succeed.

HA HA. Are you serious? Waymo has a lot of real world autonomous driving. Waymo has over 20M miles of real world fully autonomous driving! And Waymo has already gotten permission from regulators to roll out driverless robotaxis in some limited areas with just millions of miles. So, they have already gotten regulatory approval with millions of real world driving, not billions and not simulations. So no, you don't need 10B miles to prove that your system is safe. That is a totally fake number that Elon pulled out his a$$.

Keep in mind that Tesla does not have FSD. So they have billions of non-FSD miles that they are using to try to create FSD. That's probably why Elon says 10B miles. Once you have FSD, like Waymo has, you don't need 10B miles to prove that it is safe.

 

[gluu]

HA HA. Are you serious? Waymo has a lot of real world autonomous driving. Waymo has over 20M miles of real world fully autonomous driving! And Waymo has already gotten permission from regulators to roll out driverless robotaxis in some limited areas with just millions of miles. So, they have already gotten regulatory approval with millions of real world driving, not billions and not simulations. So no, you don't need 10B miles to prove that your system is safe. That is a totally fake number that Elon pulled out his a$$.

Which regulators have approved driverless robotaxis? Chandler, Arizona? How many miles did Waymo have to drive in Chandler before getting approval? Multiply that by every city in the United States/World before approval is achieved. According to this link:

How Many Cities Are In The Us.

There are 20,000 cities in the United States. Suppose Waymo only needed 1 million miles in Chandler before getting approval. 20e3 * 1e6 = 20e9, aka 20 Billion. Elon's number holds up under this napkin math, and if anything underestimates.

For US-wide regulatory approval Elon's number makes sense, even by your own argument.

Also, Waymo is not even making money in Chandler! Didn't they recently shut the program down? I'm not sure how they will ever scale.

 

[diplomat33]

Which regulators have approved driverless robotaxis? Chandler, Arizona? How many miles did Waymo have to drive in Chandler before getting approval? Multiply that by every city in the United States/World before approval is achieved. According to this link:

How Many Cities Are In The Us.

There are 20,000 cities in the United States. Suppose Waymo only needed 1 million miles in Chandler before getting approval. 20e3 * 1e6 = 20e9, aka 20 Billion. Elon's number holds up under this napkin math, and if anything underestimates.

For US-wide regulatory approval Elon's number makes sense, even by your own argument.

It does not work that way. You don't do 1M miles in city A then do another 1M miles in city B etc.. A lot of driving cases will repeat themselves. Once you've validated that your self-driving car can handle a case in one city, it can handle that same case in another city. You don't need new miles to test the same case again. And you don't need to test in all 20,000 cities because again, some cities will be the same in terms of autonomous driving. You just need a good enough sample of cities to cover all the cases, and enough total miles. So if you have a large enough fleet of self-driving cars operating in a lot of cities with a certain number of total autonomous miles, that will be good enough. You don't need 10B miles. Heck, there are some towns/cities that Tesla does not have a lot of cars in.

L5 does not currently exist, so it is false to say that LIDAR is a requirement. If anything, LIDAR suffers from a chicken and egg problem. Currently there are no major applications that I can see of for LIDAR that will drive its costs down the learning curve. In order for the price to be driven down, one must already have L5 working to get appropriate volumes, but in order to get L5 working, one likely needs LIDAR to have already been driven down the cost curve and made available in a large scale testing fleet.

This is total nonsense. We already have applications for lidar that are driving down the cost. Waymo has L4 robotaxis. And some consumer cars are getting lidar next year because the cost is low enough. The cost of lidar has already come down dramatically. Waymo's main lidar went from $75,000 to $7,500. Lidar for FSD is only a couple thousand now and will get cheaper as more cars get it. Velodyne is now selling lidar for ADAS for only $100. Full Page Reload

 

[gluu]

It does not work that way. You don't do 1M miles in city A then do another 1M miles in city B etc.. A lot of driving cases will repeat themselves. Once you've validated that your self-driving car can handle a case in one city, it can handle that same case in another city. You don't need new miles to test the same case again. And you don't need to test in all 20,000 cities because again, some cities will be the same in terms of autonomous driving. You just need a good enough sample of cities to cover all the cases, and enough total miles. So if you have a large enough fleet of self-driving cars operating in a lot of cities with a certain number of total autonomous miles, that will be good enough. You don't need 10B miles. Heck, there are some towns/cities that Tesla does not have a lot of cars in.



This is total nonsense. We already have applications for lidar that are driving down the cost. Waymo has L4 robotaxis. And some consumer cars are getting lidar next year because the cost is low enough. The cost of lidar has already come down dramatically. Waymo's main lidar went from $75,000 to $7,500. Lidar for FSD is only a couple thousand now and will get cheaper as more cars get it. Velodyne is now selling lidar for ADAS for only $100. Full Page Reload

Do you know exactly how the approval process goes? I actually don't know myself and you seem more confident about how it would go. If the marginal number of miles needed to get approval is low, I would be surprised and not supported by statistical rigor. If you want to make statistical arguments, it is very obvious that the number of miles needed is very large.

https://www.rand.org/content/dam/rand/pubs/research_reports/RR1400/RR1478/RAND_RR1478.pdf

As for LIDAR costs, I think you're mistaken. I suggest you read this:

Why spinning lidar sensors might be around for another decade

 

[diplomat33]

Do you know exactly how the approval process goes? I actually don't know myself and you seem more confident about how it would go. If the marginal number of miles needed to get approval is low, I would be surprised and not supported by statistical rigor. If you want to make statistical arguments, it is very obvious that the number of miles needed is very large.

https://www.rand.org/content/dam/rand/pubs/research_reports/RR1400/RR1478/RAND_RR1478.pdf

Thanks for the resource.

Keep in mind that every State is different in terms of regulatory approval. Some States have no formal approval process. So in those States, you definitely would not need billions of miles to deploy robotaxis.

I was disagreeing with your point that Waymo will use simulations to get approval. Clearly, Waymo has real world driving data and has gotten some approval to deploy in limited cases, like in Chandler, AZ. So Waymo has been able to deploy at least in geofenced areas without needing 10B miles and using real world driving, not simulations. They can continue to collect more millions of miles to get approval as they expand their geofenced areas to include more cities.

I find this table very informative. Thanks.

It worth noting that since Tesla does not have FSD yet, none of the billions of miles of AP count toward this validation. It is only when Tesla achieves FSD that they can start counting autonomous miles towards proving that it is safe enough.

As for LIDAR costs, I think you're mistaken. I suggest you read this:

Why spinning lidar sensors might be around for another decade

I was talking about solid state lidar. It is much cheaper than the spinning lidar. Yes, spinning lidar has some advantages over solid state lidar. That's why Waymo has the one spinning lidar on the roof. But that does not mean that FSD cars can't use the cheaper solid state lidar. We see several autonomous cars with it. The bottom line is that the cost of lidar is coming down and will come down as lidar gets mass produced. Your own article says so:

"Ultimately, a lidar unit's single-unit price doesn't matter that much. The long-run goal for all of these lidar companies is to sell sensors in units of thousands or even millions for use in consumer vehicles. A carmaker buying lidar units in batches of 10,000 will obviously get a big discount from the single-unit price."

So even if we say that FSD cars will need the more expensive spinning lidar because they are better than the solid state lidar, the price of the spinning lidar will still come down when they get mass produced. It's economics.

 

[Buckminster]

L5 does not currently exist, so it is false to say that LIDAR is a requirement. If anything, LIDAR suffers from a chicken and egg problem. Currently there are no major applications that I can see of for LIDAR that will drive its costs down the learning curve. In order for the price to be driven down, one must already have L5 working to get appropriate volumes, but in order to get L5 working, one likely needs LIDAR to have already been driven down the cost curve and made available in a large scale testing fleet. How will Waymo ever show regulators that their vehicles are 10x safer than human? By showing simulations? If I were a regulator I would scoff, simulations are always doomed to succeed. Tesla has an obvious method, it simply needs to deploy the software and incur many miles and show that interventions/accidents are less than x%. Elon estimates this to be 10 billion miles. Do some quick math about how much money this would cost, to outfit vehicles with LIDAR and hire enough drivers to do this validation. By my estimations, this would cost in the 10's of billions of dollars to achieve. Not only that, once it is achieved, Tesla could still underprice Waymo if they are successful in their strategy.

Tesla's strategy works because it allows the Autopilot team solve issues incrementally. It is a harder path than using a LIDAR based system no doubt, but at least success is one of the possible options. I see no such ability for LIDAR based systems to achieve commercial success.

Great point - if LIDAR was really useful it would have found it's way into multiple products outside of automotive in a big way. Cost is no longer an excuse now that prices have come down a fair bit.

 

[diplomat33]

Great point - if LIDAR was really useful it would have found it's way into multiple products outside of automotive in a big way. Cost is no longer an excuse now that prices have come down a fair bit.

No, it is not a great point. Lidar is used a lot outside of autonomous driving. Here is a list 100 applications that lidar is used for today:
100 Real-World Applications of LiDAR Technology

So yes, lidar has already proven to be very useful in autonomous driving and outside of autonomous driving.