Autopilot 3.0 Hardware: When and what?

[kurdakov]

Thank you. 18 inches is pretty good! Much better than I had read. I still don't feel like it's good enough to fit in a 12 ft lane.

then one more link http://gizmodo.com/a-new-technique-makes-gps-accurate-to-an-inch-1758457807 - gps + inertial navigation -> 1 inch accuracy.

 

[Forty Creek]

Well if we don't see a substantial improvement soon with their NAV system, it won't matter how accurate the sensors become.

 

[Mo City]

nvidia plans to launch update of their autonomous driving hardware by the end of 2017 Introducing NVIDIA Xavier | NVIDIA Blog .

As Tesla likely will follow cooperation with nvidia, that means either by the end of 2017 or beginning of 2018 there will be next iteration.

How it will be called (2.1 or 3.0 ) is another question

As compared to Drive PX 2, Xavier is not so much a performance improvement as a power, size and cost improvement. I doubt very much it will represent AP 3.0 unless there are significant other hardware changes.

Apparently the capabilities of Drive PX 2 will be enough for the foreseeable future.

However, I have the same wonder on why rearview radar wasn't included in AP 2.0. That's a question Elon should answer.

 

[kurdakov]

As compared to Drive PX 2, Xavier is not so much a performance improvement as a power, size and cost improvement. I doubt very much it will represent AP 3.0 unless there are significant other hardware changes.

Apparently the capabilities of Drive PX 2 will be enough for the foreseeable future.

I think much depends on what will be actually achieved with AP 2.0 - currently this is an open question. Of cause, AP 2.0 will perform most of autonomous level 3 and 4 driving functions, but then there is a question - what corner cases it might not solve.
if there will be apparent deficiencies then possibly a new hardware will be used, say Oryx new detector ( already mentioned it, but another link Oryx nano antennas allow autonomous cars to see farther, better ) - being cheaper and more compact then possibly it will be used for both front and back detections.

So - all new hardware will depend how successful will prove current AP 2.0 hardware, and nobody knows. That will be more apparent by the end of 2017 - then new iteration will follow: if everything works ok - then just a cheaper nvidia computer will be used, if not - then new sensor hardware will be added.

 

[Reeler]

I think regulators will demand full redundancy. 360 degree radar on top of the 360 degree cameras.

 

[bob_p]

Tesla is claiming the AP 2.0 is sufficient to pass validation for FSD and also receive regulatory approval. Realistically, they can't guarantee that either will happen.

While they've shown an impressive demo, that was in a relatively controlled environment (they didn't even have the navigation software running for the route). Anyone who's developed major software products knows that there's a huge difference between a "proof of concept" demo and a real product - and until Tesla has been able to get the FSD software through testing, they can't be sure they have sufficient hardware in AP 2.0 to achieve their goals.

Regulatory approval is another challenge, outside of their control. To get approval, there might even be a mandate to install additional hardware - such as communications between FSD vehicles.

Even if Tesla is never able to fully validate AP 2.0 to achieve FSD or they don't get regulatory approval, if they utilize the AP 2.0 hardware to provide an increased driver assist - in almost all driving conditions, that may be enough to justify the additional $3K to enable FSD with purchase of a new Tesla.

 

[LargeHamCollider]

I think regulators will demand full redundancy. 360 degree radar on top of the 360 degree cameras.

If a safety improvement of greater than 2x is clearly demonstrated Tesla will have an extremely powerful moral argument in favor of regulatory approval. That argument will win the day in most places.

 

[3Victoria]

I think it is pretty obvious that we are in the development phase. I suspect the current AP hw is sufficient for their goals. I agree that new hw will be added/substituted as costs decline. Xavier will allow use of multiple chips to increase the number of sensor channels, and I would expect multiple modalities including extended spectrum cameras, radar, LIDAR, ultrasonics, etc to be used for redundancy and extended use in bad weather. In summary, expect advances as technology allows.

 

[Alketi]

The real question is whether or not the current hardware suite will achieve L5 AND be approved by regulators.

If the answer is YES, then future hardware versions are significantly less relevant and probably wouldn't happen for a while. Consider that Tesla is (unfortunately) still using the Tegra 3 to power its main display -- 4 years and counting.

If the answer is NO, then I would expect a hardware revision as soon as they realize their shortcoming, probably in the 2018/2019 timeframe.

If I had to guess I'd say it's more likely than not that Tesla's current hardware won't be certified for L5 driverless.

There are just too many unknowns and the regulations and redundancy requirements aren't even spec'd yet. If it turns out this way, then current vehicles (including early-gen Model 3s) could STILL be fully self-driving with a person in the driver seat, but not allowed to be summoned remotely nor to join the future Tesla Mobility network.

 

[timdorr]

GPS with SBAS or LBAS and inertial smoothing is. I fly an aircraft fairly regularly that uses SBAS and inertial smoothing, I've never seen the system more than 18 inches off (except right after it starts up and all the satellites have not yet been acquired), LBAS is even more accurate. Tesla will definitely use SBAS with inertial smoothing, I wouldn't be surprised if they have some sort of LBAS built in as well.

SBAS: Satellite Based Augmentation System
LBAS: Local (no, not "Locally") Based Augmentation System.

GPS/SBAS is good for airplanes because there aren't any construction crews or kids playing in the clouds, nor things like tunnels or multi-level bridges. Unfortunately, we have to deal with those things on the ground, so at best those systems are good for navigation but not for active control of the car.

You need a local, on-the-ground detection system to make it work. GPS and a networked geotagging database are good to tell you which routes to take or to avoid a certain lane of traffic which is traditionally slower to drive in. But you need localized, autonomous systems to know to avoid kids playing street hockey or that some dumbass is going 35 in a 55

 

[Panu]

Two quotes from Tesla configurator page:

"self-driving in almost all circumstances"

"system is designed to be able to conduct short and long distance trips with no action required by the person in the driver's seat"

So they clearly say that it will not work in all circumstances and there will need to be a person in the driver's seat.

A lot of hardware upgrades are still needed. Most important at this point would be a system that will keep all cameras clean in all weather conditions. Currently in Finland cars are covered with wet dirt.

 

[3Victoria]

It would extremely silly for them to claim 'all circumstances' -- can you claim that you can drive your car in all circumstances? It would also be hard for someone not in the driver's seat to take any meaningful actions -- so I am not sure they are stating they need a driver -- BUT at this stage they must state what they did because things are not nailed down.

 

[Troy]

why rearview radar wasn't included in AP 2.0. That's a question Elon should answer.

Until he does, I will try to attempt to answer it. There are three reasons why a rear radar is not included:

• Currently, radars don't have enough range to perform automatic lane changes safely (1) using the Clearance Method (2).
• Automatic lane changes with the Detection Method (3) are safer and easier.
  
• Enabling lane changes with the Clearance Method would risk Autopilot being banned in Germany and potentially elsewhere in Europe (4).

(2) Clearance Method: An automatic lane change without driver input is performed by making sure there is no vehicle approaching from behind at high speed on the target lane.

(3) Detection Method: An automatic lane change without driver input is performed only if another vehicle is detected in the target lane that drives around the same speed and is a safe distance behind.

(1) Currently, radars have 200 meters range. For example, Daimler's Intelligent-Drive hardware includes a 200 meters rear radar (source).

On the German Autobahn cars can travel at high speeds. For example, the Mercedes S-Class has an electronically limited top speed of 250 km/h (155 mph). According to this website, a car driving at 250 km/h on dry asphalt would take 351 meters to come to a complete stop. At 164 km/h, it would take 151 meters. Therefore slowing down from 250 km/h to 164 km/h would take 351-151= 200 meters.

In other words, the current 200 meters radar range would allow automatic lane changes only if the self-driving car is traveling at 164 km/h (102 mph) or higher speeds. Only then you can be sure that the speed difference between the self-driving car and the bullet car will be small enough for it to slow down in time. This means automatic lane changes wouldn't be possible in most situations.

(4) A few months ago, The German Federal Highway Research Institute (BAST) tested Tesla Autopilot 1.0 cars and suggested that the Transport Minister should ban Autopilot in Germany. One of the reasons was because "the car's sensors do not detect far back enough during a passing maneuver" (source). This is of course, a misunderstanding because in Autopilot 1.0 the driver needs to make sure the lane is clear before initiating a lane change. However, it demonstrates that they were testing the rear visibility (Autopilot 1.0 has only 4 meters rear visibility).

---

To summarize, Autopilot 2.0 cars will do automatic lane changes only if there is at least one other car behind it on the target lane within 100 meters range that drives around the same speed. For example, let's say the Tesla is driving at 80 mph and another car is 50 meters behind in the left lane, approaching at 84 mph. In this situation, the Tesla would do a lane change by itself without the driver doing anything because the car knows with certainty that the distance is enough. However, in the same situation, if there were no other cars behind within 100 meters range, the Tesla wouldn't do an automatic lane change because there might be cars more than 100 meters away approaching at much higher speeds.

I think Tesla's solution is very smart. It is smart in a sneaky way. You wouldn't normally think of using the other car behind you like a satellite to gather remote information. With the clearance method, the problem was, what if there is a bullet car approaching on the target lane just outside the rear visibility range? The same problem doesn't exist here because even if there is such a car, it's not going to hit the Tesla. It's going to hit the car behind the Tesla.

As a side note, to support my theory, let me add a quote by Elon. In this video at 8:36 Elon says the following:

Elon: The net effect of Enhanced Autopilot is, you should be able to go from freeway on-ramp to exit as well as transitioning between multiple freeways and passing and maneuvering around other cars without touching anything with Enhanced Autopilot.

 

[calisnow]

@Troy - detailed explanation of the situation in Germany - thank you. But if I read your post correctly it seems like you believe that AP 2.0 will not use the "clearance method" even with its rear view cameras - and will only change lanes with no driver input if it detects a car in the adjacent lane traveling at a safe speed behind it - but will not change lanes if it detects no car at all.

Where do you get this information? I have never heard it said that AP 2.0 will not change lanes if detects no cars whatsoever in the next lane.

 

[Reeler]

If anyone believes Tesla's/EM's claims that AP 2.0 will have L5 autonomy, let's think about their track record. AP 1.0 was extensively advertised by Tesla as having blind spot detection. Now, you cannot find any mention of blind spot detection on their site. It is clearly in their prior press releases and can be seen in old snapshots of their web site, but their current site attempts to revise history.

I predict the claims about AP 2.0 will be used to sell their snake oil and then conveniently backpedaled upon like many things from Tesla. I would probably do the same if I were in their shoes, but we customers should not be so easily fooled.

The second they have L5 with regulatory approval in Colorado, I will upgrade but until then I will not drink the Kool-aide.

 

[Troy]

@calisnow, that's my theory based on two criteria:

1. The quote I added by Elon in my previous message.

2. Autopilot 2.0 has only 100 meters rear visibility (source). Therefore it would only be able to do lane changes safely using the clearance method when the Tesla is driving at 221 km/h (137 mph) or higher speeds. Practically that means never. Why? It is basic maths. According to this website a car driving at 250 km/h (155 mph) would take 351 meters to come to a complete stop. When that same car starts slowing down and drops to 221 km/h, from that point onward it takes 251 meters to come to a complete stop. Just enter 221 km/h to the calculator to see this. Therefore slowing down from 250 km/h (155 mph) to 221 km/h (137 mph) would take 100 meters.

We know that cars like the Mercedes S-Class have a top speed of 250 km/h (155 mph) and they can drive at that speed on the Autobahn. Therefore to avoid getting rear-ended by that Mercedes that happens to be 101 meters behind, the Tesla would have to be driving at 221 km/h (137 mph) or higher speeds to give the Mercedes enough braking distance.

In other words, the clearance method doesn't work anyway, which is why Tesla is not even trying. If they were trying they would have used the 200 meters rear-radar like Daimler or the 250 meters camera they use for front view. But even then it wouldn't work because it starts to make sense if the range is around 300-350 meters. Instead, they went with 100 meter rear cameras. To me, that shows they will only use the detection method.

By the way, the clearance method and detection method are terms I have come up with to explain the two different approaches between Daimler and Tesla. You won't find more information if you search for these terms.

 

[LargeHamCollider]

GPS/SBAS is good for airplanes because there aren't any construction crews or kids playing in the clouds, nor things like tunnels or multi-level bridges. Unfortunately, we have to deal with those things on the ground, so at best those systems are good for navigation but not for active control of the car.

You need a local, on-the-ground detection system to make it work. GPS and a networked geotagging database are good to tell you which routes to take or to avoid a certain lane of traffic which is traditionally slower to drive in. But you need localized, autonomous systems to know to avoid kids playing street hockey or that some dumbass is going 35 in a 55

And that's the primary function of the inertial system, inertial nav systems can go for a few minutes with accuracy at the end of the time period of well under 1 foot, overpasses/bridges etc. will not be an issue.

There will be roads, however, at the bottom of steep canyons or at the edge of steep cliffs or which go through very long tunnels which will probably be too much for the GPS/Inertial systems unaided, here is where you'd need the cameras to recognize certain points with known, pre-programmed lat/long to periodically correct the INS for drift.

 

[3Victoria]

@Troy Thank you. I am not sure the math is right, but I am too lazy to do it myself. Since the Autobon has no maximum speed, how does anyone change lanes?

 

[calisnow]

@calisnow,

Autopilot 2.0 has only 100 meters rear visibility (source)

That I did not know - I assumed that the cameras had more range than radar, not less. You are right - 100 meters rear visibility is not that much. Your logic is good, however from a practical point of view I cannot imagine Tesla offering "full self driving" but crippling the ability to change lanes by reducing it to situations only when the car senses vehicles behind it. I don't really have an answer to your challenge on the safety aspect except to say that everything is risk vs. reward.

Here in the U.S.:

1 - I cannot remember any time in my memory that I have encountered a vehicle doing close to 155 mph in California - and I drive 30,000 miles per year. Does it happen? I suppose once in a while. However there are a couple things to say about this:
2 - The Tesla could take countermeasures against a speeding car by also increasing its own speed when it "realizes" the car behind is closing too quickly.
3 - Some non-trivial percentage of the drivers closing from behind too quickly to stop will swerve into the next lane to avoid you.
4 - The combination of numbers 1, 2 and 3 mean that, in my opinion:
5 - The risk of being rear ended by a car speeding up behind you in the next lane because your Tesla can see only 100 meters is very, very small - at least here in the U.S. Even in the rare case in which it might happen the speeding car will probably be slowing significantly, and the Tesla accelerating to avoid it and the impact will be minimal. I would predict that 100 meters rear range is sufficient to keep impact risk at or below that of typical human drivers - who may have greater rear range in their eyes but who also frequently fail to check their mirrors at all, are looking at their phones, etc.

Maybe this would be much more dangerous in Germany and maybe Tesla will program the cars differently there.

 

[Troy]

I cannot imagine Tesla offering "full self-driving" but crippling the ability to change lanes by reducing it to situations only when the car senses vehicles behind it.

If you tap the turn signal while in Autopilot, it will still do a lane change like it does now. However, it will do fully automatic lane changes without driver input while passing and maneuvering around other cars like Elon said here. This is enough for driverless use under the Tesla Network system. The description in the Design Studio says "automatically change lanes without requiring driver input." Technically they are delivering on what they are advertising.

Notice that overly optimistic and simplified advertisement is not new for Tesla. Look how they were advertising Autopilot 1.0 lane changes. I wrote about it last year here. Autopilot 1.0 advertisement gave the impression that the car could see behind. This was actually the main reason why The German Federal Highway Research Institute misunderstood Autopilot 1.0.

The risk of being rear-ended by a car speeding up behind you in the next lane because your Tesla can see only 100 meters is very, very small - at least here in the U.S.

Yes but the car is already doing fully automated lane changes when there are other cars behind it on the target lane. That already covers most of the highway driving in the USA. Tesla can't make major functionality changes for different regions. The danger of getting rear-ended by a bullet car is very real. Just watch the following video. If Tesla was trying to use the clearance method, they would have added longer rear-visibility. I think I have now added enough data to support my theory.