Tesla autopilot HW3

[dgatwood]

The question becomes "What's the minimum number of functioning sensors the vehicle needs to safely pull over?"

In theory? One front camera plus either ultrasonics or a rear-facing camera on the shoulder side.

If you have ultrasonics and a side camera, you can put on your turn signal, begin repeatedly honking your horn to get the attention of the cars around you, then slowly inch over sideways, knowing that the driver of any vehicle that might be beside you will be so surprised that he/she will brake or speed up to get the heck out of the way, and knowing that if you move slowly enough, even if the driver doesn't, you will detect that vehicle with the ultrasonics or rear-facing side camera before hitting it.

If you have a rear-facing side camera and a front camera, you do the same thing, but slow down rapidly first to ensure that any car that might be beside you will end up in front of you. After that, you only (practically) have to worry about cars that are overtaking you.

 

[Daniel in SD]

In theory? One front camera plus either ultrasonics or a rear-facing camera on the shoulder side.

If you have ultrasonics and a side camera, you can put on your turn signal, begin repeatedly honking your horn to get the attention of the cars around you, then slowly inch over sideways, knowing that the driver of any vehicle that might be beside you will be so surprised that he/she will brake or speed up to get the heck out of the way, and knowing that if you move slowly enough, even if the driver doesn't, you will detect that vehicle with the ultrasonics or rear-facing side camera before hitting it.

If you have a rear-facing side camera and a front camera, you do the same thing, but slow down rapidly first to ensure that any car that might be beside you will end up in front of you. After that, you only (practically) have to worry about cars that are overtaking you.

I would say in theory you could do it with zero functioning sensors 99% of the time. Just remember everything that’s happening and pull over or stop in the lane.
Remember you’ve got a system that can go hundreds of thousands of miles without an accident. If you have a sensor failure you only have to drive for less than ten seconds before stopping. You’ve got perfect information from all the cameras before the failure. I think the sensor redundancy concerns are way overblown. Cars have almost no redundant systems right now and they break down all the time.

 

[dgatwood]

No doubt in my mind. I feel like they would have made changes when introducing the FSD chip if Elon had not promised full self driving with the current sensor suite.

IMO, the only sensors they really should consider adding are side-facing cameras on the front bumper, which would be almost trivially easy to retrofit (as in "ten minutes with a drill, contact cement, and wire clips" simple).

An interior camera facing forwards from the middle of the car would be useful as a backup in certain weather conditions, but is far from critical. Similarly, it would be somewhat helpful to have side-facing cameras on the rear bumper as well, for use when backing out of straight nose-in parking spots, but again, that's probably a pretty low priority.

 

[OPRCE]

Really the only 100% safe bet would be that HW4 involves a process shrink. Saves power, saves space, and in theory could increase speeds. Added features beyond shrinking the die are anyones guess though.

It seems likely that once they have the software mostly all worked out on HW3, some processor-intensive features will be transposed into dedicated silicon for better speed at lower power in HW4, e.g. the methodology Karpathy mentioned for mapping depth info onto all incoming vision pixels to emulate a 3D LiDAR point cloud, thus making it effectively real-time. This I suspect is at least part of what Bannon meant when he prompted Elon with the word "safety" as HW4 was being discussed during the Autonomy presentation. Night vision light enhancement could be another such candidate task.

Here are some relevant papers:

1. Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras
Depth from Videos in the Wild: Unsupervised Monocular Depth...

2. Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D...

3. Night Vision with Neural Networks
Learning to See in the Dark
[ see demo video ]

Anyone else care to speculate what Bannon could have been implying? [I really doubt he was giving away that there is a sensor-suite upgrade needed/planned, as Elon would have leapt across the stage and chopped him in the throat right there]

 

[ChrML]

IMO, the only sensors they really should consider adding are side-facing cameras on the front bumper, which would be almost trivially easy to retrofit (as in "ten minutes with a drill, contact cement, and wire clips" simple).

Any camera added to the bumper will likely be useless half the year in northern countries. If it won't work all the time, there's no point having it.

 

[Mackaypett]

How about putting that spare processor to work creating a 3D model of the surroundings on the move and holding that in memory long enough so that things that disappear from line of sight of the cameras are still in the car’s awareness?

 

[dgatwood]

Any camera added to the bumper will likely be useless half the year in northern countries. If it won't work all the time, there's no point having it.

I assume you mean from road salt? I kind of doubt it. That would be in front of the wheels, not behind it, and presumably such a camera would be recessed into the bumper slightly as well.

Besides, if it won't work all the time, the worst case would be having to tell the user, "I can't turn this corner/pull out of this parking space, so I'm going to stay stopped. Take over for a moment." That's not really a serious failure, but for the 99% of the time that it works, it would provide extra safety.

 

[Inanchy]

IMHO...

If you are going to own a Tesla and be happy, you have to reset your time scale to a couple of years at a time, a time scale of a year or even months won't work out so well for you. You'll find a way to be perpetually miserable.

It's almost hard not to believe that with the investment and assets they have in place today FSD won't become *something* unqiue and pretty great in 2 years or so, but if your hanging on to every update, waiting with bated breath for big news --well, despite anything you heard in the past, that's not going to happen.

Advanced Summon might be about sometime this fall, and that would be quite interesting, I hope, but this is a multi-year game. Our investment was a bet that they'd eventually get it right (on HW3, I sure hope) --but come on now, its a bet. I know Musk said a lot of things, and that can be frustrating, but best to enjoy the coolest and most advanced vehicle on the road RIGHT NOW... and let go of exactly what is coming and when.

Despite being awesome, no one could realistically have bought a Tesla in the last 2-5 years and not thought they were buying very early stage tech, and everything that comes with it. If you did, you were mistaken. I know my 6 month old Tesla will be significantly outdated in 5 years from now (FSD/AP4 HW will not likely be upgradable on my car by then). It's the nature of business, and technology --and this is what we all signed up for.

If you really hate the experience, cut your losses and just sell it. Sit on sidelines for a couple of years and buy something from whoever gets there first --but I think its going to be Tesla, even the analysts are collecting enough information (slowly) to figure out they can actually do this. If your really, really deeply angry, sell it and get a lawyer and see of you can start a class action suit against Tesla --that won't make FSD come any faster (if it got class action status, it might even slow it down further).

But this won't make you happy either --drive and smile friends, you can say you were in the thick of it when you tell your kids and grandkids how cars used to require humans to drive them, and you had one of the first that drove pretty much by itself (or better... : )

Isn’t there an old saying that the pioneers are the ones with the arrows in their back? No offense to native Americans meant. We bought a 2017 Model S with AP 2.0. We did not opt for EAP or FSD Software, but hoped fir it in the near future. Prices have increased, EAP is now AP with fewer functions. I’m still optimistic but not sure HW 3.0 will actually be available to us if we pay for FSD at this point anyway. Maybe we should just settle for AP

 

[dgatwood]

Has anybody else noticed what appears to be sporadic speed limit sign reading behavior? I've noticed several streets I drive on, for which the database incorrectly says 25 MPH, *sometimes* now show the correct maximum 30 MPH speed (but not always), and on one highway in which the speed limit has always been 65, at one point, it said 55, which means either A. it didn't update the maximum speed based on location for about five miles or B. it incorrectly read the truck speed limit sign.

It might be a fluke, but I'm starting to wonder if they're running 1% experiments to test speed sign recognition.

 

[LeoBley]

IMHO...

If you are going to own a Tesla and be happy, you have to reset your time scale to a couple of years at a time, a time scale of a year or even months won't work out so well for you. You'll find a way to be perpetually miserable.

It's almost hard not to believe that with the investment and assets they have in place today FSD won't become *something* unqiue and pretty great in 2 years or so, but if your hanging on to every update...

--that won't make FSD come any faster...

For what it is worth, here is my FSD HW3 intel:

5/6/2019 I picked up S from Tesla Fremont Service
- the requested FSD upgrade to our 10/2016 was deferred due to lack of part availability
- on 5/7 Tesla Engineer was scheduled to get the first FSD HW3 upgrade to their 3 for verification testing (i.e., install procedure and functionality)
- ETA for start of HW 3 upgrade program was 2 weeks

They said we would be contacted based on our vehicle’s VIN to schedule the HW3 upgrade. I’m hoping we are early since our car is one of the first / oldest FSD vehicles with HW 2.5 (which can be upgraded).

 

[CK_Stuggi]

Good news they're starting early stages of HW3 upgrades. I'd really like to get some info on how long they estimate the fleet upgrade to take and when to expect upgrades in different markets. Would at least give the impression they still care about those "pioneers", something they didn't show much since Model 3 hit the stage.

 

[Bladerskb]

It seems likely that once they have the software mostly all worked out on HW3, some processor-intensive features will be transposed into dedicated silicon for better speed at lower power in HW4, e.g. the methodology Karpathy mentioned for mapping depth info onto all incoming vision pixels to emulate a 3D LiDAR point cloud, thus making it effectively real-time. This I suspect is at least part of what Bannon meant when he prompted Elon with the word "safety" as HW4 was being discussed during the Autonomy presentation. Night vision light enhancement could be another such candidate task.

Here are some relevant papers:

1. Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras
Depth from Videos in the Wild: Unsupervised Monocular Depth...

2. Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D...

3. Night Vision with Neural Networks
Learning to See in the Dark
[ see demo video ]

Anyone else care to speculate what Bannon could have been implying? [I really doubt he was giving away that there is a sensor-suite upgrade needed/planned, as Elon would have leapt across the stage and chopped him in the throat right there]

No he meant. ASIL-D. None of what you posted requires a new hardware to do.

 

[OPRCE]

No he meant. ASIL-D. None of what you posted requires a new hardware to do.

1. Do you know HW3 is not ASIL-D compliant already? It would seem a little strange to have gone to all that trouble without bringing it up to the automotive standard, no?

2. Also, the new SoC was billed as built in conformity to "AEC Q100", where does that fit in?

3. What specifically would you expect to be different about HW4 then?

 

[Wooloomooloo]

How about putting that spare processor to work creating a 3D model of the surroundings on the move and holding that in memory long enough so that things that disappear from line of sight of the cameras are still in the car’s awareness?

Well firstly the 2nd processor is supposed to offer full redundancy as well as a checksum, so if it was doing another task it woul dnot be able to do that. Secondly, in a dynamic environment, it would be difficult to model where everything should be within a few seconds of sensor failure, especially without a pre-mapped view of all fixed objects (which is the LIDAR approach already dismissed).

Regardless, the chances of total input failure is at least an order of magnitude lower than the chance of a driver being completely incapacitated, so the cost/benefit of total redundancy would be minuscule.

 

[willow_hiller]

Good news they're starting early stages of HW3 upgrades. I'd really like to get some info on how long they estimate the fleet upgrade to take and when to expect upgrades in different markets. Would at least give the impression they still care about those "pioneers", something they didn't show much since Model 3 hit the stage.

Do you mean the retrofits? I haven't heard of any starting yet.

 

[tmoz]

IMHO...

If you are going to own a Tesla and be happy, you have to reset your time scale to a couple of years at a time, a time scale of a year or even months won't work out so well for you. You'll find a way to be perpetually miserable.

It's almost hard not to believe that with the investment and assets they have in place today FSD won't become *something* unqiue and pretty great in 2 years or so, but if your hanging on to every update, waiting with bated breath for big news --well, despite anything you heard in the past, that's not going to happen.

Advanced Summon might be about sometime this fall, and that would be quite interesting, I hope, but this is a multi-year game. Our investment was a bet that they'd eventually get it right (on HW3, I sure hope) --but come on now, its a bet. I know Musk said a lot of things, and that can be frustrating, but best to enjoy the coolest and most advanced vehicle on the road RIGHT NOW... and let go of exactly what is coming and when.

Despite being awesome, no one could realistically have bought a Tesla in the last 2-5 years and not thought they were buying very early stage tech, and everything that comes with it. If you did, you were mistaken. I know my 6 month old Tesla will be significantly outdated in 5 years from now (FSD/AP4 HW will not likely be upgradable on my car by then). It's the nature of business, and technology --and this is what we all signed up for.

If you really hate the experience, cut your losses and just sell it. Sit on sidelines for a couple of years and buy something from whoever gets there first --but I think its going to be Tesla, even the analysts are collecting enough information (slowly) to figure out they can actually do this. If your really, really deeply angry, sell it and get a lawyer and see of you can start a class action suit against Tesla --that won't make FSD come any faster (if it got class action status, it might even slow it down further).

But this won't make you happy either --drive and smile friends, you can say you were in the thick of it when you tell your kids and grandkids how cars used to require humans to drive them, and you had one of the first that drove pretty much by itself (or better... : )

This is similar to buying a new laptop. You have the coolest laptop for about 3 months. Everyone is envious of you. Then, after about a year, many others seem to have cooler laptops. At the 3 yr point, your laptop seems antiquated, and now you look forward to your next purchase when you can leapfrog them.

 

[tmoz]

At some point, vehicle to vehicle communications will be a big deal. That will be yet another HW update, but I don't think we have to put that on our 'radar screen' until 3-5 yrs from now, but it is coming.

 

[willow_hiller]

At some point, vehicle to vehicle communications will be a big deal. That will be yet another HW update, but I don't think we have to put that on our 'radar screen' until 3-5 yrs from now, but it is coming.

No reason close-proximity communications couldn't be done with the Wifi or Bluetooth antennas, and far-field communications (e.g. traffic jams upcoming) wouldn't be routed through the LTE.

 

[tmoz]

No reason close-proximity communications couldn't be done with the Wifi or Bluetooth antennas, and far-field communications (e.g. traffic jams upcoming) wouldn't be routed through the LTE.

I would think LTE would be too unreliable, meaning the car would need a transmitter. I'm sure there is a better hardware solution out there, especially 5 yrs from now.
I've got a 4 yr old S 85D. The latest Model S incarnation has been the first version that is seriously tempting me to trade up. I think I will stay married to my 85D for another 4 yrs if I can stay faithful that long despite more sexy upgrades to the model S.

 

[willow_hiller]

I would think LTE would be too unreliable, meaning the car would need a transmitter. I'm sure there is a better hardware solution out there, especially 5 yrs from now.
I've got a 4 yr old S 85D. The latest Model S incarnation has been the first version that is seriously tempting me to trade up. I think I will stay married to my 85D for another 4 yrs if I can stay faithful that long despite more sexy upgrades to the model S.

True. I was watching the PBS News Hour the other day and one guest was saying that 5G networks would be necessary for autonomous vehicles. I'm hoping that he meant vehicle-to-vehicle communications, and not some sort of "cloud-autonomy" where a car is just equipped with cameras and a 5G antenna, and all the decision-making processing is done in the cloud.