Can you clarify, does he work for tesla or is he expressing his opinion on what others are doing?
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
New AU FSD vrs USA FSD post June 21
- Thread starter Wesleyb
- Start date
I researched it and no he’s not a tesla employee, but he is a tesla investor….so keep those messages positive.
Its interesting that mobile eye, who created the first tesla autopilot (which was and for many tesla owners still is very good) and are global leaders, also agree a vision only system will work. They do however make it expressely clear that their system adds radar and/or lidar for redundancy. They dont have confidence in a system without redundancy. You can read more on their website.
Importantly, mobile eye are developing AV systems. They dont just have hypothetical chats.
Its interesting that mobile eye, who created the first tesla autopilot (which was and for many tesla owners still is very good) and are global leaders, also agree a vision only system will work. They do however make it expressely clear that their system adds radar and/or lidar for redundancy. They dont have confidence in a system without redundancy. You can read more on their website.
Importantly, mobile eye are developing AV systems. They dont just have hypothetical chats.
Fair assessment paulp. I don’t usually agree with you, but here you’ve done your research and posted a well balanced and well thought out opinion.
Now onto the mobileye point. Think about lidar and vision as two circles of a Venn diagram. Vision has benefits exclusive to vision, lidar has benefits in areas exclusive to lidar and then there is a sizeable overlap where both sensor types are capable of perception.
The point Tesla (and others in the ML industry) are making is that the vision part is eating into lidar circle at an increasing rate. It won’t ever be a perfect combined circle of overlap but it’s getting close. There are things that vision can do that lidar will never be able to do (read a stop sign for example). But the ability for vision to synthesise the lidar output is getting to the point there is very little marginal benefit to true lidar (and now radar) for the task at hand.
So, take the favourite situation by most fusion proponents— dense fog — and think about how an autonomous car would drive. Let’s say it does have a lidar or radar with perfect fusion (this in unto itself is incredibly difficult problem and prone to failure) it will in that case, be able to, in theory, very accurately detect the surrounding vehicles and maybe pedestrians. However you won’t be able to see traffic lights, road lines, traffic cones etc. so in that case, it’s not safe to drive at a rate that’s beyond the capabilities of vision anyway. Radar/Lidar can tell you how far away an object is, it can’t tell you any attributes of that object nor the context. Vision on the other hand, can do a superhuman level of estimating distance to an object (not lidar level but why is that useful? Humans have no idea if a car is 50m or 60m away— it provides zero benefit to be accurate at every distance) but in turn can give you the context, properties etc.
Now take an autonomous car with vision only. Given a sufficiently trained set of NNs, it can predict a visibility impairment and then calculate a safe driving speed/trajectory similar to that of a component human. It will still be able to respond to a set of cones closing off a lane, a flashing orange traffic light to indicate cross traffic or any other higher level driving task lidar provides zero insight to. If the fog is truly bad enough that the well tuned NN predicts an impairment that is too great to drive safely, it certainly will be too unsafe for a human or continue driving also.
I have no doubt MobilEye are leaders in the industry. They will have a level 5 offering eventually and they will be the best at sensor fusion. However that doesn’t mean Tesla can’t do it without sensor fusion.
that’s absolutely not true. There is no database of objects to reference when the NN is making inferences. There are objects that are used in training but this is purely for training. The resulting NN will have an intrinsic ability to predict distances no matter what the object is. Think about the vanishing line optical illusion, that’s what it’s replicating. Things like shadows, horizon etc are all used internally by the NN to make the prediction.
radar is also terrible at vertical resolution and detecting stationary objects as it has the same response as the background.
If one camera is obscured it would safely pull over. Forward facing cams are behind the wiper. The rider might be asked to clean the camera and it will continue driving. Detecting obscured cameras and finding a spot to pull over is an amazingly simple problem compared to 99.9999% of all autonomous KMs accident free.
there is no way any government is going to approve autonomous vehicles on the basis that the occupant who may be a 10 year old child or a drunk has to get out mid journey and wash the cameras, especially if a competing brand can demonstrate triple redundancy that can still work if any of the vision/radar/lidar fail And when that dpredundancy doesnt come at a cost premium.Indeed there is no autonomous fast moving human carrying device that I’m aware of without complete full system redundancy. Even the humble lift car has two distinct methods to prevent freefall.
Also which autonomous car carrying passengers and no driver has achieved 99.9999% accident free around our streets? That more an aspiration and a theory. Its not yet reality.
Why do you think the government would require 10 year olds to have the ability to ride by themselves in a robo taxi before the robo taxi can be of significant societal value ?
what happens if you stick your 10 year old child in a taxi now and the driver has a heart attack? Should the government ban taxis?
also the triple redundancy of vision, lidar, radar isn’t an overlap as I’ve said. You can’t get vision from lidar, but you can get lidar from vision. Even if your AV had all 3 sensors, if a camera fails you will need to clean that camera because it’s the most important sensor. If that sensor isn’t obscured isn’t critical to drive, then having a vision based failure mode is just as safe as supplementing with lidar/radar. So again, having lidar/radar doesn’t fix anything and only makes it more complex and error probe.
Cameras being obscured is solvable problem via operational means. If a camera gets dirty, there are failure modes the system can fall back to… just like an elevator system. That’s a strange argument to make by the way, most people were incredibly negative and fearful towards automated elevators in the 1920s because they were misunderstood too. There was all kind of FUD surrounding them too.
Assuming 3mil HW3 on the road, Tesla can demonstrate whatever number the relevant authorities want. As I’ve said, this is about data, not your perception of safe/unsafe. Safety is an emergent property of the loss function. It would take Tesla a week/month/quarter to collect 1 million kms of shadow mode data in whatever city to validate performance.
Suggest you go and advise Professor Shashua.
Mobileye True Redundancy | Realistic Path for AVs at Scale
Mobileye's two separate self-driving systems, one based on cameras alone, and one based on radar-LiDAR alone, provide True Redundancy for safer, more scalable AV systems.
Shock-On-T
⚡️⚡️⚡️⚡️⚡️
10 year old is just an example of a passenger that couldn't be expected to clean the cameras. The car could just as easily be carrying no one at all, or a blind adult etc etc.
Shock-On-T
⚡️⚡️⚡️⚡️⚡️
Yes, it is an overlap.
In fact, this thread started in particular lamenting the loss of radar, which can bounce under the car in front and 'see' through fog.
Radar cruise control is mainstream and very reliable at avoiding the usual 'run up the back of someone' accident. Seems a shame to lose it.
Suggest you go and advise Professor Shashua.
Mobileye True Redundancy | Realistic Path for AVs at Scale
Mobileye's two separate self-driving systems, one based on cameras alone, and one based on radar-LiDAR alone, provide True Redundancy for safer, more scalable AV systems.www.mobileye.com
mobileye system requires HD maps in order to drive in LIDAR only mode and will stop at every intersection if there are no other cars to give confirmation of traffic light control. Their approach is totally valid it’s just not as scalable. MobilEye readily admit driving with cameras only is not only possible but preferable. The lidar system is there to appease conservative OEM partners.
lidar system is like a set of guarantees on the vehicles position relative to the environment. With a lidar stack, you can guarantee that it won’t hit a curb (assuming HD map is up to date) as it has a ~mm accurate 3D map of where the curb is ahead of time.
it’s also slightly better in object detection recall, although that gap is quickly closing, given advancements in 4D annotation and labelling.
so assuming the vision only bird's eye view net is able to determine where curbs, the intersection layout, where other agents are etc then LIDAR provides zero incremental value. In fact, the complexity of resolving arguments between sensors is a brittle trade off in what to trust.
The 3 front facing cameras are behind the wind screen wiper. If the repeater cams become obscured, the driving policy takes this into account to get to a safe place.
who knows maybe the robo taxi repeater modules will have a little wiper? Maybe this can be easily installed by fleet operators before they send their car off to be a robotaxi?
This is a totally solvable operational challenge. Not some smoking gun of why vision only is not possible.
Have you read any of my other posts where I address this? If you are driving through this supposed fog where you can't see a few metres in front of you, no amount of additional sensors will make that scenario safe.
It will be just as important to be able to see the colour of traffic light or the hazard lights on a car as it is accurate distances to arbitrary points.
Is not like a vision system assumes that it has perfect weather all the time and if there is fog it drives full speed blissfully unaware at the impeded visibility. Through simple annotation and training cycles, a NN can even when the road surface is totally covered in snow or fog make estimations of where the road goes and then drive slow enough to minimise risk of collision if an object enters detection range suddenly. This is exactly what humans do today except a CNN will be able to do it with incredible attention, 360 view and 1/10th human reaction time.
Shock-On-T
⚡️⚡️⚡️⚡️⚡️
That's my point exactly.
The website no longer says it, but it used to say that the current cars are capable of joining the "Tesla Network" or whatever their robotaxi service was called.
I just don't think they ever will.
Maybe the next gen cars will.
To be fair, you really dont have a clue how intel mobile eye’s new proprietary system works as they simply dont publish their IP, and given their number of clients its seems they all think Its very scalable.
The lidar systems purpose is outlined on their website, and it seems intel and the lidar partner have come up with a low cost solution to it. They also make it clear that vision is the basis of the system.
Wol747
Active Member
I think the thing that will possibly kill it is not safety, but what happens when the car can't decide what to do in the many "edge cases". No problem when there's a "driver" but robotaxis? And again, no problem if there are one or two - but fleets of vehicles, stopping - safely - because they can't negotiate something?
ShockOnT's correct - robotaxis aren't going to be on our roads on our current hardware and probably not in my lifetime.
ShockOnT's correct - robotaxis aren't going to be on our roads on our current hardware and probably not in my lifetime.
Its certainly possible that current gen cars are stuck as a L2 system. However the L2 capability would cover the entire driving task. I would be pretty happy with that outcome even if I can't send my car off to make me $20k per year in robo taxi.
It may be due to governments and regulations that Tesla don't launch a robo taxi service in Australia.
As I've said, the LIDAR approach MobilEye have is perfectly valid. I have no issues with sensor fusion and I have no doubt it works well. LIDAR and to a certain extent radar gives you a shortcut to say 50% of the self driving task being solved but no value in obtaining the 50% --> 99.9999% of the task. This is because the remaining percentages (especially the nines after the decimal point) can only be solved through a generalised AI solution that is able to comprehend a road system that is designed for humans. Almost all the "edge cases" people refer to are not going to be solved by additional sensors and the ones that can, are too dangerous to drive at any speed that's different to a human anyway.
Actually this is no entirely true. A driver makes mistakes too. The worst accident Waymo have reported was when a safety driver ignored the fact that the car was slowing down for a green light and punched the accelerator. The car was actually slowing down because another vehicle was about to run a red light as cross traffic and the autonomous system had correctly predicted this. In that case, it would have been better to simply not have a driver in the car at all.
There will be disagreement with the human driver based purely on comfort and preference. This happens way, way more frequently that emergency disengagement (usually because drivers are good citizens and disengage early). Remote tele-operators like what Waymo use can be a better way to guide a RoboTaxi out of a tricky situation than it is for a human being the wheel.
Another example of this is a car broken down in a lane that you have to give way to. In order to proceed the car would need to break the road rules and to do that safely, understand that the car is broken down. Does LIDAR help with this? No, the only way to resolve this is to have a sufficiently trained NN with an output that predicts whether a car is inactive or active, or have remote tele-operators give guidance to the car when it has uncertainty. Either way, only a very well trained vision and NN can work in this case.
Short term pain for long term gain.
A pseudo-lidar CNN will give AP a depth map that a radar unit could only dream of providing. It will also correlate pixels to depth (which currently it cannot do, there are seperate radar responses and seperate vision inputs)
This means (as I've already explained in depth a few posts back) that the AP stack will not only be able to detect the car in front of the car but also a myriad of other partially occluded objects that may stationary, different radar reflectivity etc.
I find it so funny how people demand that AP improves constantly yet are not willing to accept the cost to this improvement.
Shock-On-T
⚡️⚡️⚡️⚡️⚡️
Lidar is line-of-sight. Can't see through fog or bounce under car in front to see occluded second vehicle.
Anyway, Tesla is not replacing radar with lidar, it's just replacing radar with improved visual software. That improved visual software relies on the cameras, which aren't ready yet.
Let's put it another way: if we ever see a fleet of robotaxis on AP3 I will buy you a coke.
Similar threads
