Videos like this are one reason why camera only system are flawed.
Http://www.youtube.com/watch?v=VE29qw4pLaA
Http://www.youtube.com/watch?v=VE29qw4pLaA
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
MobilEye's approach
- Thread starter lunitiks
- Start date
stopcrazypp
Well-Known Member
I believe Subaru is the only one using a camera-only system however. Others at least have a radar (or multiple).
One forward radar with narrow fov is practically useless. first of all, that is literally a camera only system in urban environment when you are doing driving maneuvers. For example a U turn, or turning in the middle yellow lane. There are dozen more examples.
Lastly look at all the accidents that AP cars get into today and the rear ends because radar and camera doesn't see the car ahead.
that tesla radar is good for highway driving when you are in one lane and even with that it sometimes fails.
Sure, to the the end user, a WDR image and HDR image may look similar, but on a sensor level things are a bit different. The terminology is confusing, though - partly because a lot of things are marketed as HDR. A WDR camera sensor adjusts the gain automatically for the parts of the cell which are under or over exposed to create a more balanced image... this is a pretty effective technique, but has some drawbacks. The effect often creates a halo effect around high contrast objects, the SDR sensor can produce a noisy image in low-light, and it can still be relatively slow to adjust to changing light conditions. Where WDR excels is in static camera situations, like security cameras.
A HDR sensor can 'see' a much broader spectrum in one go and the resulting captured image contains all of that info (again, to us looking at things on a computer screen the image may look similar, but there's a lot more info "under" there that the computer can use). HDR sensors can outperform our eyes. Then, there's the grey area of HDR lenses, but that's pretty futuristic stuff.
Indeed, you can create an HDR image by using multiple images taken with different exposures (usually 3) and merging them together to get the desired effect, and is a popular technique in modern photography.
Obviously the cameras they use in HW2 are enough for what Tesla want to do with the current cars.
Anyway, originally I was just wondering why they don't use more generally cutting edge camera tech in these situations, given the high price and critical function they play.
Off topic, but I wonder if the resolution and DR difference in the HW2 cameras vs Mobileye's are partly to explain why we don't have auto-headlight dimming yet... Mobileye's technique relied on recognition of the specific bloom/flare signature of headlights on the camera sensor - basically understanding quite specifically the blobs of overexposed bright light. Maybe Tesla's HW2 cameras see that differently, or not as over exposed blobs... and as such the technique has to be tweaked or reinvented to compensate for what the HW2 cameras see. I have no idea... just musing.
Kant.Ing
Member
Mobileye's solution keeps amazing me, their computer vision since several years ago can tell the cars' 4 sides and representing them as 3D boxes/cubes reliably and it can also predict the cars traverse courses. However, nVidia's Drive Net and Tesla's FSD demo flag cars as 2D rectangles and might be insufficient realizing their courses and so that the color identifiers keep flashing between "in-path" and static objects (in Tesla Vision's FSD time-lapse video). Add that Mobileye also claims their computer vision can tell where the pedestrians are looking at. It's correct to think that using cameras alone can replace human vision, but I believe we have to aim higher to outperform human's vision and senses with a backup vision or database for the surroundings. Nowadays people consider sensing is pretty much done, true in general, but I am afraid not at molecular level. This is sensing alone.
Trivial though, but this might also explain why AP1 can display the cars in the adjacent lanes with good perspectives correctly.
Driving policy definitely involves more behavior learning and needs a high level of social skills, not only to all other cars but also to its own occupants. Even some humans are bad at this. Every little bit of information derived from sensing and mapping can contribute to its friendliness a lot. And I believe this is the key that will differentiate these FSD approaches, plus (maybe) new harness and/or active suspension for the occupants and cargo.
Trivial though, but this might also explain why AP1 can display the cars in the adjacent lanes with good perspectives correctly.
Driving policy definitely involves more behavior learning and needs a high level of social skills, not only to all other cars but also to its own occupants. Even some humans are bad at this. Every little bit of information derived from sensing and mapping can contribute to its friendliness a lot. And I believe this is the key that will differentiate these FSD approaches, plus (maybe) new harness and/or active suspension for the occupants and cargo.
Mobileye's EyeQ3 vision is no doubt very impressive... however, I wouldn't presume that the computer vision capacities *as of EyeQ3/AP1* are better than Tesla Vision's current in-development status (which is not what we have in our cars right now). Remember - only a tiny fraction of that is exposed to us via AP2 at the moment... Of course, Mobileye have massively upped their game since EyeQ3 and remain at the forefront of the industry.
As far as I know, Mobileye's human gesture and sight-line detection is a feature of EyeQ4. This isn't a part of the EyeQ3/AP1 hardware suite... however, pedestrian detection very much is... as well as signs, animals etc...
However, I would be very very surprised if Tesla Vision isn't capable of detecting these objects. Whether or not it reacts to them currently, or displays them on the UI side of things is a different matter; but I would almost guarantee that it is capable of seeing them.
Computer vision took a total u-turn with ImageNet, followed by improvements with AlexNet and so on... Facebook have been doing incredible things... as have Microsoft, Google etc. Basically, big cash flowed into this space since the EyeQ3 development days.
So, I have almost no doubt that Tesla Vision can distinguish between a car and a motorbike. It can probably even tell you what kind of motorbike it is, and what year it was made... and whether or not a guy or a girl is riding it, if they're looking at you, and whether they're happy or sad. Oh, and also where their clothes are from.
As Nvidia's CEO attests: "It's not a detection problem". Computer vision for autonomous cars is practically a solved problem. The issue now is the AI that's required to actually drive the car... and that's a whole other kettle of fish where Mobileye may well still have the lead (I think they will have long since moved on from trying to be leaders in computer vision) but ... who knows?!
Perhaps one reason that AP2 currently doesn't display cars in adjacent lanes is that they're waiting until they use all the cameras - or at least most of them. Once all the cameras are activated (at least all the forward facing), they could probably do some cool things and hit with a very meaningful and impactful update to the UI. It would be one of the most 'obvious' differentiators between the EAP and AP1, and I imagine it would go down very well with customers and sales people alike.
As far as I know, Mobileye's human gesture and sight-line detection is a feature of EyeQ4. This isn't a part of the EyeQ3/AP1 hardware suite... however, pedestrian detection very much is... as well as signs, animals etc...
However, I would be very very surprised if Tesla Vision isn't capable of detecting these objects. Whether or not it reacts to them currently, or displays them on the UI side of things is a different matter; but I would almost guarantee that it is capable of seeing them.
Computer vision took a total u-turn with ImageNet, followed by improvements with AlexNet and so on... Facebook have been doing incredible things... as have Microsoft, Google etc. Basically, big cash flowed into this space since the EyeQ3 development days.
So, I have almost no doubt that Tesla Vision can distinguish between a car and a motorbike. It can probably even tell you what kind of motorbike it is, and what year it was made... and whether or not a guy or a girl is riding it, if they're looking at you, and whether they're happy or sad. Oh, and also where their clothes are from.
As Nvidia's CEO attests: "It's not a detection problem". Computer vision for autonomous cars is practically a solved problem. The issue now is the AI that's required to actually drive the car... and that's a whole other kettle of fish where Mobileye may well still have the lead (I think they will have long since moved on from trying to be leaders in computer vision) but ... who knows?!
Perhaps one reason that AP2 currently doesn't display cars in adjacent lanes is that they're waiting until they use all the cameras - or at least most of them. Once all the cameras are activated (at least all the forward facing), they could probably do some cool things and hit with a very meaningful and impactful update to the UI. It would be one of the most 'obvious' differentiators between the EAP and AP1, and I imagine it would go down very well with customers and sales people alike.
Mobileye brags a lot about "their" 8 camera setup, "their" triple camera, "their" cloud learning/mapping, etc. The same principles Tesla is using. Did the two companies come up with these ideas at the same time? Is Mobileye stealing ideas from Tesla, or is it vice versa? Reasons to belive there will be an epic legal battle in the near future?
Mobileye will certainly be suing tesla in the future. That's almost definite. They have lot of patents and everything they introduced years ago, tesla is using. Right now to the exact details.
If there is blatant infringement and you sit there twiddling your thumbs then you are a fool.
Please elaborate on why you think this because I know a thing or two about patent litigation.
stopcrazypp
Well-Known Member
I'm also curious what patents Tesla might be violating. Having a triple FOV front view is not a new idea (there are other examples, although largely using radar). And someone posted the mobileye triple cam setup in some other thread and the layout is different than Tesla's. My understanding is that patents must be very specific (can't just patent a general idea).
AP2.0 Cameras: Capabilities and Limitations?
I'm also curious what advantage there is in holding off from suing if Mobileye already knows Tesla is violating their patents.
I haven't reviewed any of the patent claims but just because you have a patent doesn't make it valuable, useful or able to withstand scrutiny. Like all things, some patents are better than others.
I doubt Mobileye's particular portfolio would be anything other than art that can be a practiced around or FRAND.
Mic drop.
I doubt Mobileye's particular portfolio would be anything other than art that can be a practiced around or FRAND.
Mic drop.
Mobileye has a horde of patents relating to driverless cars specifically vision. you think they won't come for tesla when the time is right?
here are some of their patents.
Patents by Assignee Mobileye Vision Technologies Ltd. - Justia Patents Search
here is mobileye 8 camera system that tesla copied right down to exact camera placement.
36mins 24secs
Tesla for one doesn't even have AP1 equivalent AP2 nor have they started using their 8 cameras. Companies usually wait till the product is ready to be sold to consumers or when the company being sued is profitable in order to get the most from them. No point suing a company when they are in early development. Its the software that matters and their software right now is half baked. 1% of Self driving software is done and maybe 25% of ADAS is done. They literally just started doing maps. That's pretty bare bone to be suing for especially if alot of what you are suing for is software based. Take for example the oculus lawsuit which happened after Facebook bought them out.
Secondly, mobileye doesn't want to be involved in dispute now that they are in the period of gathering partners.
There is a reason that there have not been one patent lawsuit over self driving cars. Because no one has one yet. But when its here, I promise you, you will see alliances and patent agreements and lawsuits left and right. Companies will team up to attack other companies and the patent trolls will be resurrected Just as you saw with the smartphone era. Right now, no one is making money off it, so there's nothing to litigate for.
Last edited:
The criticism of Lidar mapping is the need to drive specialized vehicles around to create the maps. There was already a company driving around specialized vehicles even before LIDAR mapping: Google
Individual companies, and especially Tesla, don't have Google's resources to drive specialized cars everywhere. Fleet/shared camera mapping has become a thing to try and match Google's mapping capabilities. It might work, but Google's approach is simpler, surer and more mature.
Tesla is now playing catchup to mobileye who is playing catchup to Waymo. Zealots do the math and find Tesla in the autonomous driving lead.
Individual companies, and especially Tesla, don't have Google's resources to drive specialized cars everywhere. Fleet/shared camera mapping has become a thing to try and match Google's mapping capabilities. It might work, but Google's approach is simpler, surer and more mature.
Tesla is now playing catchup to mobileye who is playing catchup to Waymo. Zealots do the math and find Tesla in the autonomous driving lead.
stopcrazypp
Well-Known Member
The patents you talked about (triple cam enclosure) and ideas (8 camera placement, I didn't see a patent that covers this) is hardware. The video you show doesn't actually show the camera placement (where it mounts on the car), only the angles. However, the angles are different on the Tesla. The forward facing side cameras cover the side view for Tesla, while the rear facing side cameras cover the side view for Mobileye.
Well Mobileye already had a very nasty and public breaking with Tesla, not sure if a patent lawsuit would be any worse.
I guess you missed the big lawsuit between Waymo and Uber? It makes sense to sue now and get an injunction to freeze the process of development (as Waymo is doing) since everyone is racing to get there first. If Mobileye can get an injunction to force Tesla to stop selling vehicles with supposedly infringing hardware, that will set Tesla back far more than waiting for Tesla to succeed in developing the system.
Alphabet's Waymo files patent and trade secret lawsuit against Uber - IPWatchdog.com | Patents & Patent Law
Also I'm not a lawyer, but I always heard that if you don't litigate in a timely manner you may be considered as abandoning your claim (@croman would know better about that).
those are just some of their patents.
same angle
let me post a video.
I didn't miss anything. that lawsuit deals strictly with trade secrets that was stolen by a former employee and has nothing to do with actual patents. If you actually followed any part of that lawsuit, you will know this. That lawsuit will also evolve to be a criminal investigation.
Last edited:
Similar threads
- Article
