-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
Neural Networks
- Thread starter lunitiks
- Start date
Thank you @jimmy_d I was waiting for this.
Excellent! verygreen mentioned something like this too.
These are for precision localization, which is necessary to use HD maps.
Excellent point.
I mean, it I wouldn't say it is recognizing them functionally....
It is detecting that there IS NOT road there, which I say is different from detecting that there IS an unknown/unclassified object there.
So this was exactly my first reaction... and I still think it is possible that this is the output of a NN. But then I thought, this path could be the lanes(output from NN) then some very simple algorithm that merges them into a path. Which would explain why they behave that way, if they are a directly correlated to the lane output which is probabilistic with very little processing of them.
But certainly its very possible that the 2nd layer NN that understands the shape of lanes is also outputting this path.
I am also not sure AP uses the orange predicted path to steer the vehicle. I think its possible they are testing this probabilistic path in parallel with using some other system for generating and following a trajectory. Or this is purely a visualization/dev tool.
+1
+1
Yes many ADAS systems are using HD maps to improve performance,
But Every commercial self driving product /service that is planning on making some kind of launch... commercial service or consumer product uses HD maps, and I expect Tesla to do the same as Autopilot matures.
And the evidence from a few years ago, I am not sure what you are referring too, but if I am thinking of the same thing you are, then I would not call that HD maps, but just regular maps.... (using a rough map and GPS to extract various road attributes and meta data vs using centimeter accurate maps with localization assets for the AV to detect to localize and use then use the map for path planning and even steering the vehicle)
I am also very curious to see evidence to that, @verygreen has from time to time said he has seen traces of newer HD maps in recent updates, but nothing that appears to be enabled yet.
Excellent! verygreen mentioned something like this too.
These are for precision localization, which is necessary to use HD maps.
Excellent point.
I mean, it I wouldn't say it is recognizing them functionally....
It is detecting that there IS NOT road there, which I say is different from detecting that there IS an unknown/unclassified object there.
So this was exactly my first reaction... and I still think it is possible that this is the output of a NN. But then I thought, this path could be the lanes(output from NN) then some very simple algorithm that merges them into a path. Which would explain why they behave that way, if they are a directly correlated to the lane output which is probabilistic with very little processing of them.
But certainly its very possible that the 2nd layer NN that understands the shape of lanes is also outputting this path.
I am also not sure AP uses the orange predicted path to steer the vehicle. I think its possible they are testing this probabilistic path in parallel with using some other system for generating and following a trajectory. Or this is purely a visualization/dev tool.
+1
+1
Yes many ADAS systems are using HD maps to improve performance,
But Every commercial self driving product /service that is planning on making some kind of launch... commercial service or consumer product uses HD maps, and I expect Tesla to do the same as Autopilot matures.
And the evidence from a few years ago, I am not sure what you are referring too, but if I am thinking of the same thing you are, then I would not call that HD maps, but just regular maps.... (using a rough map and GPS to extract various road attributes and meta data vs using centimeter accurate maps with localization assets for the AV to detect to localize and use then use the map for path planning and even steering the vehicle)
I am also very curious to see evidence to that, @verygreen has from time to time said he has seen traces of newer HD maps in recent updates, but nothing that appears to be enabled yet.
im.thatoneguy
Member
I would say that the lack of evidence is proof that it's not. The yellow drive path appears to be completely constructed by the info being displayed. I would assume that we would see evidence of it using the maps to "cheat" and have premonitions of what is up ahead beyond what we see (at least in the highway demos). Instead it acts 'surprised' by every new twist and turn of the lane.
Right, verygreen said all the references to maps seem to be not enabled. So it is clear that the current version of 8.1 is not using HD maps
strangecosmos
Non-Member
Thanks for your analysis, Jimmy!
Could it be that Autopilot knows the curve of the road ahead — before actually seeing it — because it’s using HD maps? This is one of the intended uses of HD maps: to know what’s up ahead before the car’s sensors see it.
Could it be that Autopilot knows the curve of the road ahead — before actually seeing it — because it’s using HD maps? This is one of the intended uses of HD maps: to know what’s up ahead before the car’s sensors see it.
Tam
Well-Known Member
Even on straight (not curvy) intersections:
When I bought AP2 in March 2017, it was really bad at any breaks in lane markings such as expected at most intersections.
Now, 1.5 years later, it's much better but it is still unable to navigate in certain wide intersections that don't have lane markings.
jimmy_d
Deep Learning Dork
I'm also inclined to the view that the path prediction shown on the video is being generated entirely from video input. I don't actually know, personally, if there's any evidence of maps being used - I defer to others on that. I just thought the question was worth posing since the path prediction seems to be very good, and I wanted to know if I could be confident that it definitely wasn't looking ahead by using some sort of maps input.
That the question seems relevant is, to me, a symptom that the feature is impressive.
jimmy_d
Deep Learning Dork
I should add that, in my review of the binaries I was unable to find any evidence of map input being used and I consider that to be pretty strong evidence that it is not. But of course, if @verygreen has seen map information flowing into AP2 then that should also be weighed.
pyraca
Member
Observations driving on AP and whether it uses Maps or not.
1. When in exit only lanes from HW (like US 101N at Marine Parkway) the car will slow down and correctly take the exit on its own. It will stop if there are cars at the traffic light at the end of that exit. Once light turns green, cars ahead move and turn right, I am the only one heading forward, it does make this split second decision to go forward & then it drives through what appears to be pretty unmarked intersection (albeit at 25Mph). it somehow must know that the exit lane is being taken, and that most likely comes from maps, no ?
2. When I took a trip up north through and was at Shasta, CA the driving in the hills was interesting on AP since the car did in some cases slow down when taking curves in these curvy parts. It was a deliberate, ahead of the turn slow down by 5-10mph from the top AP speed that I had set. I would have blown by without speed reduction ... again it could have been visual only but that behavior I thought changed at some point through one of the releases after the butter smooth AP from march this year.
3. Counter argument is that on some exits, especially tight right loop interchanges (South bound 101, to 237 at Mt View, CA) the car will slow down but does not always make the loop in a way that I find comforting. A good driver will adjust the steering wheel in such cases minimally and in some will find the "right" position to take that loop - the car kinda keeps going forward and guessing almost.
4. Highway interchanges (long right or left turns between highways) are taken also with reduced speed but not to the extend that exits are.
1. When in exit only lanes from HW (like US 101N at Marine Parkway) the car will slow down and correctly take the exit on its own. It will stop if there are cars at the traffic light at the end of that exit. Once light turns green, cars ahead move and turn right, I am the only one heading forward, it does make this split second decision to go forward & then it drives through what appears to be pretty unmarked intersection (albeit at 25Mph). it somehow must know that the exit lane is being taken, and that most likely comes from maps, no ?
2. When I took a trip up north through and was at Shasta, CA the driving in the hills was interesting on AP since the car did in some cases slow down when taking curves in these curvy parts. It was a deliberate, ahead of the turn slow down by 5-10mph from the top AP speed that I had set. I would have blown by without speed reduction ... again it could have been visual only but that behavior I thought changed at some point through one of the releases after the butter smooth AP from march this year.
3. Counter argument is that on some exits, especially tight right loop interchanges (South bound 101, to 237 at Mt View, CA) the car will slow down but does not always make the loop in a way that I find comforting. A good driver will adjust the steering wheel in such cases minimally and in some will find the "right" position to take that loop - the car kinda keeps going forward and guessing almost.
4. Highway interchanges (long right or left turns between highways) are taken also with reduced speed but not to the extend that exits are.
It hasn't been confirmed yet, but the expectation is that V9 is probably using some form of "HD" maps (how HD is tbd) specifically for highways (Navigation on Autopilot).
@jimmy_d above was only analyzing and discussion V8 NNs. I don't think anyone has gotten a hold of the new NNs yet. And no one has published a public breakdown of the V9 firmware.
HD maps can mean different things to different development groups.
It is often meant to mean localization. Ie. GPS tells you where you are within a few feet (if it is working correctly, sometimes it isn’t, in which case it’ll be off by 30 feet or so), but to get accuracy within a few centimeters, you need something else. The something else that Mobileye uses is a “map” of side of road signs. The speed limit signs, turn ahead signs, mile markers, points of interest signs, etc. Any sign that can be uniquely identified within a, say, 50 foot radius. The vision systems looks for and tracks these signs and does a continuous range estimation to them via visual cues (how big they are getting relative to the speed the car is driving). This is then compared to its HD map for the area and via dead recogning, can tell the AP software where it is within an inch or so.
I’ve often thought this is an important missing ingredient from Tesla’s AP. Among other things, it is the difference between driving on a road you’ve never driven before and one you’ve driven a dozen times before. Humans drive new roads much more cautiously than roads they know well. It is asking a lot for a vision system to drive an unknown road as well as an experienced human when the vision system doesn’t know what the road should do in 100 feet.
It is often meant to mean localization. Ie. GPS tells you where you are within a few feet (if it is working correctly, sometimes it isn’t, in which case it’ll be off by 30 feet or so), but to get accuracy within a few centimeters, you need something else. The something else that Mobileye uses is a “map” of side of road signs. The speed limit signs, turn ahead signs, mile markers, points of interest signs, etc. Any sign that can be uniquely identified within a, say, 50 foot radius. The vision systems looks for and tracks these signs and does a continuous range estimation to them via visual cues (how big they are getting relative to the speed the car is driving). This is then compared to its HD map for the area and via dead recogning, can tell the AP software where it is within an inch or so.
I’ve often thought this is an important missing ingredient from Tesla’s AP. Among other things, it is the difference between driving on a road you’ve never driven before and one you’ve driven a dozen times before. Humans drive new roads much more cautiously than roads they know well. It is asking a lot for a vision system to drive an unknown road as well as an experienced human when the vision system doesn’t know what the road should do in 100 feet.
Exactly, they are creating / using maps, or at-least getting ready to start creating/using HD maps
Similar threads
- Article
