.48 feels like AP2 finally passed AP1

[Ben W]

Cross-posting from a different thread: this is my brand-new Model 3 with 2017.48.15. Autopilot is attempting to navigate the transition from 101W to 405S, and failing twice within 30 seconds. I had expected Autopilot to be able to handle this because it's essentially just lane-keeping: a single lane to stay in, the nav knew I was taking this route, the speed wasn't very fast, and there was a car in front of me to follow. But first the car tried to drive into the left guardrail on the turn, then 30 seconds later it tried to drive into a bunch of posts on the right shoulder. Curious what you make of this. Was I expecting too much of Autopilot here? Do you think .52 would improve this?

 

[cyphear]

Was I expecting too much of Autopilot here? Do you think .52 would improve this?

Yes, these looks like pretty normal situations where autopilot would fail. .52 is not out, but I doubt .50 would do much better. You'll learn where it is an is not good soon, and anticipate this.

 

[croman]

Yes, these looks like pretty normal situations where autopilot would fail. .52 is not out, but I doubt .50 would do much better. You'll learn where it is an is not good soon, and anticipate this.

.48.15 is the most recent model 3 fw. It's equivalent to 50.3 for ap2 S or x. AP isn't perfect and changes so much fw to fw. I agree that experience with the system is critical to understanding likely behavior and adjusting use accordingly.

 

[GoTslaGo]

Cross-posting from a different thread: this is my brand-new Model 3 with 2017.48.15. Autopilot is attempting to navigate the transition from 101W to 405S, and failing twice within 30 seconds. I had expected Autopilot to be able to handle this because it's essentially just lane-keeping: a single lane to stay in, the nav knew I was taking this route, the speed wasn't very fast, and there was a car in front of me to follow. But first the car tried to drive into the left guardrail on the turn, then 30 seconds later it tried to drive into a bunch of posts on the right shoulder. Curious what you make of this. Was I expecting too much of Autopilot here? Do you think .52 would improve this?

Initially the lane turned to the right more sharply, which is why your 3 wanted to drive into the guardrail the first time.

The second instance, the color difference between the asphault (dark and light) likely took priority for the lane rather than the posts on the right shoulder. It was trying to follow the “lane” it recognized/created with the dark/light road.

If you could move your dash cam back to capture your 3s blue/gray lines and car icon, then I would suspect that you would see the car recognize the dark/light road as the main road with “blue” lane markers—thus driving you into the posts.

 

[Tintorera]

Curious what you make of this. Was I expecting too much of Autopilot here? Do you think .52 would improve this?

Based on my experience of EAP, this was expected. Seems to me you are giving more playroom for the EAP than what I’m comfortable with.

As others have pointed out, be careful with how much confidence you give EAP. As your familiarity and experience with the system increase, you gain more knowledge as to it’s capabilities and limitations.

Stay safe and enjoy your new ride!

 

[daktari]

Sorry, but in my experience, those speeds in that kind of turns are not something AP2 can handle atm.
It performs better a bit below speed limit. In S and X speed is easily adjustable with the cruise stalk 3 is missing.

 

[jimmy_d]

Hadn't seen "the blue car" myself using AP2 either. Interestingly, this AP2 (50.3) got thrown off by what on the IC looks like another car crossing its lane and it briefly visualised that car in blue: youtube.com/watch?v=lJwp__0T2nI&t=22m40s

More subject matter from that video: duct taping a few sensors (and side camera’s - although, not relevant here yet, rendering the car’s right-hand side blind) doesn’t disable auto lane change. More like the opposite: allows for more dangerous AP behaviour.

Interesting video - haven't seen that. Based on what @verygreen has seen the side cameras aren't active during lane changes, so this fits with that. And there wasn't even a vision NN attached to the side cameras until really recently, so it's probably safe to say that the car was relying on ultrasonics to validate the availability of adjacent lanes up until very recently and very probably still is.

 

[BigD0g]

It still is, the calibration is still at 0% for side repeaters on .53

 

[jimmy_d]

Cross-posting from a different thread: this is my brand-new Model 3 with 2017.48.15. Autopilot is attempting to navigate the transition from 101W to 405S, and failing twice within 30 seconds. I had expected Autopilot to be able to handle this because it's essentially just lane-keeping: a single lane to stay in, the nav knew I was taking this route, the speed wasn't very fast, and there was a car in front of me to follow. But first the car tried to drive into the left guardrail on the turn, then 30 seconds later it tried to drive into a bunch of posts on the right shoulder. Curious what you make of this. Was I expecting too much of Autopilot here? Do you think .52 would improve this?

Yeah, I see similar failures in recent AP2 on MS, and also on AP1 on MS. Not at all surprised that you're seeing them in M3.

The first one was classic: you hit the decreasing radius portion of the ramp at speed and that never turns out well. On some ramps the car will slow before the curve tightens and on those you have a decent chance of pulling it off, but it will fail close to 100% of the time if you hit a ramp at highway speed and don't experience substantial braking before the curve tightens. The second one I'd describe as a lane hugging fail since the car seemed to just gradually lose the curve as the radius decreased on what is otherwise a pretty tame curve. Aside from hill cresting I think this is the biggest fail in AP2 as of 50.2 - AP1 doesn't really see this failure.

You didn't mention your speed and it's hard to tell from the video. Generally AP1 or AP2 can do a typical 90 degree+ freeway-to-freeway ramp at 40MPH or maybe a bit above, but it seems like most of the FWY-FWY ramps in LA don't drop the speed enough for lane following to handle them gracefully.

BTW - On AP2 the presence of a lead car doesn't have any effect on laneholding AFAIK, and even on recent AP1 it only helps if the lead car isn't too much further around the curve than you. On tighter ramps AP1 loses the lead car and TACC can even suddenly accelerate in those situations.

I like to play around with ramps as a lane-following benchmark even though they don't work very well, but I haven't seen them improve any in quite a while. Ten months ago when AP first started braking for curves automatically I was hopeful that we'd see the ramps become generally doable but that never materialized.

 

[jimmy_d]

It still is, the calibration is still at 0% for side repeaters on .53

I assume you mean 50.3 there.

That's interesting about there being a calibration phase on the repeaters. Based on stuff @verygreen showed me (code snippets and the NN prototxt files) I have more or less convinced myself that 'calibration' for main and narrow is undistortion and raster alignment for those two cameras to enable raster aligned stereo. What I've heard about when calibration is needed and when it happens matches pretty nicely with calibrating a precise alignment between main and narrow, which is definitely needed for raster aligned stereo depth extraction. And the fact that the the same NN is used for both kind of suggests that the FOVs are being matched up already, which also supports the idea that stereo depth is being extracted. (Though that would be in a separate process from the NN).

Of course, there are other kinds of useful camera calibrations that we can imagine, but if we just restrict ourselves to pixel mapping alignment of overlapping FOV cameras the repeaters wouldn't have anything to match up with until the pillars become active.

You have an opinion on any of this?

 

[BigD0g]

I assume you mean 50.3 there.

That's interesting about there being a calibration phase on the repeaters. Based on stuff @verygreen showed me (code snippets and the NN prototxt files) I have more or less convinced myself that 'calibration' for main and narrow is undistortion and raster alignment for those two cameras to enable raster aligned stereo. What I've heard about when calibration is needed and when it happens matches pretty nicely with calibrating a precise alignment between main and narrow, which is definitely needed for raster aligned stereo depth extraction. And the fact that the the same NN is used for both kind of suggests that the FOVs are being matched up already, which also supports the idea that stereo depth is being extracted. (Though that would be in a separate process from the NN).

Of course, there are other kinds of useful camera calibrations that we can imagine, but if we just restrict ourselves to pixel mapping alignment of overlapping FOV cameras the repeaters wouldn't have anything to match up with until the pillars become active.

You have an opinion on any of this?

Apologies, yes .50.3, there whole release numbering is ...

I have a theory, but I'm not a NN guy at all, and I merely like to dabble around. My idea is this, and I haven't seen code "yet" to give me a high degree of confidence of my idea. What if they two side repeaters also aligned to give stereoscopic vision behind the car, the back ends of the car is known. So there is no reason, they can't form a reverse stereoscopic vision in the rear to help with decisioning for fast approaching objects from the rear. Once the object get's close enough it can fall back to sonar and regular images. And this is all wild speculation, and I haven't seen any calibration code to give me confidence in my thinking.

 

[jimmy_d]

Apologies, yes .50.3, there whole release numbering is ...

I have a theory, but I'm not a NN guy at all, and I merely like to dabble around. My idea is this, and I haven't seen code "yet" to give me a high degree of confidence of my idea. What if they two side repeaters also aligned to give stereoscopic vision behind the car, the back ends of the car is known. So there is no reason, they can't form a reverse stereoscopic vision in the rear to help with decisioning for fast approaching objects from the rear. Once the object get's close enough it can fall back to sonar and regular images. And this is all wild speculation, and I haven't seen any calibration code to give me confidence in my thinking.

Since the repeaters are very close to the sides of the car, and the sides of the car are nearly parallel, the FOVs of the two repeaters won't overlap until you get very far from the back of the car - which more or less renders stereo moot for them as a camera pair. However the FOV of the repeaters will overlap with the rear, which gives the possibility using repeater / rear as a stereo pair to get the distance to an object to the side and rear of the vehicle within maybe 100 feet or so. The problem there is that rear plus repeater makes for a highly asymmetric pair so the processing needed is much heavier than what you need with a symmetric pair of cameras. I suspect that they won't use that in the near term because it's such a hard thing from a processing point of view - but maybe eventually. I sat down at one point to try to see if there was anyplace that AP2 *couldn't* do stereo with an asymmetric pair - and it turns out that you can do it everywhere except for a narrow band straight to the left of the car and straight to the right. Every other direction has at least two cameras covering it.

Are you thinking about the lane change / approaching vehicle problem?

You may be aware of this, but even without stereo it's possible to determine the distance to an approaching vehicle using just a single repeater. There's a computational technique called depth from context which, more or less, estimates distance based on the apparent size of an object in the FOV and it's location relative to other objects, pavement marks, etc. This is more or less how people judge the distance to other vehicles and their relative speed when driving a car. For reference, human stereo depth perception is only good out to a few car lengths; beyond that distance humans determine depth based on context and this same approach produces similar results when done with NNs. For things really close to the car - under 30 feet or so - stereo is better, but in that zone Tesla also has sonar. Detecting an approaching car in an adjacent lane is something that needs to work at distances of 30ft to maybe 200ft, and in this zone context is better than stereo.

Thanks for the feedback.

 

[buttershrimp]

Cross-posting from a different thread: this is my brand-new Model 3 with 2017.48.15. Autopilot is attempting to navigate the transition from 101W to 405S, and failing twice within 30 seconds. I had expected Autopilot to be able to handle this because it's essentially just lane-keeping: a single lane to stay in, the nav knew I was taking this route, the speed wasn't very fast, and there was a car in front of me to follow. But first the car tried to drive into the left guardrail on the turn, then 30 seconds later it tried to drive into a bunch of posts on the right shoulder. Curious what you make of this. Was I expecting too much of Autopilot here? Do you think .52 would improve this?

I think you came in too hot on the first disengagement and it probably would have handled it great if it were no more than 5 mph above speed limit maybe 10 mph with the road cambered like that.... your second disengagement surprised me though, it usually is would get stuff like that even with the mismatched pavement. Eventually software should auto adjust speed on that turn gracefully, my guess is when HD maps are enabled. The job posting @lunitiks posted on tesla for an inertial turn specialist (or whatever the hell its called) makes sense

 

[Electroman]

Cross-posting from a different thread: this is my brand-new Model 3 with 2017.48.15. Autopilot is attempting to navigate the transition from 101W to 405S, and failing twice within 30 seconds. I had expected Autopilot to be able to handle this because it's essentially just lane-keeping: a single lane to stay in, the nav knew I was taking this route, the speed wasn't very fast, and there was a car in front of me to follow. But first the car tried to drive into the left guardrail on the turn, then 30 seconds later it tried to drive into a bunch of posts on the right shoulder. Curious what you make of this. Was I expecting too much of Autopilot here? Do you think .52 would improve this?

Forget Autopilot. You should be crazy to be driving, even manually, and coming in too fast on that curve. I bet you were driving 10 to 15 mph over the limit on that curve. Try the same at speed limit or at most 5++, come back and report.

The 2nd one is obvious. The asphalt to concrete (or whatever the surface) color change was rally confusing for AP, especially after the lines faded.

 

[MarcusMaximus]

I think you came in too hot on the first disengagement and it probably would have handled it great if it were no more than 5 mph above speed limit maybe 10 mph with the road cambered like that.... your second disengagement surprised me though, it usually is would get stuff like that even with the mismatched pavement. Eventually software should auto adjust speed on that turn gracefully, my guess is when HD maps are enabled. The job posting @lunitiks posted on tesla for an inertial turn specialist (or whatever the hell its called) makes sense

The second one is pretty much the same situation that caused the 1 forced disengagement I had coming back to San Jose from San Diego.

 

[GoTslaGo]

Cross-posting from a different thread: this is my brand-new Model 3 with 2017.48.15. Autopilot is attempting to navigate the transition from 101W to 405S, and failing twice within 30 seconds. I had expected Autopilot to be able to handle this because it's essentially just lane-keeping: a single lane to stay in, the nav knew I was taking this route, the speed wasn't very fast, and there was a car in front of me to follow. But first the car tried to drive into the left guardrail on the turn, then 30 seconds later it tried to drive into a bunch of posts on the right shoulder. Curious what you make of this. Was I expecting too much of Autopilot here? Do you think .52 would improve this?

One more note. I would suggest you forward a copy of this to Tesla. I typically send detailed notes (location, speed, time, etc...). I think the video would be helpful for them to see. I usually send to [email protected].

Thanks for posting!

 

[Ben W]

I think you came in too hot on the first disengagement and it probably would have handled it great if it were no more than 5 mph above speed limit maybe 10 mph with the road cambered like that.... your second disengagement surprised me though, it usually is would get stuff like that even with the mismatched pavement. Eventually software should auto adjust speed on that turn gracefully, my guess is when HD maps are enabled. The job posting @lunitiks posted on tesla for an inertial turn specialist (or whatever the hell its called) makes sense

The video is deceiving because of the wide angle and very slow traffic on the freeway; I'd guess I was only going about 35-40mph (same speed as the car in front of me) on the straightaway before the curve. I had assumed that the Autopilot would slow down to match the car in front of me, and it did slightly at first, but then seemed to accelerate toward the guardrail (though it may merely have stopped braking).

The second disengagement surprised me too, since the right lane line was very clearly marked, and the difference in pavement was more subtle. I will forward the video and details to Tesla and see what they think. (Of course they will know exactly how fast I entered the curve )

 

[BigD0g]

Are you thinking about the lane change / approaching vehicle problem?

Yes, because of no rear facing radar, I figure they need something more then monocular vision looking back, I haven't done the math to know for sure, but I imagine at highway speeds the gaps close pretty quickly if say your coming from an on ramp to the highway thats running full bore at 77 mph.

I think my biggest fear with the current arraignment is merge events and how they will be handled, it seems to me to be a huge blindspot in the system until the straightaway, as there is no radar and there no camera pointing like the b-pillars but towards the rear. But again, I'm not NN or vision guy, this is just my theories based on what I see, and more importantly what I don't see.

The more time goes on, the more I feel swindled, these cars including AP2.5 will never have FSD, it's just not going to happen. We "may" get lvl4 for highway driving, but I don't see this system taking us door to door, at least not without some serious hardware retrofits. I think the camera approach Tesla is trying to take can most certainly work, but for it to really work, they need more coverage of the rear of the vehicle, just not perpendicular to the backside of the car. And, sadly, I do think they need at least 1 rear facing radar, but that's probably for another thread.

Yes, I'm also aware of the depth from context stuff, it's actually the comma.ai job interview. You solve the problem, greater then .4 I believe and your in like Flynn. Thought, I've spent the last 30 minutes or so trying to find it, no luck.

But thank you for your insights, the deep expertise you bring to the community are very englightening!

 

[Joe F]

Yes, because of no rear facing radar, I figure they need something more then monocular vision looking back, I haven't done the math to know for sure, but I imagine at highway speeds the gaps close pretty quickly if say your coming from an on ramp to the highway thats running full bore at 77 mph.

You sure about that being a problem? Unless you're hauling a trailer behind a MX, I know my MS, and I'm sure yours as well, accelerates fast enough that an on-ramp merge concern is a thing of the past. Maybe don't come to a full stop first?

I think my biggest fear with the current arraignment is merge events and how they will be handled, it seems to me to be a huge blindspot in the system until the straightaway, as there is no radar and there no camera pointing like the b-pillars but towards the rear. But again, I'm not NN or vision guy, this is just my theories based on what I see, and more importantly what I don't see.

Isn't that what the cameras on the front fenders are for? What am I missing?

 

[bebbiXpress]

Yes, because of no rear facing radar, I figure they need something more then monocular vision looking back, I haven't done the math to know for sure, but I imagine at highway speeds the gaps close pretty quickly if say your coming from an on ramp to the highway thats running full bore at 77 mph.

I'm a little concerned about that too. Is there enough information to safely change lanes? I mean on parts of German highways there are no speed restrictions, you can go as fast as your car will go. So, if you're driving behind a truck with 80km/h (~50mph) and wanna pass it, it's possible that there's another coming with 250km/h (~155mph) or even a higher speed (yes, sadly there are some "folks" who think they can handle 200mph on a public highway).