Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register
This site may earn commission on affiliate links.
How are your disengagements. Are they happening at the same location everytime? Any evidence the drive is getting better on the same firmware (44) which can happen thru better maps.
That's a good question, and it makes me realize that a lot of my disengagements are simply because I don't want to scare someone behind me if I look like I'm weaving.
In general, there are a few factors that affect the likelihood of needing to disengage:
.
1) Base Speed - it's amazing how many fewer disengagements occur if you go the speed limit, or no more than 5 mph above,
2) Day and night are big factors, and I find my AP is best at night and in the rain. ... I'm sure this has something to do with slower traffic at night and in rain, but also less visual noise (if I'm describing that correctly).
3) My subjective impression that the car is learning a bit as time goes on.... my guess is that this is placebo effect once you get beyond the 25 miles after a firmware update where it seems to be recalibrating.
4) AP2 and 17.44 is definitely improving, I think there is a clear difference after 17.40 whereas before it seemed like it was two steps forward one step back... now it feels like 5 steps forward and 1 step back (for example, my summon no longer opens my garage door when I wait in the alley while I do my warmup stretches :) now, I have to open the door first... freaking Elon.
5) There are times it seems like the processor is more efficient than others.... kind of like if you boot up your computer and try to open too many programs before it's done warming up... there are times where the car is like it is taken out of a nap... just like a human. If the car is preconditioned according to time of day, it seems to be ready to roll as soon as you get in.
Some things I wonder about.... @verygreen had an interesting data capture regarding some confusion between image uploads labeled slope up and slope down, and this makes sense to me because the car struggles most at the bottom of a hill. Look at this video at 1:00... it consistently makes me almost disengage at the bottom of the hill, just before it goes up again.
 
One thing’s for damn sure - there will be no rap* in any TJ vids.

* Remember, you can’t spell “crap” without “rap”.

Love it... this is exactly the effect I'm going for... the what the hell? I find that there is a fine line between finding a song that is slightly catchy and downright offensive. Occasionally, you come across this gem....
 
2) Day and night are big factors, and I find my AP is best at night and in the rain. ... I'm sure this has something to do with slower traffic at night and in rain, but also less visual noise (if I'm describing that correctly).

Interestingly, this has been my experience as well! I think I noted it somewhere on TMC too when my first long trip on .36 went so well. It was darker and the weather was worse than previously on the same firmware.

But it had nothing to do with traffic density in my case. The daytime testing had been in as low traffic as well. Is it possible the darkness and the car's lights outlining the shiny lane markings are simply a more contrast-rich environment for the car's cameras?

Now, of course, this doesn't tell us much about the ability to detect obstacles, but lane-following-wise it might well make sense that in darkness with headlights it is easier to see the lane.
 
  • Like
Reactions: buttershrimp
My educated guess is that their NN doesn't have memory from the previous frame. They do averaging of the NN output, which is why you see the road bouncing around on the IC display, but the NN itself takes each video frame without reference to the previous one.

That is basically crippling the NN. It has to deal the camera image that is poorer than the human eye in many respects, and it doesn't get any hints from previous frames and the way parts of the image move.
 
  • Like
Reactions: buttershrimp
My educated guess is that their NN doesn't have memory from the previous frame. They do averaging of the NN output, which is why you see the road bouncing around on the IC display, but the NN itself takes each video frame without reference to the previous one.

That is basically crippling the NN. It has to deal the camera image that is poorer than the human eye in many respects, and it doesn't get any hints from previous frames and the way parts of the image move.

Agree / disagree. The caffe NN that we have prototxt files for only analyzes a single frame. It has no provision for feeding in state from previous frames or any other variables that might influence it's interpretation of the frame. So yes, *that* NN has no memory and almost certainly is not going to be sufficient for making driving decisions because of that fact. So in that sense it's crippled.

But my read on how it's being used seems to be a bit different from yours, if I understand where you're coming from.

I think *that* NN is only being used in a fashion akin to any other vision processing algorithm - it extracts some set of features from a frame and passes those features to a downstream process that makes driving decisions. It's easy to demonstrate that the downstream process maintains a memory of previous frames and uses that information to inform driving decisions. The simplest one is to just look at how long the system waits to make a lane change after you click the lever under various lane marking circumstances. That delay is not dependent on the road view at the moment you click the lever, but rather on how the road has looked over the last five seconds or so. If the car has seen a clear lane for several seconds already it will change lanes instantly. If not it will wait until it sees several seconds of clear lane or it will time out.

The reason I say *that* NN is we don't have much visibility into what is involved in the downstream process. It might well involve another NN that takes a stack of periodic observations or an RNN that maintains state memory. If that NN is not a caffe NN then we won't have prototxt for it and we wouldn't necessarily know that it exists. The existence of a prototxt file is just an aspect of how caffe builds distributable sets - it's not a requirement for an NN to be present. Of course the downstream could be heuristic too, or it could use some other ML algorithms. We don't know.
 
Agree / disagree. The caffe NN that we have prototxt files for only analyzes a single frame. It has no provision for feeding in state from previous frames or any other variables that might influence it's interpretation of the frame. So yes, *that* NN has no memory and almost certainly is not going to be sufficient for making driving decisions because of that fact. So in that sense it's crippled.

But my read on how it's being used seems to be a bit different from yours, if I understand where you're coming from.

I think *that* NN is only being used in a fashion akin to any other vision processing algorithm - it extracts some set of features from a frame and passes those features to a downstream process that makes driving decisions. It's easy to demonstrate that the downstream process maintains a memory of previous frames and uses that information to inform driving decisions. The simplest one is to just look at how long the system waits to make a lane change after you click the lever under various lane marking circumstances. That delay is not dependent on the road view at the moment you click the lever, but rather on how the road has looked over the last five seconds or so. If the car has seen a clear lane for several seconds already it will change lanes instantly. If not it will wait until it sees several seconds of clear lane or it will time out.

The reason I say *that* NN is we don't have much visibility into what is involved in the downstream process. It might well involve another NN that takes a stack of periodic observations or an RNN that maintains state memory. If that NN is not a caffe NN then we won't have prototxt for it and we wouldn't necessarily know that it exists. The existence of a prototxt file is just an aspect of how caffe builds distributable sets - it's not a requirement for an NN to be present. Of course the downstream could be heuristic too, or it could use some other ML algorithms. We don't know.

I agree that is what they are likely doing.

The problem is that cameras only offer mono vision, with no depth perception. The usual way of getting around that is to use multiple frames from different positions.

You can kinda fake it with one image. Google does it for "portrait mode" on its latest phones, but even with a dedicated image processor it takes several seconds and only blurs the background, i.e. it only has foreground/background and not an estimate of 3D position.

Without that information they are a long, long way from building the 3D model of the environment they need for FSD.
 
Agree / disagree. The caffe NN that we have prototxt files for only analyzes a single frame. It has no provision for feeding in state from previous frames or any other variables that might influence it's interpretation of the frame. So yes, *that* NN has no memory and almost certainly is not going to be sufficient for making driving decisions because of that fact. So in that sense it's crippled.

But my read on how it's being used seems to be a bit different from yours, if I understand where you're coming from.

I think *that* NN is only being used in a fashion akin to any other vision processing algorithm - it extracts some set of features from a frame and passes those features to a downstream process that makes driving decisions. It's easy to demonstrate that the downstream process maintains a memory of previous frames and uses that information to inform driving decisions. The simplest one is to just look at how long the system waits to make a lane change after you click the lever under various lane marking circumstances. That delay is not dependent on the road view at the moment you click the lever, but rather on how the road has looked over the last five seconds or so. If the car has seen a clear lane for several seconds already it will change lanes instantly. If not it will wait until it sees several seconds of clear lane or it will time out.

The reason I say *that* NN is we don't have much visibility into what is involved in the downstream process. It might well involve another NN that takes a stack of periodic observations or an RNN that maintains state memory. If that NN is not a caffe NN then we won't have prototxt for it and we wouldn't necessarily know that it exists. The existence of a prototxt file is just an aspect of how caffe builds distributable sets - it's not a requirement for an NN to be present. Of course the downstream could be heuristic too, or it could use some other ML algorithms. We don't know.
Perfect timing! @JimmyD and friends.... I shot this video today and it is my favorite to date because it's nuts on so many levels.... starting 10 seconds into the video until just after the toll booth.... especially when you realize that they had blocked off the left toll booth spot.... it's easy to say it was just following the car, but I'm not so sure because the car felt like it initially wanted to go through the left side..... (the worst part of the video was when I messed up the merge at the end by trying to signal while on autopilot to get on the highway properly )
 
Perfect timing! @JimmyD and friends.... I shot this video today and it is my favorite to date because it's nuts on so many levels.... starting 10 seconds into the video until just after the toll booth.... especially when you realize that they had blocked off the left toll booth spot.... it's easy to say it was just following the car, but I'm not so sure because the car felt like it initially wanted to go through the left side..... (the worst part of the video was when I messed up the merge at the end by trying to signal while on autopilot to get on the highway properly )

You are a brave, brave man
 
I shot this video today

I enjoy your videos, and comparing how AP2 is coming along with my AP1 experience.

Sorry to be really picky ...

Found it hard to see your Dash in that one - I like viewing when AP is on/off, your turn signals, the speed, and what AP thinks of the lane markings and surrounding vehicles (where it seems to me AP1 still does a better job of displaying on the dash, at least).

Perhaps slightly higher camera position, angled down a bit? maybe you were just into-the-sun a bit on this trip.
 
I enjoy your videos, and comparing how AP2 is coming along with my AP1 experience.

Sorry to be really picky ...

Found it hard to see your Dash in that one - I like viewing when AP is on/off, your turn signals, the speed, and what AP thinks of the lane markings and surrounding vehicles (where it seems to me AP1 still does a better job of displaying on the dash, at least).

Perhaps slightly higher camera position, angled down a bit? maybe you were just into-the-sun a bit on this trip.
How about these two?


 
  • Like
Reactions: generalenthu
Really surprised AP2 initiated a passing move on that vehicle moving into the left turning lane. AP1 freaks out if the vehicle in front is in the lane at all (actually, not sure I've tried that exact thing, but if a car is moving from middle lane to slow lane, is only a foot or so "still in middle lane", and I turn-signal AP1 to move from outside lane to middle lane AP1 jumps on the brakes. Also, pretty sure I've pressed accelerator when car in front has almost completed departing my lane, to encourage AP1 into that space and resume my cruising speed, and AP1 hates it ... until the vehicle in front is 100% out of the lane. Clearly AP must be able to plot its trajectory, so looks like AP2 has that right already :)

How about these two?

The night ones make it really easy to see the dash :) but I'm becoming increasingly worried that this is all a cover and the real project is to change everyone's music tastes!

Video #2:

Tesla Music Streaming Surprise.

Haha ... you need to get that fixed!
 
  • Love
Reactions: buttershrimp
Really surprised AP2 initiated a passing move on that vehicle moving into the left turning lane. AP1 freaks out if the vehicle in front is in the lane at all (actually, not sure I've tried that exact thing, but if a car is moving from middle lane to slow lane, is only a foot or so "still in middle lane", and I turn-signal AP1 to move from outside lane to middle lane AP1 jumps on the brakes. Also, pretty sure I've pressed accelerator when car in front has almost completed departing my lane, to encourage AP1 into that space and resume my cruising speed, and AP1 hates it ... until the vehicle in front is 100% out of the lane. Clearly AP must be able to plot its trajectory, so looks like AP2 has that right already :)



The night ones make it really easy to see the dash :) but I'm becoming increasingly worried that this is all a cover and the real project is to change everyone's music tastes!

Video #2:



Haha ... you need to get that fixed!
Here you go
 
upload_2017-12-6_8-10-37.png

Oops