AP2.0 Cameras: Capabilities and Limitations?

[croman]

Onto actual things worth discussing, anyone notice how 18.4.1 is only going to AP2 and not AP2.5? I believe that is because they have a working AI (or are testing that) for the repeater cams and AP2.5 repeater cams are color and might need a different AI.

Any speculation on that? Will AP2.5 have yet another feature that requires "catch-up"?

I'm surprised that whatever Tesla is working on with additional cameras does not contribute, in any way, to safety or added functionality (in a reliable manner). You'd think that additional data would contribute a large leap in function (yet another set of eyes for lane keeping as well as lane changing safety) at minimum.

 

[chillaban]

It's exactly that. Collecting everything wouldn't just be wasteful, it'd be actively harmful. The vast majority of the time your car is just traveling normally. If you trained on all of that, you're going to overfit the model to those circumstances and completely wash out the challenging instances you actually care about.

Choosing how much of which type of data to feed to a model during training is actually a big topic generally in ML. You have to carefully balance the different classes of input to ensure you don't overfit the model to specific circumstances just because of selection bias.

But I see a distinction between what's being collected vs what's being *used*/fed into training.

Waymo/Google collects a ton of data so they could, at some point in the future, say "oh hmm I want to see how well the system is taking 20 degree curves in the sunlight on an incline" and they could collect data on this, or run simulations against this library of data. Tesla would have to send out a bounty to collect such data that they hadn't been collecting before.

Of course when you have a large fleet the difference may not be huge between the two approaches.

 

[J1mbo]

Onto actual things worth discussing, anyone notice how 18.4.1 is only going to AP2 and not AP2.5? I believe that is because they have a working AI (or are testing that) for the repeater cams and AP2.5 repeater cams are color and might need a different AI.

Any speculation on that? Will AP2.5 have yet another feature that requires "catch-up"?

I thought they had implemented a number of colour profiles for the cameras, so would think they would use a profile that works with the NN. (Whilst running a different, colour-sensitive NN in shadow mode.. sorry, couldn't resist )

 

[lunitiks]

The illusion of «shadow mode»: Your car ‘observes’ your every move. It ‘plans ahead’ and ‘determines what it would do’ if it was in control. But it isn’t - it’s in the secret «shadow mode». Your 40 x mobileye, easily upgradeable super computer in the glove box either ‘learns’ or ‘gloats’ when you deviate from what it’s planned. Keep this up long enough, and AI will rise from it!

Guess what? The car doesn’t learn anything.

It doesnt get better by itself. Theres no training going on inside your Tesla. Not in ‘live’ mode, not in shadow mode, not in any mode.

Your car can LOG stuff you do (did), and send the logs or snapshots back to tesla at an appropriate time later if/when your wifi connected.

God knows what they do (or dont do) with this data!!!

I know 99% of this forum couldn't give a bigger turd about this guy, but now PM_YOUR_NIPS_PAPER posts this (as usual, pointing out he's some kind of insider - I dunno about that, but):

Autopilot vision scientist here.

No, it does not actively learn (online learning on a car means death to the driver). Nor is any alert triggered for manual review.

The only thing Tesla collects are short (few second) clips of you driving, about one clip per week, randomly when you drive.

The calibration process at the beginning of a brand new AP car simply measures camera pitch/angle variances and engine sensitivity. Has nothing to do with your driving style.

The data is more useful to Autopilot Maps (roads, 3D) rather than controls or vision. Mainly due to labeling bottlenecks and annotation quality from in-house and 3rd party vendors. Yes, Tesla outsources this too. All tech companies do.

 

[MarcusMaximus]

I know 99% of this forum couldn't give a bigger turd about this guy, but now PM_YOUR_NIPS_PAPER posts this (as usual, pointing out he's some kind of insider - I dunno about that, but):

That all actually sounds pretty reasonable to me, and in line with what I'd expect. From what I've seen posted from those with access to the APE, the current networks that do exist(but aren't yet being run) are focused on vision, not controls. For vision, since they haven't actually released anything yet outside emulating AP1, it's not terribly surprising that they'd rely on a smaller internal curated dataset right now. That said, they aren't likely just throwing away the data after building maps. They're pretty certainly building up a database that's slowly being labeled for vision and will someday be used for controls.

EDIT: I will add, though, this is yet another different label this person has given themselves. The information given sounds reasonable, but I suspect it's a guess by someone not actually affiliated with the company in any way.

 

[chillaban]

That all actually sounds pretty reasonable to me, and in line with what I'd expect. From what I've seen posted from those with access to the APE, the current networks that do exist(but aren't yet being run) are focused on vision, not controls. For vision, since they haven't actually released anything yet outside emulating AP1, it's not terribly surprising that they'd rely on a smaller internal curated dataset right now. That said, they aren't likely just throwing away the data after building maps. They're pretty certainly building up a database that's slowly being labeled for vision and will someday be used for controls.

EDIT: I will add, though, this is yet another different label this person has given themselves. The information given sounds reasonable, but I suspect it's a guess by someone not actually affiliated with the company in any way.

Agreed. What he says makes sense based off what others have said and found. He seems like he just reads the same things that we all read and pretends to be an insider in the process.

 

[BigD0g]

Sounds like an imposter selling @verygreen's findings.

 

[chillaban]

Sounds like an imposter selling @verygreen's findings.

Gotta love the Internet. You can pretend to be an Internet expert by parroting facts you found, and others will believe you because they'll find the same things online

 

[run-the-joules]

Couple of interesting anecdotal data points for y'all. I have a Model 3 that is currently being serviced running 2017.50.12 and in the meantime I have a late 2017-build Model S (I BELIEVE it's AP2.5 but will confirm when I leave work if someone can tell me how) with 2018.2, and the Model S is *dramatically* worse on local roads that the Model 3 has no issue with whatsoever. I'll try and get some video demonstrations if I don't have to take the S back today.

 

[P85Dave]

Sounds like an imposter selling @verygreen's findings.

Wish we could all chip in and buy @verygreen a new car so we can get regular updates

 

[chillaban]

Couple of interesting anecdotal data points for y'all. I have a Model 3 that is currently being serviced running 2017.50.12 and in the meantime I have a late 2017-build Model S (I BELIEVE it's AP2.5 but will confirm when I leave work if someone can tell me how) with 2018.2, and the Model S is *dramatically* worse on local roads that the Model 3 has no issue with whatsoever. I'll try and get some video demonstrations if I don't have to take the S back today.

I would guess that the Model 3's more nimble handling and smaller size gives it a lot more margin. It's not unlike the human equivalent of driving a GMC Yukon vs a Tesla Roadster.

Anecdotally, I once took a tech demo test drive of the then-new 2015 S Class with the stereoscopic cameras. The engineer explained that the car is pretty much AutoCorrecting everything that you're doing as well as making minor suspension tweaks. He turned that off, and the car became unbelievably more difficult to control properly (e.g. stay centered around curves, stability after hitting pavement grooves, etc)

 

[run-the-joules]

I would guess that the Model 3's more nimble handling and smaller size gives it a lot more margin. It's not unlike the human equivalent of driving a GMC Yukon vs a Tesla Roadster.

Anecdotally, I once took a tech demo test drive of the then-new 2015 S Class with the stereoscopic cameras. The engineer explained that the car is pretty much AutoCorrecting everything that you're doing as well as making minor suspension tweaks. He turned that off, and the car became unbelievably more difficult to control properly (e.g. stay centered around curves, stability after hitting pavement grooves, etc)

Well, the notable things I've seen aren't little things like "I wish it were a bit more centered", but rather things like completely losing its mind when going through an intersection that the Model 3 handles. Specifically there's one I go through every day and the Model 3 has never failed to pick up the lane again on the other side, whereas the Model S reacted by just swerving to the left towards the raised median. I was prepared for it, but it's a pretty drastic difference in performance.

Again, just anecdotal data so not worth much, but still interesting to me.

 

[verygreen]

Well, the notable things I've seen aren't little things like "I wish it were a bit more centered", but rather things like completely losing its mind when going through an intersection that the Model 3 handles. Specifically there's one I go through every day and the Model 3 has never failed to pick up the lane again on the other side, whereas the Model S reacted by just swerving to the left towards the raised median. I was prepared for it, but it's a pretty drastic difference in performance.

Again, just anecdotal data so not worth much, but still interesting to me.

Note that you are also in CA, so they might have enabled the ap2 maps in CA. I wonder if it's any different in other areas

 

[run-the-joules]

Note that you are also in CA, so they might have enabled the ap2 maps in CA. I wonder if it's any different in other areas

Good question, and also brings to another interesting difference. When I take the exit to work, the Model 3 tells me to turn left at the stop, then do a U-turn and continue on my way. The Model S is like "turn right", which is the correct choice. I *did* file a bug against on Monday and on Tuesday I had the Model S but I find it *exceptionally* unlikely they corrected the routing that quickly. I should get my 3 back tomorrow and I'll report back on Friday if it's been fixed there as well. My belief until then is that they're using different maps.

 

[verygreen]

Good question, and also brings to another interesting difference. When I take the exit to work, the Model 3 tells me to turn left at the stop, then do a U-turn and continue on my way. The Model S is like "turn right", which is the correct choice. I *did* file a bug against on Monday and on Tuesday I had the Model S but I find it *exceptionally* unlikely they corrected the routing that quickly. I should get my 3 back tomorrow and I'll report back on Friday if it's been fixed there as well. My belief until then is that they're using different maps.

these are different maps. Navigation maps on S use old Navigon maps and navigation maps on 3 use new tesla mps that are crowdsourced from mapbox/valhalla and are not always better (when I tried them navigon certainly had a better route to one of my destinations in the comparison video).

But the maps I am talking abut are autopilot-specific maps, it does not give you directions, it's for ap to know what lanes are where, speed limits and such. S/X mostly use old style adas maps and ap2 maps are mostly disabled (And are only scantily present in California anyway), so it's possible model 3 has them enabled when you are in California so it feels better on the roads.

 

[run-the-joules]

these are different maps. Navigation maps on S use old Navigon maps and navigation maps on 3 use new tesla mps that are crowdsourced from mapbox/valhalla and are not always better (when I tried them navigon certainly had a better route to one of my destinations in the comparison video).

But the maps I am talking abut are autopilot-specific maps, it does not give you directions, it's for ap to know what lanes are where, speed limits and such. S/X mostly use old style adas maps and ap2 maps are mostly disabled (And are only scantily present in California anyway), so it's possible model 3 has them enabled when you are in California so it feels better on the roads.

Right on. I didn't realize that the routing data isn't based on the AP maps. Neat.

 

[lunitiks]

News:

@sumitkgarg recently reported that his SC replaced one of his/her side repeaters. The part number he/she quotes (11251061-99-A) is new, i.e. not the same that's been in the parts manuals since ~ Oct. 2016 (1034344-99-A and 1034344-20-B).

It's a 2.5 model S

 

[lunitiks]

Btw - and I know this 'might' be a coincidence - but @sumitkgarg's new side repeater part number has the same first four digits as the autopilot 2.5 computer (ape 2.5) part number; "1125"

 

[ishareit]

News:

@sumitkgarg recently reported that his SC replaced one of his/her side repeaters. The part number he/she quotes (11251061-99-A) is new, i.e. not the same that's been in the parts manuals since ~ Oct. 2016 (1034344-99-A and 1034344-20-B).

It's a 2.5 model S

Yes. I (he) used to get random cruise control disabled errors. They checked the logs and replaced this part. It might be too early to say but since this repair, I haven’t received these random errors.

 

[generalenthu]

So, can someone humor me on how the cars decide to upload snippets? Presumably this doesn't happen randomly. I suppose only when the car detects an unexpected event, the clip / context is uploaded.

So how do you define something as unexpected, unless you have an expectation? Who is creating the expectation. Calling it The Shadow mode could range from reasonable to charitable. But it's hard to say nothing of that sort exists.

I see @verygreen disputes the existence of the mode, but the question here is not one of existence, but it's capability. And to that, none of us has a clue.

Sorry to quote myself, but realized I posted in the wrong thread.