Time to crack her open and get inside the network for AP2

[appleguru]

I was just about to suggest that. They've prob switched the loose cubby connector from direct CID access to APECU access and from there CID access. Kind of what they did with rv-cam (no longer hooked up to CID directly but to APECU)

And actually... the colors make some sense too. Tesla probally decided they wanted to have different colors for different segments to make things easier/harder to connect to the wrong place...

So they made backup camera (not Ethernet, but same connector) blue.
Made IC to CID black
Made CID to diag cable white

But then they needed to add an APE. So, they made a weird white to purple cable so they could keep using the same cables at service centers with toolbox ;-)

 

[appleguru]

While this is possible, it is somewhat cumbersome.
The locked port is done by ethernet switch with basic vlan-capable ports on the cid board (I only looked in great details on my ap1 bench setup). So the switch is connected to gateway on the master port and diag is connected to a restricted port, all traffic from restricted port flows to the master port and master decides which packets to let through or not (only lets in the magic unlock packet in locked state).
cid node is connected to fully unrestricted port and ic is connected to partially restricted port (the only restriction I saw so far is no ic<->gtw communications are allowed)

Now if you reuse the diag port as the link to ape, how are you going to do the locking? you cannot tell if the incoming datagram came from ape/lb/ap2b or the diag port... You need to somehow propagate the locking/unlocking logic to ape/lb but I am not quite sure I see this logic anywhere, though I guess if it's hidden withing gateway and it just commands lb to do more things, that's possible, just a lot more cumbersome. Somebody needs to take a look at the harness, I guess.

Which is a big reason why I'm still not 100% convinced they're doing any VLAN configuration or port "locking"/disabling. After all, even with the diag port locked down, they still need to listen over that locked port for incoming requests from their diagnostics software to unlock the port... so they can't just disable it outright.

Based on what I know so far, I think they're locking down the port(s) via firewall rules on each connected component.

 

[verygreen]

Which is a big reason why I'm still not 100% convinced they're doing any VLAN configuration or port "locking"/disabling. After all, even with the diag port locked down, they still need to listen over that locked port for incoming requests from their diagnostics software to unlock the port... so they can't just disable it outright.

Based on what I know so far, I think they're locking down the port(s) via firewall rules on each connected component.

the firewall is separate.

The way it works is ethernet port for diag ethernet on the switch is marked as "always send data from here to gateway port" in locked mode.
gateway rtos gets these packets (tagged with source port) and drops them on the floor if they don't match the unlock packet properties (udp port) otherwise it forwards it further.

Now in addition to that CID has it's own firewall - but it is just a guard against people connecting to in-car network directly (i.e. installing an extra ethernet switch into the ic line or even just disconnecting ic and trying to do some stuff pretending to be the ic).
If you look at the firewall rules - you'll see thy don't block any arp traffic or multicast traffic and in fact do not even have any broadcast traffic blocking, but locked diag is dead silent, no arp discovery, nothing, unlock it and suddenly you see lots of broadcasts, arp discovery works and so on.

 

[BigD0g]

So, some bad news I was heading home from work and plotting out the ape approach to start probing, got home and my wife was heading off to her business dinner, which I knew and was counting on....

However my wife decided to take Jarvis to dinner.... no hacking for me tonight

 

[lunitiks]

Thanks. It's 00.15 over here so that means I can safely go to bed

 

[appleguru]

the firewall is separate.

The way it works is ethernet port for diag ethernet on the switch is marked as "always send data from here to gateway port" in locked mode.
gateway rtos gets these packets (tagged with source port) and drops them on the floor if they don't match the unlock packet properties (udp port) otherwise it forwards it further.

Now in addition to that CID has it's own firewall - but it is just a guard against people connecting to in-car network directly (i.e. installing an extra ethernet switch into the ic line or even just disconnecting ic and trying to do some stuff pretending to be the ic).
If you look at the firewall rules - you'll see thy don't block any arp traffic or multicast traffic and in fact do not even have any broadcast traffic blocking, but locked diag is dead silent, no arp discovery, nothing, unlock it and suddenly you see lots of broadcasts, arp discovery works and so on.

Got it... so, my conclusion is that the gateway must have some slightly different logic for AP2 cars given how they are physically wired. Unless the APE duplicates some of the gateway's functionality for filtering traffic on the diag port?

Remember that these aren't really switches... more like separate NICs on a Linux box, that happen to have IPs on the same subnet... so I'm not convinced ARPs/etc would normally get forwarded. Maybe they're doing something with arp_filter?

Could also just be different on AP2 cars... @BigD0g, if you do a packet capture on the diag port... do you see anything at all?

 

[verygreen]

Got it... so, my conclusion is that the gateway must have some slightly different logic for AP2 cars given how they are physically wired. Unless the APE duplicates some of the gateway's functionality for filtering traffic on the diag port?

There's most definitely another ethernet switch in ape, and the lb is the ape-gateway as I now believe, so it must be controlling things there.
So I guess it's possible cid gateway instructs lb or whatever is it controlling ape switch to let the traffic in or not.
I know you cannot ping gateway from ape (in fact service_api there does it all the time as a way to determine if we are locked or not apparently).
It's also impossible to reach lb from cid (but possible to ping lb from ape) which seems to imply a similar setup to that of cid-gtw.

Remember that these aren't really switches... more like separate NICs on a Linux box, that happen to have IPs on the same subnet... so I'm not convinced ARPs/etc would normally get forwarded. Maybe they're doing something with arp_filter?

They are most definitely switches. Here's the datasheet: http://www.marvell.com/switching/assets/marvell_linkstreet_88E6060_datasheet.pdf

Could also just be different on AP2 cars... @BigD0g, if you do a packet capture on the diag port... do you see anything at all?

there's nothing at all on AP2 cars, same as on AP1 (I have tried both).

 

[apacheguy]

@verygreen - I'm interpreting what you are saying as the ape is on a separate VLAN as the CID. So you cannot access CID services from the ape Ethernet port without it being unlocked?

 

[verygreen]

@verygreen - I'm interpreting what you are saying as the ape is on a separate VLAN as the CID. So you cannot access CID services from the ape Ethernet port without it being unlocked?

Not really.
The whole VLAN thing is very superficial.

Basically consider that you have a flat network inside of the car, just some of the ports are restricted in some ways.
For example of restrictions:
On cid side switch:
ic can talk to anything but gateway
diag port normally cannot talk to anything at all unless it's just hta one udp port (but once unlocked it is fully unrestricted) (on AP1)
ape link can talk to cid and ic, but apparently not gtw (unless that's limited on lb side?) unless the diag ethernet is unlocked

on ape side:
ape can talk to lb and to the link that leads into the cid switch
diag port cannot talk to anything but that unlock udp port in locked state
apparently nothing else can talk to lb

 

[BigD0g]

I missed something somewhere, lb?

 

[apacheguy]

I missed something somewhere, lb?

It's another node on the network.

.100 cid
.101 ic
.102 gw
.103 lb
.104 ape

Pretty sure that's right.

 

[BigD0g]

I understand that piece, just trying to understand what lb refers to in terms of the network.

 

[appleguru]

I understand that piece, just trying to understand what lb refers to in terms of the network.

Based on my current understanding, the gateway (gw) is in the same housing/on the same board as the CID. And the lb seems to perform similar functions/at the very least additional intrusion protection and ethernet filtering and is in the same housing/board as the APE.

So, from expanding my text 'one line' from before:

The dotted red lines represent internal switch connections inside the components.... I could/should probably include a separate switch block in the APE and CID sections for completeness, but for practical purposes based on the way Tesla seems to protect/filter/route traffic, this seems to be mostly correct.

 

[appleguru]

There's most definitely another ethernet switch in ape, and the lb is the ape-gateway as I now believe, so it must be controlling things there.
So I guess it's possible cid gateway instructs lb or whatever is it controlling ape switch to let the traffic in or not.
I know you cannot ping gateway from ape (in fact service_api there does it all the time as a way to determine if we are locked or not apparently).
It's also impossible to reach lb from cid (but possible to ping lb from ape) which seems to imply a similar setup to that of cid-gtw.


They are most definitely switches. Here's the datasheet: http://www.marvell.com/switching/assets/marvell_linkstreet_88E6060_datasheet.pdf


there's nothing at all on AP2 cars, same as on AP1 (I have tried both).

Do both the CID and the APE have a (the same?) switch chipset built in? Does the CID and/or APE enumerate one NIC? or multiple? (Ignoring the additional USB Ethernet devices connected to the CID for a moment)

 

[verygreen]

Do both the CID and the APE have a (the same?) switch chipset built in? Does the CID and/or APE enumerate one NIC? or multiple? (Ignoring the additional USB Ethernet devices connected to the CID for a moment)

I don't know what switch chipset is in ape, but potentially something different since the link state is gigabit and 88E6060 is 10/100 switch.
ape has a gige nic as part of SoC it appears.
cid does not have any NICs, everything is usb (i.e. there's parrot that's usb ethernet device and there's usb-ethernet that's connected to the in-car switch (toucan)).
there's also usb-wlan, but it does not have a kernel driver installed so unused for now.

 

[appleguru]

Looking at the photos from the other thread, the APE chipset definitely seems different (is a different shape/size). I see two Marvell ICs on the board; the larger is likely the switch chipset. Can't quite make it out, but I see.. XXXAX321-1FJX

Inside the NVIDIA PX2 board on my HW2 AP2.0 Model S (with Pics!)

Not sure what the GigE links are good for when they can only physically connect externally via the Fakra connectors at 100mbps (only 4 wires/2 pairs), though I suppose if they are using GMII or similar for a direct gigabit connection internally on the APE?

 

[HX_Guy]

I love smart people.

 

[lunitiks]

View attachment 248450

Good work

I dug up some photos, and I’ll try to highlight some differences between AP1 and AP2:

The Ethernet DIAG jack – easily accessible below CID – is a Rosenberger D4Z001-000-B (White). AKA the “easy” port or “Toolbox” port. It seems to be identical on AP1 and AP2. Which makes sense, considering that the service centers / rangers frequently have to deal with both configs…

The DIAG cable terminates differently on AP1 vs. AP2 cars, however.

On AP1 cars, it feeds directly into the back of CID. The connector on the CID side seems to have changed color at some point, because I’ve seen pictures where it’s white and pictures where it’s blue. Here’s the white one:

On AP2 however, the Toolbox cable terminates into the Autopilot 2 ECU (“PX2”). The connector on the ECU side is a Rosenberger D4K20A-1D5A5-D (Bordeaux, or Purple).

As you can see from the picture above, the AP2ECU also has a white Rosenberger D4Z001-000-B. This is, according to the wiring diagrams, “Ethernet Gateway”, and connects to the CID.

On the CID side, the socket should be Bordeaux / Purple color

 

[BigD0g]

So, with a heavy heart, I can say the approach I was planning on taking to gain CID access was apparently closed by Tesla with the 8.0 release..... So, a bit of set back their, but we'll find another way in....

 

[croman]

So, with a heavy heart, I can say the approach I was planning on taking to gain CID access was apparently closed by Tesla with the 8.0 release..... So, a bit of set back their, but we'll find another way in....