AP2.0 Cameras: Capabilities and Limitations?

[Bladerskb]

As I said when measuring GPU performance generally we use FP32 so as far as FP32 the Pascal Titan X and 1080Ti are faster than Drive PX2 that's undeniable.

There's no question that the Drive PX2 will outperform the TitanX in FP16. Just so you know 1080 != 1080 Ti btw

Now did you hear about Google's new TPU 2.0 180 TFLOPS (at possibly FP16 instead of the older one that only did 8 bit ops). I want one so bad....

Since deep learning utilizes FP16 hence in this discussion Drive PX2 > them.
If this was a gaming forum where we are discussing using graphics card for games then drive px2 is inferior because games utilizes fp32, but since we are talking about self driving cars which uses deep learning then drive px2 is superior.

I think we are in agreement.

And yeah i did see that chip and its interesting that last yeear Google said it needed 50 TFLOPS of performance.
How self-driving cars will drive demand for high-end chips

 

[MP3Mike]

Are there reasons why we haven't seen a PX2 board extracted out of a HW2 car yet?

I'm really surprised that we haven't since we know at least one person, @wk057, has purchased a totaled AP2 car. I can't imagine he didn't crack the APU unit open, he must just not have shared it.

 

[Bladerskb]

@JeffK Also looks like the same limitation is on all 10 series

List of Nvidia graphics processing units - Wikipedia

 

[JeffK]

@JeffK Also looks like the same limitation is on all 10 series

List of Nvidia graphics processing units - Wikipedia

Yes, it's how they differentiate the consumer cards

 

[KyleDay]

If you're decent with handling electronics and taking apart stuff then there's no reason you can't open it up to take some pics. If you don't feel comfortable then it's up to you. I'm sure such pics would be featured on Electrek and other news sites so prepare for 15 min of fame.

Abso-frigging-lutely not! Please, please, PLEASE do it. I'll provide you with all the info and step-by-step stuff you need

Please provide me the info and steps and I'll get on it.

 

[lunitiks]

Allright I'll write u up in a pm

 

[verygreen]

Also while we are waiting on those teardown pictures:

00:01.0 PCI bridge: NVIDIA Corporation Device 10e5 (rev a1) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 104
   Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
   I/O behind bridge: 00001000-00001fff
   Memory behind bridge: 50800000-51ffffff
   Prefetchable memory behind bridge: 0000000058000000-000000006fffffff
   Capabilities: [40] Subsystem: NVIDIA Corporation Device 0000
   Capabilities: [48] Power Management version 3
   Capabilities: [50] MSI: Enable- Count=1/2 Maskable- 64bit+
   Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
   Capabilities: [80] Express Root Port (Slot+), MSI 00
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [140] L1 PM Substates
   Kernel driver in use: pcieport

01:00.0 3D controller: NVIDIA Corporation Device 1c36 (rev a1)
   Subsystem: NVIDIA Corporation Device 11cb
   Flags: bus master, fast devsel, latency 0, IRQ 104
   Memory at 51000000 (32-bit, non-prefetchable) [size=16M]
   Memory at 60000000 (64-bit, prefetchable) [size=256M]
   Memory at 58000000 (64-bit, prefetchable) [size=32M]
   I/O ports at 1000 [size=128]
   [virtual] Expansion ROM at 50800000 [disabled] [size=512K]
   Capabilities: [60] Power Management version 3
   Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
   Capabilities: [78] Express Endpoint, MSI 00
   Capabilities: [100] Virtual Channel
   Capabilities: [250] Latency Tolerance Reporting
   Capabilities: [128] Power Budgeting <?>
   Capabilities: [420] Advanced Error Reporting
   Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
   Capabilities: [900] #19
   Kernel driver in use: nvgpu

and also on the lb front.
It seems to be a very simplistic device of some sort that only listens on telnet port, cannot se what command it accepts.
It sends and receives data that canrx/cantx listen to, and that's how (gps info?) propagates around. lb is referenced in gps service by ip on cid.

Nmap scan report for lb (192.168.90.104)
Host is up (0.00016s latency).
Not shown: 1065 closed ports
PORT   STATE SERVICE
23/tcp open  telnet
MAC Address: 00:11:22:33:44:55 (Unknown)
No exact OS matches for host (If you know what OS is running on it, see http://nmap.org/submit/ ).
TCP/IP fingerprint:
OS:SCAN(V=6.47%E=4%D=5/21%OT=23%CT=1%CU=42453%PV=Y%DS=1%DC=D%G=Y%M=001122%T
OS:M=5921D1CF%P=aarch64-redhat-linux-gnu)SEQ(SP=14%GCD=1%ISR=20%CI=I%II=RI%
OS:TS=U)SEQ(SP=14%GCD=1%ISR=2C%II=RI%TS=U)OPS(O1=M400%O2=M400%O3=M400%O4=M4
OS:00%O5=M400%O6=M400)WIN(W1=1FA0%W2=1FA0%W3=1FA0%W4=1FA0%W5=1FA0%W6=1FA0)E
OS:CN(R=Y%DF=N%T=FF%W=1FA0%O=M400%CC=N%Q=)T1(R=Y%DF=N%T=FF%S=O%A=S+%F=AS%RD
OS:=0%Q=)T2(R=N)T3(R=Y%DF=N%T=FF%W=1FA0%S=O%A=S+%F=AS%O=M400%RD=0%Q=)T4(R=Y
OS:%DF=N%T=FF%W=1FA0%S=A%A=S%F=AR%O=%RD=0%Q=)T5(R=Y%DF=N%T=FF%W=1FA0%S=A%A=
OS:S+%F=AR%O=%RD=0%Q=)T6(R=Y%DF=N%T=FF%W=1FA0%S=A%A=S%F=AR%O=%RD=0%Q=)T7(R=
OS:Y%DF=N%T=FF%W=1FA0%S=A%A=S+%F=AR%O=%RD=0%Q=)U1(R=Y%DF=N%T=FF%IPL=38%UN=0
OS:%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)IE(R=Y%DFI=S%T=FF%CD=S)

Network Distance: 1 hop

# telnet lb
lb>

10:55:54.290245 IP 192.168.90.104.49153 > 192.168.90.103.27694: UDP, length 38
10:55:54.292534 IP 192.168.90.103.44930 > 192.168.90.104.28204: UDP, length 19
10:55:54.297267 IP 192.168.90.104.49153 > 192.168.90.103.27694: UDP, length 19
10:55:54.297617 IP 192.168.90.103.44930 > 192.168.90.104.28204: UDP, length 57
10:55:54.300255 IP 192.168.90.104.49153 > 192.168.90.103.27694: UDP, length 38
10:55:54.302570 IP 192.168.90.103.44930 > 192.168.90.104.28204: UDP, length 19
10:55:54.307548 IP 192.168.90.103.44930 > 192.168.90.104.28204: UDP, length 38
10:55:54.310247 IP 192.168.90.104.49153 > 192.168.90.103.27694: UDP, length 38
10:55:54.317582 IP 192.168.90.103.44930 > 192.168.90.104.28204: UDP, length 19
10:55:54.320243 IP 192.168.90.104.49153 > 192.168.90.103.27694: UDP, length 38
10:55:54.322537 IP 192.168.90.103.44930 > 192.168.90.104.28204: UDP, length 19
10:55:54.325266 IP 192.168.90.104.49153 > 192.168.90.103.27694: UDP, length 19
10:55:54.326234 IP 192.168.90.104.49153 > 192.168.90.103.27694: UDP, length 19

 

[KyleDay]

Currently disassembling dash...hold please.

 

[lunitiks]

 

[KyleDay]

Just a teaser. I have it out of the car! Let me disassemble it and see what's under all this metal shielding.

 

[lunitiks]

 

[DamianXVI]

Did some more experiments on color calculation in the camera images from @verygreen sets (hope you don't mind I get back to this for a moment, when the discussion already progressed). I ended up finding a fixed brightness range of grayscale and red channels for each camera group to crop pixel values to, so the computation is very simple. This gives fairly good and consistent results (but not perfect). There is no additional normalization or color balancing per image. Both green and blue are represented by cyan in a final image.

Below are some examples. This time with proper gamma correction applied, so that images from this kind of camera would be correctly shown on our screens.

Orange and white road marking lines, green or red light ahead:



Red light, then green light:




Red and green lights:

 

[DamianXVI]

Looking into blinding sun, but can you see the difference between red, orange and green on traffic light warning signs:



Red sign seen in the back facing camera (there is no stop sign in the set):



Combined intersection and parking lot scenes, just like the ones made by @bjornb:

 

[lunitiks]

This thread right here is on steroids

 

[KyleDay]

Getting inside. It's a single cpu board. Not two.

 

[MP3Mike]

Getting inside. It's a single cpu board. Not two.

What is the part number of your unit? (What revision is it?)

 

[KyleDay]

Rev 01? Here's a pic

 

[lunitiks]

That's the PCB part no., not the one I mentioned earlier (which is the complete assembly). There should be a white sticker underneath the enclosure.

Anyway:

 

[JeffK]

My heart is pounding!

 

[KyleDay]

Long post coming...