TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC
Start a Discussionhttps://teslamotorsclub.com/tmc/tags/

Is Tesla using machine learning for AP?

Discussion in 'Autonomous Vehicles' started by BeesKnees, Jun 14, 2017.

  1. BeesKnees

    BeesKnees Member

    Joined:
    Dec 13, 2016
    Messages:
    68
    Location:
    Oregon
    Greetings,

    I believe that with Mobileye there was a machine learning approach in play for AP. Post parting of the ways, there was lots of talk of "shadow" mode data collection and such. Do we believe that the plan forward for Tesla is machine learning based? Not, do we think is "should be". I'm wondering if it "is". What do we know?

    Thanks.
     
  2. chillaban

    chillaban Active Member

    Joined:
    May 5, 2016
    Messages:
    1,895
    Location:
    Bay Area
    What exactly do you mean?

    With regards to the computer vision approaches used by both MobileEye and Tesla Vision for image recognition, both are inherently machine learning style approaches, whether you'd like to call them neural nets, deep learning, etc etc etc.

    However, in terms of the actual driving control algorithms, neither the one Tesla implemented in AP1, AP2, nor the stock one from MobileEye is machine learning per se. Those are human coded algorithms / control loops leveraging the data produced by the AI vision model.
     
    • Informative x 1
  3. mongo

    mongo Member

    Joined:
    May 3, 2017
    Messages:
    340
    Location:
    Michigan
    I know that AP2 has multiple cameras, Nvidia processing power, a network connection to Tesla HQ, and that it requires drivers to give near continuous feedback regarding it's operation when using AP.

    Since I don't work there, I don't know if they are taking the force feedback and braking inputs from the driver and feeding them into a NN to improve the control algorithm. I also don't know if they are using the drivers' control strategy and comparing it to what AP would do if it were active. I also don't know if they have flagged certain drivers as good examples of driving and other as bad examples. I do know I would be surprised if they weren't. ;-)
     
    • Informative x 1
  4. BeesKnees

    BeesKnees Member

    Joined:
    Dec 13, 2016
    Messages:
    68
    Location:
    Oregon
    What exactly do I mean is always a good question for me. But this answer is what I'm looking for. Thanks. No AP algorithm. Vision systems are yes.
     
  5. BeesKnees

    BeesKnees Member

    Joined:
    Dec 13, 2016
    Messages:
    68
    Location:
    Oregon
    This possible NN for AP is what I'm curious about. I know Nvidia has done some full end to end machine learning on AP at least as an experiment. Wondering about Tesla control algorithm. Other reply says nope.

    Thanks.
     
  6. chillaban

    chillaban Active Member

    Joined:
    May 5, 2016
    Messages:
    1,895
    Location:
    Bay Area
    I mean, I think an AI based control algorithm is interesting but for now I think that's still a science experience / research field. I think for now, the industry expects aircraft Autopilot style design, where software is explicitly written by humans to obey the rules of traffic, and can be formally verified and proven.

    There's been a lot of interesting research in driving simulators and in video games for having a pure neural net trained the way you'd train a human. But I think that is years away from being a practical alternative.

    To date I don't know of anyone in this industry actually productizing such an approach, and that includes MobileEye and Waymo.
     
  7. Tam

    Tam Active Member

    Joined:
    Nov 25, 2012
    Messages:
    2,676
    Location:
    Visalia, CA
  8. RubberToe

    RubberToe Supporting the greater good

    Joined:
    Jun 28, 2012
    Messages:
    1,539
    Location:
    Pasadena, Ca
  9. chillaban

    chillaban Active Member

    Joined:
    May 5, 2016
    Messages:
    1,895
    Location:
    Bay Area
    That's data gathering for training the camera vision neural nets, though.

    What the OP is referring to is, for example, Follow the Rainbow Road - Training AI to Play Mario Kart | NVIDIA Blog

    In Autopilot terms, you'd basically tell the car "ok here's two pedals that take inputs 0% to 100%, and a steering wheel that takes a force value. Here's millions of hours of footage of what outputs humans give for a given goal and visual input. Go learn!"

    It's an interesting field of research, but not really something that I see any major player trying.
     
  10. JeffK

    JeffK Well-Known Member

    Joined:
    Apr 27, 2016
    Messages:
    5,350
    Location:
    Indianapolis
    One thought might be to check out the various other threads on autopilot and machine learning... we've had pages and pages of conversation on this topic.
     
  11. BeesKnees

    BeesKnees Member

    Joined:
    Dec 13, 2016
    Messages:
    68
    Location:
    Oregon
    Thanks for the conversation. I did look and found some threads. Did not see anything that talked about current Tesla reality. Thus this post. Thanks again.
     
  12. stopcrazypp

    stopcrazypp Well-Known Member

    Joined:
    Dec 8, 2007
    Messages:
    9,164
    #12 stopcrazypp, Jun 14, 2017
    Last edited: Jun 14, 2017
    You are unlikely to find anyone that can say what Tesla is really doing unless an employee leaks it. Everyone so far has been guessing based on known information (aka they would have to qualify with "likely" rather than "is" as you are requesting).
     
  13. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    954
    Location:
    TN
    Here's the list of convolution kernels (I think this is called) from 17.17.4 that are run on nvidia cuda side:
    Code:
    b2f.cubin
    bbias.cubin
    bgemm_128x128x8_TN_vec.cubin
    bgemm_32x32x32_TN_vec.cubin
    bpool.cubin
    copy_transpose_f4_27_4_4_26_40_0_3_1_4_2.cubin
    copy_transpose_f4_5_16_16_26_40_0_3_1_4_2.cubin
    f2b.cubin
    i8conv_3x3_20x4_16_N1_q8.cubin
    i8conv_3x3d2_20x4_16_N1_q8.cubin
    i8conv_5x5_20x4_8_N1_q8.cubin
    i8conv_5x5d2_20x4_8_N1_q8.cubin
    sbias.cubin
    sconv_direct_fprop_64x32_N1_bias_relu.cubin
    sconv_winograd_2x2_3x3_32x32_K256_W40_Q40_N1_bias_relu.cubin
    sgemm_32x32x32_TN.cubin
    sgemm_32x32x32_TN_vec.cubin
    sgemm_tn_128x128_vec.cubin
    
    Here's mapping to more readable names:
    Code:
      name: "conv1"
      class_name: "neon_convolution_kernel"
      kernel_name: "sconv_direct_fprop_64x32_N1_bias_relu"
      module_name: "sconv_direct_fprop_64x32_N1_bias_relu"
      name: "convert_to_int8"
      class_name: "tesla_conversion_kernel"
      kernel_name: "f2b"
      module_name: "f2b"
      name: "pool1"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3"
      module_name: "bpool"
      name: "conv2_1x1_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "conv2_1x1"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_128x128x8_TN_vec"
      module_name: "bgemm_128x128x8_TN_vec"
      next_kernel_spec_name: "conv2_1x1_bias"
      name: "conv2_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3_20x4_16_N1_q8"
      module_name: "i8conv_3x3_20x4_16_N1_q8"
      name: "pool2"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3"
      module_name: "bpool"
      name: "inception_3a_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_3a_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_128x128x8_TN_vec"
      module_name: "bgemm_128x128x8_TN_vec"
      next_kernel_spec_name: "inception_3a_merge_bias"
      name: "inception_3a_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3_20x4_16_N1_q8"
      module_name: "i8conv_3x3_20x4_16_N1_q8"
      name: "inception_3a_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5_20x4_8_N1_q8"
      module_name: "i8conv_5x5_20x4_8_N1_q8"
      name: "inception_3a_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_3a_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_3a_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_3a_pool_proj_bias"
      name: "inception_3b_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_3b_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_128x128x8_TN_vec"
      module_name: "bgemm_128x128x8_TN_vec"
      next_kernel_spec_name: "inception_3b_merge_bias"
      name: "inception_3b_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3_20x4_16_N1_q8"
      module_name: "i8conv_3x3_20x4_16_N1_q8"
      name: "inception_3b_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5_20x4_8_N1_q8"
      module_name: "i8conv_5x5_20x4_8_N1_q8"
      name: "inception_3b_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_3b_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_3b_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_3b_pool_proj_bias"
      name: "pool3"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3"
      module_name: "bpool"
      name: "inception_4a_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4a_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_128x128x8_TN_vec"
      module_name: "bgemm_128x128x8_TN_vec"
      next_kernel_spec_name: "inception_4a_merge_bias"
      name: "inception_4a_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3_20x4_16_N1_q8"
      module_name: "i8conv_3x3_20x4_16_N1_q8"
      name: "inception_4a_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5_20x4_8_N1_q8"
      module_name: "i8conv_5x5_20x4_8_N1_q8"
      name: "inception_4a_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_4a_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4a_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_4a_pool_proj_bias"
      name: "inception_4b_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4b_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_128x128x8_TN_vec"
      module_name: "bgemm_128x128x8_TN_vec"
      next_kernel_spec_name: "inception_4b_merge_bias"
      name: "inception_4b_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3_20x4_16_N1_q8"
      module_name: "i8conv_3x3_20x4_16_N1_q8"
      name: "inception_4b_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5_20x4_8_N1_q8"
      module_name: "i8conv_5x5_20x4_8_N1_q8"
      name: "inception_4b_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_4b_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4b_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_4b_pool_proj_bias"
      name: "inception_4c_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4c_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_4c_merge_bias"
      name: "inception_4c_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3_20x4_16_N1_q8"
      module_name: "i8conv_3x3_20x4_16_N1_q8"
      name: "inception_4c_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5_20x4_8_N1_q8"
      module_name: "i8conv_5x5_20x4_8_N1_q8"
      name: "inception_4c_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_4c_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4c_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_4c_pool_proj_bias"
      name: "inception_4d_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4d_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_4d_merge_bias"
      name: "inception_4d_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3_20x4_16_N1_q8"
      module_name: "i8conv_3x3_20x4_16_N1_q8"
      name: "inception_4d_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5_20x4_8_N1_q8"
      module_name: "i8conv_5x5_20x4_8_N1_q8"
      name: "inception_4d_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_4d_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4d_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_4d_pool_proj_bias"
      name: "inception_4e_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4e_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_4e_merge_bias"
      name: "inception_4e_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3_20x4_16_N1_q8"
      module_name: "i8conv_3x3_20x4_16_N1_q8"
      name: "inception_4e_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5_20x4_8_N1_q8"
      module_name: "i8conv_5x5_20x4_8_N1_q8"
      name: "inception_4e_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_4e_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_4e_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_4e_pool_proj_bias"
      name: "pool4"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_5a_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_5a_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_5a_merge_bias"
      name: "inception_5a_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3d2_20x4_16_N1_q8"
      module_name: "i8conv_3x3d2_20x4_16_N1_q8"
      name: "inception_5a_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5d2_20x4_8_N1_q8"
      module_name: "i8conv_5x5d2_20x4_8_N1_q8"
      name: "inception_5a_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_5a_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_5a_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_5a_pool_proj_bias"
      name: "inception_5b_merge_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_5b_merge"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_128x128x8_TN_vec"
      module_name: "bgemm_128x128x8_TN_vec"
      next_kernel_spec_name: "inception_5b_merge_bias"
      name: "inception_5b_3x3"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_3x3d2_20x4_16_N1_q8"
      module_name: "i8conv_3x3d2_20x4_16_N1_q8"
      name: "inception_5b_5x5"
      class_name: "device_q8_convolution_kernel"
      kernel_name: "i8conv_5x5d2_20x4_8_N1_q8"
      module_name: "i8conv_5x5d2_20x4_8_N1_q8"
      name: "inception_5b_pool"
      class_name: "tesla_pooling_kernel"
      kernel_name: "bmaxpool_3x3_s1x1"
      module_name: "bpool"
      name: "inception_5b_pool_proj_bias"
      class_name: "tesla_bias_scale_kernel"
      kernel_name: "bbias"
      module_name: "bbias"
      name: "inception_5b_pool_proj"
      class_name: "openai_gemm_kernel"
      kernel_name: "bgemm_32x32x32_TN_vec"
      module_name: "bgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "inception_5b_pool_proj_bias"
      name: "convert_to_fp32"
      class_name: "tesla_conversion_kernel"
      kernel_name: "b2f"
      module_name: "b2f"
      name: "conv_pool_26x40_bias"
      class_name: "tesla_bias_kernel"
      kernel_name: "sbias_128x1_relu"
      module_name: "sbias"
      name: "conv_pool_26x40"
      class_name: "openai_gemm_kernel"
      kernel_name: "sgemm_32x32x32_TN"
      module_name: "sgemm_32x32x32_TN"
      next_kernel_spec_name: "conv_pool_26x40_bias"
      name: "conv_pool_13x20_bias"
      class_name: "tesla_bias_kernel"
      kernel_name: "sbias_128x1_vec_relu"
      module_name: "sbias"
      name: "conv_pool_13x20"
      class_name: "openai_gemm_kernel"
      kernel_name: "sgemm_32x32x32_TN_vec"
      module_name: "sgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "conv_pool_13x20_bias"
      name: "conv_pool_9x14_bias"
      class_name: "tesla_bias_kernel"
      kernel_name: "sbias_128x1_relu"
      module_name: "sbias"
      name: "conv_pool_9x14"
      class_name: "openai_gemm_kernel"
      kernel_name: "sgemm_32x32x32_TN"
      module_name: "sgemm_32x32x32_TN"
      next_kernel_spec_name: "conv_pool_9x14_bias"
      name: "conv_pool_5x7_bias"
      class_name: "tesla_bias_kernel"
      kernel_name: "sbias_128x1_vec_relu"
      module_name: "sbias"
      name: "conv_pool_5x7"
      class_name: "openai_gemm_kernel"
      kernel_name: "sgemm_32x32x32_TN_vec"
      module_name: "sgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "conv_pool_5x7_bias"
      name: "conv_PSP"
      class_name: "neon_convolution_kernel"
      kernel_name: "sconv_winograd_2x2_3x3_32x32_K256_W40_Q40_N1_bias_relu"
      module_name: "sconv_winograd_2x2_3x3_32x32_K256_W40_Q40_N1_bias_relu"
      name: "deconv_16x16"
      class_name: "neon_compound_deconvolution_kernel"
      kernel_name: "sgemm_tn_128x128_vec"
      module_name: "sgemm_tn_128x128_vec"
      next_kernel_spec_name: "deconv_16x16_shuffle"
      name: "deconv_16x16_shuffle"
      class_name: "neon_transpose_kernel"
      kernel_name: "copy_transpose_f4_5_16_16_26_40_0_3_1_4_2"
      module_name: "copy_transpose_f4_5_16_16_26_40_0_3_1_4_2"
      next_kernel_spec_name: "deconv_16x16_bias"
      name: "deconv_16x16_bias"
      class_name: "tesla_bias_kernel"
      kernel_name: "sbias_128x1_vec"
      module_name: "sbias"
      name: "reshape_to_input_27x104x160"
      class_name: "openai_compound_deconvolution_kernel"
      kernel_name: "sgemm_32x32x32_TN_vec"
      module_name: "sgemm_32x32x32_TN_vec"
      next_kernel_spec_name: "reshape_to_input_27x104x160_shuffle"
      name: "reshape_to_input_27x104x160_shuffle"
      class_name: "neon_transpose_kernel"
      kernel_name: "copy_transpose_f4_27_4_4_26_40_0_3_1_4_2"
      module_name: "copy_transpose_f4_27_4_4_26_40_0_3_1_4_2"
      next_kernel_spec_name: "reshape_to_input_27x104x160_bias"
      name: "reshape_to_input_27x104x160_bias"
      class_name: "tesla_bias_kernel"
      kernel_name: "sbias_128x1_vec"
      module_name: "sbias"
    
    They are used with some sort of a neural net if this message is to be believed in the code: "constructured Tesla conv kernels for network".
    I don't think they train the net in the car, though I don't know much about the subject.
    There's a ~30M binary blob of the trained network data they use instead that is supplied.
     
    • Informative x 3
  14. BeesKnees

    BeesKnees Member

    Joined:
    Dec 13, 2016
    Messages:
    68
    Location:
    Oregon
    Fun. Thanks

    And to the msg on expectations, I do not expect complete clarity. Not my first user group : ) but any info on what we know, think and believe is useful.

    I am puzzling on ML strategies in general and adding what my car MAY be doing to the pile couldn't be passed up.
     
  15. Wander

    Wander Member

    Joined:
    Mar 28, 2015
    Messages:
    30
    Location:
    Gelderland
    #15 Wander, Jun 14, 2017
    Last edited: Jun 14, 2017
    That are indeed parts that are used with neural nets. Quite an interesting list, they seem to be using stuff from OpenAI, one of Elon Musks other compagnies and (parts of) Inception, which is a neural network architecture from Google.

    It is very unlikely that they are using the cars to train the network, they don't have the bandwidth (shuffling around video files and neural networks is a bit much for a lte modem) nor do they have the computing power to train a big neural network.

    Edit: here is a link to the Google inception neural network paper:
    https://arxiv.org/pdf/1409.4842.pdf

    You can recognize a lot of the kernels from Tesla in the inception paper
     
    • Informative x 1
    • Like x 1
  16. 3Victoria

    3Victoria Active Member

    Joined:
    May 8, 2016
    Messages:
    1,524
    Location:
    Victoria, British Columbia
    Here's my guesses:
    AP1: Mobileye processes raw video to recognize objects, Tesla supplies the driving logic.
    AP2 now: Tesla uses AI to recognize objects, and supplies the driving logic to achieve AP1 parity.
    AP2 future: Tesla uses deep end-to-end AI, with tweaks, to achieve level 3/4.

    The demo videos used the second method, but this code/AI code is not fully developed, so is only good for demos. It requires very large datasets, which they will collect from S/X/3s and simulations, to train the AI. The AI training will be done on large fast computers. The resulting AI/models will downloaded to the car to be executed by the NVidia AI/video processors.
     
  17. JeffK

    JeffK Well-Known Member

    Joined:
    Apr 27, 2016
    Messages:
    5,350
    Location:
    Indianapolis
    Be careful when you use the term end-to-end AI because many people think end-to-end reinforcement learning which would absolutely not work for proper self driving even though it works for little demos like Nvidia's BB8.
    More than likely it'd be a combination of a few different types of neural networks and some logic.
     
    • Like x 1
  18. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    954
    Location:
    TN
    Another interesting bit, the trained net data in 17.11.45 was ~14m, so it more than doubled to 17.17.4, I wonder what's the change in 17.22.46...
     
  19. chillaban

    chillaban Active Member

    Joined:
    May 5, 2016
    Messages:
    1,895
    Location:
    Bay Area
    It's also worth noting that the ME architecture uses a custom ASIC to run the computationally-intensive parts of the neural net, and the firmware portion runs on a few MIPS cores that pale in comparison to a Linksys router. I'm not sure how much they can achieve in terms of fundamental updates to recognition algorithms in the EyeQ3 hardware out there.

    AP2 runs their neural nets on a purely software-defined GPU + ARM CPU setup. In theory this makes Tesla's architecture more flexible to change and improve with software updates, including dramatic redesigns of their vision pipeline should they choose to do so in the future.
     
  20. BigD0g

    BigD0g Member

    Joined:
    Jan 12, 2017
    Messages:
    553
    Location:
    VA
    Sign recognition in shadow mode perhaps?
     

Share This Page