TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

Tesla's new FSD patent to optimize image processing for deep learning

Discussion in 'Autopilot & Autonomous/FSD' started by diplomat33, Dec 27, 2019.

  1. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,667
    Location:
    Terre Haute, IN USA
    Tesla has a new patent called “Data Pipeline and Deep Learning System for Autonomous Driving”. Essentially, the patent optimizes the process of image processing for deep learning.

    Some interesting quotes from the article:

    "The flowchart below describes what the process of the vehicle learning the information would look like. “Receive Sensor Data” is the first portion of this process. Then, data will be broken down and pre-processed for the system to then begin its “Deep Learning Analysis.” The results will then be passed along to the vehicle’s Artificial Intelligence Processor to be utilized during vehicle control."

    [​IMG]

    "In another process, the series of information that is retrieved from these images will be compared to data compiled from other Tesla users on a global scale. This will alleviate concerns that drivers may have that the system could perform the wrong process when driving autonomously. The aim of the patent is to create a safe driving experience and improve upon the already solid performance of Tesla’s autonomous driving software, and do so in a process that is more efficient than before."

    "By using this process, Tesla is able to maintain as much resolution as possible from the images captured by its vehicles’ cameras and sensors. This then allows the Neural Network to more efficiently learn from the data packets that it is receiving. This allows the Neural Network to work with better images in a more efficient manner as well, which opens the doors to faster autonomous driving improvements."

    Tesla's new data pipeline and deep learning patent paves way for quicker autonomous driving improvements

    Hopefully, this patent helps Tesla speed up their FSD development.

    .
     
    • Informative x 2
    • Like x 2
  2. Eno Deb

    Eno Deb Active Member

    Joined:
    Aug 17, 2018
    Messages:
    2,597
    Location:
    SF Bay Area
    At a cursory glance that looks like a very broad patent that will be difficult to defend in litigation. Interesting that they are covering Lidar in one of their claims. But generally, filing a patent doesn't necessarily mean that any or all of the claims will end up in the product.
     
    • Like x 2
    • Informative x 1
  3. strangecosmos2

    strangecosmos2 Koopa Troopa

    Joined:
    Nov 24, 2019
    Messages:
    177
    Location:
    New Donk City
    The language is extraordinarily generic and legalistic. Hard to tell if there's anything in this patent we didn't already know.
     
  4. Eno Deb

    Eno Deb Active Member

    Joined:
    Aug 17, 2018
    Messages:
    2,597
    Location:
    SF Bay Area
    If you look at the claims the key is really in claim 1 (the other claims are either dependent claims or more detailed versions of claim 1), specifically this:

    "providing each component image of the plurality of component images as a different input to a different layer of a plurality of layers of an artificial neural network to determine a result; and using the result of the artificial neural network to at least in part autonomously operate the vehicle. "

    This seems to be essentially what Karpathy has recently talked about in some presentations about the structure of their neural nets (obviously after Tesla filed he patent). Multi-layer neural nets are actually well known prior art and thus probably not patentable, but perhaps not in the context of autonomous cars (although that seems doubtful too).
     
    • Helpful x 2
    • Informative x 1
    • Like x 1
  5. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,667
    Location:
    Terre Haute, IN USA
    While multi-layered NN's are not new, it seems what Tesla did is develop a method or technique for feeding different component images to different layers in the NN. So one layer of the NN gets one component image, another layer of the NN gets a different component image. That's what earned them this patent. So presumably, this specific technique for distributing different components images to different layers of a NN is something new.
     
    • Informative x 1
  6. DanCar

    DanCar Active Member

    Joined:
    Oct 2, 2013
    Messages:
    1,616
    Location:
    SF Bay Area
    Not sure where the word "earned" comes from. From my perspective they haven't earned anything. They are just claiming something in a patent filing. If they win a court case defending this, or someone decides to pony up something based on the patent, then they will have earned something.
    Someone might claim if they are granted a patent they have earned something but the bar is low for filing patents and many frivolous patents are filed each year.
     
    • Helpful x 1
    • Like x 1
  7. strangecosmos2

    strangecosmos2 Koopa Troopa

    Joined:
    Nov 24, 2019
    Messages:
    177
    Location:
    New Donk City
    Multi-layer (i.e. deep) neural networks have been ubiquitous since around 2012 and today “neural network” is synonymous with “deep neural network” or “multi-layered neural network”.

    What is notable about Tesla's neural network is that it's a multi-task neural network. (See Karpathy's ICML 2019 talk.) Tesla didn't invent the idea of multi-task networks, so I would guess/hope that idea by itself isn't patentable.
     
    • Informative x 1
  8. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,667
    Location:
    Terre Haute, IN USA
    Again, the patent is not for multi-layered NN. The patent is for distributing composite images to different layers in the NN, among other things.
     
    • Helpful x 1
  9. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,667
    Location:
    Terre Haute, IN USA
    By earned, I just mean that the patent was granted.
     
    • Informative x 1
  10. strangecosmos2

    strangecosmos2 Koopa Troopa

    Joined:
    Nov 24, 2019
    Messages:
    177
    Location:
    New Donk City
    Was the patent actually granted or is the patent application still pending?
     
  11. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,667
    Location:
    Terre Haute, IN USA
    The patent was published yesterday.
     
    • Informative x 2
  12. strangecosmos2

    strangecosmos2 Koopa Troopa

    Joined:
    Nov 24, 2019
    Messages:
    177
    Location:
    New Donk City
    Does published mean granted?

    Isn't it the opposite of composite images, i.e. images decomposed into parts?
     
    • Informative x 1
  13. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,667
    Location:
    Terre Haute, IN USA
    Not necessarily. Patents are published before they are granted.

    The patent says "decomposing the received image into a plurality of component images". So I interpret that to mean that it takes one image image and breaking it up into several images.
     
    • Helpful x 1
    • Informative x 1
  14. strangecosmos2

    strangecosmos2 Koopa Troopa

    Joined:
    Nov 24, 2019
    Messages:
    177
    Location:
    New Donk City
    Right, and a composite image is made by taking parts of different images and stitching them together into one image. It's the opposite of a composite image.
     
  15. Eno Deb

    Eno Deb Active Member

    Joined:
    Aug 17, 2018
    Messages:
    2,597
    Location:
    SF Bay Area
    This is an international patent application (PCT process), which cannot in itself turn into granted patent (only national patent offices can grant patents). Basically, it's a process of preparing a unified application that can later be used to apply for a patent in multiple treaty countries of your choice. The publication (together with the results of an international search) is the final step of this "international phase" before the potential national patent applications.
     
    • Informative x 4
    • Like x 1
  16. Bladerskb

    Bladerskb Senior Software Engineer

    Joined:
    Oct 24, 2016
    Messages:
    2,014
    Location:
    Michigan
    #16 Bladerskb, Dec 27, 2019
    Last edited: Dec 27, 2019
    No the patent hasn't been granted. Plus It takes years to get a patent granted.
    Secondly this is an absolutely useless patent that is being as usual hyped up by the Tesla fanbase as being some 'game changer'.

    While others have hundreds some even thousands of SDC related patents.
    Tesla's SDC related patents can almost be counted on one hand. Yet every single one of them have be hyped.

    ''paves way for quicker autonomous driving improvements" lol seriously? is there any truth in the Tesla Land?
     
  17. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,667
    Location:
    Terre Haute, IN USA
    I stand corrected on the first part. I was wrong about the patent being granted.

    But how is the process described in the patent of "providing each component image of the plurality of component images as a different input to a different layer of a plurality of layers of an artificial neural network to determine a result; and using the result of the artificial neural network to at least in part autonomously operate the vehicle." useless? Surely it would have some use to Tesla.
     
    • Informative x 1
  18. Bladerskb

    Bladerskb Senior Software Engineer

    Joined:
    Oct 24, 2016
    Messages:
    2,014
    Location:
    Michigan
    #18 Bladerskb, Dec 27, 2019
    Last edited: Dec 27, 2019
    Because others have hundreds some even thousands of SDC related patents. Google/Waymo has hundreds, maybe thousands. Mobileye for example has hundreds maybe thousands and these are useful patents that could block an actual competitor from doing something vital.

    Tesla's SDC related patents can almost be counted on one hand. Yet every single one of them is hyped up as some game changer. The article you posted said this patent 'paves way for quicker autonomous driving improvements' and that it 'making sure that it will evolve at an even faster rate in the future.'

    This patent has nothing to do with that, its simply a patent on prepping data before feeding it into a NN. The spin that is put on any little thing that Tesla does to push agenda and narrative that their NN is on an exponential growth or that they are ahead is so gross.
     
  19. Jackl1956

    Jackl1956 Active Member

    Joined:
    May 11, 2013
    Messages:
    1,767
    Location:
    Los Angeles
    You weren’t the sunny kid in your class. Were you?
     
    • Like x 2
  20. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,215
    Location:
    Snohomish, WA
    Where are you seeing that?

    This is what he linked to, and it's not an article at all.
    WO2019245618 DATA PIPELINE AND DEEP LEARNING SYSTEM FOR AUTONOMOUS DRIVING

    That's not really what they're trying to patent. It's more about how the data is optimized/customized for the layer it's used on.

    At the end of the day it's just another SW Patent which typically is a patent on obvious things. I'm sure you've dealt with the Head of IP asking people to think of anything that can be protected. Where the patent is mostly defensive posturing. Where you try to line up your patents with a competitors patent in some kind of Cold War like stand off. So they're often about extremely trivial things, and probably won't hold up to scrutiny.
     
    • Informative x 2

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC