Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Programming Language Wars: Python/C/C++ et al.

This site may earn commission on affiliate links.

MC3OZ

Active Member
Jul 25, 2019
4,880
29,906
QLD Australia
Cathie Wood from Ark Investments discussing Tesla this week. Tesla New York on Twitter

She opined that Tesla’s solar roof business is good but that they think their autonomous vehicle development is where the real value growth will come from.

Just on autonomy I'm not sure if we discussed a tweet form Elon mentioned that some of the software developer recruitment for the FSD team (and other areas) was to rewrite Python code into C++.

The exact quote being:-
Our NN is initially in Python for rapid iteration, then converted to C++/C/raw metal driver code for speed (important!). Also, tons of C++/C engineers needed for vehicle control & entire rest of car. Educational background is irrelevant, but all must pass hardcore coding test.

Being for a software developer background this statement makes me optimistic for several reasons:-
  1. The strategy of rapid development followed by speed optimisation is very sound.
  2. You do speed optimisation closer to the finish line...
To be clear it is not obvious if it was always converted to C++ prior to release, it always was, then this conversion process is the current bottleneck, and the python version is ahead of what we have in our cars...

Either way it is bullish, it is either going to get faster, or better... and my guess is this is going to free up resources within the team to better focus them on the end result.

I know software developers will be challenged by this and keen to join the team, 20 years ago I would have paid my airfare to the US for an interview even with a 10% chance of landing the job.... These days I know I'm a bit to old for a full on role like this...
 
Last edited:
Just on autonomy I'm not sure if we discussed a tweet form Elon mentioned that some of the software developer recruitment for the FSD team (and other areas) was to rewrite Python code into C++. ... Being for a software developer background this statement makes me optimistic for several reasons:

That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)
 
That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)

Well I guess I'm easily pleased....

My assumption is productivity is higher in a high level language like Python when things are rapidly being iterated, once the design becomes more concrete, you migrate to C++...
 
That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)
That is, in fact, exactly the situation where it's chthonic.


On 6-month review, 19 Oct: Dang! That was one of my all-time greats. Too bad too few saw and appreciated it.:mad::D
 
Last edited:
That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)

I don’t think they are converting python code to C++. They are using PyTorch to experiment with training methods, then using a C++ trainer to train the NN at scale.

The trainer is the software 2.0 equivalent of the compiler. It turns source (labeled images and video) into an executable (NN). PyTorch is a great compiler for development on smaller data sets but C++ is used for nightly builds.

Thats my guess.
 
Just on autonomy I'm not sure if we discussed a tweet form Elon mentioned that some of the software developer recruitment for the FSD team (and other areas) was to rewrite Python code into C++.

The exact quote being:-


Being for a software developer background this statement makes me optimistic for several reasons:-
  1. The strategy of rapid development followed by speed optimisation is very sound.
  2. You do speed optimisation closer to the finish line...
To be clear it is not obvious if it was always converted to C++ prior to release, it always was, then this conversion process is the current bottleneck, and the python version is ahead of what we have in our cars...

Either way it is bullish, it is either going to get faster, or better... and my guess is this is going to free up resources within the team to better focus them on the end result.

I know software developers will be challenged by this and keen to join the team, 20 years ago I would have paid my airfare to the US for an interview even with a 10% chance of landing the job.... These days I know I'm a bit to old for a full on role like this...


No way they are just now using C. The embedded code running on the Nvidia and HW3 is assuredly in C. I think this really means nothing.
 
That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)

Disagree.

Python for rapid prototyping of algorithms is the de facto standard nowadays, because you can modify your algorithms much more quickly than in C/C++. The key is of course to have experience in understanding how you develop your algorithms can be readily replicated with the same precision in C.

All the main deep learning packages are in Python (tensorflow, pytorch). Those are what Tesla uses.

It's not even hard to write python code that writes your desired C code, I do it all the time! Just have to make sure your cast your variables correctly for memory optimization
 
Last edited:
No way they are just now using C. The embedded code running on the Nvidia and HW3 is assuredly in C. I think this really means nothing.

This is the quote from Elon:-
Our NN is initially in Python for rapid iteration, then converted to C++/C/raw metal driver code for speed (important!).

You are right that the code running in the car is probably C, but at some stage in the development cycle the NN, or related software, is in Python, or something like Python..

I still see hiring more C++ developers as a good thing... and a likely sign of future progress..
 
This is the quote from Elon:-


You are right that the code running in the car is probably C, but at some stage in the development cycle the NN, or related software, is in Python, or something like Python..

I still see hiring more C++ developers as a good thing... and a likely sign of future progress..

Converting Python NN weights and dot products to C isn't that complicated... Neural nets seem complicated but the math is really easy. That is not really the speed optimization to focus on.

The key is when you see them obsessing over pruning their massive neural network to achieve similar accuracy while reduces the number of weights / computations. And limiting the memory scope of some layers (drop from 32 bit to 16 or 8 bit). Deepsscale acquisition is for this.
 
OT more FSD discussion:

I am very bullish that if Tesla can achieve accurate FSD, it will be very bullish on share price.

I am not confident at all that this is happening soon. Frankly, I have no idea how anyone here can have high confidence, unless you work in computer vision and are familiar with this specific type of problem.

As background, I work in building machine learning / deep learning algorithms that end up running on memory constrained systems.

I was always skeptical about FSD on HW2 in 2016 because of computational limitations. Images weren't even proccessed at full resolution - they were downsampled. Not a good start.

Before HW3 came out, @jimmy_d was excited because some big neural net was uncovered that was used for all cameras and took not one, but 2 full scale images as input.

Exciting yes, but 2 full sized images is a far cry from 6 seconds of video (if say that is what Dojo is going to be training on).

My point being, I think we should be excited that Tesla has the data and possibly getting the computational resources to train a great FSD net. But I don't see any evidence that it would work on HW3.

I am curious about peoples thoughts on why they think differently, other than "Elon said so"

Basically, I'm confident that you need to solve vision to develop FSD and very confident that Tesla will have the lowest cost base......
So I don't see a competitor achieving FSD significantly earlier and 6-12 months earlier would be no big deal, so lots of upside at some stage.

On when it will be achieved, I'm less confident, my confidence is based on the fact that guys like Keller, Elon and Karpathy seem confident and the fact that development versions may be a few steps ahead...
Even when it works, I see regulatory approval as a long haul which will take at least 2 years.....

Hiring the Deepscale team and this focus on C++ mean they are trying to squeeze a solution into HW3 and working hard on optimising performance. I'm confident that they at least have something worth optimising..Perhaps Deepscale are also squeezing EAP in HW2...

I also expect HW4 is under development and Tesla will do whatever it takes to succeed...What is more important is outside of HW2/HW3/HW4 all of the sensors and other components are adequate.

If it is hard problem that takes longer to solve, I actually see that as favouring Tesla, except for customers who have purchased FSD not being happy...

So what Tesla needs is a version of FSD good enough to satisfy private buyers and good enough to apply for regulatory approval. My guess is around the end of this year might be possible, otherwise it takes as long as it takes, which could even be several more years..
 
Last edited:
That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)

Well I guess I'm easily pleased....

My assumption is productivity is higher in a high level language like Python when things are rapidly being iterated, once the design becomes more concrete, you migrate to C++...

That is, in fact, exactly the situation where it's chthonic.

I don’t think they are converting python code to C++. They are using PyTorch to experiment with training methods, then using a C++ trainer to train the NN at scale.

The trainer is the software 2.0 equivalent of the compiler. It turns source (labeled images and video) into an executable (NN). PyTorch is a great compiler for development on smaller data sets but C++ is used for nightly builds.

Thats my guess.

No way they are just now using C. The embedded code running on the Nvidia and HW3 is assuredly in C. I think this really means nothing.

Disagree.

Python for rapid prototyping of algorithms is the de facto standard nowadays, because you can modify your algorithms much more quickly than in C/C++. The key is of course to have experience in understanding how you develop your algorithms can be readily replicated with the same precision in C.

All the main deep learning packages are in Python (tensorflow, pytorch). Those are what Tesla uses.

It's not even hard to write python code that writes your desired C code, I do it all the time! Just have to make sure your cast your variables correctly for memory optimization

This is the quote from Elon:-


You are right that the code running in the car is probably C, but at some stage in the development cycle the NN, or related software, is in Python, or something like Python..

I still see hiring more C++ developers as a good thing... and a likely sign of future progress..


Converting Python NN weights and dot products to C isn't that complicated... Neural nets seem complicated but the math is really easy. That is not really the speed optimization to focus on.

The key is when you see them obsessing over pruning their massive neural network to achieve similar accuracy while reduces the number of weights / computations. And limiting the memory scope of some layers (drop from 32 bit to 16 or 8 bit). Deepsscale acquisition is for this.

PyTorch is a Python front end to a bunch of C++ libraries. You can also interface to them using the C++ front end:
Using the PyTorch C++ Frontend — PyTorch Tutorials 1.4.0 documentation
The PyTorch C++ frontend is a pure C++ interface to the PyTorch machine learning framework. While the primary interface to PyTorch naturally is Python, this Python API sits atop a substantial C++ codebase providing foundational data structures and functionality such as tensors and automatic differentiation. The C++ frontend exposes a pure C++11 API that extends this underlying C++ codebase with tools required for machine learning training and inference.
The C++ frontend is not intended to compete with the Python frontend. It is meant to complement it. We know researchers and engineers alike love PyTorch for its simplicity, flexibility and intuitive API. Our goal is to make sure you can take advantage of these core design principles in every possible environment, including the ones described above.
Above being low latency, highly multithreaded enviroments, or existing C++ codebases.

AKA libtorch:
PyTorch C++ API — PyTorch master documentation
Installing C++ Distributions of PyTorch — PyTorch master documentation
 
That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)
Regarding Perl, not Python, but another powerful scripting based language (before it went off the rails IMHO), there was a powerful supporter of Perl, Tom Christianson, who had "The Rule of E": he challenged anyone to come up with a problem that he couldn't solve in Perl with more than a factor of e (2.71828...) performance slowdown over any other language. He claimed that he never lost the challenge, and often the Perl code was actually faster. Probably because he was just a better programmer... but hey. The other possibility is that you might try approaches in Perl/Python that you'd never think of trying in C++, but which actually work better in the end.
 
Well I guess I'm easily pleased....

My assumption is productivity is higher in a high level language like Python when things are rapidly being iterated, once the design becomes more concrete, you migrate to C++...

There is also a dependency on what you're actually doing. One of the reasons you use Python for the fast iteration is you're doing machine learning / data science type work. There is a relatively large pool of people that know how to use Python for that kind of work. WHen you add a C++ dependency on the skill, you shrink that pool of people pretty significantly.

As best I can tell, it's almost a binary choice - either Python + machine learning skill, or C++ / software engineering skill. The number of people with both is small enough that you use them to do the translation for the software engineers that are reimplementing the work done on the Python side.


Then again, maybe Tesla is such an attractive place to work that they can get all of the outstanding engineers that have both skills they want, and they can do their model development in C++ from the beginning.

(I'm one of the Python + machine learning people, that would be starting with "Hello World" in C++)
 
There is also a dependency on what you're actually doing. One of the reasons you use Python for the fast iteration is you're doing machine learning / data science type work. There is a relatively large pool of people that know how to use Python for that kind of work. WHen you add a C++ dependency on the skill, you shrink that pool of people pretty significantly.

As best I can tell, it's almost a binary choice - either Python + machine learning skill, or C++ / software engineering skill. The number of people with both is small enough that you use them to do the translation for the software engineers that are reimplementing the work done on the Python side.


Then again, maybe Tesla is such an attractive place to work that they can get all of the outstanding engineers that have both skills they want, and they can do their model development in C++ from the beginning.

(I'm one of the Python + machine learning people, that would be starting with "Hello World" in C++)

Yes it is interesting, I'm on the C++ side myself and have had experience with systems that needed that needed to be fast and reliable.....
I don't have Machine learning experience .... I expect quality C++ programmers can be phased into the project, and learn to do the translation and optimisation, in the meantime the Python side can proceed as normal.....

It is also fairly likely that they can find bottlenecks that are a priority for a raw C++ implementation.....

I agree on the attractiveness of Tesla as a work place, this project and the mission ... people love a challenge and doing valuable and interesting work that achieves something. This is a rare project that that achieves something of lasting worth, and looks great on the CV.
I also know from personal experience working with a team of talented, motivated very smart people doing interesting work is a rare pleasure, it is very memorable and probably the exception rather than the rule... So I am sure Elon has plenty of people wanting to apply ... and the best of that talent will make a difference.
 
Disagree.

Python for rapid prototyping of algorithms is the de facto standard nowadays, because you can modify your algorithms much more quickly than in C/C++. The key is of course to have experience in understanding how you develop your algorithms can be readily replicated with the same precision in C.

All the main deep learning packages are in Python (tensorflow, pytorch). Those are what Tesla uses.

It's not even hard to write python code that writes your desired C code, I do it all the time! Just have to make sure your cast your variables correctly for memory optimization

I swear, if hardcore Python advocates were in the construction industry, they'd insist that the fastest way to build a skyscraper would be first to engineer it for and build it out of wood, because building it out of wood is faster than steel, and then tear it down and redo the engineering and construction out of steel. ;)

You don't write the same program twice if you can avoid it. You just don't. It's nothing more than extra work. I've been down this path way too many times.

And don't get me wrong - I don't dislike python, I use it all the time. It's one of my three go-to languages (bash for very simple problems, python for problems of moderate complexity / performance / memory needs, C++ for problems of great complexity with significant performance / memory needs). But discovering that you chose the wrong language and have to switch languages is the absolute worst. And it's all too easy to gloss over a language's shortcomings until you yourself hit them.

BTW, to anyone who wants to pretend that Python isn't a hog....

>>> a=1
>>> sys.getsizeof(a)
28
>>> a=[]
>>> sys.getsizeof(a)
72
>>> a=[1]
>>> sys.getsizeof(a)
80
>>> a={}
>>> sys.getsizeof(a)
248

Numpy helps, but it's still insufficient for many tasks.

I could do a performance benchmark for you if I wanted, too. C++ is generally about 10x faster, give or take half an order of magnitude depending on the task. There's a lot of things for which even C (let alone Python) can't do that C++ can due to greater ability to inline code in algorithms (for example, try benchmarking qsort vs. std::sort).

One of the worst aspect of Python in particular is you have no granular control over your data structures. Python lists are equivalent to std::vector. You have no ability to choose between, say, std::vector, std::array, std::deque, std::list, and std::forward_list (Numpy's are like std::array, but for a very limited functionality subset). These not only have different memory footprints, but outright different O(N)s for different operations! And here we're not even going into the fact that you can template C++ data structures over different allocators or extend them to new functionality. The same applies to dictionaries vs. std::map or std::hashmap or std::multimap - very different memory and performance footprints, and if python doesn't have a builtin means for doing something (for example, "find the nearest entry to a particular key"... eg. std::lower_bound, std::upper_bound), you're utterly screwed from a performance standpoint.

When it comes to performance and memory, Python is like, "Hey, want to drive in a nail? Here's a hammer! Hey, want to drive in a screw? Here's a hammer! Hey, want to tighten a bolt? Here's a hammer!"

The whole thing about Python being "rapid prototyping" vs. C++ is IMHO an oversold sales pitch. And remember, I'm a person who likes Python here! Python is great until it utterly screws you over. And even when that doesn't happen, Python has a lot of things which tend to make it more likely to bite you the more complicated the program gets. The lack of variable typing for example make things simpler early on but can come back to bite you later on - the longer and more complicated the program, the more likely that's going to happen. A complex program should not only be strictly typed, but all variables outright sanity checked at the start of each function.

Mind you, I'd definitely recommend Python as a "language for beginners" over C++, and as a language for people whose primary job isn't programming.

(I also often find that people bashing C++ in favour of Python are unfamiliar with the features of modern C++ standards)

Are you safely back? Was there something to see besides a decaying parking lot, a scurrying rat and a cop who wants to know what you're doing there at 3:00 a.m.?

Sign on the door at the Honda dealership stating that they've moved, that's all. Stopped by the car lot at the port just because ;) It'll be interesting for sure to monitor that lot starting in a couple weeks! :)
 
Last edited:
The whole thing about Python being "rapid prototyping" vs. C++ is IMHO an oversold sales pitch. And remember, I'm a person who likes Python here! Python is great until it utterly screws you over. And even when that doesn't happen, Python has a lot of things which tend to make it more likely to bite you the more complicated the program gets. The lack of variable typing for example make things simpler early on but can come back to bite you later on - the longer and more complicated the program, the more likely that's going to happen. A complex program should not only be strictly typed, but all variables outright sanity checked at the start of each function.

Fully agreed.

BTW., the main reason that PyTorch is so good for fast prototyping in the machine learning context isn't because Python is the best to program neural networks in, but because under PyTorch there are C++ and even GPU libraries that can offload the actual training to something that is orders of magnitude faster and more scalable than Python.

When constructing machine learning architectures in PyTorch you are not really programming in the classic sense, you are using template data structures and template libraries to construct layouts and feedback loops with pretty much predetermined low level data structures but variable topologies - which something other than a Python runtime will number-crunch over ...

I.e. Python is not the primary language of PyTorch or machine learning - it's a flexible, programmable, extremely useful front-end.
 
Last edited:
That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)

You might want to check out Swift at some point. It’s pretty mature now.

In my more recent life developing and publishing iOS apps including a couple of full featured and highly rated ones, I can attest that Swift is far superior to Objective-C. You can be vastly more productive in Swift and the resulting code, even without tests or using Apple’s analysis tools (i.e. Instruments), is much more robust.

Except for one thing, I would use Swift for everything (that one thing isn’t scripting, Swift allows that). If I were to ever return to a former life writing code simulating neural models or, really, any code that implemented some hard core, secret sauce type dense computational code, I might still use C or a derivative thereof. Granted that is a big exception and the one that likely would matter to you, but that exception won’t last much longer.

One stated goal of the current Swift team is to “kill off” C++ at Apple. This was in one of the recorded WWDC Swift language session videos this year or last (don’t recall which session).

I suspect that Chris Lattner only lasted six months at Tesla because he probably wanted to move Tesla to Swift and there was resistance at Tesla. Lattner was the main person behind the creation of Swift at Apple.

I can understand why Tesla might prefer not to have Swift for their Software 2.0 neural network code and use C/C++ for the time being. However, that likely leaves a lot of code that might be more productively written in Swift.

As an aside, Apple has also made considerable progress on concurrency for practical use.
 
Last edited:
  • Informative
Reactions: JusRelax
I swear, if hardcore Python advocates were in the construction industry, they'd insist that the fastest way to build a skyscraper would be first to engineer it for and build it out of wood, because building it out of wood is faster than steel, and then tear it down and redo the engineering and construction out of steel. ;)

You don't write the same program twice if you can avoid it. You just don't. It's nothing more than extra work. I've been down this path way too many times.

And don't get me wrong - I don't dislike python, I use it all the time. It's one of my three go-to languages (bash for very simple problems, python for problems of moderate complexity / performance / memory needs, C++ for problems of great complexity with significant performance / memory needs). But discovering that you chose the wrong language and have to switch languages is the absolute worst. And it's all too easy to gloss over a language's shortcomings until you yourself hit them.

BTW, to anyone who wants to pretend that Python isn't a hog....

>>> a=1
>>> sys.getsizeof(a)
28
>>> a=[]
>>> sys.getsizeof(a)
72
>>> a=[1]
>>> sys.getsizeof(a)
80
>>> a={}
>>> sys.getsizeof(a)
248

Numpy helps, but it's still insufficient for many tasks.

I could do a performance benchmark for you if I wanted, too. C++ is generally about 10x faster, give or take half an order of magnitude depending on the task. There's a lot of things for which even C (let alone Python) can't do that C++ can due to greater ability to inline code in algorithms (for example, try benchmarking qsort vs. std::sort).

One of the worst aspect of Python in particular is you have no granular control over your data structures. Python lists are equivalent to std::vector. You have no ability to choose between, say, std::vector, std::array, std::deque, std::list, and std::forward_list (Numpy's are like std::array, but for a very limited functionality subset). These not only have different memory footprints, but outright different O(N)s for different operations! And here we're not even going into the fact that you can template C++ data structures over different allocators or extend them to new functionality. The same applies to dictionaries vs. std::map or std::hashmap or std::multimap - very different memory and performance footprints, and if python doesn't have a builtin means for doing something (for example, "find the nearest entry to a particular key"... eg. std::lower_bound, std::upper_bound), you're utterly screwed from a performance standpoint.

When it comes to performance and memory, Python is like, "Hey, want to drive in a nail? Here's a hammer! Hey, want to drive in a screw? Here's a hammer! Hey, want to tighten a bolt? Here's a hammer!"

The whole thing about Python being "rapid prototyping" vs. C++ is IMHO an oversold sales pitch. And remember, I'm a person who likes Python here! Python is great until it utterly screws you over. And even when that doesn't happen, Python has a lot of things which tend to make it more likely to bite you the more complicated the program gets. The lack of variable typing for example make things simpler early on but can come back to bite you later on - the longer and more complicated the program, the more likely that's going to happen. A complex program should not only be strictly typed, but all variables outright sanity checked at the start of each function.

Mind you, I'd definitely recommend Python as a "language for beginners" over C++, and as a language for people whose primary job isn't programming.

(I also often find that people bashing C++ in favour of Python are unfamiliar with the features of modern C++ standards)



Sign on the door at the Honda dealership stating that they've moved, that's all. Stopped by the car lot at the port just because ;) It'll be interesting for sure to monitor that lot starting in a couple weeks! :)

Fully agreed.

BTW., the main reason that PyTorch is so good for fast prototyping in the machine learning context isn't because Python is the best to program neural networks in, but because under PyTorch there are C++ and even GPU libraries that can offload the actual training to something that is orders of magnitude faster and more scalable than Python.

When constructing machine learning architectures in PyTorch you are not really programming in the classic sense, you are using template data structures and template libraries to construct layouts and feedback loops with pretty much predetermined low level data structures but variable topologies - which something other than a Python runtime will number-crunch over ...

I.e. Python is not the primary language of PyTorch or machine learning - it's a flexible, programmable, extremely useful front-end.

Sorry for general signal processing and data science problems you start in a high level language to explore and determine what methods are going to work best for the data you have.

Have to load varied forms of data? Massive pain in C, easy with accessible Python libraries.

Want to interact and plot data in C++? LOL what a pain.

Want to quickly test out different filtering techniques or other DSP stuff to see what works best with your data? Easy in Python and open source packages. Wayy slower in C.

Yes if you know what model architecture you are going to use, just code it in C++.

But most data driven algorithm problems in data science require so much data cleaning, partitioning, and analysis, and these are way faster in Python. This is a key point. The running of final algorithm chosen is a small piece of the puzzle.

The proper analogy is you are going to have to construct a building 10 times, but it's 10 times faster in Python. So you build it 9 times in Python, then once in C, and end up ahead.

That's why Python is do popular, and growing in the data science community.
 
Sorry for general signal processing and data science problems you start in a high level language to explore and determine what methods are going to work best for the data you have.

Have to load varied forms of data? Massive pain in C, easy with accessible Python libraries.

Want to interact and plot data in C++? LOL what a pain.

Want to quickly test out different filtering techniques or other DSP stuff to see what works best with your data? Easy in Python and open source packages. Wayy slower in C.

Yes if you know what model architecture you are going to use, just code it in C++.

But most data driven algorithm problems in data science require so much data cleaning, partitioning, and analysis, and these are way faster in Python. This is a key point. The running of final algorithm chosen is a small piece of the puzzle.

The proper analogy is you are going to have to construct a building 10 times, but it's 10 times faster in Python. So you build it 9 times in Python, then once in C, and end up ahead.

That's why Python is do popular, and growing in the data science community.

That's not really "coding" though that @KarenRei was referring to, but data analysis.

To be fair, the engineering of machine learning systems actually is data analysis to a large degree - much of the real "coding" is done by the training and the network in the end, we often don't even understand what algorithm it has learned.
 
That makes you optimistic? Ugh, no. Never start a project in Python that it's obvious you're going to have to end up doing in C++. I've a number of times had the misfortune of starting projects in Python and then later discovering that I had to switch to C++, and it was a huge waste of time every single time. :(

Python = Best for simple projects where memory and/or performance are not critical
C++ = Best for complex projects where memory and/or performance are critical
Cython = Where you started a project in python and then discovered that memory and/or performance are critical and that you really screwed up and don't have the time to rewrite the whole thing ;)

I would never choose c++ over c