6.0 - Public Beta

[Zythryn]

Bob, you don't know how many beta testers they have.
As someone else stated, they very well may have added more recently. If there is a NDA, we would never know.

As for bugs, you could add a million beta testers and are not guaranteed a bug free release.

I would agree with you that improvement is always good. I also feel Tesla has more information than you about how their beta program works and how it can be improved.

 

[JohnQ]

As Tesla continues to sell more cars - moving beyond the "early adopters", the start-stop process of distributing releases will break down and there will be a growing community of owners dissatisfied with getting releases with obvious bugs.

Since obvious bugs are still getting into the software releases - then their testing process needs improvement - and software releases need more testing - and with a growing number of configurations (including multiple languages), the additional testing can be done either through increasing the time for testing each new release OR increasing the number of people doing the testing.

Since we don't want them to slow down software releases (especially since there are multiple areas where the Tesla software is considerably behind - such as navigation and the audio system) - then by taking advantage of more of their owners willingness to help out during the beta testing process - they should be able to find more bugs during the testing process and improve the quality of the releases being distributed - without slowing down the release process.

Yes, they can improve the quality of their releases. But, more beta testers does not equal fewer bugs. Those who manage testing and QA can speak more intelligently than I about the process and constraints. Indeed, software is released with bugs all the time (even known bugs). Incuding those with large public betas. That's why patches and service packs are so common.

 

[bob_p]

More beta testers does not necessarily result in fewer bugs.

However, increasing the number of experienced beta testers, combined with more communication from the development group to the testers to help focus testing on the areas of greatest change (and risk of new problems) - should be able to produce better results than Tesla's track record (and I do have considerable experience in managing groups that both develop and test a wide range of software systems - which have avoided the start-stop-resume release process Tesla is using).

There are other options that Tesla should also consider for improving software quality - but in the short term - higher quality testing should help.

 

[Zythryn]

More beta testers does not necessarily result in fewer bugs.

However, increasing the number of experienced beta testers, combined with more communication from the development group to the testers to help focus testing on the areas of greatest change (and risk of new problems) - should be able to produce better results than Tesla's track record (and I do have considerable experience in managing groups that both develop and test a wide range of software systems - which have avoided the start-stop-resume release process Tesla is using).

There are other options that Tesla should also consider for improving software quality - but in the short term - higher quality testing should help.

You don't know the current level of quality nor number or experience of the current testers.

I agree with you that improvements are always good. However, I don't see how you can suggest specific areas for improvement without any idea of the current beta process.

 

[jerry33]

Yes, they can improve the quality of their releases. But, more beta testers does not equal fewer bugs. Those who manage testing and QA can speak more intelligently than I about the process and constraints. Indeed, software is released with bugs all the time (even known bugs). Incuding those with large public betas. That's why patches and service packs are so common.

Many companies believe that shipping something quickly is more important than shipping something that works.

 

[JohnQ]

Many companies believe that shipping something quickly is more important than shipping something that works.

I've found that it's a balancing act and there are many reasons why a a particular piece of software makes it to market with bugs.

 

[bob_p]

You don't know the current level of quality nor number or experience of the current testers.

I agree with you that improvements are always good. However, I don't see how you can suggest specific areas for improvement without any idea of the current beta process.

Until Tesla is able to issue a release without having to resort to pausing the release process and resuming after adding an emergency fix - their software development and release process is flawed.

There are likely multiple areas where Tesla should be considering improvements in their software process - and improvements earlier in the process could avoid the start-stop-resume releases we are seeing. However, because their testing is missing major problems - requiring them to make emergency fixes after they start releasing new software - then the easiest and fasted change they could implement would be to focus on their beta testing - and improve that.

Longer term, it's likely their entire development process could benefit from improvements.

While the topic here has been the stutter stepping of releases - a larger problem is the slow pace of introducing new functionality - and while additional resources could help accelerate that - it's not unusual that a better software process could also help accelerate the release of new functionality.

They should be able to release better software - and do it faster...

 

[tomas]

Until Tesla is able to issue a release without having to resort to pausing the release process and resuming after adding an emergency fix - their software development and release process is flawed.

There are likely multiple areas where Tesla should be considering improvements in their software process - and improvements earlier in the process could avoid the start-stop-resume releases we are seeing. However, because their testing is missing major problems - requiring them to make emergency fixes after they start releasing new software - then the easiest and fasted change they could implement would be to focus on their beta testing - and improve that.

Longer term, it's likely their entire development process could benefit from improvements.

While the topic here has been the stutter stepping of releases - a larger problem is the slow pace of introducing new functionality - and while additional resources could help accelerate that - it's not unusual that a better software process could also help accelerate the release of new functionality.

They should be able to release better software - and do it faster...

I believe it is wrong to speak with such absolute confidence about flaws in process without knowledge of Tesla's development, testing, and release process. My perception is that Tesla is providing functionality at a reasonable pace. When I think about how much has been added over this last year, it seems to me more than I've gotten on any other single OS or software product I own. There are a lot of things I'd still like to see improved, and I think everyone has their pet peeve list. After watching for > 1 year, I think the release, adjust, release approach IS intentional, and I think - as long as safety-related bugs are not getting into general releases and responsiveness is pretty quick - it is an acceptable approach. I know that a high % of forum members are/were in software and consider themselves authorities. I'm part of that group, so not ignorant of software testing and release approaches...

 

[FlasherZ]

I'm in the industry as well. fast + good + cheap == a constant. I'm pretty sure that no matter which configuration of the three are used, you will find people that complain.

Increase quality by placing more emphasis and money on testing, and people will complain about feature velocity (oh wait, they already are complaining).
Increase feature velocity by placing more emphasis and money on development, and people will complain about quality (oh wait, they already are complaining).
Increase both quality and feature velocity at a higher product cost, and people will complain about the cost (oh wait, they already are complaining).

I rather like the mix they're choosing. We get new features at a very rapid pace - when I think about the difference between v3.2 when I received the car to v5.9 in 17 months, I'm amazed at what has changed and those around me are as well... There are minor bugs but they tend to be annoyances rather than car-disabling bugs that leave the cars stranded... The cost is at the right target in that over 150,000 to 200,000 miles I can get its price down to the equivalent of a $30-50k ICE vehicle (gas & maintenance savings)...

All things considered, it's my opinion that Tesla has a good mix.

 

[Kraken]

They seem to work magic... They just need to find a way to magically increase the value of the constant. :biggrin:

 

[brianman]

Many companies believe that shipping something quickly is more important than shipping something that works.

Many consumers drive that belief as well.

 

[tezco]

I'm in the industry as well. fast + good + cheap == a constant. I'm pretty sure that no matter which configuration of the three are used, you will find people that complain.

All things considered, it's my opinion that Tesla has a good mix.

I think I've received a total of two software updates over the two and a half years of Leaf ownership. Guess that would = cheap!

 

[SCW-Greg]

Okay, bring on 6.0, I'm ready!

Looking for (in order of easy pickings):
Random play music
Music play lists
Using proper album art (inboard)
Improved GUI (time to drop Skeuomorph - Wikipedia, the free encyclopedia)
Best in class adoption of Apple - CarPlay
Terrain, weather, and traffic aware navigation
SDK for 3rd apps

 

[tomas]

Okay, bring on 6.0, I'm ready!

Looking for (in order of easy pickings):
Random play music
Music play lists
Using proper album art (inboard)
Improved GUI (time to drop Skeuomorph - Wikipedia, the free encyclopedia)
Best in class adoption of Apple - CarPlay
Terrain, weather, and traffic aware navigation
SDK for 3rd apps

While we're fixing audio, how about fixing the bug that doesn't properly recognize FLAC tags with punctuation or accents in them? 20% of my songs "don't exist" from an album or artist view.