Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Reinventing service...

brianman

Burrito Founder
Nov 10, 2011
17,533
2,997
While you're general point is on the mark, IMO, there's one part I disagree with:
Tesla tells everyone that currently has a Model S that a software upgrade will enable a 33% increase in supercharging speed. That enhancement later turns out to only apply to certain folks, and Tesla very well knew all along that they wouldn't be releasing the upgrade to these folks and intentionally decided not to mention this fact. A large debate ensues because several owners feel they were misled.
We don't know the underlined. It's quite likely the discovered it after the announcement. I'm not saying this excuses them. As soon as the line(s) of code were checked into the firmware source that branched the behavior for some batteries vs. others, they began walking down a road of "not living up to what they promised". I'm not saying they should have announced it immediately, but they didn't announce it at all -- which is what I have a problem with.

When Tesla promises something and "TMC discovers" the promise was broken rather than Tesla owning up to it ahead of time, it erodes trust in Tesla among their strongest supporters. An incredibly poor choice for a company led by some very intelligent market disrupters. I expected better from Elon, JB, and Jerome, for example.
 

tomas

Out of warranty...
Oct 22, 2012
4,267
3,856
Chicago/Montecito
Last chapter on A vs B batteries and supercharging yet to be written, but I am hopeful there will be an accommodation based on info I can share with affected owners via pm.
 

ckessel

Active Member
Jan 15, 2011
4,452
284
We don't know the underlined.
I've written software for 20 years as a living, from FDA regulated medical equipment with long release cycles to rapid fire near daily updates in startups. I've made a study of software processes, going back to school while working and getting a Masters degrees specifically focused on software engineering processes, both heavy and lightweight.

I give that preface to say that I'm comfortable claiming with some confidence there is absolutely no way they didn't know the A/B issues well before the May announcement. Software development just does not work that way.
 

Atebit

Member
Jan 7, 2014
286
26
PA
I give that preface to say that I'm comfortable claiming with some confidence there is absolutely no way they didn't know the A/B issues well before the May announcement. Software development just does not work that way.

Well, I'm not sure that statement applies anymore in the Fire! Aim! Ready! world that is Agile Development. The entire premise is based on shipping something that's not done & let letting the users tell you what to fix, so you don't "waste time" fixing "the wrong things". As if.

Similar to the reasons code is so exploitable today... why bother checking/limiting how many bytes I'm writing into this 16 byte buffer...I know I'll only ever write 16 byt.àęÿœ[email protected]@t!p0wned
 

ckessel

Active Member
Jan 15, 2011
4,452
284
Well, I'm not sure that statement applies anymore in the Fire! Aim! Ready! world that is Agile Development. The entire premise is based on shipping something that's not done & let letting the users tell you what to fix, so you don't "waste time" fixing "the wrong things". As if.
Actually, that's not quite the purpose of agile development. The purpose of agile development is prioritized development in an environment with rapidly changing or unknown priorities. That may be churn from customers or from other sources (such as a rapidly changing new market). The theory is that of X items, some subset of those are core to your business and you can focus in on those and narrow the field of later features as time goes by. There are pros and cons, but generally it's good for greenfield development where the trail has not been well blazed with products to copy.

Which is not to be confused with iterative development, which is about incremental development. Agile is a form of iterative development and is an umbrella term that covers a number of methodologies, while iterative development is more of a core best practice. Agile is an iterative variant specifically for environments with high requirements churn or uncertainty. It's also for an environment where being wrong isn't game over as you can pull it and try again.

Extreme programming (and some other ultra-agile), which some often confuse with the more generic agile movement, is an agile methodology, but very specifically tailored originally to development where the customer is co-located. It's this particular situation where the early customer feedback is a central part of the methodology because, in theory, the customer is co-located with the development group nearly 100% of the time. I'm not a fan and almost no one does pure XP because it's almost academic in it's purity of requirements. Kent beck states you have to do XP in all it's specifics or it doesn't work, but it's all but impossible to fulfill each criteria that Beck lays out.

While most agile methodologies encourage customer feedback, it's by releasing minimal initial sets of core business functionality. That is _not_ software that's "not done", it's software that's feature poor. However, you need tolerant customers that believe in your potential to give feedback on initial release that aren't, frankly, competitive at that point. Telsa's very initial software release would qualify. For example, they didn't even have per-driver settings in their first release. They also got to use Roadster owners for very early, almost proof-of-concept, software.

We won't even discuss Waterfall, which is all but discredited at this point. Though, interestingly, Winston Royce's vision of it wasn't as anti-iterative as it's adoption became (see my article at Software History: Waterfall – The Process That Wasn’t Meant To Be | Online Business Systems).

In an environment like Tesla, I'd guess they are definite strong iterative, focusing on core business values first (hence, why the damn music system still doesn't have a shuffle). They probably have adopted various Agile practices which fit well with most organizations, but they do not have frequent updates (daily or weekly), so they are likely on a large-chunk delivery schedule that coordinates across multiple groups. Such coordinated releases require a non-trivial lead time, which means things like A/B pack software changes are incredibly unlikely to have snuck in and surprised anyone.
 
Last edited:

deonb

Active Member
Mar 4, 2013
4,057
4,209
Redmond, WA
I've written software for 20 years as a living, from FDA regulated medical equipment with long release cycles to rapid fire near daily updates in startups. I've made a study of software processes, going back to school while working and getting a Masters degrees specifically focused on software engineering processes, both heavy and lightweight.

I give that preface to say that I'm comfortable claiming with some confidence there is absolutely no way they didn't know the A/B issues well before the May announcement. Software development just does not work that way.

I also have over 20 years in software development and have never worked on nor seen a product where there weren't new issues discovered in the field after release.

The development team may have only tested on 'B' packs all the time, then someone got the idea to test on 'A' packs at the last minute and saw that it did active damage to the pack. Or worse, it was actually released, and an A pack got damaged by a 120kW SuperCharger and the event was hushed.

Remember that the 120 kW charging only became active about 2 months after the initial promise, so this may have been due to issues like this that was discovered in the field.

Tesla R&D is very much a bleeding edge team - considering that there was no Battery swapper prototype at the time of Elon's announcement, and the actual swapper didn't work until it was within hours of the demo. So I would not bat an eye if you tell me that they didn't test on A packs until the very last minute, or later.
 

apacheguy

S Sig #255
Oct 21, 2012
5,082
1,243
So Cal
Ok, let's assume that Tesla was innocent and didn't know that A packs wouldn't be able to charge at 120 kW. Why then was there a random batch of "last minute" A batteries that got pushed out the door just prior to the announcement in May? After May, ALL batteries were B or better. Seems like too much of a coincidence to me.

But even in this scenario, I don't understand why Tesla thought they had their bases covered. The press release and announcement were very clearly directed towards all customers, not just new ones.
 

ckessel

Active Member
Jan 15, 2011
4,452
284
So I would not bat an eye if you tell me that they didn't test on A packs until the very last minute, or later.
If you truly believe that, you should be terrified of any software update. If you truly think Tesla isn't testing on every variant in the field, then you're assuming they might very well literally blow up your battery when you head to the Supercharger due to a lack of testing.

I'm sure Tesla takes such validation incredibly seriously, both as a culture since Elon has touted safety repeatedly and for legal reasons (recalls and bad PR and such). It's precisely because of that, that I find it all but impossible to believe that A/B pack charging differences came as a surprise to Tesla. Hell, they wouldn't even have rolled an A/B label if they didn't know there was a significant difference.
 

deonb

Active Member
Mar 4, 2013
4,057
4,209
Redmond, WA
Well, I'm not sure that statement applies anymore in the Fire! Aim! Ready! world that is Agile Development. The entire premise is based on shipping something that's not done & let letting the users tell you what to fix, so you don't "waste time" fixing "the wrong things". As if.

Similar to the reasons code is so exploitable today... why bother checking/limiting how many bytes I'm writing into this 16 byte buffer...I know I'll only ever write 16 byt.àęÿœ[email protected]@t!p0wned

Most exploitable code was written in the days before agile development, in the 70's to 90's where security meant user accounts and access control, and preventing end-users from performing administrative functions on company hardware.

No developers back that foresaw that people will eventually run their code in an environment where every single point of input will try and actively sabotage you.

Code quality today as a whole is MUCH higher, even with agile methodology. There's just more of it, and more to gain by attacking it than there used to be.

- - - Updated - - -

If you truly believe that, you should be terrified of any software update. If you truly think Tesla isn't testing on every variant in the field, then you're assuming they might very well literally blow up your battery when you head to the Supercharger due to a lack of testing.

I'm sure Tesla takes such validation incredibly seriously, both as a culture since Elon has touted safety repeated and for legal reasons (recalls and bad PR and such). It's precisely because of that, that I find it all but impossible to believe that A/B pack charging differences came as a surprise to Tesla. Hell, they wouldn't even have rolled an A/B label if they didn't know there was a significant difference.

I am terrified of any software update from them - that's why I let you'all try it out for 2 months before I upgrade each time :).

Tesla Ownership wasn't at all aware that with the height adjustment change that the ride height would still lower at 100mph. This seems to have just somehow gotten in there. We all know how bad communication internally in Tesla is - how would the testers in the field know what to test for if the developers don't tell them? And if there is such bad communication between the developers and the ownership team, I don't see how there would be any better communication between the developers and the testers.

We all just hope there is, but I bet you can't point to any part of Tesla and say: "Wow - look at how effective the communication is between those 2 departments".
 

Atebit

Member
Jan 7, 2014
286
26
PA
Actually, that's not quite the purpose of agile development. The purpose of agile development is prioritized development in an environment with rapidly changing or unknown priorities. That may be churn from customers or from other sources (such as a rapidly changing new market). The theory is that of X items, some subset of those are core to your business and you can focus in on those and narrow the field of later features as time goes by. There are pros and cons, but generally it's good for greenfield development where the trail has not been well blazed with products to copy.

Perhaps that's what he manifesto says, but I've yet to see it practiced that way. Having a release every two weeks makes the executives happy, though. And I would disagree that the product is merely "feature poor"...if it doesn't do what the customer needs it to do, it's broken, and not ready for prime time.

Developers can talk about sprints all they want, but when you're the guy on the ground helping the customer try to use their "feature poor" software, it's a different kind of sprint, often assisted by the customer's Size 10 Regular.
 

brianman

Burrito Founder
Nov 10, 2011
17,533
2,997
I give that preface to say that I'm comfortable claiming with some confidence there is absolutely no way they didn't know the A/B issues well before the May announcement. Software development just does not work that way.
Going to have to disagree with you here. Medical software is very different from the rate of software development Tesla seems to be following.

I would be frightened if Medical software had the charging bugs that they recently introduced in Model S firmware, for example.

- - - Updated - - -

Tesla Ownership wasn't at all aware that with the height adjustment change that the ride height would still lower at 100mph.
I was going to mention that example as well but held off.
 

ckessel

Active Member
Jan 15, 2011
4,452
284
Perhaps that's what he manifesto says, but I've yet to see it practiced that way. Having a release every two weeks makes the executives happy, though. And I would disagree that the product is merely "feature poor"...if it doesn't do what the customer needs it to do, it's broken, and not ready for prime time.

Developers can talk about sprints all they want, but when you're the guy on the ground helping the customer try to use their "feature poor" software, it's a different kind of sprint, often assisted by the customer's Size 10 Regular.
Releasing every two weeks (or 4) isn't specifc to Agile, but rather to particular implementations. Namely, Scrum. Other variations, like Kanban, are geared towards continuous deliveries.

Neither of which make sense for a delivery schedule like Tesla's. If an organization is using 2 week deliveries to release broken software to external customers that aren't interested in that type of beta interaction, then the organization is definitely flawed.

There are many organizations that develop software very, very badly regardless of what name they choose to mislabel their process.

- - - Updated - - -

Going to have to disagree with you here. Medical software is very different from the rate of software development Tesla seems to be following.
True, though not really your original point, which was that maybe Tesla didn't know about the A/B issues.

Tesla knew they had A/B issues. The packs were rolled and the software handled it. This wasn't an "oh, we're so agile, we forgot". If we're being very generous, we could say that perhaps management that made the announcements weren't in the loop.

Tesla releases much more like a off the shelf software company. Every few months with minor updates, maybe a couple times a year with large updates.
 

SUPRKAR

Member
Feb 28, 2013
220
27
New Jersey
quote_icon.png
Originally Posted by ckessel viewpost-right.png I give that preface to say that I'm comfortable claiming with some confidence there is absolutely no way they didn't know the A/B issues well before the May announcement. Software development just does not work that way.

brianman
Going to have to disagree with you here. Medical software is very different from the rate of software development Tesla seems to be following.

I would be frightened if Medical software had the charging bugs that they recently introduced in Model S firmware, for example.

Well if they did not know and I do NOT believe that, then why not make an announcement or contact A battery owners and explain, not hide in the closet with silly 4 minute difference answers which we are all getting. Also Tesla could do something about it and I don't care what the cost is. I read they want to be fair to the Chinese with the pricing and just made a change in the vehicle price to accommodate some exchange rate for another market. What about the USA market and the early buyers that put their faith in Tesla when they did not even have a car to sell. There are a number of items Tesla has backed out of and I don't like it. Not because I can not live without those things but because I put my trust and faith in Tesla and I feel they did not act fairly by not being open. They hold back the negative and I understand why but it still is not right. If you make an announcement, Lighted Visors or 120k supercharging, pano roof shade, 4 usb ports etc. then keep your word or explain why. Not saying anything is like telling the early buyers or go @#$%#.
 

Ugliest1

S85: "Sparky"
Aug 19, 2013
1,401
1,645
Victoria BC Canada
/subscribed. I prepaid for four years too. I'll be interested to see how all this plays out. I consider Tesla to be a pretty ethical company with the good of the consumer in mind. I feel certain they'll do whatever it takes to make this fair for everyone.

I also paid for 4 years' service, ranger service, and am waiting the next four to see re the extended warranty. I like not having to worry about anything for 4 (maybe 8) years, even when travelling. IMO it's too early to feel betrayed or scammed for the 4 year service $ because it's only speculation at this point that we paid $600 for *only* a pair of wiper blades. Maybe cut them a break -- they're inventing everything from scratch, including a nearly-no-maintenance car and a turn-the-industry-on-its-head business model -- they HAD to cover their bases in the early going with some kind of maintenance fee, and it looks like they are now discovering they didn't need it so much. But they have 100 balls in the air and how can they do everything all at once. Maybe they'll extend the prepaid service period for another four years, or something (so cost ends up $300/yr), and/or other inventive solutions. Per bareyb: "I feel certain they'll do whatever it takes to make this fair for everyone".

... For my part, I actually prefer that they are aspirational, aim high, and occasionally fall short. That's how we got this impossible ride to begin with....

+1. They are aiming where no one has aimed before, and it's a jungle out there, and they are bound to get tangled in a few vines occasionally. Some vines take a while to sort out. Their strategic decisions have always seemed to me to be on the mark and *with integrity*, even things like the over-sale of inventory cars which turned out to be a mis-step that needed to be dealt with (and the policy change in January presumably was a reasonable solution). The entire business they are executing is so complicated, I'm willing to give them time.
 

Gizmotoy

Active Member
Sep 16, 2013
3,661
867
Bay Area, CA
Going to have to disagree with you here. Medical software is very different from the rate of software development Tesla seems to be following.

I would be frightened if Medical software had the charging bugs that they recently introduced in Model S firmware, for example.

At the companies I've worked for, medical device and aircraft software was all developed using formal verification techniques. Serious stuff. Prove there's a problem. Prove you've found the problem. Prove your solution fixes the problem and is the easiest fix possible. Prove the fix didn't break anything else. And so on. Even the simplest change takes FOREVER.

There's no way Tesla can be that vigorous and still provide us with all the additional stuff we want. The price of rapid development is that sometimes bugs and regressions creep in.
 

ElSupreme

Model S 03182
Jan 13, 2012
4,303
92
Atlanta, GA
Just as a data point. My service (25k) had ~$120 worth of parts.

It also included installing those parts.
Rotating my tires.
Allignment.
Wash, vacuum.
Inspection (brakes, pads, tires, other).

And plenty of 'warranty work' and service bulletin work.

All in all $600 seems about right for the work done.
 

AmpedRealtor

Well-Known Member
Jun 30, 2013
6,383
3,355
Phoenix, AZ
Just as a data point. My service (25k) had ~$120 worth of parts.

It also included installing those parts.
Rotating my tires.
Allignment.
Wash, vacuum.
Inspection (brakes, pads, tires, other).

And plenty of 'warranty work' and service bulletin work.

All in all $600 seems about right for the work done.

Warranty and service bulletin work should not be factored into the $600 because all owners are eligible to receive those free of charge.
 

RedMatador

Member
Apr 15, 2013
58
4
SF Bay Area
Just as a data point. My service (25k) had ~$120 worth of parts.

It also included installing those parts.
Rotating my tires.
Allignment.
Wash, vacuum.
Inspection (brakes, pads, tires, other).

And plenty of 'warranty work' and service bulletin work.

All in all $600 seems about right for the work done.

Can you itemize the $120 parts? You may be able to help Elon justifying the $600 service costs to europeans :)
 

About Us

Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.

Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


SUPPORT TMC
Top