TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker or making a Paypal contribution here: paypal.me/SupportTMC

FW Beta testing and bug reporting

Discussion in 'Model S: User Interface' started by mnx, Aug 15, 2013.

  1. mnx

    mnx 2013 P85

    Joined:
    May 6, 2009
    Messages:
    2,281
    Location:
    Ancaster, Canada
    How does one become a beta tester?
     
  2. gg_got_a_tesla

    gg_got_a_tesla Model S: VIN P65513, Model 3 Res Holder

    Joined:
    Jan 29, 2010
    Messages:
    6,097
    Location:
    Redwood Shores, CA
    Some early Signatures were apparently handpicked for this.

    I'd love to become a beta tester too if anyone at Tesla's reading this. Ben/Dave/Nigel, please PM me if there's any other way to sign up :)
     
  3. NigelM

    NigelM Recovering Member

    Joined:
    Apr 3, 2011
    Messages:
    13,257
    Location:
    Sarasota, FL
    No use asking me, I'm not on the magic list!
     
  4. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    Ditto.
     
  5. dflye

    dflye S Sig Perf 414, VIN 814

    Joined:
    Oct 7, 2011
    Messages:
    569
    Location:
    Raleigh, NC
    To me, that is utter BS when it comes to QA where the real testers seem to end up being owners vs Tesla's QA FW/SW group.

    The folks that have had their vehicles for many months now are very aware of every existing quirk, and would pick up on any new quirks in a heartbeat.

    Some new owner who is completely green to the entire legacy of prior software updates would have no clue what was a longtime existing feature, a new feature, a new bug, a regression of an old bug, etc.

    Even more annoying to me being a Sig owner in the SW industry; both their UI design and testing groups could definitely use a bit more "road testing" in a lot of cases... they obviously aren't trying a "hit the target" test in a vehicle going down a bumpy road and accidentally disconnecting a call when aiming for the answer button, as the default of detecting a press that misses the target button is to drop the call! Bad UI design led to a poor implementation followed by QA apparently not done in the real world.

    One of many interface failures that if a regular user complains about, the whining is lost in the noise, but if there was a beta tester or otherwise sanctioned source of bug reporting, crap like that would actually get fixed!
     
  6. AlMc

    AlMc 'Senior Moments' member

    Joined:
    Apr 23, 2013
    Messages:
    5,378
    Location:
    Delaware
    Agreed, But I thought there were/are beta testers? I would like to believe (may I am naïve) that they would have some existing owners testing the upgrades before general release. If so, they are giving feedback and getting rid of the bugs.

    I figured beta testers would be a combo of Sig owners, 60/85 KW battery, perf vs non perf people.
     
  7. dflye

    dflye S Sig Perf 414, VIN 814

    Joined:
    Oct 7, 2011
    Messages:
    569
    Location:
    Raleigh, NC
    That is definitely true that multiple long-time existing owners are beta testers, but apparently they have lower standards than I when it comes to acceptable UI!

    After identifying and documenting how to reproduce any new faults with a beta release, I'd be bitching non-stop about the existing UI issues that have been there since day 0!
     
  8. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    You're making a pretty large assumption that they (beta testers) aren't doing that as well. You're making an even larger assumption that it somehow directly affects whether Tesla decides to prioritize fixing that change high enough that they hold the release for it. And finally you're then insulting the beta testers as having "lower standards".

    That's a lot of leaps to make because you happen to be disappointed with one particular behavior that is non-critical to the car's behavior as a transport vehicle.
     
  9. dflye

    dflye S Sig Perf 414, VIN 814

    Joined:
    Oct 7, 2011
    Messages:
    569
    Location:
    Raleigh, NC
    If Tesla had a public site where folks could view existing bugs/feature requests, and let hoi polloi up-vote/down-vote those requests, I think you'd find I'm FAR more nit-picky than your typical beta tester. Just ask anyone I've worked with in SW in the last few decades, I don't accept cutting corners for any reason as an excuse for delivering an inferior product.

    That isn't to say that the existing beta testers aren't already doing a great job, I'm just saying that there is no publicly available way for the general public to know what is known to Telsa as a bug or feature request, to provide ranking of those requests, etc. Basically, there is no visibility into the process, when they have a VAST audience of experts in the field who would gladly be providing pro bono detailed jira (or similar system) tickets documenting any quirk they've found.

    If you've ever tried reporting a bug or requesting a feature through either email or phone channels, after the first few tries, you give up, as the folks on the other end apparently have absolutely no clue how to deal with technical details.
     
  10. brianman

    brianman Burrito Founder

    Joined:
    Nov 10, 2011
    Messages:
    15,487
    I have (with Tesla) and have gotten acceptable responses thus far. Sparse/non-in-depth responses, but acceptable. Then again, I don't expect to get a direct line to the engineering teams.

    - - - Updated - - -

    Now I can agree with you.

    What I took issue with was in the prior post was assumptions about the beta tester behavior based on not seeing your desired fixes implemented.

    And, no I'm not one, though I'd like to be. If for nothing else than to explore REST while they adjust/expand it rather than after it's already baked in a given release.
     
  11. dflye

    dflye S Sig Perf 414, VIN 814

    Joined:
    Oct 7, 2011
    Messages:
    569
    Location:
    Raleigh, NC
    Sorry, didn't mean to imply that beta testers aren't racking up huge volumes of tickets in the internal Tesla feature tracker (whether through direct access or some watered down process hopefully involving an engineer not of the junior grade), just saying that on the anal-retentive scale for UI design and implementation, I'm usually waaaaay off the far end of the chart. :eek:

    Now if I could just get the ear of someone involved with the Navigon integration with regards to when they will support custom POI loading, I'd be in heaven! Attempting that through the regular Tesla channels on numerous occasions has gotten me absolutely nowhere, so for the time being I use an old TomTom when I need to do any real navigation, as the current navigation system is a toy at best.
     
  12. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    Having done software development and product delivery for a couple of decades... no, having random people vote their issues up/down almost never gives you good data to base your productization decisions on.
    Since you are mentioning Jira... do you think that Google is actually fixing the bugs that joe user reports in the public Jira for Android? Yeah right, let's rethink that argument as well.
    I have yet to report a bug to Tesla - but not because there isn't a public bug tracker, but because overall it was clear that they were laser focused on getting the European cars to market. So reporting bugs in 4.5 seemed like an utter waste of everyone's time.

    Now once we have 5.0 rolled out, yet, I will be reporting all the little issues that I find and yes, a public forum to see what other bugs have been reported would be nice. So let's do this here! This forum has been exceptionally good at managing some of the processes that Tesla doesn't want to provide in public (for whatever reason). It should be easy enough to track the bugs for them as well (and avoid duplicate reporting of the same bugs over and over again).

    I'll start a new thread for that (and will need some moderator help to turn the post into a wiki so people can edit the summary, not just add comments at the end.
     
  13. ElSupreme

    ElSupreme Model S 03182

    Joined:
    Jan 13, 2012
    Messages:
    4,279
    Location:
    Atlanta, GA
    Well maybe they don't wan UI feedback. Maybe they want bug feedback.

    Not saying that is the right way to do it, but I could definitely see that desire.
     
  14. viperboy

    viperboy Member

    Joined:
    Dec 18, 2012
    Messages:
    121
    Location:
    United States

    I get this daily. I prefer the term "broken as designed". This typically occurs when the requirements / design are incomplete, written in a vaccuum (without proper vetting) and/or people just dont think. I find an equal mix of the two.

    Edit: a majority of these cases are when the designed function doesnt execute as planned. The un-happy path if you will...
     
  15. dirkhh

    dirkhh Middle-aged Member

    Joined:
    Jul 7, 2013
    Messages:
    3,645
    Location:
    Portland, OR, USA
    I love this term... and stole it for the bug tracking wiki :)
     

Share This Page