FSD may require a hardware upgrade...

[MTOman]

Or does anyone have some other explanation as to how they pulled it off?

I think (hope) it's real, but if it's not wouldn't it be possible someone sat in the backseat and drove it? Assuming their engineers can hook a laptop up and control the car it wouldn't be hard to imagine them having a joystick system like this in the backseat and driving by wire:

One reason I think it's real is when it makes a mistake and comes to a complete stop for people walking on the side of the road (starting around 3:50):

That's a mistake I can totally see FSD making, much like how AP sometimes slows down around a curve because it detects a car or truck in the adjacent lane. I suppose they could be really slick and include fake mistakes like that, but I tend to think it's really driving itself and stopped because it thought the people were in the road.

 

[lunitiks]

Thanks for posting that video again @MTOman. Watching it again, notice how erratic the IC animation is. Ping-pong, undetected cars, ghost lane markings, bicyclists interpreted as cars etc.

Reminds me of what I first thought when I saw the demo: How the heck do we know they're using production HW or even code that's intended for FSD. Is the AP1 code for IC running separately while some Nvidia demoware is actually doing the leg work?

 

[stopcrazypp]

I think your interpretation of the data in the report relative to how the video was made is highly debatable. Ultimately, it will come out, likely during discovery if this isn’t resolved in a reasonable timeframe (I.e. during the expected lifecycle of the lease/purchase).

As for a new video, no one wants another scripted, staged demo. People want transparency and communication.

Go ahead and dispute the interpretation, but I'll lay out the facts again, as it appears many people simply aren't aware of them.

10/9/16: "Tesla product unveiling on the 17th (unexpected by most), followed by Tesla/SolarCity on the 28th"
Elon Musk on Twitter

10/16/16: "Moving the Tesla announcement to Wednesday. Needs a few more days of refinement."
Elon Musk on Twitter

10/19/16 5:00PM (Pacific): FSD announcement
All Tesla Cars Being Produced Now Have Full Self-Driving Hardware

10/19/16 5:41PM: "Will post video of a Tesla navigating a complex urban environment shortly. That was what took the extra couple of days."
Elon Musk on Twitter

10/20/16 12:24AM: "Still working on the video. Fully autonomous drive demo only completed several hours ago."
Elon Musk on Twitter

10/20/16: First FSD video released
Full Self-Driving Hardware on All Teslas
You can see visible cuts (might still be from same take just different camera, but cuts nonetheless) and that it was dry weather. Watching it again now, driver had his hands hovering ready to hold the wheel. Check it yourself.

7 mile route taken in the first video in Palo Alto:
Route Taken During Tesla Self Driving Demonstration Video • r/teslamotors

This article (from a big Tesla naysayer at that) analyzes the video in depth in context to the disengagement report, and has an eyewitness that saw Tesla filming on 10/14-17th in Palo Alto, and also independently verified the route is correct.
CA DMV Report Sheds New Light On Misleading Tesla Autonomous Drive Video - DailyKanban

It is possible to independently verify that it rained in Palo Alto 10/14-16, and weather cleared up 17th onward.
Weather History for Palo Alto, CA | Weather Underground

11/18/2016: Second Video Released. You can see there are no visible cuts. The weather is foggy. Driver had his hands on his lap the entire time.
Tesla Self-Driving Demonstration

Disengagement report:
https://www.dmv.ca.gov/portal/wcm/c...a/Tesla_disengage_report_2016.pdf?MOD=AJPERES
October: 530 miles, 169 disengagements on wet roads, 13 on dry roads, total 182
Wet Disengagement dates: 10/14 (37), 10/15 (86), 10/16 (46)
Dry Disengagement dates: 10/17 (7), 10/19 (1), 10/22 (5)
November: 20 miles, 0 disengagements
(Don't these dates and conditions look familiar?)

Draw your own conclusions. Hopefully this was informative to at least some people.

 

[Swift]

Go ahead and dispute the interpretation, but I'll lay out the facts again, as it appears many people simply aren't aware of them.

10/9/16: "Tesla product unveiling on the 17th (unexpected by most), followed by Tesla/SolarCity on the 28th"
Elon Musk on Twitter

10/16/16: "Moving the Tesla announcement to Wednesday. Needs a few more days of refinement."
Elon Musk on Twitter

10/19/16 5:00PM (Pacific): FSD announcement
All Tesla Cars Being Produced Now Have Full Self-Driving Hardware

10/19/16 5:41PM: "Will post video of a Tesla navigating a complex urban environment shortly. That was what took the extra couple of days."
Elon Musk on Twitter

10/20/16 12:24AM: "Still working on the video. Fully autonomous drive demo only completed several hours ago."
Elon Musk on Twitter

10/20/16: First FSD video released
Full Self-Driving Hardware on All Teslas
You can see visible cuts (might still be from same take just different camera, but cuts nonetheless) and that it was dry weather. Watching it now, driver had his hands hovering ready to hold the wheel. Check it yourself.

7 mile Route taken in the first video in Palo Alto:
Route Taken During Tesla Self Driving Demonstration Video • r/teslamotors

This article (from a big Tesla naysayer at that) analyzes the video in depth in context to the disengagement report, and has an eyewitness that saw the filming that happened on 10/14-17th, and also independently verified the route.
CA DMV Report Sheds New Light On Misleading Tesla Autonomous Drive Video - DailyKanban

It is possible to independently verify that it rained in Palo Alto 10/14-16, and weather cleared up 17th onward.
Weather History for Palo Alto, CA | Weather Underground

11/18/2016: Second Video Released. You can see there are no visible cuts. The weather is foggy. Driver had his hands on his lap the entire time.
Tesla Self-Driving Demonstration

Disengagement report:
https://www.dmv.ca.gov/portal/wcm/c...a/Tesla_disengage_report_2016.pdf?MOD=AJPERES
October: 530 miles, 169 disengagements on wet roads, 13 on dry roads, total 182
Wet Disengagement dates: 10/14 (37), 10/15 (86), 10/16 (46)
Dry Disengagement dates: 10/17 (7), 10/19 (1), 10/22 (5)
November: 20 miles, 0 disengagements
(Don't these dates and conditions look familiar?)

Draw your own conclusions. Hopefully this was informative to at least some people.

First, could I just ask that you complete your FSD timeline with Tesla/Elon’s remaining FSD communications in the 10 months between 11/18/2016 and now?

Second, no one knows what actually went down to create that video. Again, if there were 0 disconnects for 20 miles and it worked perfectly around pedestrians, traffic lights, intersections, etc, etc, then why haven’t we see or heard anything of substance on FSD in the 10 months since?

 

[mmd]

I think (hope) it's real, but if it's not wouldn't it be possible someone sat in the backseat and drove it? Assuming their engineers can hook a laptop up and control the car it wouldn't be hard to imagine them having a joystick system like this in the backseat and driving by wire:

One reason I think it's real is when it makes a mistake and comes to a complete stop for people walking on the side of the road (starting around 3:50):

That's a mistake I can totally see FSD making, much like how AP sometimes slows down around a curve because it detects a car or truck in the adjacent lane. I suppose they could be really slick and include fake mistakes like that, but I tend to think it's really driving itself and stopped because it thought the people were in the road.

Good point! It's sad that we're discussing this here now after 9 months. At that time, I knew it was AP 0.5 when MobileEye got out abruptly. I thought, using the blue screen and other video ricks could achieve this, but your method seems even easier to fake it. It isn't impossible to intentionally add a minor mistake to make it look real. Tesla spent months to make that video and a lot was riding on it. I mean, if you want to make a made up facial image real, you just add some skin blemishes, a pimple, and/or a mole. Same principle can be applied here too.

Consumer watchdog released a video around that time. Search "watchdog tesls" on youtube. It boldly claimed that the self driving demo was completely staged. Now that we are here after 10 months of that demo video, I wonder what information they might have had at that time.

 

[AnxietyRanger]

As any AP2 owner will tell you there's literally no way that happened with any software or hardware we have/had/promised to have in a single shot. I can't drive in the carpool lane on the 405 which gets millions of tesla miles annually without having a disengagement over 5 miles, it's all over the place with lines etc and that is considered basic stiff for ap1 which I still drive in regularly. There is clearly zero fleet "learning" going on which is a real bummer.

Not sure what they did to get that video but my x isn't capable of driving safely on freeways with good markings let alone with pedestrians and stop signs etc...

I disagree.

I think the FSD video was real code running on real AP2 hardware. I don't think it was faked. How much of that code was Tesla's and how much Nvidia's, and how much of that code was abnormally pre-trained or pre-configured for that route, now THOSE are more realistic questions IMO...

EAP is completely different codebase, it tells us nothing. Basically a quick stopgap solution to manage without the missing MobilEye chip they had hoped to use for a while after AP2 came along.

 

[dknisely]

Has anything come out about that video since it was released?

It looked so promising but now it makes no sense that they had it working that good when all this time later AP2.0 has major problems from what I've read -- since I don't have it myself. But I can't imagine it was fraudulently produced. That would involve too many people and someone would talk -- it always happens -- and I can't accept that Tesla is that stupid, or would cross that line. So I don't buy that it's scripted and staged even though that is possible it's highly improbable to me. Then again, I'd have never predicted that VW would make software to fool regulators when testing for emissions. I guess you can never underestimate the brazenness and stupidity when it comes to corporate greed.
...

It did not look impressive to people who understand that stuff. It was obvious how it worked and how far that approach is from any kind of actual "FSD." These days, that's not a lot more than an undergraduate computer vision project, assuming you have the hardware platform to control the car. It looks just like every self-driving startup project demo reel.

 

[J1mbo]

It did not look impressive to people who understand that stuff. It was obvious how it worked and how far that approach is from any kind of actual "FSD." These days, that's not a lot more than an undergraduate computer vision project, assuming you have the hardware platform to control the car. It looks just like every self-driving startup project demo reel.

Let's see some links then!

 

[Canuck]

It did not look impressive to people who understand that stuff. It was obvious how it worked and how far that approach is from any kind of actual "FSD." These days, that's not a lot more than an undergraduate computer vision project, assuming you have the hardware platform to control the car. It looks just like every self-driving startup project demo reel.

So can you give us the layman's summary please?

 

[lunitiks]

Pretty-please

 

[stopcrazypp]

First, could I just ask that you complete your FSD timeline with Tesla/Elon’s remaining FSD communications in the 10 months between 11/18/2016 and now?

I only have those tweets from the article, I don't have the free time to dig through all of Elon's tweets to look it up (I don't even have Twitter accoint). Feel free to do so if you want. I only have the video info because I discussed it in another thread.

Second, no one knows what actually went down to create that video. Again, if there were 0 disconnects for 20 miles and it worked perfectly around pedestrians, traffic lights, intersections, etc, etc, then why haven’t we see or heard anything of substance on FSD in the 10 months since?

Are you suggesting the report was fraudulent? I'm operating under the assumption that it's truthful. Again the next goalpost is the cross country trip (and FSD aside, Tesla must critically make sure Model 3 launch goes well). They seem to be selling just fine, so no need for more hype at the moment.

 

[3Victoria]

Here's my somewhat educated guess of the second demo:
The car is running AP2 which controls the IC.
It is also running a deep neural-network (NN) model that handles object recognition and driving logic.
This dNN model was hacked to respond to mapping and route software that was used to plan the route. ( Several days of testing and bug fixing occurred in October ... see failures of routing and mapper functions in California report. )

While a demo will handle many circumstances, it will not handle all. That requires incremental work on the DNN and extensive retraining.

Present cars have completely separate codebase which includes a NN for object recognition and linear code for driving functions. The former replaces the Mobileye chip used in AP1.

A general solution is much harder than that demo, but EM seems confident that it is achievable.

In summary:
AP1 = MB chip plus Tesla driving logic
AP2 = Tesla NN plus Tesla driving logic
FSD = Tesla dNN with integrated Tesla mapping and route planning.

 

[Swift]

I only have those tweets from the article, I don't have the free time to dig through all of Elon's tweets to look it up (I don't even have Twitter accoint). Feel free to do so if you want. I only have the video info because I discussed it in another thread.

I'll make it easy on you. Elon only had one relevant tweet on FSD after the 11/16 demo. Here it is:

Nothing since. Nothing. Not a good feeling for those who paid for FSD in 2016 based on the hype, video, etc, especially if it's on a car that was leased and the months are ticking away.

 

[Swift]

Are you suggesting the report was fraudulent? I'm operating under the assumption that it's truthful.

I operated under that assumption (that it was truthful) when I bought the car based on all the marketing. It's actually one of the key reasons I bought the car. Tesla had to know how way far off they were from actually getting something to a regulator at the time so that's a huge issue for me. The lack of communication makes it much worse.

 

[MP3Mike]

I operated under that assumption (that it was truthful) when I bought the car based on all the marketing. It's actually one of the key reasons I bought the car. Tesla had to know how way far off they were from actually getting something to a regulator at the time so that's a huge issue for me. The lack of communication makes it much worse.

Of course they knew it was a ways off which is why they said the next demonstration would be a coast to coast drive that they would release by the end of this year. So it seems that they made it clear that there is no way that FSD would be released before 2018.

 

[Swift]

Of course they knew it was a ways off which is why they said the next demonstration would be a coast to coast drive that they would release by the end of this year. So it seems that they made it clear that there is no way that FSD would be released before 2018.

Just like how explicitly clear they were that EAP would be done by 12/31/2016. Thankfully I took a screenshot of that before they changed the text.

As far as FSD's timeline, take a look at Elon's pretty clear tweet above.

 

[conman]

Andrej Karpathy on Twitter

We're hiring strong ML/CV/Roboticists for the Tesla Autopilot Vision team. We ship autonomy at scale. Join us: [email protected]

 

[dknisely]

Andrej Karpathy on Twitter

We're hiring strong ML/CV/Roboticists for the Tesla Autopilot Vision team. We ship autonomy at scale. Join us: [email protected]

Yeah, I'll bet you are. Bwahaha!

 

[dknisely]

So can you give us the layman's summary please?

You need to break it down frame-by-frame and blow them up to see how much they let you actually make out in the feature detection. One of the things these demos do is speed them up, which hides a lot of detail. It looks fantastic (just like many other similar demos, most notably those from NVIDIA) on the surface, but the devil is in the details of exactly what features were being recognized (correctly) and actually semantically "understood." Moreover, you need to really assess how much the car was actually DOING anything in response to the feature recognitions (other than aborting the many failed attempts that are publicly documented before they got the "keeper."

If they had slowed the video down, it would look less impressive and would also reflect the not-so-silky-smooth driving. But that's OK -- it's not what the dog says, but the fact that the dog can talk that is impressive! Kudos for Tesla engineers to pull off a demo like this. Basing an FSD claim around this is less advisable, IMHO. I want Tesla to succeed here, and I DO believe they can make driving MUCH safer with AP2.0 and 2.5 hardware. I'd just rather see them pull back from the FSD claim and focus development on solid L3.

You also have to recognize that no matter how "spontaneous" those Mountain View routes look, they are completely scripted and immaculately scoped out and orchestrated. I'm guessing that every pedestrian, dog, bike, and even many cars are scripted and queued to the script. It's not FAKED, but rather just scripted. I'm sure the car is honestly responding to the events presented to it, but the script is designed around what the car is known to be able to do. Also, for every live run attempt, there are thousands of simulated runs based on captured video feeds of the route to prepare and customize the code FOR THAT ROUTE. You can't prepare like that for EVERY ROUTE POSSIBLE (unless you are Google ).

There are actually open source code bases that will monitor video feeds and do the type of feature recognition (cars, people, bikes, etc.), and draw boxes around them. That's not the hard part. Making it actionable and reliable enough to trust is an enormous technical challenge! To my knowledge, there aren't any self-driving tests that aren't supervised by a driver standing by to take over publicly disclosed except for Waymo, and those are based on an extraordinary 3D mapping of every driven area down to centimeters, along with years of advanced machine learning to dictate a conservative driving strategy. That approach has been the antithesis of Tesla's, at least until recently. I think we started to see a sea-change in their thinking when they realized that it was essential to create a micro-localization map based on crowdsourced data, but that won't come to fruition (IMHO) for a long time, and won't enable FSD until several AP HW generations in the future.

I'm disappointed that it now appears that my AP2 car won't see much of the fruits, but I took the risk and lost the bet. I always knew that things would be all over after M3 hit (because Tesla is now primarily a M3 and MY company), but I thought we'd get a bit more L3 stuff before July. Oh well... I was wrong.

 

[stopcrazypp]

I'll make it easy on you. Elon only had one relevant tweet on FSD after the 11/16 demo. Here it is:

Nothing since. Nothing. Not a good feeling for those who paid for FSD in 2016 based on the hype, video, etc, especially if it's on a car that was leased and the months are ticking away.

I believe this was discussed in another thread: they said explicitly on the order page even from launch that there is absolutely no estimated timetable for release (this is different from EAP). Personally, I was led to believe FSD will come no sooner than the end of this year (because the cross country demo won't happen until the end of the year).

I would go with that instead of Elon's tweets, where his estimates have been completely off many times: one issue I remember reading in other threads is the browser update, which still isn't here. On that particular tweet, it had been analyzed to death elsewhere (Elon was talking about some differentiating features, not L4/L5 driving).