Andrej Karpathy

[Terminator857]

https://twitter.com/x/status/1510229609937264650

Quote: I forgot how cool European cities are. More compact, denser, more unique / interesting, cleaner, safer, pedestrian/bike friendly, a lot more pedestrian only plazas with people relaxing / hanging out. A lot more of outside is an outdoor living space, not just transportation space.
/quote
Funny how is knocking cars indirectly and directly knocking streets for cars. I agree with Karpathy that U.S. should be more pedestrian, bicycle friendly, and less car friendly.

 

[Terminator857]

https://twitter.com/x/status/1511448391263338500

I had a great time chatting with @karpathy yesterday! We discussed a range of topics, from synthetic biology to ML conferences. What's funny is that we are about 2 hrs from each other but we got to meet up in Amsterdam airport!

 

[Terminator857]

https://twitter.com/x/status/1511729310746374146

@karpathy: Exactly. But NLP has run far ahead of vision on showing impressive transfer learning to tasks outside of the self-supervised objective. Vision is bit behind I think partly due to required scale (many many more pixels than words). Papers like MAE are close

@ranig: Text is already “compressed by design” so much easier to process in scale. Vision requires lots more processing power so easy to see why text is leading the way. On the positive side, it allows vision to skip “mistakes”
made in text

@karpathy: Yes in NLP humans did the hard work of compression into discrete tokens. In vision the pixels are extra plentiful, raw (uncompressed), and also have a lot more distracting entropy - e.g. structure in clouds, trees, etc. Could simulate it in NLP by sprinkling in 10X random tokens.

 

[Terminator857]

https://twitter.com/x/status/1511716250883346445

Quote: Highly amusing conspiracy theories. I am interested in language models because they reveal techniques for training very big Transformer neural nets, and today's AI both language and vision (or anything else) is all just Transformers due to convergence see

 

[enemji]

I have been saying the same on the FSD forums, especially the Phantom Braking issue. There is too much information to process, which leads to it making decisions on factors which we humans are ignoring very conveniently as we have been trained. The Tesla is still in training, and being trained by the FSD Beta group who have a high safety score.

 

[heltok]

 

[DayTrippin]

https://twitter.com/x/status/1510229609937264650

Quote: I forgot how cool European cities are. More compact, denser, more unique / interesting, cleaner, safer, pedestrian/bike friendly, a lot more pedestrian only plazas with people relaxing / hanging out. A lot more of outside is an outdoor living space, not just transportation space.
/quote
Funny how is knocking cars indirectly and directly knocking streets for cars. I agree with Karpathy that U.S. should be more pedestrian, bicycle friendly, and less car friendly.

Really helps to put in context that so many European cities were built many centuries before cars existed. Maybe barely big enough for people walking or on horseback (without a cart). A lot of the streets suck on a bicycle too.

I will be the first to say I hate densely populated cities. I hate hearing all my neighbors fighting, yelling, playing their music, etc. It is like listening to hundreds of conversations in your head all the time. It takes a real toll on you. Most cities don't have enough green space. Add in that just because you have less cars, doesn't mean you don't have congestion in other ways. I have been in a lot of non-US cities where there was even more congestion with bikes than with cars.

As for safety, can't say I agree with all that either. I worked in Denmark and Sweden (and other place in Europe). I was in Malmo for a while. Don't just have to worry about shootings, but hand grenade attacks there along with bombings. The locals kept telling me Malmo was safe. Just don't go to the outskirts of town. Don't go out at night, if female walk in pairs, don't dress provocatively, etc. etc.

Sweden | Analysis of rise in bombings and gun violence

Organised crime appears to be the primary factor behind a rise in the frequency of bombings and gun violence in Sweden in recent years.

www.dragonflyintelligence.com

God help you if you are in the police department (or anyone else) and talk about the increased number of rapes and what groups are committing them in Sweden as well. Europe is its own worst enemy in many ways. The US is getting there too.

 

[enemji]

https://twitter.com/x/status/1511729310746374146

@karpathy: Exactly. But NLP has run far ahead of vision on showing impressive transfer learning to tasks outside of the self-supervised objective. Vision is bit behind I think partly due to required scale (many many more pixels than words). Papers like MAE are close

@ranig: Text is already “compressed by design” so much easier to process in scale. Vision requires lots more processing power so easy to see why text is leading the way. On the positive side, it allows vision to skip “mistakes”
made in text

@karpathy: Yes in NLP humans did the hard work of compression into discrete tokens. In vision the pixels are extra plentiful, raw (uncompressed), and also have a lot more distracting entropy - e.g. structure in clouds, trees, etc. Could simulate it in NLP by sprinkling in 10X random tokens.

That is what I have been saying- there is too much data to process into usable information and that is why things like Phantom Braking are occurring.

 

[Terminator857]

10 years ago Karpathy was pessimistic about computer vision:

The state of Computer Vision and AI: we are really, really far away.

Musings of a Computer Scientist.

karpathy.github.io

Now Karpathy says the problem can be solved in the future.

https://twitter.com/x/status/1522639577118240769

 

[Terminator857]

Andrej says FSD saved him from collision with motorcycle.

https://twitter.com/x/status/1528584867675373569

Quote:
I’ve seen similar events play out in clip telemetry many times, but a few minutes ago is the first time Autopilot prevented an almost certain collision for me personally. Experiencing it in real life is something else.

I was lane changing while a motorcyclist very aggressively lane changed and accelerated from behind the car behind me. Autopilot aborted the lane change and was right.

 

[enemji]

Kar-pathy: Literally means Lord/Head of Kar.

 

[cwerdna]

To repost here:

https://twitter.com/x/status/1547332300186066944

 

[DayTrippin]

Nobody saw that coming...

 

[enemji]

 

[jebinc]

Nobody saw that coming...

AMF!

 

[DanCar]

Andrej has been working out or was it Wegovy?

https://twitter.com/x/status/1600582722228875264

 

[DanCar]

Andrej tells how to creat chat GPT from scratch.

https://twitter.com/x/status/1615398117683388417

 

[DanCar]

Karypathy rejoining OpenAI

https://twitter.com/x/status/1623476659369443328

 

[DanCar]

Karpathy gives a talk on the state of GPT.

Microsoft Build–Join us May 23–25, 2023

Learn to harness what's next for developers with expert speakers and sessions.

build.microsoft.com