🤖 Foundation models are a big-tech oligopoly - the substrate gets given away

“basically these big companies have decided, no, we're just gonna make all these models extremely good, extremely useful and very, very free. And so a lot of the resources are going there to subsidize, economically subsidize, and by implication, economically destroy the value of that category. That's going to be good for startups.”

“So whenever I see a chip maker and a cloud provider come together to put in a lot of money, it's essentially round tripping cash.”

“they're trialing, Friedberg, on average, six, seven, eight language models before they pick one, and they're not picking OpenAI every time.”

“when you think about what these hyperscalers are going to do, they're not gonna spend a billion, they're not gonna spend 10 billion. They'll spend a hundred billion dollars, right, in order to be in this race.”

“Foundational models are getting disrupted every other week. They're being decreased in size, parameters are being reduced. They're being commoditized. You can run these things on M2 chips.”

“But the most important thing, I think, to remember is that where the real value gets accrued is five, six, seven years later when the software and services companies show up and create a huge moat. And those are the Googles and the Facebooks and the Apples of the world.”

“I think in order for you to be able to compile that model, to generate that initial instantiation, you're still running it in a cluster of thousands of GPUs.”

“I think it's more of a moat where, because it's not that the ladder comes up and nobody else can get in, but the regulations are going to be a pretty big moat around major incumbents who know they qualify for this because they're going to write these standards.”

“There's this fuzzy gray area where a lot of people can find utility in a lot of different products, and then the one with the better distribution wins.”

“The model runs on 700 gigs. That's less data than fits on my iPhone. So I could take that model, I could take the parameters of that model, and I could create an entirely new version, I could fork it, and I could do something entirely new with it.”

“And then you have the distribution endpoints of which there are many whose economic incentives are very high, right? So Facebook doesn't want to just sit around and have all this traffic go to ChatGPT.”

“Maybe it just all ends up accreting to the big companies who can make massive investments in this space.”

“It may all end up with Microsoft and Google. I mean, this may end up looking like iOS and Android at the foundation model level.”

“The market has not resolved to the right answer with all the other big tech problems because they're monopolies.”

“And you know, as LLMs aren't right now, they feel like they're this monopoly held by Google and held by Microsoft and OpenAI. I think very quickly, like all technologies, they will commoditize.”

“I think that you'll see the emergence of these various models that are actually optimized for various ways of thinking or political leanings.”

“The good news is that the Google monopoly has finally been cracked. The bad news is that it's Microsoft and even bigger monopoly. That's the one that's done it. But it just shows how vulnerable all these big tech companies are.”

“This is something that we've said before, which is that ChatGPT is an incredibly important innovation, but it's an element of a platform who will get quickly commoditized because everybody will compete over time.”

“So my point is, David's right, the huge companies, I think, will create the substrates.”

“I don't think startups are going to be able to create the AI themselves, but they might be able to benefit from the APIs.”