🤖 Models commoditize — proprietary data is the only AI moat

“Those are the kinds of moves in business that we will see in the next five to ten years that I find much more exciting and trying to figure out how to play in that space.”

“Apple has all that watch data, if they could pair that with Epic's data set, what could they do together? So this is going to be like, this is the new oil is going to be data.”

“So to your point, that's really valuable because that's real work that Google or Microsoft or OpenAI won't do. Right. And if you have that and you bring it to the problem, you can probably make money. There's a business there to be built.”

“If you can be the first out there in a given vertical with a proprietary dataset and then you get the advantage, the moat of reinforcement learning, that would be the way to create, I think, a sustainable business.”

“And one of the advantages that I think businesses are going to latch on to, which we talked about historically, is novelty in their data in being able to build new systems and new models that aren't generally available.”

“When you can have large, unique data sets that you can then model using these tools and these capabilities and be predictive about what the next product iteration should be, it can really change the value and the trajectory of your business.”

“So I am thinking there's going to be a really good business created in taking the open source projects and forking them and verticalizing them like, you know, Sacks's one that's doing dental work, you know, like this makes sense to me.”

“Where it's really powerful is over time, right? If it's got your last six sets of x-rays over a, whatever, six-year period, it can detect changes that are probably, you know, hard for a human to see.”

“I'm saying, when you look at transformer architecture today, every LLM that you write on the same corpus of underlying data for training will get to the same answer.”

“You have to remember, all these models are open-sourced and none of them mean anything in the absence of the data you give it to train on.”

“I thought it would be an interesting tactic for AI startups to use if they're trying to get a hold of proprietary training data.”

“So the point is, I think we've talked about this for a while, but all of these models will converge in the absence of highly unique data, right? What I've been calling these white truffles. So if you can hoard white truffles, your model will be better. Otherwise, your model will be the same as everybody else's model.”

“If you look at certain data sets, Reddit, Stack Overflow for programming, and Quora, these things are going to be worth a fortune.”

“The extensibility, the integration of live data, and the integration with Google's very unique data set is what's so powerful that they have access to flight data that they have integrated, YouTube transcript data.”

“So I've used that as a way, a forcing function for me to try to prove that there's really these two barbells, it's a barbell, bookends. One is a silicon and one of these data providers. That's probably an investible thing that's defensible.”

“I do agree with Chamath's point. Data's the new oil. Whatever data set you have that you have that's unique is incredibly powerful in terms of defensibility and could help you to outpace one of the generic models.”

“every enterprise wants its own internal chatbot. It wants its own internal ChatGPT, where the employees can ask questions, and the AI has access to all of the company's information”

“there is no business on Earth that has more data than Google. YouTube is the richest data repository, digital data repository on Earth.”

“So my refined thoughts today are sort of what my initial guess was when we started talking about AI a year ago, which is the picks and shovels providers can make a ton of money. And the people that own proprietary data can make a ton of money.”

“eventually when you're scouring the open Internet, it doesn't matter how fast or how big you scour the open Internet. The open Internet is a bounded space. And so you get to the same answer.”

“aren't you worried that somebody with an agenda and a balance sheet could now basically gobble up all kinds of training data that make all models crappy? And then they basically put their layer of interpretation on critical information for people?”

“I think the open Internet has enough data that there isn't going to be a monopoly on information by someone spending money for content from third parties.”

“Because I think that there's a great deal of capability that emerges in the fine-tuning and the unique data that certain people may have to make that one tool better than the rest. And therefore, everyone will end up using this one lawyer service or this one accounting service or what have you.”

“you have probably the most prolific set of training data that has ever been created in the entire world to make these models kick ass”

“the accuracy or the value of these models will probably shift to these proprietary sources of training data that you could get that others can't”

“I definitely don't think it will be an arms race for data because when the models get smart enough at some point, it shouldn't be about more data, at least not for training.”

“because you have this data on your phone, all your iMessages, all your documents, all your photos, videos, music collection, there's a unique set of data on that phone to make your personal LLM that's going to do extraordinary things for you”

“Don't you agree that the data set is not so cool? Uber has the data. DoorDash has the data.”

“And having all this local data is a huge advantage for Apple. They've got your messages, your phone, your calendar, your photos, your app behavior, the data inside of your wallet. All of this gives them a huge, huge advantage.”

“foundational models are quickly becoming a consumer surplus. Every model is roughly the same. They keep getting better and better, but they're also approaching these asymptotic returns.”

“we're going to create synthetic data, we're going to do all kinds of other things that are going to mean that no one's particular data is really going to matter”

“The third piece is that all of these models basically run out of viable data to differentiate themselves, and it basically becomes a race around synthetic information and synthetic data, which is a cost problem.”

“So the problem that Expedia has is the same that booking and a bunch of these other folks have, which is that the principal heartbeat of the company, flight information and other things, are licensed to them by third parties. And so what they are is a UI and a front door.”

“And the actual advantage is going to be in the IP and owning content. And the really smart thing to do would be for somebody to go by Reddit, Quora, the New York Times, the Washington Post and Disney and take all that IP and then not allow other people to use it, sue the hell out of them every time they try.”

“Tesla has an extraordinary advantage that they were really pressure to put cameras on everything years ago, and that gives them this ability to build models that do self-driving. So I think that there's a lot more data advantage that arises in certain industry segments than others, and that's where the moat will lie, and that moat will allow you to actually build better products that get you a more persistent advantage in gathering more data.”

“At some point, the amount of data becomes the long pole in the tent. At some point, the quality of the algorithms becomes a long pole in the tent, and more compute is not going to change that.”

“They have such a data advantage and such a deep integration into people's lives because they use three or four services.”

“Onavo was a small data service provider, but what it did is it had a panel of phones, and we as investors could see which apps people were using, and the data was incredibly valuable because it was the only service that gave you true engagement data. ... Eventually, it sold to Facebook, and Facebook used it internally and didn't allow anybody else to use it.”

“This is great service. A lot of people rely on it. He buys it, shuts it down for everybody else, gets the tool for himself, gets the data for himself.”

“That has huge implications because if you think about all these other companies, what has Lama been doing? They just spent 15 billion to buy 49% of scale AI. That's exactly a bet on human knowledge. What is Gemini doing? What is OpenAI doing? What is Anthropic doing? So all these things come into question.”

“The most important thing I think as a VC that you said, as we've been debating for years, should we invest in companies like Scale, or WorkCore, or any of these surge? The truth is, I think there's a very short half-life on human-label data. And so, everybody who's investing in these companies, just looking at revenue traction, really didn't understand that there may be a year, two years, three years max, when anybody uses human-label data for maybe anything.”

“Grok 5 and for sure Grok 6 will not use Common Crawl. It will not use the Internet. It'll just be an enormous amount of synthetic data.”