Reddit, Inc.

So I think that in order to train a model, you need access to information. And the Internet ain't what it used to be in terms of being a source of information. And so IP is becoming more valuable. You're not, I think everybody by this time expected all the foundational models to have all this healthcare information. But if I'm Mayo Clinic, I'm not giving Microsoft or Google or OpenAI my IP, because that's what brands me. And so there's going to be a lot of money available there.

don't try to hold out for money on the training side of things, because we're going to create synthetic data, we're going to do all kinds of other things that are going to mean that no one's particular data is really going to matter. What you should be is on freshness, on brand, on other things

I think the conversation has been historically very caught up on training data, but it will increasingly become more about what happens at inference time.

Chamath oppose ×3 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E174 · 2024-04-12 ▶ 50:37

It turns out that OpenAI transcribed over a million hours of YouTube videos to train GPT-4... if Google is not going to sue OpenAI, then this is a moot point

Chamath support ×2 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E170 · 2024-03-15 ▶ 24:20

I don't know if anybody's actually explored this, but if it is true and Google decides they have an issue with it, that's not good for these folks.

Sacks oppose ×2 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E168 · 2024-03-01 ▶ 52:37

There's just a huge number of vendors of content. And so, models will need to buy some, but as long as they can get some, they don't need to have all. And therefore, it's basically highly competitive among suppliers, and there's a very limited number of buyers. So that tends to be the buyers.

Friedberg oppose ×3 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E168 · 2024-03-01 ▶ 46:13

And I'm not sure how you get paid a continuous licensing stream for that content. Once you've trained the model, the content gets old, it gets stale at some point in a lot of cases, like news. And then eventually, if you don't have a high quality, continuous stream of content, it's not worth as much anymore. ... And so every year, all the old data becomes worth even less.

Jason support ×3 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E168 · 2024-03-01 ▶ 50:40

Reddit, Quora, Stack Overflow, they're going to just get taken out. I think this is going to be the new model. ... I think they're going to get taken out. I think these businesses will become too valuable because they do have ongoing content that just keeps getting generated.

Chamath support ×3 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E168 · 2024-03-01 ▶ 42:01

And I just said that we should call this TAC 2.0, except now what Google is doing is, instead of paying for search, they're actually paying for your data and saying, give it to me so that I can train my models and make it better. ... So if you're an entrepreneur building a website or building an app that has really unique training data or really unique data, you'll be able to license and sell that. And that'll be an incremental revenue stream to everything you do in the near future.

Chamath support ×3 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E167 · 2024-02-23 ▶ 1:06:34

like what Google did with Reddit, we're now going to spend $60 billion a year licensing training data, right? We're going to scale this up by a thousand fold ... we are going to be the truth tellers in this new world of AI

Sacks support ×2 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E164 · 2024-02-02 ▶ 1:12:25

It's hard to know exactly how valuable that is because we're still in the early innings, but I mean, they can definitely do something with that data. Grok's whole competitive advantage is having exclusive access to Twitter's data

Jason support ×2 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E164 · 2024-02-02 ▶ 1:11:37

there is this concept that Reddit has the greatest pool of data for large language models ... They have talked about they want to get paid for licensing and that if you want to use their data for your language money, you got to get permission.

Jason support ×2 ▲ on 🤖 Training-data owners (NYT, Reddit, X, YouTube) win 2024 E160 · 2024-01-06 ▶ 24:32

My biggest winner in 2024: training data owners like the New York Times, Reddit, X, YouTube... language models are hitting parity... the real value is going to be in the training data... I think it's going to be a nine-figure settlement and an ongoing licensing fee.