More hopeful signs for AI content licensing
Two recent developments are pointing the way towards a web-wide content licensing and payment trend for AI training.
Two days ago, publishers including Reddit, Yahoo, Medium, Quora, Ziff-Davis and O’Reilly announced support for RSL (really simple licensing), a scheme by which site owners could include AI licensing terms in their site’s robots.txt file. Shelly Palmer’s explanation here is extremely clear. The RSL collective will develop a payment model similar to ASCAP for the music industry. This initiative is similar to what website security vendor Cloudflare announced earlier.
That’s the carrot. There’s also the stick. Yesterday Encyclopedia Brittanica and its subsidiary Merriam-Webster sued the AI search engine Perplexity, claiming its answers ripped off their content.
It’s going to take a little while, but I think this will all settle out
AI companies will eventually realize that they can get further by making deals with online content providers rather than fighting with them. This points to a new economic model for web content based on direct payments rather than crappy ads.
AI is growing at an unprecedented rate. These new licensing schemes have the potential catch on remarkably fast as well, because they’ll add less friction than lawsuits. They point the way towards a mutually beneficial model in which content providers that create quality content get rewarded for fueling the best AI-model answers.
Another hopeful sign is that these schemes are based in existing infrastructure and standards like Cloudflare and robots.txt. We don’t have to reinvent the web to make it AI-ready — we just need to adapt it.
Note as well that these same schemes allow any content owner to opt out. If you set your payment terms very high, or just say your site doesn’t permit AI scraping, you can escape the whole ecosystem.
The alternative is not just endless lawsuits. It’s the erosion and eventual destruction of quality content sites as their traffic withers and their ad revenues evaporates. AI companies built on that content would also then eventually collapse in a mound of slop. So it’s in the AI companies’ best interest to find a mechanism that preserves the content they depend on.
Of course, if this doesn’t work, we could end up in a world where the only content available to AI crawlers is free because it’s evil, partisan, and fake. I’d prefer to avoid that. And I think the AI companies would, too.