Data Marketplaces & Intermediaries

‘Really Simple Licensing’ wants to set AI training terms in robots.txt

Source: RSL Collective · Sep 25, 2026

A new open standard called Really Simple Licensing (RSL) is trying to make AI-training permissions machine-readable at the source. Instead of every data deal being a bespoke negotiation, RSL lets a publisher declare its AI-training license terms directly in robots.txt — the same file that already tells crawlers what they may access. It's backed by the RSL Collective, co-founded by Eckart Walther and Doug Leeds, with early adopters including Reddit, Yahoo, and Medium.

The ambition is a governance layer for training data that operates automatically at crawl time: rights and terms expressed in a standard machine can read and act on, rather than buried in contracts negotiated one publisher at a time. If it gains traction, permission stops being an afterthought and becomes part of the infrastructure of the open web.

The direction of travel is unmistakable and favorable to anyone building on rights-cleared data: the industry is steadily moving toward explicit, enforceable, machine-readable data-use permissions. A world where consent and licensing are standardized and checkable is one where a corpus built on documented, specific consent from the start is the norm buyers expect — not the exception they hope for.

Key Points

• RSL (Really Simple Licensing) is an open standard letting publishers declare machine-readable AI-training license terms in robots.txt
• Backed by the RSL Collective (co-founders Eckart Walther and Doug Leeds); early adopters include Reddit, Yahoo, and Medium
• Aims to move licensing from one-off negotiations to a crawl-level, automated governance layer
• Part of the broader shift toward explicit, enforceable data-use permissions