# Cooperative machine traffic — Agent Delivery Network — ADN

← Back to Help The web's machine traffic problem is not a technology problem. It is an incentive problem. Sites block bots because bots cost money and provide no signal. Agents evade blocks because they need content to do their job. Both sides lose.
 ADN changes the incentives with a cooperative invitation model .
 Invitation over interception
 Three approaches exist for handling machine traffic:
 Approach How it works Problem
 Identity-gated Serve signed bots well, block everyone else 90%+ of traffic is unsigned
 Hosted proxy Point a hostname at ADN; all traffic on it flows through us Uptime on the hostnames you proxy
 Invitation + edge connector Serve anonymous traffic correctly; edge connector captures traffic that ignores invitations Requires compatible edge stack
 ADN chose invitation plus hosted proxy or edge connector because it serves the entire traffic spectrum — from anonymous curl scripts to Web Bot Auth agents — without sending ordinary browser HTML through the gateway on edge-connector paths.
 How invitation works
 Sites publish discovery artefacts that tell agents where to find structured content:
 Manifest at /.well-known/adn.json — links to all machine-readable resources
 Navigation at /llms.txt — page index in Markdown
 Link headers on HTML responses — RFC 8288 discovery without scraping
 DNS-AID at _agent.<domain> — TXT record pointing to MCP endpoint
 Agents that honour these invitations get better content in better formats — without the gateway ever intercepting their traffic.
 Content negotiation as universal fallback
 Only 3 of 7 major agents send Accept: text/markdown by default. ADP supports URL suffix fallback:
 GET /docs/api/index.md
GET /docs/api.md
GET /docs/api.dclg
GET /index.dclg Agents can also negotiate DocLang via Accept: application/vnd.doclang.document+xml for structured XML with explicit headings, tables, and code blocks.
 Even a totally anonymous script gets correct Markdown or DocLang with zero identity — graded by behaviour score, not blocked.
 The identity ladder
 Cooperation is rational at every tier:
 Tier 0 (anonymous): Full content via negotiation. Conservative rate limit. Visible path to register.
 Tier 1 (verified bot): Class-specific defaults. Higher throughput for known crawlers.
 Tier 2 (rDNS/IP range): Bulk corpus access for training crawlers with confirmed identity.
 Tier 3 (registered agent): Better limits across every site on the network. Free registration at https://registry.agent-delivery.network/register.
 Tier 4 (Web Bot Auth): Highest trust. Commercial terms available.
 Every tier is independently useful. No tier gates basic service.
 Why agents register
 Registration takes 30 seconds and buys something immediately:
 • Visibly higher rate limits on the first request
 • Access to MCP tools and bulk corpus on participating sites
 • Listed in the public agent directory
 • Machine-readable terms acceptance recorded for compliance
 The network effect: every additional site makes registration more valuable to agents. Every additional agent makes the network more valuable to sites.
 Why sites cooperate
 Sites that publish discovery artefacts and enable hosted proxy or an edge connector get:
 • Origin offload — machine traffic served from edge cache, not your servers
 • Per-class control — different policies for training crawlers vs live fetches
 • Token savings — agents get Markdown instead of parsing HTML nav chrome
 • Analytics — total vs machine traffic, formats served, and What to fix : ranked URLs with format fit, missing index, and optional auto-fix JSON-LD on pricing paths
 The free efficiency scan at https://www.agent-delivery.network shows the ROI before you commit.
 Edge connector safety contract
 For sites that need full offload while keeping custom routing, the edge connector runs at the customer's edge with a fail-open guarantee :
 • Classification happens in Cloudflare Snippet or Next.js middleware
 • If classification fails → request goes to origin
 • If gateway is down → request goes to origin
 • Ordinary browser HTML → always goes to origin
 • Requests that ask for Markdown, DocLang, or JSON are served that format (that is not a machine label)
 10 automated tests verify fail-open behaviour. This is not optional — it is a safety contract.
 Machine-readable compliance
 Every degradation response includes upgrade instructions:
 {
 "error": "rate_limited",
 "upgrade": {
 "register": "https://registry.agent-delivery.network/register",
 "terms": "https://gateway.example.com/terms.json"
 },
 "retry_after": 3600
} No silent walls. No mystery 403s. Cooperation has a clear, frictionless path.
 Ask ADN