SpendPreflight / Crawler

A public, unpaid health check for x402 endpoints

We operate SpendPreflight. Scout is our endpoint discovery feature, currently being tested in isolated staging before production review. Our crawler visits HTTP resources listed publicly by Coinbase Bazaar or on x402scan server pages. A listing and a working payment challenge do not prove that a paid service will deliver its advertised result.

User agent: SpendPreflight-Crawler/1.0 (+https://spendpreflight.com/bot)

• The Health Index uses GET. Scout uses the GET or POST method in each public listing; POST sends only an empty JSON object. Neither uses credentials, signatures or payment headers. We never pay third-party endpoints.
• At most one request per host every ten seconds, including robots.txt, with longer advertised crawl delays honored. We respect robots.txt and stop when it is unavailable or denies access.
• Public HTTP status, TLS verification, latency, x402 challenge fields, domain registration age and OFAC payee matches are recorded. Endpoint response bodies, cookies and request bodies are not retained.
• Scout stores the latest public listing metadata, aggregate 30-day activity, quote and screening observation in D1; it removes listings after 30 days without rediscovery. Scout never stores caller search text or request bodies. The separate Health Index history covers seven days (up to 28 observations per resource); cloud snapshots expire within 48 hours if refreshes stop. Public RDAP dates are reused for seven days. Older public observations may remain in a stopped snapshot for up to nine days; local development snapshots may be retained for troubleshooting. Missing or skipped probes remain unknown.

To exclude an endpoint, publish User-agent: SpendPreflight-Crawler and Disallow: /your-path in robots.txt. For corrections or removal, contact contact@spendpreflight.com.

Privacy · Terms