crawl-allowed-check
Is a URL crawlable? Fetches the site's robots.txt and evaluates the given path for a specific user-agent (Googlebot, GPTBot, ClaudeBot, Bingbot or any token; default *), returning allowed/disallowed, the matching rule, the group used, crawl
Go to this service →Our latest check (2026-10-11)
- Answering
- Yes — valid x402 payment terms
- Time to answer
- 81 ms
- Price
- $0.010 per call
- Paying buyers, last 30 days
- 2 (directory's own records)
- Paid calls, last 30 days
- 3
- Category
- Web page readers and scrapers
- Address
- https://intel.rallylive.ca/monitor/robots-allowed
Accuracy
Accuracy scores are published once a seller has enough paid checks against an independent record (at least 7 scored answers over at least 7 days). Until then no score is shown, and a missing score is never a pass.
For agents
This page as JSON: /sellers/e09648d28372.json. Paid lookup by address: GET /api/seller?resource=… ($0.001, x402).