api.strale.io
Fetch and parse a website's robots.txt. Returns rules per user-agent, sitemaps, crawl delays, and whether all crawlers are blocked.
Go to this service →Our latest check (2026-10-14)
- Answering
- Yes — valid x402 payment terms
- Time to answer
- 136 ms
- Price
- not stated
- Paying buyers, last 30 days
- 2 (directory's own records)
- Paid calls, last 30 days
- 5
- Category
- Web page readers and scrapers
- Address
- https://api.strale.io/x402/robots-txt-parse
Accuracy: paid answers checked against an independent record
Not yet in our paid checks. A seller we have not checked has neither passed nor failed.
For agents
This page as JSON: /sellers/4355668ebc1f.json. Paid lookup by address: GET /api/seller?resource=… ($0.001, x402).