{"resource":"https://web.openverbs.com/v1/robots","service":"Check robots.txt crawl rules","areas":["web_search"],"answering":true,"http_status":402,"latency_ms":410,"price_usdc":0.006,"buyers_30d":3,"calls_30d":3,"description":"Fetch a site's robots.txt and evaluate crawl permissions: given a URL (plus optional extra paths) and a user-agent, return whether each path is allowed or disallowed, the rule that matched, the user-agent group, the crawl-delay and any decl","id":"634d27c2f431","category":"Web page readers and scrapers","accuracy":null,"tool_calls":null,"checked_day":"2026-10-11","page":"https://velliumandco.com/sellers/634d27c2f431"}