{"resource":"https://api.vextorium.com/v1/robots-txt","service":"Vextorium","areas":["web_search","ai_inference"],"answering":true,"http_status":402,"latency_ms":96,"price_usdc":0.003,"buyers_30d":2,"calls_30d":2,"description":"Parse a site's robots.txt: whether a user-agent may crawl a path, which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot...) are blocked, and declared sitemaps. Check before your agent scrapes.","id":"d0dd22793dba","category":"Web page readers and scrapers","accuracy":null,"tool_calls":null,"checked_day":"2026-10-11","page":"https://velliumandco.com/sellers/d0dd22793dba"}