{"resource":"https://api.vextorium.com/robots-txt-pago","service":"Vextorium","areas":["web_search","ai_inference"],"answering":true,"http_status":402,"latency_ms":78,"price_usdc":0.003,"buyers_30d":3,"calls_30d":3,"description":"Parse a site's robots.txt: whether a user-agent may crawl a path, which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot...) are blocked, and declared sitemaps. Check before your agent scrapes.","id":"da98e5c7ef6b","category":"Web page readers and scrapers","checked_day":"2026-10-11","page":"https://velliumandco.com/sellers/da98e5c7ef6b"}