BOT DIRECTORY
Known bots and crawlers
See which bots and crawlers RequestIntel recognises, who runs them, and whether they can be verified.
Check a crawler request →| Bot | Organisation | Category | Verification |
|---|---|---|---|
| AhrefsBot | Ahrefs | SEO crawler | Can be verified |
| Amazonbot | Amazon | AI crawler | Can be verified |
| Amzn-SearchBot | Amazon | Search crawler | Can be verified |
| Amzn-User | Amazon | AI crawler | Can be verified |
| ClaudeBot | Anthropic | AI crawler | Can be verified |
| Claude-SearchBot | Anthropic | AI crawler | Can be verified |
| Claude-User | Anthropic | AI crawler | Can be verified |
| Applebot | Apple | Search crawler | Can be verified |
| iTMS | Apple | Other bot | Can be verified |
| Baiduspider | Baidu | Search crawler | Can be verified |
| Better Uptime Bot | Better Uptime | Monitoring bot | Can be verified |
| Bytespider | ByteDance | AI crawler | Identification only |
| CensysInspect | Censys | Security scanner | Can be verified |
| CCBot | Common Crawl | AI crawler | Can be verified |
| Datadog Synthetics | Datadog | Monitoring bot | Can be verified |
| DataForSeoBot | DataForSEO | SEO crawler | Can be verified |
| Discordbot | Discord | Social bot | Can be verified |
| DuckAssistBot | DuckDuckGo | AI crawler | Can be verified |
| DuckDuckBot | DuckDuckGo | Search crawler | Can be verified |
| AdsBot-Google | Search crawler | Can be verified |