BOT DIRECTORY
Known bots and crawlers
See which bots and crawlers RequestIntel recognises, who runs them, and whether they can be verified.
Check a crawler request →| Bot | Organisation | Category | Verification |
|---|---|---|---|
| bingbot | Microsoft | Search crawler | Can be verified |
| MistralAI-Index | Mistral AI | AI crawler | Can be verified |
| MistralAI-Training | Mistral AI | AI crawler | Identification only |
| MistralAI-User | Mistral AI | AI crawler | Can be verified |
| DotBot | Moz | SEO crawler | Identification only |
| ChatGPT Agent | OpenAI | AI crawler | Can be verified |
| ChatGPT-User | OpenAI | AI crawler | Can be verified |
| GPTBot | OpenAI | AI crawler | Can be verified |
| OAI-AdsBot | OpenAI | AI crawler | Can be verified |
| OAI-SearchBot | OpenAI | AI crawler | Can be verified |
| PerplexityBot | Perplexity | AI crawler | Can be verified |
| Perplexity-User | Perplexity | AI crawler | Can be verified |
| Pingdom | Pingdom | Monitoring bot | Can be verified |
| Pinterestbot | Social bot | Can be verified | |
| SemrushBot | Semrush | SEO crawler | Identification only |
| SEOkicks-Robot | SEOkicks | SEO crawler | Identification only |
| SISTRIX Crawler | SISTRIX | SEO crawler | Can be verified |
| Site24x7 | Site24x7 | Monitoring bot | Can be verified |
| Slackbot-LinkExpanding | Slack | Social bot | Identification only |
| StatusCake | StatusCake | Monitoring bot | Can be verified |