WEB CRAWLER VERIFICATION API

Identify and verify web crawlers.

Identify known web crawlers from HTTP request information, verify claimed identities using supported IP and signature checks, and detect impersonated traffic through a REST API.
POST /v1/inspect
curl https://api.requestintel.dev/v1/inspect \
  -H "Authorization: Bearer ri_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "ip": "66.249.66.1",
    "user_agent": "Googlebot/2.1"
  }'
Response
{
  "is_bot": true,
  "bot": {
    "name": "Googlebot",
    "organization": "Google",
    "category": "search_crawler"
  },
  "identity": {
    "status": "verified"
  }
}
REST APIIPv4 + IPv6Batch lookupsVerified / unverified / spoofedRequest details discarded

SUPPORTED CRAWLERS

Explore supported web crawlers.

Browse supported crawlers, who runs them, and what kind of traffic they send.

CrawlerOrganisationType
GPTBotOpenAIAI crawler
OAI-SearchBotOpenAIAI crawler
ClaudeBotAnthropicAI crawler
GooglebotGoogleSearch crawler
bingbotMicrosoftSearch crawler
DiscordbotDiscordSocial bot
Browse crawler directory →

IDENTITY VERIFICATION

A claimed crawler identity needs verification.

RequestIntel separates the identity a request claims from whether that identity can actually be verified.

Claimed crawler

UNVERIFIED
Claimed identity
Known crawler
Matched
Verification
Not verified
Unverified

Identity: unverified

Verified crawler

VERIFIED
Claimed identity
Known crawler
Matched
Verification
Verified
Matched

Identity: verified

Send a request. Get an identity result.

01

Inspect

Send the request details to RequestIntel.

02

Identify

Match the request to a supported bot or crawler.

03

Verify

Check whether the claimed identity can be verified.

04

Return

Get a result such as verified, unverified, or spoofed.

PRIVACY

We don't store the request details you send us.

RequestIntel processes inspection data when you call the API. IP addresses, User-Agent strings, headers, URLs, request bodies, and per-request results are not stored. We keep aggregate usage totals for billing and limits.

  • No inspected IP history
  • No User-Agent history
  • No per-request result history
  • Aggregate usage only

Try it free with 1,000 lookups a month.

All plans use the same detection and verification. Paid plans give you more usage, API keys, and team members.

View pricing →

Free

1,000 API lookups / month

$0

Developer

25K API lookups / month

$7/month

Pro

250K API lookups / month

$15/month

REQUESTINTEL API

Know what's making the request before your app decides what to do.