REQUESTINTEL API

RequestIntel API

Identify supported bots and crawlers, then understand whether a claimed identity is verified, unverified, or spoofed. Inspection data is processed ephemerally and is not stored.

Base URL

https://api.requestintel.dev/v1
Try a one-off crawler check →

Quickstart

Send a User-Agent and, when available, an IP address to /v1/inspect using your RequestIntel API key.

Quickstart
curl -X POST https://api.requestintel.dev/v1/inspect \
  -H "Authorization: Bearer ri_live_xxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "ip": "20.171.207.12",
    "user_agent": "GPTBot/1.2"
  }'
Response
{
  "request_id": "req_01K...",
  "is_bot": true,
  "bot": {
    "id": "openai-gptbot",
    "name": "GPTBot",
    "organization": "OpenAI",
    "category": "ai_crawler",
    "purposes": ["training"]
  },
  "identity": {
    "status": "verified",
    "signals": [
      { "type": "user_agent", "result": "match" },
      { "type": "official_ip_range", "result": "match" }
    ]
  },
  "network": null
}

Signed bot requests

ChatGPT Agent and Google-Agent can supply Signature, Signature-Input, and Signature-Agent headers. Forward their exact values, every covered header, and the original method and absolute URL. RequestIntel checks the signature against the operator's public keys; header presence alone does not verify a bot. An ordinary User-Agent plus a supported signer header can identify ChatGPT Agent from its signer header.

Signed request examples
{
  "user_agent": "Mozilla/5.0",
  "original_request": {
    "method": "GET",
    "url": "https://your-site.example/article"
  },
  "headers": {
    "Signature-Agent": "\"https://chatgpt.com\"",
    "Signature-Input": "sig1=(\"@authority\" \"signature-agent\" \"@method\");created=1791158400;expires=1791158700;keyid=\"<operator-key-id>\";alg=\"ed25519\";tag=\"web-bot-auth\"",
    "Signature": "sig1=:<base64-signature-from-incoming-request>:"
  }
}

The JSON values illustrate the header format. Replace the signature, key ID, times, label, and covered components with values from the incoming request; these examples are not valid signatures. Missing, expired, or invalid signatures do not establish verification. Forward the externally visible URL before proxy rewrites. Only signed components are authenticated; signature verification does not guarantee freshness against replay within its validity window or authenticate an unsigned path, IP, or body. DuckAssistBot signature verification awaits official signer provenance; its existing IP verification still applies.

Authentication

Authenticate every API request with an active API key created in your RequestIntel account.

Authorization: Bearer ri_live_xxxxxxxxx

Treat API keys as secrets and never expose them in client-side code.

POST/v1/inspect

Inspect one request.

User-Agent identifies the claimed crawler. Send a non-empty User-Agent and include an IP address when available. For signed bots, also forward the original request details and covered headers.

Request body
{
  "ip": "20.171.207.12",
  "user_agent": "GPTBot/1.4"
}
Response
{
  "request_id": "req_01K...",
  "is_bot": true,
  "bot": {
    "id": "openai-gptbot",
    "name": "GPTBot",
    "organization": "OpenAI",
    "category": "ai_crawler",
    "purposes": ["training"]
  },
  "identity": {
    "status": "verified",
    "signals": [
      { "type": "user_agent", "result": "match" },
      { "type": "official_ip_range", "result": "match" }
    ]
  },
  "network": {
    "asn": 8075,
    "organization": "Microsoft Corporation"
  }
}
ParameterTypeRequiredDescription
ipstringNoIPv4 or IPv6 address to inspect. Include it when available for network verification.
user_agentstringYesCopy the incoming request's User-Agent header, for example GPTBot/1.2. This field cannot be missing or empty.
original_requestobjectNoOptional for inspection. To verify a signature, include both method and url from the original request before rewrites, for example GET and https://your-site.example/article.
headersobjectNoOptional for inspection. To verify a signature, include Signature, Signature-Input, Signature-Agent and every covered header, copied exactly from the incoming request.

When no supported bot is identified, bot, identity, and network are null. is_bot: false does not prove the request came from a human.

Integration examples

Choose your server runtime. Every example calls /v1/inspect directly with native fetch.

›Show integration examples

Node.js 20+: inspect a request

Run this on your server with native fetch. Modern Node/server runtimes support the included five-second AbortSignal timeout.

Node.js 20+: inspect a request
// Node.js 20+ (run this only on your server)
async function main() {
  const apiKey = process.env.REQUESTINTEL_API_KEY;
  if (!apiKey) {
    console.error("REQUESTINTEL_API_KEY is not configured");
    process.exitCode = 1;
    return;
  }

  const response = await fetch("https://api.requestintel.dev/v1/inspect", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      ip: "203.0.113.24",
      user_agent: "Mozilla/5.0 ...",
    }),
    signal: AbortSignal.timeout(5000),
  });

  const result = await response.json().catch(() => null);
  if (!response.ok) {
    console.error(result?.error?.message ?? `Request failed: ${response.status}`);
    process.exitCode = 1;
    return;
  }

  console.log(result.identity?.status ?? "not a supported bot");
}

main().catch(() => {
  console.error("RequestIntel could not be reached. Check the network and try again.");
  process.exitCode = 1;
});

The Node.js examples use AbortSignal.timeout(5000), which is available in modern Node/server runtimes. Omit it or use your platform's cancellation support in runtimes that do not provide it.

MCP integration

Integrate RequestIntel with compatible development tools using the Model Context Protocol (MCP). Access the same crawler verification functionality available through our REST API.

Endpoint

https://api.requestintel.dev/mcp

Tool

inspect_request

Client setup

Open the setup for the assistant you use. Keep your API key in an environment variable, never a repository, prompt, or AGENTS.md file.

›Add RequestIntel to Codex

Codex supports remote HTTP MCP servers. The base command is codex mcp add requestintel --url https://api.requestintel.dev/mcp. RequestIntel also needs bearer authentication, so use this complete command instead.

Codex setup
export REQUESTINTEL_API_KEY="ri_live_..."

codex mcp add requestintel --url https://api.requestintel.dev/mcp \
  --bearer-token-env-var REQUESTINTEL_API_KEY

codex mcp list

Now ask: Use RequestIntel to check whether this request is really GPTBot: IP: 20.171.207.12 User-Agent: GPTBot/1.4

Optional: add Use the RequestIntel MCP server whenever you need to identify or verify a web bot or crawler. to AGENTS.md. Do not put an API key there.

›Add RequestIntel to Claude Code

Claude Code supports remote HTTP MCP servers. Its basic command is claude mcp add --transport http requestintel https://api.requestintel.dev/mcp --header "Authorization: Bearer …". For RequestIntel, add the following user-level configuration to ~/.claude.json instead. It reads the bearer token at runtime from the environment that launches Claude Code.

~/.claude.json
{
  "mcpServers": {
    "requestintel": {
      "type": "http",
      "url": "https://api.requestintel.dev/mcp",
      "headersHelper": "printf '{\"Authorization\":\"Bearer %s\"}' \"$REQUESTINTEL_API_KEY\""
    }
  }
}

Set REQUESTINTEL_API_KEY in that environment, then run claude mcp list. Inside Claude Code, use /mcp to confirm the connection.

Now ask: Use RequestIntel to check whether this IP and User-Agent belong to a verified crawler.

›Add RequestIntel to Cursor

Cursor reads global MCP configuration from ~/.cursor/mcp.json. Add this configuration and set REQUESTINTEL_API_KEY in the environment Cursor starts with. This keeps the key out of a project file.

~/.cursor/mcp.json
{
  "mcpServers": {
    "requestintel": {
      "url": "https://api.requestintel.dev/mcp",
      "headers": {
        "Authorization": "Bearer ${env:REQUESTINTEL_API_KEY}"
      }
    }
  }
}

Restart or reload Cursor, then ask: Check this request with RequestIntel and tell me whether the crawler identity is verified.

MCP Reference

Endpoint: https://api.requestintel.dev/mcp. Authenticate with Authorization: Bearer ri_live_….

inspect_request arguments
{
  "ip": "20.171.207.12",
  "user_agent": "GPTBot/1.4"
}

Pass a non-empty User-Agent, optional IP address, and original signature headers/request details when available. IP alone cannot establish an identity.

MCP calls use the same RequestIntel account rate limits, monthly lookup allowance, lookup accounting, and verified, unverified, and spoofed identity semantics as POST /v1/inspect.

POST/v1/inspect/batch

Inspect up to 100 requests at once.

Every item must include a non-empty User-Agent and may include an IP address or original signature details. The output order matches the input order.

Request body
{
  "requests": [
    {
      "ip": "20.171.207.12",
      "user_agent": "GPTBot/1.2"
    },
    {
      "ip": "192.0.2.7",
      "user_agent": "Mozilla/5.0"
    }
  ]
}
Response
{
  "results": [
    {
      "request_id": "req_01K...",
      "is_bot": true,
      "bot": {
        "id": "openai-gptbot",
        "name": "GPTBot",
        "organization": "OpenAI",
        "category": "ai_crawler",
        "purposes": ["training"]
      },
      "identity": {
        "status": "verified",
        "signals": [
          { "type": "user_agent", "result": "match" },
          { "type": "official_ip_range", "result": "match" }
        ]
      },
      "network": null
    }
  ]
}

Integration examples

Choose your server runtime to send a batch directly to /v1/inspect/batch.

›Show integration examples

Node.js 20+: batch inspection

Send up to 100 inspections in one request with native fetch.

Node.js 20+: batch inspection
async function main() {
  const apiKey = process.env.REQUESTINTEL_API_KEY;
  if (!apiKey) {
    console.error("REQUESTINTEL_API_KEY is not configured");
    process.exitCode = 1;
    return;
  }

  const requests = [
    { ip: "203.0.113.24", user_agent: "Mozilla/5.0 ..." },
    { ip: "2001:db8::24", user_agent: "ExampleBot/1.0" },
  ];
  if (requests.length > 100) {
    console.error("Batch requests are limited to 100 items");
    process.exitCode = 1;
    return;
  }

  const response = await fetch("https://api.requestintel.dev/v1/inspect/batch", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ requests }),
    signal: AbortSignal.timeout(5000),
  });

  const result = await response.json().catch(() => null);
  if (!response.ok) {
    console.error(result?.error?.message ?? `Request failed: ${response.status}`);
    process.exitCode = 1;
    return;
  }

  // result.results is in the same order as requests.
}

main().catch(() => {
  console.error("RequestIntel could not be reached. Check the network and try again.");
  process.exitCode = 1;
});

A batch containing 100 items is one HTTP request for rate limiting, but consumes 100 API lookups from the account's monthly allowance.

Batch quota checks are atomic. If the account does not have enough remaining monthly allowance for the entire batch, the whole batch is rejected before inspection begins. No items are processed and no lookup allowance is consumed. For example, if an account has 7 lookups remaining and sends a batch containing 10 items, the full batch is rejected.

POST/v1/ua

Match one User-Agent.

Match one User-Agent against RequestIntel's supported crawler identities. This is claim matching only: it does not perform IP, DNS, or ASN verification.

Request body
{
  "user_agent": "GPTBot/1.2"
}
Response
{
  "matched": true,
  "bots": [
    {
      "id": "openai-gptbot",
      "name": "GPTBot",
      "organization": "OpenAI",
      "category": "ai_crawler"
    }
  ]
}

The response contains at most one bot, selected using RequestIntel's deterministic User-Agent match. A result with matched: false returns an empty bots array.

POST/v1/ip

Match one IP address.

Match one IPv4 or IPv6 address against supported official crawler network ranges. This is range matching only: it does not establish that a crawler claim is verified.

Request body
{
  "ip": "20.171.207.12"
}
Response
{
  "matched": true,
  "bots": [
    {
      "id": "openai-gptbot",
      "name": "GPTBot",
      "organization": "OpenAI",
      "category": "ai_crawler"
    }
  ]
}

An IP address can match more than one supported bot. In that case, every matched bot is returned. RequestIntel does not expose the ranges or source records used for matching.

Response meanings

These definitions apply to every response returned by /v1/inspect and each item in /v1/inspect/batch.

is_bot: true means RequestIntel matched the request to a supported known bot or crawler identity. is_bot: false means RequestIntel did not match the request to a supported known bot or crawler identity; it does not prove that the request came from a human. In that case, bot, identity, and network are null.

verified
The claimed bot identity was independently verified.
unverified
The request matched a known bot identity, but RequestIntel could not verify it.
spoofed
The claimed identity conflicts with an authoritative verification check.

Bot purposes

bot.purposes is an array describing what the bot is used for. A bot can have multiple purposes. When no supported bot is identified, bot is null.

ValueMeaning
trainingCollect content for AI model training.
searchRetrieve content to support search results or answers.
user_fetchFetch content in response to a user request.
indexingBuild or update a search index.
link_previewGenerate previews of shared links.
monitoringCheck availability, uptime or site behavior.
seoAnalyze websites for search-engine optimization.
securityScan websites for security issues.
scrapingExtract or collect data from websites.
otherOther bot activity outside these purposes.

bot.organization is the organisation operating the identified crawler.

network.organization is the organisation associated with the source IP's network/ASN. Network enrichment may be null, is not guaranteed for every request, and does not mean the crawler operator owns that infrastructure.

Errors & limits

StatusCodeMeaning
400invalid_requestThe request is missing or has invalid input.
401unauthorizedThe API key is missing or invalid.
429monthly_limit_exceededThe account has reached its monthly lookup allowance.

Request-rate limit

API requests are rate-limited per account at the rate included with your plan. All active API keys share that account-wide limit; a batch is one rate-limited HTTP request.

Monthly lookup allowance

Monthly usage is counted per inspected item at the account level. This allowance is separate from the request-rate limit.

Monthly-limit error response
{
  "error": {
    "code": "monthly_limit_exceeded",
    "message": "monthly limit exceeded"
  }
}

Monthly limits are hard limits. RequestIntel does not automatically charge overages. Browse supported bots and crawlers to see the public reference coverage.

Privacy

RequestIntel processes request data when you call the API. Raw inspected IPs, User-Agent strings, headers, URLs, request bodies, results, and per-request histories are not stored. Aggregate usage data is kept for billing and limits.