BOT PROFILE

CCBot

Common Crawl AI crawler
Can be verified
← Browse bot directory

Overview

Operator / organisation
Common Crawl
Category
AI crawler
Purposes
AI training, other automated activity
Intelligence checked

CCBot is operated by Common Crawl for AI training and other automated activity.

What does CCBot do?

AI training

AI training crawlers collect material for use in developing models. This purpose is distinct from returning search results or fetching a page in response to a user request.

Verification and limitations

RequestIntel supports independent verification of requests claiming to be CCBot. Availability does not mean every request using this identity is verified; check the result for each request.

Identity verification does not grant access to your site. Apply your own access policy after evaluating the request.

Common Crawl crawler documentation →

Check a CCBot request

Send the request to RequestIntel and get the identity result back.