Overview
- Operator / organisation
- Common Crawl
- Category
- AI crawler
- Purposes
- AI training, other automated activity
- Intelligence checked
CCBot is operated by Common Crawl for AI training and other automated activity.
What does CCBot do?
AI training
AI training crawlers collect material for use in developing models. This purpose is distinct from returning search results or fetching a page in response to a user request.
Verification and limitations
RequestIntel supports independent verification of requests claiming to be CCBot. Availability does not mean every request using this identity is verified; check the result for each request.
Identity verification does not grant access to your site. Apply your own access policy after evaluating the request.
Common Crawl crawler documentation →Check a CCBot request
Send the request to RequestIntel and get the identity result back.