Crawl Activity
SEO Compare — seocompare.co.uk
Bing crawl ran 15h ago · 23 Apr, 06:00 · runs daily 06:00
5
Total Bing Crawls
0
Pages Indexed (Bing)
15
AI Bots Allowed
0
AI Bots Blocked
Crawler Activity
Bing (bars, left axis) vs AI bots (lines, right axis)
AI Crawler Access
GPTBot allowed
ChatGPT-User allowed
Google-Extended allowed
GoogleOther allowed
ClaudeBot allowed
Claude-Web allowed
PerplexityBot allowed
Bytespider allowed
CCBot allowed
cohere-ai allowed
Amazonbot allowed
anthropic-ai allowed
FacebookBot allowed
Applebot-Extended allowed
Bingbot allowed
Bing Index Growth
Server-Side Bot Hits
Real crawler visits detected at the server level (last 11 days)
4815
Total Bot Hits
457
AI Crawler Hits
600
Search Crawler Hits
15
Unique Bots
Bot Breakdown
Other Crawler 3360
Applebot 420
Other Bot 225
MajesticBot 164
GPTBot 158
PerplexityBot 137
YandexBot 106
ClaudeBot 76
ChatGPT-User 73
Googlebot 51
Bingbot 23
GoogleOther 12
Facebook 8
WhatsApp 1
Bytespider 1
Daily Hits
23 Apr
151 AI 17 search 6 seo 258
22 Apr
11 AI 31 search 4 seo 689
21 Apr
41 AI 12 search 1 social 4 seo 702
20 Apr
58 AI 51 search 1 social 4 seo 435
19 Apr
20 AI 119 search 1 social 12 seo 157
18 Apr
30 AI 35 search 1 social 23 seo 415
17 Apr
83 AI 112 search 2 social 4 seo 520
16 Apr
7 AI 85 search 1 social 8 seo 689
15 Apr
23 AI 87 search 1 social 12 seo 522
14 Apr
18 AI 28 search 1 social 4 seo 78
13 Apr
15 AI 23 search 83 seo 350
Live Crawler Access Test (HTTP Response)
Tests if each bot actually gets a 200 response when visiting the homepage
GPTBot 200
ClaudeBot 200
PerplexityBot 200
Googlebot 200
Bingbot 200
Browser 200
Topic Clusters (4 detected)
/blog/
32
pages in cluster
/questions/
42
pages in cluster
/insights/
8
pages in cluster
/stats/
21
pages in cluster
Terminology Consistency (H1 Analysis)
search
123x (38.3%)
agencies
96x (29.9%)
visibility
43x (13.4%)
best
39x (12.1%)
chatgpt
30x (9.3%)
ratings
28x (8.7%)
compared
26x (8.1%)
audit
25x (7.8%)
check
23x (7.2%)
optimisation
22x (6.9%)