Robots.txt Tester

Inspect how graventonsolutions.com controls crawler access, blocked paths, sitemap references and AI crawler rules.

Preview

Score: 92

Blocking CSS or JS may prevent search engines from rendering pages correctly.

Get your full report + exact fixes

See what’s hurting your SEO and how to fix it step by step.

Full breakdown
Actionable fixes
Prioritized next steps

Robots.txt Status Present Score 92 /100 · Strong

Domain graventonsolutions.com

robots.txt URL https://graventonsolutions.com/robots.txt

Last analyzed June 16, 2026

View Full Robots.txt

Open robots.txt in a new tab

Download robots.txt

Robots.txt Content Preview

User-agent: *
Allow: /

# ââ Private / non-public paths âââââââââââââââââââââââââââ
Disallow: /admin/
Disallow: /assets/inc/
Disallow: /assets/data/
Disallow: /assets/phpmailer/
Disallow: /db/
Disallow: /.env
Disallow: /.env.example

# ââ Utility / partial HTML (not real pages) ââââââââââââââ
Disallow: /header.html
Disallow: /footer.html
Disallow: /thankyou.html
Disallow: /test.html
Disallow: /test1.html
Disallow: /testnav.html

# ââ AI crawlers â explicitly allowed for GEO âââââââââââââ
User-agent: GPTBot
Allow: /
Disallow: /admin/

User-agent: ChatGPT-User
Allow: /
Disallow: /admin/

User-agent: ClaudeBot
Allow: /
Disallow: /admin/

User-agent: Claude-Web
Allow: /
Disallow: /admin/

User-agent: anthropic-ai
Allow: /
Disallow: /admin/

User-agent: PerplexityBot
Allow: /
Disallow: /admin/

User-agent: Googlebot-Extended
Allow: /
Disallow: /admin/

User-agent: cohere-ai
Allow: /
Disallow: /admin/

User-agent: Meta-ExternalAgent
Allow: /
Disallow: /admin/

User-agent: Applebot-Extended
Allow: /
Disallow: /admin/

# ââ Sitemap âââââââââââââââââââââââââââââââââââââââââââââââ
Sitemap: https://www.graventonsolutions.com/sitemap.xml

User-agent Rules

User-agent(s)	Allowed paths	Disallowed paths
*	/	/admin/ /assets/inc/ /assets/data/ /assets/phpmailer/ /db/ /.env /.env.example /header.html /footer.html /thankyou.html /test.html /test1.html /testnav.html
gptbot	/	/admin/
chatgpt-user	/	/admin/
claudebot	/	/admin/
claude-web	/	/admin/
anthropic-ai	/	/admin/
perplexitybot	/	/admin/
googlebot-extended	/	/admin/
cohere-ai	/	/admin/
meta-externalagent	/	/admin/
applebot-extended	/	/admin/

Blocked and Allowed Paths

Blocked paths	/admin/ /assets/inc/ /assets/data/ /assets/phpmailer/ /db/ /.env /.env.example /header.html /footer.html /thankyou.html /test.html /test1.html /testnav.html
Allowed paths	/
Crawl-delay	No Crawl-delay directive detected.

Sitemaps Detected

https://www.graventonsolutions.com/sitemap.xml

AI Crawler Policy

At least one AI crawler (such as GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot or Google-Extended) appears to be blocked by robots.txt.

Issues Found

Blocking CSS or JS may prevent search engines from rendering pages correctly.

Recommendations

Review your AI crawler policy for GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot and Google-Extended to ensure it matches your content strategy.
Ensure important pages, CSS and JavaScript assets are crawlable so search engines can fully render your site.