Robots.txt Tester

Inspect how globalinvestment.ir controls crawler access, blocked paths, sitemap references and AI crawler rules.

Preview

Score: 92

Sitemap URL appears to be unreachable: https://globalinvestment.ir/sitemap.xml

Get your full report + exact fixes

See what’s hurting your SEO and how to fix it step by step.

Full breakdown
Actionable fixes
Prioritized next steps

Robots.txt Status Present Score 92 /100 · Strong

Domain globalinvestment.ir

robots.txt URL https://globalinvestment.ir/robots.txt

Last analyzed May 25, 2026

View Full Robots.txt

Open robots.txt in a new tab

Download robots.txt

Robots.txt Content Preview

# Global Investment Iran — robots policy
# Allows search engines and AI/answer engines to crawl, summarize and cite the site.

User-agent: *
Allow: /

# Generative & answer engines — explicit allow
User-agent: GPTBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: anthropic-ai
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: CCBot
Allow: /

User-agent: Bingbot
Allow: /

User-agent: DuckDuckBot
Allow: /

Sitemap: https://globalinvestment.ir/sitemap.xml

User-agent Rules

User-agent(s)	Allowed paths	Disallowed paths
*	/	No explicit Disallow rules.
gptbot	/	No explicit Disallow rules.
chatgpt-user	/	No explicit Disallow rules.
oai-searchbot	/	No explicit Disallow rules.
google-extended	/	No explicit Disallow rules.
perplexitybot	/	No explicit Disallow rules.
claudebot	/	No explicit Disallow rules.
anthropic-ai	/	No explicit Disallow rules.
applebot-extended	/	No explicit Disallow rules.
ccbot	/	No explicit Disallow rules.
bingbot	/	No explicit Disallow rules.
duckduckbot	/	No explicit Disallow rules.

Blocked and Allowed Paths

Blocked paths	No Disallow paths detected.
Allowed paths	/
Crawl-delay	No Crawl-delay directive detected.

Sitemaps Detected

https://globalinvestment.ir/sitemap.xml

AI Crawler Policy

No explicit blocks were detected for common AI crawlers (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended).

Issues Found

Sitemap URL appears to be unreachable: https://globalinvestment.ir/sitemap.xml

Recommendations

Document your AI crawler policy explicitly in robots.txt so future bots know how to treat your content.
Ensure important pages, CSS and JavaScript assets are crawlable so search engines can fully render your site.