Robots.txt Tester

Inspect how hacecuentas.com controls crawler access, blocked paths, sitemap references and AI crawler rules.

Preview

Score: 92

  • At least one user-agent has Disallow: / which blocks the entire site.

Get your full report + exact fixes

See what’s hurting your SEO and how to fix it step by step.

  • Full breakdown
  • Actionable fixes
  • Prioritized next steps

No spam. One email with your report and next steps.

Robots.txt Status Present Score 92 /100 · Strong
Domain hacecuentas.com
robots.txt URL https://hacecuentas.com/robots.txt
Last analyzed June 17, 2026

View Full Robots.txt

Open robots.txt in a new tab

Download robots.txt

Robots.txt Content Preview

# Hacé Cuentas — robots.txt
#
# Política 2026 (revisada 2026-05-01):
# Maximizar visibilidad en search ENGINES tradicionales + AI search
# (ChatGPT, Claude, Gemini, Perplexity, etc.). En 2026 el tráfico
# orgánico tradicional cae mientras AI search crece 10x — conviene
# permitir LLMs para ser citados como fuente.
#
# Bloqueamos solo scrapers comerciales sin valor SEO (Bytespider,
# Diffbot, Omgili, etc.) y rutas técnicas internas.

# ──────── Default (todos los bots) ────────
User-agent: *
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

# ──────── Search engines tradicionales ────────
User-agent: Googlebot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

User-agent: Bingbot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

User-agent: DuckDuckBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

User-agent: Applebot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

User-agent: YandexBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

# ──────── AI / LLM search bots — PERMITIDOS (cambio de política 2026-05-01) ────────
# Razón: aparecer en AI Overviews + ChatGPT + Claude + Perplexity respuestas.
# El tráfico desde AI search convierte 2-27x premium vs organic search.

# OpenAI: GPTBot (training), ChatGPT-User (live citations), OAI-SearchBot (search)
User-agent: GPTBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

User-agent: ChatGPT-User
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

User-agent: OAI-SearchBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Anthropic: ClaudeBot (training), Claude-Web (research mode), anthropic-ai (legacy)
User-agent: ClaudeBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

User-agent: Claude-Web
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

User-agent: anthropic-ai
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Google Gemini (separado de Googlebot — opt-in explícito para AI training)
User-agent: Google-Extended
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

# Perplexity (search engine basado en LLMs, alta conversión)
User-agent: PerplexityBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

User-agent: Perplexity-User
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Apple Intelligence (Siri, Apple Search)
User-agent: Applebot-Extended
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Common Crawl — base de training data de muchos LLMs públicos (Llama, Mistral, etc.)
User-agent: CCBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Meta (Llama models)
User-agent: Meta-ExternalAgent
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

User-agent: FacebookBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# You.com
User-agent: YouBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Amazon (Amazon Q + Alexa AI — ahora cita fuentes y manda tráfico,
# revisado 2026-05-13: mismo tratamiento que GPTBot / ClaudeBot)
User-agent: Amazonbot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/
Disallow: /embed/
Disallow: /embed.js
Disallow: /search-index.json

# ──────── Search engines regionales (LATAM-relevant via diaspora + viajes) ────────

# Naver (Corea — usa búsqueda propia, no Bing)
User-agent: Yeti
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Seznam.cz (Chequia — buscador propio top en CZ)
User-agent: SeznamBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Google adicional bots (AdsBot, Mobile, etc)
User-agent: GoogleOther
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Huawei Petal Search (mercados emergentes)
User-agent: PetalBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Mojeek (independent search engine, privacy-focused — usa crawler propio)
User-agent: MojeekBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Sogou (Tencent search — usuarios chinos en LATAM)
User-agent: Sogou web spider
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Exalead (Dassault Systèmes, used in France/EU)
User-agent: Exabot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# ──────── AI search emergentes 2026 ────────

# Yep (Ahrefs new search engine, 2024+)
User-agent: YepBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Phind (dev-focused AI search)
User-agent: PhindBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Andi (conversational AI search)
User-agent: Andibot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Komo AI
User-agent: KomoBot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Qwant (Francia — independent EU search)
User-agent: Qwantify
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Marginalia Search (independent, slow web)
User-agent: search.marginalia.nu
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# Stract (open source search engine)
User-agent: stractbot
Allow: /
Allow: /api/calcs-index.json
Allow: /api/calc/
Disallow: /admin
Disallow: /api/

# ──────── Bloqueados: scrapers comerciales sin valor SEO ────────
# Estos no traen tráfico ni mejoran AI visibility — solo consumen ancho de banda.

# Bytespider (TikTok/ByteDance scraper — no agrega tráfico, opaco)
User-agent: Bytespider
Disallow: /

# Diffbot (commercial data scraper)
User-agent: Diffbot
Disallow: /

# ImagesiftBot (image scraping comercial)
User-agent: ImagesiftBot
Disallow: /

# Omgili / Omgilibot (scraping for sale, no valor SEO)
User-agent: Omgilibot
Disallow: /

User-agent: Omgili
Disallow: /

# Timpibot (AI scraper poco transparente)
User-agent: Timpibot
Disallow: /

# Cohere (no es buscador público, training-only)
User-agent: cohere-ai
Disallow: /

User-agent: cohere-training-data-crawler
Disallow: /

# Mistral (Le Chat — buscador IA público con citación). Permitido explícitamente
# (paridad con .well-known/llms-allowed.txt).
User-agent: MistralAI-User
Allow: /

# ──────── Sitemap ────────
# Index principal (Google sigue los sub-sitemaps automáticamente vía sitemap index).
Sitemap: https://hacecuentas.com/sitemap.xml
# Sitemap-fresh: URLs modificadas en últimos 14 días — Bing/Yandex freshness
Sitemap: https://hacecuentas.com/sitemap-fresh.xml
# RSS feed: discovery alternativo para Yandex/Seznam/Naver y AI engines (Claude/Perplexity)
# que parsean RSS para detectar contenido nuevo. Google lo ignora silenciosamente
# (acepta solo XML sitemaps válidos) — cero impacto en crawl budget de Google.
Sitemap: https://hacecuentas.com/rss.xml
# Sub-sitemaps geo + idioma declarados explícitamente: Bing y otros crawlers
# que no auto-procesan sitemap-index los descubren acá directo. Resuelve el
# warning "Important pages missing in sitemaps" de BWT (2026-05-25).
Sitemap: https://hacecuentas.com/sitemap-co.xml
Sitemap: https://hacecuentas.com/sitemap-cl.xml
Sitemap: https://hacecuentas.com/sitemap-mx.xml
Sitemap: https://hacecuentas.com/sitemap-es.xml
Sitemap: https://hacecuentas.com/sitemap-en.xml
Sitemap: https://hacecuentas.com/sitemap-pt.xml
Sitemap: https://hacecuentas.com/sitemap-argentina.xml
Sitemap: https://hacecuentas.com/sitemap-iibb.xml

User-agent Rules

User-agent(s) Allowed paths Disallowed paths
*
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
googlebot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
bingbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
duckduckbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
applebot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
yandexbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
gptbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
chatgpt-user
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
oai-searchbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
claudebot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
claude-web
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
anthropic-ai
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
google-extended
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
perplexitybot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
perplexity-user
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
applebot-extended
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
ccbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
meta-externalagent
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
facebookbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
youbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
amazonbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
yeti
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
seznambot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
googleother
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
petalbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
mojeekbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
sogou web spider
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
exabot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
yepbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
phindbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
andibot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
komobot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
qwantify
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
search.marginalia.nu
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
stractbot
  • /
  • /api/calcs-index.json
  • /api/calc/
  • /admin
  • /api/
bytespider No explicit Allow rules.
  • /
diffbot No explicit Allow rules.
  • /
imagesiftbot No explicit Allow rules.
  • /
omgilibot No explicit Allow rules.
  • /
omgili No explicit Allow rules.
  • /
timpibot No explicit Allow rules.
  • /
cohere-ai No explicit Allow rules.
  • /
cohere-training-data-crawler No explicit Allow rules.
  • /
mistralai-user
  • /
 No explicit Disallow rules.

Blocked and Allowed Paths

Blocked paths
  • /admin
  • /api/
  • /embed/
  • /embed.js
  • /search-index.json
  • /
Allowed paths
  • /
  • /api/calcs-index.json
  • /api/calc/
Crawl-delay No Crawl-delay directive detected.

Sitemaps Detected

AI Crawler Policy

At least one AI crawler (such as GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot or Google-Extended) appears to be blocked by robots.txt.

Issues Found

  • At least one user-agent has Disallow: / which blocks the entire site.

Recommendations

  • Avoid blocking the entire site (Disallow: /); restrict only sensitive or low-value paths instead.
  • Review your AI crawler policy for GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot and Google-Extended to ensure it matches your content strategy.
  • Ensure important pages, CSS and JavaScript assets are crawlable so search engines can fully render your site.

Analyze this site with other tools

Want a website that actually generates leads?

Start a conversion-focused website project with a team that builds fast, SEO-optimized sites for real businesses.

Start project