### Marrow Fitness — robots.txt
### Last updated 2026-07-01. See /llms.txt and /ai.txt for the AI usage policy.
###
### Policy: AI answer engines and AI assistants are ALLOWED on the public
### marketing surface so Marrow is found, summarized, and cited when a gym
### owner or coach asks an assistant what software to buy. That includes
### GPTBot, ClaudeBot, PerplexityBot, and the other assistant crawlers,
### which inherit the generic rules below: public pages allowed,
### authenticated, admin, and internal surfaces disallowed. Machine-readable
### pricing lives at /pricing.json. Pure dataset scrapers with no answer
### surface stay blocked.

# ----- Dataset scrapers with no answer surface: NOT permitted -----------
User-agent: CCBot
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: Meta-ExternalFetcher
Disallow: /

User-agent: FacebookBot
Disallow: /

User-agent: cohere-training-data-crawler
Disallow: /

User-agent: Diffbot
Disallow: /

User-agent: ImagesiftBot
Disallow: /

User-agent: Omgilibot
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: Timpibot
Disallow: /

User-agent: PanguBot
Disallow: /

User-agent: PetalBot
Disallow: /

# ----- Everyone else, including AI answer engines and assistants --------
# GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-Web,
# Claude-SearchBot, anthropic-ai, PerplexityBot, PerplexityUser,
# Google-Extended, DuckAssistBot, YouBot, Amazonbot, AI2Bot and any
# other crawler follow these rules: public marketing pages are open,
# private surfaces are not.
User-agent: *
Allow: /

# Post-submit pages (no SEO value, content only relevant after a form submit)
Disallow: /thank-you/
Disallow: /lead-magnet/*/thanks/

# Internal sales kit hub (password gated, not public)
Disallow: /kit/

# Killed surfaces. Internal, dev, product, dynamic, campaign, and locale paths
# pulled off the public marketing surface per the brand simplification audit
# (2026-06-02). None of these may be indexed.
Disallow: /admin/
Disallow: /tests/
Disallow: /runbooks/
Disallow: /scripts/
Disallow: /transcripts/
Disallow: /docs/
Disallow: /gtm/
Disallow: /marketing/
Disallow: /sales/
Disallow: /migrations/
Disallow: /lead-magnet/
Disallow: /miami/
Disallow: /method/
Disallow: /properties/
Disallow: /roadmap/
Disallow: /status/
Disallow: /report/
Disallow: /account/
Disallow: /learn/
Disallow: /ondemand/
Disallow: /marketplace/
Disallow: /community/
Disallow: /pt/
Disallow: /es/
Disallow: /fr/
Disallow: /it/
Disallow: /ru/
Disallow: /b/
Disallow: /g/
Disallow: /c/
Disallow: /vs/
Disallow: /press-kit/
Disallow: /affiliate/
Disallow: /affiliates/
Disallow: /refer/

# Auth + functional surfaces (no public content)
Disallow: /auth/
Disallow: /coach/dashboard/
Disallow: /coach/athletes/
Disallow: /coach/crm/
Disallow: /coach/programs/builder/
Disallow: /coach/nutrition/
Disallow: /coach/retention/
Disallow: /coach/onboarded/
Disallow: /coach/onboarding/wizard/
Disallow: /coach/dashboard/migrations/
Disallow: /athlete/dashboard/
Disallow: /athlete/nutrition/
Disallow: /athlete/onboarding/wizard/
Disallow: /gyms/dashboard/
Disallow: /community/host/verify/

# Honeytrap routes. Real visitors never reach these. Reaching one identifies
# you as automation that ignores robots directives.
Disallow: /internal/coach-roster-export.json
Disallow: /api/v0/internal/dump
Disallow: /admin/coaches.csv
Disallow: /api/scrape-me
Disallow: /.well-known/marrow-sitemap-private.xml

Sitemap: https://marrowfitness.com/sitemap.xml