User-agent: * Disallow: /admin/ Disallow: /api/ Disallow: /settings/ Disallow: /dashboard/ # Block AI scrapers specifically User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: / # Allow Google Search to index the catalog User-agent: Googlebot Allow: / Allow: /flavor/ Allow: /category/ Allow: /profile/