User-agent: * # Uploaded customer files (project documents, training videos, page images). # Never search results; the directory is generic Django convention, so naming # it here reveals nothing that /media/ does not already imply. Disallow: /media/ # Everything else is deliberately absent. Private areas are protected by # authentication and carry a `noindex` meta tag, which is the directive that # actually keeps a page out of the index. Listing those paths here would be # counterproductive twice over: a crawler that is disallowed never fetches the # page, so it never sees the `noindex` - and the file itself is public, so each # entry would advertise an internal endpoint to anyone who reads it. Sitemap: https://breev.ai/sitemap.xml Sitemap: https://breev.ai/de/sitemap.xml Sitemap: https://breev.ai/en/sitemap.xml