User-agent: A6-Indexer User-agent: AcoonBot User-agent: AhrefsBot user-agent: Ahrefs Site Audit User-agent: Amazonbot User-agent: anthropic-ai User-agent: AwarioRssBot User-agent: AwarioSmartBot User-agent: archive.org_bot User-agent: AwarioSmartBot User-agent: Barkrowler User-agent: BLEXBot User-agent: BotBl User-agent: Botify User-agent: BrightEdge User-agent: btcrawler User-agent: Bytespider User-agent: CCBot User-agent: Chrome-Lighthouse User-agent: Cliqzbot User-agent: ClaudeBot User-agent: Claude-Web User-agent: cohere-ai User-agent: CrawlBot User-agent: Cxense User-agent: DataForSEO User-agent: DataForSeoBot User-agent: DeepCrawl User-agent: Diffbot User-agent: dotbot User-agent: DotBot User-agent: DomainCrawler User-agent: Exabot User-agent: Facebot User-agent: FacebookBot User-agent: facebookcatalog User-agent: facebookexternalhit User-agent: Google-Extended User-agent: ia_archiver User-agent: Linkdex User-agent: magpie-crawler User-agent: Majestic User-agent: Majestic-12 User-agent: meta-externalagent User-agent: MJ12bot User-agent: MojeekBot User-agent: Moz User-agent: Moz dotbot User-agent: NetpeakSpider User-agent: Nekstbot User-agent: Neevabot User-agent: omgili User-agent: omgilibot User-agent: OnPageBot User-agent: peer39_crawler User-agent: peer39_crawler/1.0 User-agent: PerplexityBot User-agent: PetalBot User-agent: RankActive User-agent: Ravenbot User-agent: RapidSpike User-agent: Rogerbot User-agent: Scrapy User-agent: Screaming Frog SEO Spider User-agent: SearchmetricsBot User-agent: SeekportBot User-agent: SEMrush User-agent: SemrushBot User-agent: SemrushBot-BA User-agent: SemrushBot-SA User-agent: SemrushBot-SI User-agent: SemrushBot-SWA User-agent: SemrushBot-CT User-agent: SEO PowerSuite LinkAssistant User-agent: SplitSignalBot User-agent: SemrushBot-COUB User-agent: Serpstatbot User-agent: SISTRIX User-agent: SiteAuditBot User-agent: sogou spider User-agent: sogou web spider User-agent: SpyFu User-agent: UpCrawler User-agent: WebMeUp User-agent: Zoominfobot Disallow: /