User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / Crawl-Delay: 20 User-agent: MJ12bot Disallow: / Crawl-Delay: 20 User-agent: BLEXBot Disallow: / Crawl-Delay: 20 User-agent: DataForSeoBot Disallow: / Crawl-Delay: 20 User-agent: Amazonbot # Amazon's user agent Disallow: / User-agent: * Disallow: /taxonomy/ User-agent: * Crawl-Delay: 20 User-agent: * Disallow: /taxonomy/ Sitemap: https://bijleshuis.be/sitemap.xml