# All robots will spider the domain User-agent: GOOGLEBOT Disallow: / User-Agent: * Disallow: /dvdxcopy.css User-agent: NPBot Disallow: / User-agent: WebCopier Disallow: / User-agent: WebZip Disallow: / User-agent: WebZip/4.0 Disallow: / User-agent: WebZip/5.0 Disallow: / User-agent: WebCopier v3.2a Disallow: / User-agent: WebCapture 2.0 Disallow: / User-agent: WebCopier v.2.2 Disallow: / User-agent: WebZIP/4.21 Disallow: / User-agent: Web Image Collector Disallow: / User-agent: Offline Explorer Disallow: / User-agent: WebCapture 2.0 Disallow: /