# All robots will spider the domain User-agent:* Disallow: # Disallow directory /files/ User-agent: * Disallow: /files/