User-agent: crawl-it Disallow: User-agent: * Disallow: /