# Allow crawling of all content User-agent: * Disallow: /recruiter/ Crawl-delay: 1