User-agent: * Disallow: /client_work/ Disallow: /html/resources/whitepapers/index.html Disallow: /clientftp/ User-agent: WebCrawler Disallow: /client_work/