# exclude help system from robots User-agent: * Disallow: /manual Disallow: /doc Disallow: /gif Disallow: /hilfe Disallow: /support-db Disallow: /webalizer Disallow: /~lukas/cgi-bin Disallow: /~lukas/asc Disallow: /~lukas/radar Allow: / # but allow htdig to index our doc-tree User-agent: susedig Disallow: # disallow stress test User-agent: stress-agent Disallow: /