Robots.txt is a file that exists on the foundation directory of each Web site and can be employed to instruct search engines like yahoo on which directories/documents of the website they will crawl and involve within their index. Seo (Search engine marketing) is the process of optimizing a web site https://technik.forum