What Is supposed by a person-agent in robots.txt is the specific kind of Website crawler that you simply provide the command to crawl. This Website crawler usually varies according to the search engine applied. When you are developing the file manually, then you have to know about the rules Employed https://seotoolstube.com/