What Is meant by a person-agent in robots.txt is the specific form of web crawler that you choose to give the command to crawl. This Website crawler commonly differs with regards to the search engine used. txt file comes in, serving like a website traffic controller for World wide web https://seotoolstube.com/