Code: p134 Severity: Notice
Description: The "robots.txt" file is commonly placed in the root directory of a system's Web server to control the actions of Web robots (often called crawlers or spiders). All robots that adhere to the Robots Exclusion Standard will check this file on your server before proceeding to index or search your site. Impact: Any HTTP server using robots. Corrective: This is not a vulnerability. Administrators should review the contents of the "robots.txt" file to check if the information is consistent with the policies of their organization.
|
