Blocking pages from scans

If you do not want Search Engine Visibility to analyze your entire website, you can block specific pages using your site's robots.txt file. This is useful if you have admin pages or forums that you don't want to optimize.

To prevent Search Engine Visibility from scanning a page, update your site's robots.txt file with the following, where page is the specific page in your site:

You can also block entire directories with the following, where directory is the name of the directory or folder you want to block:

For more information about building a robots.txt file for your site, see Selecting website pages .