Archive for November, 2007
How do I test my robots.txt file?
Mardian Purwanto - Nov 17th, 2007
You can use the Google robots.txt analysis tool in google webmaster tools to: Check specific URLs to see if your robots.txt file allows or blocks them. See if Googlebot had trouble parsing any lines in your robots.txt file. Test changes to your robots.txt file. If you don’t currently use a robots.txt file, you can create one [...]
Read More..>>