Work on your new site, without Google snooping around and indexing it.

As many times as I tell people to not develop on a live site for numerious reasons, it is still commonplace.  I admit, I do it sometimes as well. Let’s block Google with robots.txt

The problem is Google and search engines will be actively indexing your new site and invariably you will be stuck with crappy search results for a period of time after you finish your site.  Read more to find out how to fix this embarrassing error.
How many times have you looked in Google for your site, only to find the default description of Joomla being a dynamic portal showing up in your results?  I can say it’s happened to me more then once. Here, a quick peek shows literally pages and pages of this.

Joomla! default boo boo

Aside from the obvious, and actually forgetting to change your meta information, some of these sites might be in development currently.  If so, all you need to do is edit your robots.txt to look like this:

disallow all

Do not forget to change this back when your site goes live, as this configuration will mean Google will never index your site.

Robots.txt is very powerful and very useful, and often overlooked. For more information, read about it here.

About the author

Submit a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>