Ever wondered why Google returned search results that lead to sites that require a registration? How did Google index the site without a registration? Many sites want their site indexed in Google to receive more hits, so they allow Googlebots in. Because of this reason we can take advantage of this. All we have to do is disguise ourself as a Googlebot and many sites will let us in without registration.

To do this we have to change the User Agent. The user agent defines the browser and what version you are using. For example if you go to this site it will tell you what browser you are using, and it finds this info out by the user agent.

Changing the user agent is easy, to do this in Opera you have to uses these settings:

User Agent: Googlebot/2.1  Compatible: http://www.googlebot.com/bot.html

Firefox users can use this extension and import this file (right click and save) that has a huge list of user agent settings including Googlebot/2.1 and many other spiders/bots.

Internet Explorer users you will need to change the registry entries, to do this open notepad and copy this into it:

Then save it as ua.reg and double click on it.

To revert the changes back, you will need to do the same but this time copy this in:

Save as oua.reg and double click.

To see if it has worked go here and your browser should now be Googlebot v2.1

Enjoy!