Friday, August 10, 2007

Disguise as a Googlebot, enter restricted websites

>
Sometimes you conduct a search on Google and the search engine returns a number of results, but when you try to open the ones that looks the most promising you get a registration page. One thing that you can do is click on the cached content to view the page directly from Google's cache. But what if you want to view another page from the same site, not present in Google’s cache?

The Solution:

Its evident that there are some sites that allows Google but not you. So why not disguise yourself as Google? That’s exactly what we will do by changing the browser's user agent to Googlebot. Copy the following code into notepad and save it as a .reg file

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\
Internet Settings\5.0\User Agent]
@="Googlebot/2.1"
"Compatible"="+http://www.googlebot.com/bot.html"

Now double-click the file to merge it into registry. In order to change back to the default user agent use this:



Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\
Internet Settings\5.0\User Agent]
@="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

The above works only for Internet Explorer. For Firefox, type in about:config in the address bar. Then scroll down till you find "general.useragent.extra.firefox"
Double click on it and type in "Googlebot 2.1". Now this will change your user agent to the google bot. To change the user agent back to the original one just right click and click Reset.

Unfortunately, there is no way to change the user agent to Googlebot in Opera.

To check whether you have successfully changed the user agent, type the following into the address bar and hit enter.


javascript(BrowserDetect.browser)


Two popular sites where you can use this trick to gain entry are http://www.governmentsecurity.org/forum/index.php. and http://www.experts-exchange.com

GovernmentSecurity does not allow you to enter without registering, and registration is not open to public. ExpertsExchange asks you to pay for viewing the full forum and the solution to the problems discussed. Well, now you know how to get entry without paying.

Happy browsing.

6 comments:

  1. Great! But is it possible to enter this site for accessing the published papers ? Here's the page :

    http://links.jstor.org/sici?sici=0022-2909%28198721%2931%3A1%3C99%3ASEOTCO%3E2.0.CO%3B2-A&size=SMALL&origin=JSTOR-reducePage

    ReplyDelete
  2. You can enter only those websites where the webmaster has allowed Googlebot to enter. This is usually done to allow Google to index the pages so that people can search. But if the webmaster has disallowed entry of Google bots you can't enter it. The link you gave does not allow unauthorised persons to visit it and that includes Googlebot too.

    ReplyDelete
  3. Sigh!
    The trouble with the likes of me is that we use the Net extensively without knowing a bit about it!

    ReplyDelete
  4. Watch googlebot in action :D

    http://labs.involutive.com/2007/10/01/humor-involutive-how-do-you-imagine-googlebot-we/

    ReplyDelete
  5. http://www.odtuders.com
    odtuders has same problem, can you help me?..

    ReplyDelete
  6. @pyrocharcy:

    The website is of a foreign language so I can't make out what it is saying. If the google trick did not work then try getting a username/password from www.bugmenot.com

    ReplyDelete

Popular Posts