To be or not to be! Becoming the Google Bot

Posted last year, at the end of March by Dave.
Categories: Web Developement, Internet, Google.
Post Views: 712 Views.

While reading Not Another One!!, The online ramblings of Niall Donegan I had a quick look at Nialls latest post Useful Firefox Extensions and I am a big fan of Firefox. Mainly down to the ability to add extentions. And Niall was listing off his favorite extentions. And one called User Agent Switcher. And what he said caught my eye…

Another good extension which is good at turning up interesting ways of getting into sites!

Interesting ways of getting into sites, this should be an interesting read. Reading through his post Googlebot UserAgent Good For Something, he explains that the User Agent Switcher can be set to look like the Google Bot. So I installed it and he spoke about one site that he used it to get into, Unison.ie.

So after installing it and looking up Googles user agents I added the following Googlebot/2.1 (+http://www.googlebot.com/bot.html) and went onto Unison.ie to test his idea. And sure as hell, it worked. You cant get more then 2 pages into a category before then ask you to login or join.

So I want to say Thanks to Niall! :D Also check out Red Cardinals post Unison.ie Cloaking very intersting read.


Last 5 posts in Web Developement

Share and Enjoy:These icons link to social bookmarking sites where readers can share and discover new web pages.
    del.icio.usDe.lirio.usdiggFurlRedditYahooMyWeb

3 comments. Print Print Email Email Download a PDF version of this post. PDF
, ,  

Niall IRELAND

Comment on March 28th, 2007 .

Unison isn’t the only one by a long shot :) However it is the one that most annoys me most. It is senseless to have do a registration, even a free one, in order to access that content.

The other sites I have come across are using a subscription model as they are offering something that’s worth paying for. I have emailed them and told them that they are on a path to trouble though.

Dave IRELAND

Comment on March 28th, 2007 .

Im sure if you send a day browsing the net you would find loads. I was on one site and I forgot to turn it off and I was blocked saying I wasnt the google bot. So people do know that this is out there but havent paid much attention to it. Im going to try a certain big newspaper that charge 7 euro a week to view the site to see if they have fixed this hole.

Irish Developer » I’m the Google Bot…. no really!

Pingback on November 12th, 2007 .

[…] couple of months ago I posted about becoming the Google Bot, and I wanted update people to the fact that many websites are catching on to […]

Leave a comment

Comments can contain some xhtml. Names and emails are required (emails aren't displayed), url's are optional.




Note: This post is over a year old. You may want to check later in this blog to see if there is new information relevant to your comment.

It took a bunch of illegal immigrants locked in a basement, 0.624 seconds to make this page.