finding the IP's of SE's webpiders (and more)...

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • goBigtime
    Confirmed User
    • Nov 2002
    • 7761

    #1

    finding the IP's of SE's webpiders (and more)...

    When the SE's index sites like whatismyip.com that display the IP of the visitor, the spider will usually also store it's own IP in the results for the index

    From there you can just hit arin with that IP to see how much of the block the company owns.

    You could also use it to determine whos results smaller engines primarily use.




    Example: http://216.239.57.104/search?q=cache...is+my+ip&hl=en

    Then: http://ws.arin.net/cgi-bin/whois.pl?...t=64.68.82.201
    Last edited by goBigtime; 07-23-2004, 02:58 PM.
  • bringer
    i have man boobies
    • Jul 2003
    • 13082

    #2
    Resolved 64.68.82.201 to crawler15.googlebot.com
    no big secret
    333-765-551

    Comment

    • bringer
      i have man boobies
      • Jul 2003
      • 13082

      #3
      cat domain.log | grep googlebot

      this works too
      333-765-551

      Comment

      • goBigtime
        Confirmed User
        • Nov 2002
        • 7761

        #4
        Originally posted by bringer
        Resolved 64.68.82.201 to crawler15.googlebot.com
        no big secret
        __




        Don't focus so much on google.

        I just used google as an example.
        Last edited by goBigtime; 07-23-2004, 03:02 PM.

        Comment

        Working...