GoFuckYourself.com - Adult Webmaster Forum

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)
-   Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)
-   -   finding the IP's of SE's webpiders (and more)... (https://gfy.com/showthread.php?t=330347)

goBigtime 07-23-2004 03:53 PM

finding the IP's of SE's webpiders (and more)...
 
When the SE's index sites like whatismyip.com that display the IP of the visitor, the spider will usually also store it's own IP in the results for the index :)

From there you can just hit arin with that IP to see how much of the block the company owns.

You could also use it to determine whos results smaller engines primarily use.
:winkwink:



Example: http://216.239.57.104/search?q=cache...is+my+ip&hl=en

Then: http://ws.arin.net/cgi-bin/whois.pl?...t=64.68.82.201

bringer 07-23-2004 03:59 PM

Resolved 64.68.82.201 to crawler15.googlebot.com
no big secret

bringer 07-23-2004 04:00 PM

cat domain.log | grep googlebot

this works too :)

goBigtime 07-23-2004 04:00 PM

Quote:

Originally posted by bringer
Resolved 64.68.82.201 to crawler15.googlebot.com
no big secret

__
:)



Don't focus so much on google.

I just used google as an example.


All times are GMT -7. The time now is 07:45 AM.

Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc123