Bitching Google using RDF

[fnet]

If you have daily updates, and don't currently publish in XML, I can get you set up with an RSS feed that will make freshbot eat out of your hand- each of your updates should be in Google within 48 hours of posting.

I'm doing this on a contract basis, hit me up if you're interested, or figure it out yourself.

Sling me an email:

tony at fucker dot net.

[fnet]

Damn, it's been 5 minutes, and not one e-mail.

What, you need an example?

[RAM]

examples are NICE

[WiredGuy]

Prove you can get FreshBot to spider your site every 48 hours. Show me a sample URL and I'll check it out.

WG

[RAM]

Originally posted by WiredGuy
Prove you can get FreshBot to spider your site every 48 hours. Show me a sample URL and I'll check it out.

WG

Yes prove it and I'm in.

[WiredGuy]

btw, if your PR is 6 or higher you can get a FreshBot weekly spider, so I'd be really interested to see how you got a 48 hour spider coming to your site with a PR of 7 or less.

WG

[fnet]

Originally posted by WiredGuy
btw, if your PR is 6 or higher you can get a FreshBot weekly spider, so I'd be really interested to see how you got a 48 hour spider coming to your site with a PR of 7 or less.

WG

Some kind of GoogleBot (presumably freshbot) visits me daily. The updates are in the Google index within 48 hours of that visit, so technically it may be between 48-72 hours from the actual posting to appearance in the Google index.

You may know that Google recently bought Pyra Labs (Blogger)... and that news is realtime, and that Google isn't keeping up well with daily indexes like daypop and blogdex.media.mit.edu (ah, how i love the media lab.)

If that tease of information doesn't convince you, send me EMAIL and I'll email you back a link.

I'm looking for actual paid work.

I mean, the other day I was on fricking slashdot, I posted a link to that, and I still can't get people to take me at my "technical word" in this forum. Most people seem to listen with their eyes here. The people who do listen with their brains seem to prefer to scuttle off with the shavings of the ideas that are dropped, without getting into a dialog.

Again, I'm looking for actual paid work.

[fnet]

Originally posted by RAM
Yes prove it and I'm in.

Just sent this off to WiredGuy, let's see what he says after checking it out:

Subject:
Re: WiredGuy
From:
Tony <[email protected]>
Date:
Fri, 07 Mar 2003 23:46:32 -0500
To:
(wiredguy's email address)

Hey WiredGuy,

May I ask that you post a response in the thread verifying (or renouncing, for that matter) the validity of my claim?

Googlebot 2.1 fetches the index once every two days, but is more active on subdirectories on days that fall in between those visits.

64.68.82.58 - - [02/Mar/2003:01:53:17 -0500] "GET /spew/ HTTP/1.0" 200 45596 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

64.68.82.39 - - [04/Mar/2003:19:48:12 -0500] "GET /spew/ HTTP/1.0" 200 51028 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

64.68.82.58 - - [06/Mar/2003:23:58:34 -0500] "GET /spew/ HTTP/1.0" 200 63802 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

So while Freshbot visits Spew every day, it's only hitting the index every other day- still not bad for a pagerank 4 subdirectory.

These visits apply to http://www.fucker.net/spew/

Tony

WiredGuy wrote:

(something)

[extreme]

Google just loves freshness.
RDF or not doesnt matter.

[fnet]

Originally posted by extreme
Google just loves freshness.
RDF or not doesnt matter.

Nice theory, but wrong. Show me a counter example to what's actually been discussed here, on the exclusive basis of update frequency.

[extreme]

Originally posted by fnet


Nice theory, but wrong. Show me a counter example to what's actually been discussed here, on the exclusive basis of update frequency.

You don't have to believe me.

Google uses a real simple way to determine if a page might have been updated.

The frequency it is willing to check depends on pagerank though, so you can't get it to check every hour.

Here's an example from a subpage only linked from the index:
64.68.82.46 - - [23/Feb/2003:07:12:16 +0100] "GET /links.php HTTP/1.0" 200
1394 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.36 - - [27/Feb/2003:22:22:17 +0100] "GET /links.php HTTP/1.0" 200
1394 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.34 - - [28/Feb/2003:14:08:28 +0100] "GET /links.php HTTP/1.0" 200
1394 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.51 - - [02/Mar/2003:17:03:37 +0100] "GET /links.php HTTP/1.0" 200
1394 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

This is the index:
64.68.82.14 - - [20/Dec/2002:10:27:35 +0100] "GET / HTTP/1.0" 200 2713 "-"
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.66 - - [22/Dec/2002:13:07:54 +0100] "GET / HTTP/1.0" 200 2713 "-"
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.28 - - [23/Dec/2002:11:23:02 +0100] "GET / HTTP/1.0" 200 2713 "-"
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.39 - - [24/Dec/2002:09:21:15 +0100] "GET / HTTP/1.0" 200 2713 "-"
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.14 - - [25/Dec/2002:09:22:47 +0100] "GET / HTTP/1.0" 200 2713 "-"
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.38 - - [26/Dec/2002:12:56:21 +0100] "GET / HTTP/1.0" 200 2713 "-"
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.38 - - [31/Dec/2002:13:58:03 +0100] "GET / HTTP/1.0" 200 2713 "-"
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

[G Sharp]

I'd like to know as well.

[fnet]

Originally posted by extreme


You don't have to believe me.

Google uses a real simple way to determine if a page might have been updated.

The frequency it is willing to check depends on pagerank though, so you can't get it to check every hour.

Here's an example from a subpage only linked from the index:
64.68.82.46 - - [23/Feb/2003:07:12:16 +0100] "GET /links.php HTTP/1.0" 200
(...)
[31/Dec/2002:13:58:03 +0100] "GET / HTTP/1.0" 200 2713 "-"
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

Before posting your logs, please tell us what you are trying to demonstrate. You haven't stated the PR of that index page, so what you're saying is "boring", by your signature's definition.

Your visits appear to be slightly arhythmic- why the 5 day gap?

Give me a counterexample with equivalent PR, or go away.

[extreme]

Originally posted by fnet


Before posting your logs, please tell us what you are trying to demonstrate. You haven't stated the PR of that index page, so everything you're saying is hearsay at this point, aka "boring", by your signature's definition. Give me a counterexample with equivalent PR, or go away.

The site is PR4.

The files Google is re-requesting have not been updated at all. One of the things you are doing is correct and that's all that matters.

Eh, fuck it. Why am I trying to point you in the right direction anyway.

You'll find plenty of people paying for the RDF stuff even if it's not the important part. So it's not like i'm affecting your biz.

[fnet]

Originally posted by extreme


The site is PR4.

The files Google is re-requesting have not been updated at all. One of the things you are doing is correct and that's all that matters.

Post a url and I'll stand agape. Your refutation doesn't stack up as stated. You haven't shown incontrivertible evidence. All I hear are sidelong comments. If you know something speak up.