Any HTACCESS People In The House?
 

At the moment if I use the above code, Google doesn't spider our site, it spiders the redirect url (domain.com) because Googlebot has a US based IP.

So the question is..... how do I use this code without redirecting search engine spiders such as Google. I found this advice:

https://www.webmasterworld.com/apache/4276815.htm

Above URL suggests it is best to:

(1) Not redirect search engine spiders
(2) Not redirect the google IP number set

Using both options will be the safest bet but I'm a complete novice so I don't fully understand the advice. Anyone know how we need to edit this code to make it work as required?

Nice little crossword puzzle for ya....

just add an agent rule against the know bots

Well if I just take the code from the above page, I get something like this code:
Does it look like it would work?

Take out the US in the htacess. Then use a JS redirect for US on the site pages...

We want to continue using htaccess for numerous reasons its actually better... and JS can cause some issues on a JS heavy page.

Then your only other option is to deny by country range IP list for the US and edit the ranges where googlebots are located at.

You can use this generator

Block Visitors by Country | IP2Location.com

Here are the bots ranges

Robots IP Address Ranges - Googlebot, Yahoo Slurp, MSNBot

If you game googlebot with different content than the people Google search refers will see: that domain will end up banned and delisted in Google search is what that means.

So it that domain a throw away blackhat domain?

Google now crawls from international IPs as well, by the way.

No, that's not the issue. It's a mainstream site and we are redirecting, for example, US (and other non-UK) users from a UK site to the International site. We were trying to avoid search engine bots precisely so that they can still correctly index the first site.

That's what I expected but I was proven wrong. Once we instituted the htaccess redirect for US users, Googlebot indexed our site with content from the site we were redirecting the US users to which tells me that the Googlebot was redirected also. I checked our webstats on the redirecting site and found the Googlebot IP is a US IP (Mountain View).

So for our redirect to work, we need to not redirect search engines (primarily Bing & Google).

<link rel="alternate" href="http:///path/" hreflang="x-default">
<link rel="alternate" href="http:///path/" hreflang="en">
<link rel="alternate" href="http:///path/" hreflang="ru">
<link rel="alternate" href="http:///path/" hreflang="de">
<link rel="alternate" href="http:///path/" hreflang="pl">
<link rel="alternate" href="http:///path/" hreflang="es">
</head>
Let the googlebot find the right content.
Let your users select their right language.

That is the right way to do it.
Set a language cookie for the user if you want continuity.

This redirect crap is way overused and unnecessary IMHO.