Business Googlebot will start executing JavaScript

[Barry-xlovecam]

Webmaster level: All

We recently announced that our indexing system has been rendering web pages more like a typical modern browser, with CSS and JavaScript turned on. Today, we're updating one of our technical Webmaster Guidelines in light of this announcement.

For optimal rendering and indexing, our new guideline specifies that you should allow Googlebot access to the JavaScript, CSS, and image files that your pages use. This provides you optimal rendering and indexing for your site. Disallowing crawling of Javascript or CSS files in your site?s robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings.

http://googlewebmastercentral.blogsp...webmaster.html

Going to be another shake-out I guess.

As always with the clan that does not want to be evil, it?s a double-edged sword. On the one side, it makes so that content is better understood and navigable. On the other side, it makes so that federation and affiliation systems suffer and are judged harshly if no extra content is provided.

http://www.html5report.com/topics/ht...ntent-king.htm

[sarettah]

Apparently it is already doing so from about 2 days ago.

Noticed my hit counts were way up all of a sudden and my ratios were way off.

This morning, when I finally went to the raw logs found google bot following links that previosuly were not being followed. All of them were being written dynamically through javascript.

Going through right now and tagging them all as nofollow which of course I should have done originally.

.

[Barry-xlovecam]

HAHAHA
He changed his mind apparently like a girl would

Mat Cutts disinformation service ...

nofollow is probably a red flag now -- time will tell ...

[shoot twice]

Can someone just pass me all of Google's IPs?
Because I basically want to do this

order allow,deny
deny from GOOGLE
allow from all

[PeR930]

Why not just do noindex and nofollow?

[shoot twice]

Why not just do noindex and nofollow?

Because I really fucking hate google.


I would like to ban all google IPs just so I can say I did. But good lucking finding a list of their IPs.. It's like a carefully guarded secret.

[EddyTheDog]

Because I really fucking hate google.


I would like to ban all google IPs just so I can say I did. But good lucking finding a list of their IPs.. It's like a carefully guarded secret.

Just change your robots.txt file - Problem solved...

[AdultKing]

Because I really fucking hate google.

I would like to ban all google IPs just so I can say I did. But good lucking finding a list of their IPs.. It's like a carefully guarded secret.

Just a tip, Google have a LOT of IP addresses, you would need to block their net blocks instead and given they now also use IPv6 for crawling and probably growing their crawling network incrementally over time then it's a useless endeavour.

The best way to keep Google off your site and your site out of the Google index is to create a robots.txt file as below:

User-agent: Googlebot-Image
Disallow: /

User-agent: Googlebot
Disallow: /

User-agent: Mediapartners-Google
Disallow: /

That will ensure Google won't index your site and your site will disappear from the Google index.

[InfoGuy]

There are legitimate reasons to block JavaScript. Hiding email addresses is probably the most common. It's stupid for Google to penalize sites for blocking access to JavaScript encoded email addresses.

[just a punk]

can someone just pass me all of google's ips?

Something to start with...

66.249.66.36
66.249.72.214
74.125.46.88
72.14.199.225
74.125.44.92
209.85.224.84
209.85.224.95
66.249.71.4
64.233.182.90
74.125.78.91
66.102.12.95
74.125.44.89
209.85.226.84
209.85.228.87
66.249.71.42
74.125.156.86
64.233.182.82
64.233.182.84
209.85.224.89
74.125.44.81
74.125.158.82
96.253.97.166
74.125.64.82
74.125.64.81
66.102.12.89
74.125.70.80
74.125.42.86
74.125.42.82
209.85.224.99
209.85.224.83
74.125.42.83
66.102.12.86
66.249.72.243
66.249.72.231
74.125.158.81
74.125.64.90
64.233.182.83
72.14.202.91
74.125.114.81
74.125.156.82
74.125.44.85
64.233.182.88
64.233.182.80
64.233.182.87
72.14.202.82
74.125.64.91
74.125.44.87
72.14.202.84
74.125.126.86
66.249.71.97
61.213.110.210
66.249.72.176
74.125.46.89
74.125.46.94
66.102.12.88
66.102.12.90
209.85.224.80
74.125.114.84
66.249.67.99
72.14.202.87
64.233.182.81
66.102.12.80
66.102.12.94
74.125.112.85
74.125.46.92
74.125.64.88
74.125.64.89
74.125.44.83
74.125.156.87
74.125.46.95
74.125.156.81
74.125.64.87
74.125.64.94
74.125.44.86
66.102.12.93
74.125.156.84
66.102.12.83
74.125.64.85
74.125.158.87
74.125.44.84
74.125.64.95
209.85.224.87
66.249.66.24
66.102.12.81
66.102.12.84
66.102.12.92
209.85.224.91
209.85.224.88
72.14.202.89
74.125.64.84
74.125.46.93
74.125.46.90
74.125.156.83
66.102.12.85
66.249.66.164
74.125.156.85
66.249.72.233
66.102.12.87
209.85.224.85
209.85.224.93
74.125.46.87
66.249.67.123
66.249.71.193
209.85.226.80
209.85.228.93
74.125.70.84
74.125.44.90
74.125.64.92
74.125.158.86
74.125.44.91
74.125.64.93
66.102.12.82
74.125.46.86
66.249.71.231
74.125.46.82
209.85.228.95
209.85.226.95
66.249.67.102
74.125.44.82
72.14.202.90
209.85.224.92
74.125.112.83
66.249.71.22
209.85.224.96
72.14.202.85
74.125.112.82
209.85.228.82
209.85.228.92
66.249.72.35
66.249.66.6
64.233.182.89
74.125.46.80
74.125.156.80
74.125.42.87
74.125.42.80
74.125.42.84
209.85.224.94
66.249.66.100
66.249.66.144
66.102.12.91
209.85.226.94
209.85.228.94
74.125.70.87
72.14.202.88
72.14.202.86
209.85.224.81
72.14.202.83
74.125.44.94
74.125.114.89
66.249.71.54
74.125.44.80
209.85.224.90
209.85.224.97
209.85.224.82
74.125.46.85
66.249.72.65
209.85.224.86
74.125.64.86
74.125.44.93
74.125.158.83
74.125.158.84
74.125.46.83
209.85.226.83
209.85.228.85
74.125.70.83
64.233.182.86
209.85.224.98
74.125.46.81
74.125.114.82
74.125.112.94
74.125.112.89
74.125.112.84
74.125.112.88
74.125.112.95
74.125.64.80
74.125.158.80
74.125.52.88
66.249.68.176
74.125.158.85
66.249.68.149
74.125.64.83
74.125.44.95
74.125.46.91
74.125.114.91
66.249.67.181
74.125.70.85
209.85.226.86
209.85.228.81
74.125.112.80
74.125.70.90
74.125.156.89
74.125.156.91
66.249.71.246
66.249.71.240
74.125.156.95
74.125.70.82
74.125.78.95
74.125.78.85
74.125.78.94
74.125.78.93
74.125.70.91
74.125.78.89
66.249.72.202
74.125.78.84
74.125.158.89
74.125.156.93
76.217.20.69
74.125.78.88
66.249.71.10
66.249.71.250

[InfoGuy]

66.249.xxx.yyy can be Googlebot, but 66.249.16-17.yyy are faked Googlebot user-agent requests from DomainTools.com.

[shoot twice]

Something to start with...

One big list of IP addresses !!!
removed to not flood the thread

I LOVE IT THANKS

Of course you guys are right about the robots.txt.

Now I've been accused many times of being a "moralist" and I admit that I'm very comfortable on a high horse. But hey no one's perfect... Plus to be very honest probably the only traffic that comes to my little vanity site is the bots. I market myself better with a business card and face-to-face. Quite frankly I need to look at my business card to remember the web address.

Therefore my wanting to ban google is purely emotional. Because the "Do No Evil" corporation is

• 1) The biggest pirate site going and the biggest enabler of pirates. (Why don't I exploit the Internet more?? Because I don't want my stuff stolen)
  
• 2) Is in the business of putting people out of work with all their robot research. ( It doesn't take a genius to see no one is going to buy our stuff if they themselves don't get a paycheck)
  
• 3) Google doesn't give a rat's ass about privacy by scanning emails, etc. Plus they make it easy for fucking assholes to stalk women and secretly photographer/film them and upload that shit
  
• 4) They manipulate the natural evolution of other nations.

So I don't need to ban google and probably the ultimate would be to simply let the domain expire. But the militant moralist in me wants to give Google the finger.

[AdultKing]

[*]1) The biggest pirate site going and the biggest enabler of pirates. (Why don't I exploit the Internet more?? Because I don't want my stuff stolen)

It's actually a piracy agnostic search engine that has only recently changed it's position from agnostic to actively penalising sites based on DMCA requests received. So the claim it's the biggest piracy site on the net just feeds the misinformation pirates like to use to justify their own activities.

[*]2) Is in the business of putting people out of work with all their robot research. ( It doesn't take a genius to see no one is going to buy our stuff if they themselves don't get a paycheck)

James Watt helped start the industrial revolution that eventually put horse coach drivers, blacksmiths, manual textile craftswomen, street light wick lighters and other now unheard of occupations out of work. If you take the attitude you're taking then we'd still be in caves.

[*]3) Google doesn't give a rat's ass about privacy by scanning emails, etc. Plus they make it easy for fucking assholes to stalk women and secretly photographer/film them and upload that shit

Algorithmic display of ads based on email content does not immediately precede the stalking of women and the activity of people secretly photographing or filming them.

[*]4) They manipulate the natural evolution of other nations.

Well at least I now know one thing for sure. You are as stupid as most of your posts to this forum so therefore......

So I don't need to ban google and probably the ultimate would be to simply let the domain expire. But the militant moralist in me wants to give Google the finger.

Let the domain expire, someone as dumb as you are doesn't need a domain.

[Barry-xlovecam]

Something to start with...

66.249.66.36


{...}
74.125.158.89
74.125.156.93
76.217.20.69
74.125.78.88
66.249.71.10
66.249.71.250

Use the CIDR (I fixed if for you ;-) it's a lot neater.
209.85.128.0/17
61.213.64.0/18
64.233.160.0/19
66.102.0.0/20
72.14.192.0/18
74.125.0.0/16
96.253.97.166

Google sees affiliates as competition -- without any affiliates Google would sell more ads -- so long as there is web content to create an index from as consumer bait.

So you have a lose-lose proposition:

1. Feed the machine that wants to eliminate you (as an affiliate),or;
2. Eliminate the content Google can use but sacrificing the "bone" Google gives you in SE referrals.

There are other search engines you could allow to index your content on your forced terms ... this is what I meant by an "Intention Economy."

Search Engines are nothing without an Index of the Web.

[Vendzilla]

Thanx for the update, I have a new client that I'm doing some SEO for and his site is a mess. It has 20 CSS files and that's just the beginning

[shoot twice]

this is what I meant by an "Intention Economy."

Worth the read thanks!

[RyuLion]

I didn't have some of those IP's..thanks!

[just a punk]

Use the CIDR (I fixed if for you ;-) it's a lot neater.
209.85.128.0/17
61.213.64.0/18
64.233.160.0/19
66.102.0.0/20
72.14.192.0/18
74.125.0.0/16
96.253.97.166

Yep, that looks much better. Actually it was a data harvested by my bot-tracking script in 2011-2012. I've just relaunched it to dig the new IP's.

[AdultKing]

Worth the read thanks!

I'm surprised you got through the first paragraph