Business Googlebot will start executing JavaScript - GoFuckYourself.com

Barry-xlovecam · 11-01-2014, 10:12 AM

Quote:

Webmaster level: All

We recently announced that our indexing system has been rendering web pages more like a typical modern browser, with CSS and JavaScript turned on. Today, we're updating one of our technical Webmaster Guidelines in light of this announcement.

For optimal rendering and indexing, our new guideline specifies that you should allow Googlebot access to the JavaScript, CSS, and image files that your pages use. This provides you optimal rendering and indexing for your site. Disallowing crawling of Javascript or CSS files in your site?s robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings.

http://googlewebmastercentral.blogsp...webmaster.html

Going to be another shake-out I guess.

Quote:

As always with the clan that does not want to be evil, it?s a double-edged sword. On the one side, it makes so that content is better understood and navigable. On the other side, it makes so that federation and affiliation systems suffer and are judged harshly if no extra content is provided.

http://www.html5report.com/topics/ht...ntent-king.htm

sarettah · 11-01-2014, 10:25 AM

Apparently it is already doing so from about 2 days ago.

Noticed my hit counts were way up all of a sudden and my ratios were way off.

This morning, when I finally went to the raw logs found google bot following links that previosuly were not being followed. All of them were being written dynamically through javascript.

Going through right now and tagging them all as nofollow which of course I should have done originally.

.

Barry-xlovecam · 11-01-2014, 10:54 AM

HAHAHA
He changed his mind apparently like a girl would

Mat Cutts disinformation service ...

nofollow is probably a red flag now -- time will tell ...

shoot twice · 11-01-2014, 02:24 PM

Can someone just pass me all of Google's IPs?
Because I basically want to do this

order allow,deny
deny from GOOGLE
allow from all

PeR930 · 11-01-2014, 03:06 PM

Why not just do noindex and nofollow?

shoot twice · 11-01-2014, 03:12 PM

Quote:

Originally Posted by ViPeR_930

Why not just do noindex and nofollow?

Because I really fucking hate google.

I would like to ban all google IPs just so I can say I did. But good lucking finding a list of their IPs.. It's like a carefully guarded secret.

EddyTheDog · 11-01-2014, 04:13 PM

Quote:

Originally Posted by shoot twice

Because I really fucking hate google.

I would like to ban all google IPs just so I can say I did. But good lucking finding a list of their IPs.. It's like a carefully guarded secret.

Just change your robots.txt file - Problem solved...

AdultKing · 11-01-2014, 08:18 PM

Quote:

Originally Posted by shoot twice

Because I really fucking hate google.

I would like to ban all google IPs just so I can say I did. But good lucking finding a list of their IPs.. It's like a carefully guarded secret.

Just a tip, Google have a LOT of IP addresses, you would need to block their net blocks instead and given they now also use IPv6 for crawling and probably growing their crawling network incrementally over time then it's a useless endeavour.

The best way to keep Google off your site and your site out of the Google index is to create a robots.txt file as below:

Code:

User-agent: Googlebot-Image
Disallow: /

User-agent: Googlebot
Disallow: /

User-agent: Mediapartners-Google
Disallow: /

That will ensure Google won't index your site and your site will disappear from the Google index.

InfoGuy · 11-02-2014, 01:29 AM

There are legitimate reasons to block JavaScript. Hiding email addresses is probably the most common. It's stupid for Google to penalize sites for blocking access to JavaScript encoded email addresses.

just a punk · 11-02-2014, 02:17 AM

Quote:

Originally Posted by shoot twice

can someone just pass me all of google's ips?

Something to start with...

66.249.66.36
66.249.72.214
74.125.46.88
72.14.199.225
74.125.44.92
209.85.224.84
209.85.224.95
66.249.71.4
64.233.182.90
74.125.78.91
66.102.12.95
74.125.44.89
209.85.226.84
209.85.228.87
66.249.71.42
74.125.156.86
64.233.182.82
64.233.182.84
209.85.224.89
74.125.44.81
74.125.158.82
96.253.97.166
74.125.64.82
74.125.64.81
66.102.12.89
74.125.70.80
74.125.42.86
74.125.42.82
209.85.224.99
209.85.224.83
74.125.42.83
66.102.12.86
66.249.72.243
66.249.72.231
74.125.158.81
74.125.64.90
64.233.182.83
72.14.202.91
74.125.114.81
74.125.156.82
74.125.44.85
64.233.182.88
64.233.182.80
64.233.182.87
72.14.202.82
74.125.64.91
74.125.44.87
72.14.202.84
74.125.126.86
66.249.71.97
61.213.110.210
66.249.72.176
74.125.46.89
74.125.46.94
66.102.12.88
66.102.12.90
209.85.224.80
74.125.114.84
66.249.67.99
72.14.202.87
64.233.182.81
66.102.12.80
66.102.12.94
74.125.112.85
74.125.46.92
74.125.64.88
74.125.64.89
74.125.44.83
74.125.156.87
74.125.46.95
74.125.156.81
74.125.64.87
74.125.64.94
74.125.44.86
66.102.12.93
74.125.156.84
66.102.12.83
74.125.64.85
74.125.158.87
74.125.44.84
74.125.64.95
209.85.224.87
66.249.66.24
66.102.12.81
66.102.12.84
66.102.12.92
209.85.224.91
209.85.224.88
72.14.202.89
74.125.64.84
74.125.46.93
74.125.46.90
74.125.156.83
66.102.12.85
66.249.66.164
74.125.156.85
66.249.72.233
66.102.12.87
209.85.224.85
209.85.224.93
74.125.46.87
66.249.67.123
66.249.71.193
209.85.226.80
209.85.228.93
74.125.70.84
74.125.44.90
74.125.64.92
74.125.158.86
74.125.44.91
74.125.64.93
66.102.12.82
74.125.46.86
66.249.71.231
74.125.46.82
209.85.228.95
209.85.226.95
66.249.67.102
74.125.44.82
72.14.202.90
209.85.224.92
74.125.112.83
66.249.71.22
209.85.224.96
72.14.202.85
74.125.112.82
209.85.228.82
209.85.228.92
66.249.72.35
66.249.66.6
64.233.182.89
74.125.46.80
74.125.156.80
74.125.42.87
74.125.42.80
74.125.42.84
209.85.224.94
66.249.66.100
66.249.66.144
66.102.12.91
209.85.226.94
209.85.228.94
74.125.70.87
72.14.202.88
72.14.202.86
209.85.224.81
72.14.202.83
74.125.44.94
74.125.114.89
66.249.71.54
74.125.44.80
209.85.224.90
209.85.224.97
209.85.224.82
74.125.46.85
66.249.72.65
209.85.224.86
74.125.64.86
74.125.44.93
74.125.158.83
74.125.158.84
74.125.46.83
209.85.226.83
209.85.228.85
74.125.70.83
64.233.182.86
209.85.224.98
74.125.46.81
74.125.114.82
74.125.112.94
74.125.112.89
74.125.112.84
74.125.112.88
74.125.112.95
74.125.64.80
74.125.158.80
74.125.52.88
66.249.68.176
74.125.158.85
66.249.68.149
74.125.64.83
74.125.44.95
74.125.46.91
74.125.114.91
66.249.67.181
74.125.70.85
209.85.226.86
209.85.228.81
74.125.112.80
74.125.70.90
74.125.156.89
74.125.156.91
66.249.71.246
66.249.71.240
74.125.156.95
74.125.70.82
74.125.78.95
74.125.78.85
74.125.78.94
74.125.78.93
74.125.70.91
74.125.78.89
66.249.72.202
74.125.78.84
74.125.158.89
74.125.156.93
76.217.20.69
74.125.78.88
66.249.71.10
66.249.71.250

InfoGuy · 11-02-2014, 02:29 AM

66.249.xxx.yyy can be Googlebot, but 66.249.16-17.yyy are faked Googlebot user-agent requests from DomainTools.com.

shoot twice · 11-02-2014, 02:42 AM

Quote:

Originally Posted by CyberSEO

Something to start with...

One big list of IP addresses !!!
removed to not flood the thread

I LOVE IT THANKS

Of course you guys are right about the robots.txt.

Now I've been accused many times of being a "moralist" and I admit that I'm very comfortable on a high horse. But hey no one's perfect... Plus to be very honest probably the only traffic that comes to my little vanity site is the bots. I market myself better with a business card and face-to-face. Quite frankly I need to look at my business card to remember the web address.

Therefore my wanting to ban google is purely emotional. Because the "Do No Evil" corporation is

1) The biggest pirate site going and the biggest enabler of pirates. (Why don't I exploit the Internet more?? Because I don't want my stuff stolen)
2) Is in the business of putting people out of work with all their robot research. ( It doesn't take a genius to see no one is going to buy our stuff if they themselves don't get a paycheck)
3) Google doesn't give a rat's ass about privacy by scanning emails, etc. Plus they make it easy for fucking assholes to stalk women and secretly photographer/film them and upload that shit
4) They manipulate the natural evolution of other nations.

So I don't need to ban google and probably the ultimate would be to simply let the domain expire. But the militant moralist in me wants to give Google the finger.

AdultKing · 11-02-2014, 03:19 AM

Quote:

Originally Posted by shoot twice

[*]1) The biggest pirate site going and the biggest enabler of pirates. (Why don't I exploit the Internet more?? Because I don't want my stuff stolen)

It's actually a piracy agnostic search engine that has only recently changed it's position from agnostic to actively penalising sites based on DMCA requests received. So the claim it's the biggest piracy site on the net just feeds the misinformation pirates like to use to justify their own activities.

Quote:

Originally Posted by shoot twice

[*]2) Is in the business of putting people out of work with all their robot research. ( It doesn't take a genius to see no one is going to buy our stuff if they themselves don't get a paycheck)

James Watt helped start the industrial revolution that eventually put horse coach drivers, blacksmiths, manual textile craftswomen, street light wick lighters and other now unheard of occupations out of work. If you take the attitude you're taking then we'd still be in caves.

Quote:

Originally Posted by shoot twice

[*]3) Google doesn't give a rat's ass about privacy by scanning emails, etc. Plus they make it easy for fucking assholes to stalk women and secretly photographer/film them and upload that shit

Algorithmic display of ads based on email content does not immediately precede the stalking of women and the activity of people secretly photographing or filming them.

Quote:

Originally Posted by shoot twice

[*]4) They manipulate the natural evolution of other nations.

Well at least I now know one thing for sure. You are as stupid as most of your posts to this forum so therefore......

Quote:

Originally Posted by shoot twice

So I don't need to ban google and probably the ultimate would be to simply let the domain expire. But the militant moralist in me wants to give Google the finger.

Let the domain expire, someone as dumb as you are doesn't need a domain.

Barry-xlovecam · 11-02-2014, 07:42 AM

Quote:

Originally Posted by CyberSEO

Something to start with...

66.249.66.36

{...}
74.125.158.89
74.125.156.93
76.217.20.69
74.125.78.88
66.249.71.10
66.249.71.250

Use the CIDR (I fixed if for you ;-) it's a lot neater.
209.85.128.0/17
61.213.64.0/18
64.233.160.0/19
66.102.0.0/20
72.14.192.0/18
74.125.0.0/16
96.253.97.166

Google sees affiliates as competition -- without any affiliates Google would sell more ads -- so long as there is web content to create an index from as consumer bait.

So you have a lose-lose proposition:

Feed the machine that wants to eliminate you (as an affiliate),or;
Eliminate the content Google can use but sacrificing the "bone" Google gives you in SE referrals.

There are other search engines you could allow to index your content on your forced terms ... this is what I meant by an "Intention Economy."

Search Engines are nothing without an Index of the Web.

Vendzilla · 11-02-2014, 08:44 AM

Thanx for the update, I have a new client that I'm doing some SEO for and his site is a mess. It has 20 CSS files and that's just the beginning

shoot twice · 11-02-2014, 09:12 AM

Quote:

Originally Posted by Barry-xlovecam

this is what I meant by an "Intention Economy."

Worth the read thanks!

RyuLion · 11-02-2014, 09:18 AM

I didn't have some of those IP's..thanks!

just a punk · 11-02-2014, 01:06 PM

Quote:

Originally Posted by Barry-xlovecam

Use the CIDR (I fixed if for you ;-) it's a lot neater.
209.85.128.0/17
61.213.64.0/18
64.233.160.0/19
66.102.0.0/20
72.14.192.0/18
74.125.0.0/16
96.253.97.166

Yep, that looks much better. Actually it was a data harvested by my bot-tracking script in 2011-2012. I've just relaunched it to dig the new IP's.

AdultKing · 11-02-2014, 02:21 PM

Quote:

Originally Posted by shoot twice

Worth the read thanks!

I'm surprised you got through the first paragraph

11-01-2014, 10:25 AM	#2
sarettah see you later, I'm gone Industry Role: Join Date: Oct 2002 Posts: 14,073	Apparently it is already doing so from about 2 days ago. Noticed my hit counts were way up all of a sudden and my ratios were way off. This morning, when I finally went to the raw logs found google bot following links that previosuly were not being followed. All of them were being written dynamically through javascript. Going through right now and tagging them all as nofollow which of course I should have done originally. . __________________ All cookies cleared!

11-01-2014, 02:24 PM	#4
shoot twice Confirmed User Industry Role: Join Date: Sep 2014 Posts: 517	Can someone just pass me all of Google's IPs? Because I basically want to do this order allow,deny deny from GOOGLE allow from all __________________ "A slightly exposed shoulder emerging from a long silk nightgown packs more sex than two bodies naked in bed." - Bette Davis I'm a shooter and I do business in private and I only take CASH. PS I'm impossible to troll.

11-02-2014, 01:29 AM	#9
InfoGuy 80/20 Rule Industry Role: Join Date: Apr 2010 Location: Los Angeles Posts: 3,051	There are legitimate reasons to block JavaScript. Hiding email addresses is probably the most common. It's stupid for Google to penalize sites for blocking access to JavaScript encoded email addresses. __________________ Support American Heroes \| How Bad is My Batch? \| Vaccine Deaths & Adverse Reactions \| Free Speech Coalition \| <WARNING> ePayService / Guerra Capital, INC / MTACC payments \| Flirt4Free Fucks their Affiliates \| Don't do business with piece of shit Andy Alvarez from Webmaster Central / VR3000, who said: "If it was up to me, they would have shot all 30,000 of those country loving shitheads"

11-02-2014, 02:29 AM	#11
InfoGuy 80/20 Rule Industry Role: Join Date: Apr 2010 Location: Los Angeles Posts: 3,051	66.249.xxx.yyy can be Googlebot, but 66.249.16-17.yyy are faked Googlebot user-agent requests from DomainTools.com. __________________ Support American Heroes \| How Bad is My Batch? \| Vaccine Deaths & Adverse Reactions \| Free Speech Coalition \| <WARNING> ePayService / Guerra Capital, INC / MTACC payments \| Flirt4Free Fucks their Affiliates \| Don't do business with piece of shit Andy Alvarez from Webmaster Central / VR3000, who said: "If it was up to me, they would have shot all 30,000 of those country loving shitheads"

11-02-2014, 08:44 AM	#15
Vendzilla Biker Gnome Industry Role: Join Date: Mar 2004 Location: cell#324 Posts: 23,200	Thanx for the update, I have a new client that I'm doing some SEO for and his site is a mess. It has 20 CSS files and that's just the beginning __________________ Webair died? Carbon is not the problem, it makes up 0.041% of our atmosphere , 95% of that is from Volcanos and decomposing plants and stuff. So people in the US are responsible for 13% of the carbon in the atmosphere which 95% is not from Humans, like cars and trucks and stuff and they want to spend trillions to fix it while Solar Panel plants are powered by coal plants think about that

11-01-2014, 10:54 AM	#3
Barry-xlovecam It's 42 Industry Role: Join Date: Jun 2010 Location: Global Posts: 18,083	HAHAHA He changed his mind apparently like a girl would Mat Cutts disinformation service ... nofollow is probably a red flag now -- time will tell ...

11-01-2014, 03:06 PM	#5
PeR930 Confirmed User Industry Role: Join Date: Dec 2012 Posts: 283	Why not just do noindex and nofollow?

11-02-2014, 09:18 AM	#17
RyuLion ClubDomCash.com Industry Role: Join Date: Mar 2003 Location: San Diego Posts: 32,186	I didn't have some of those IP's..thanks! __________________ Adult Biz Consultant A tech head since 1995