What makes you say this? I am seriously curious. I have seen somme site ranking go down over the past few months (mainstream).
Anything to back it up?
Yeah, our server crashing last night. Its getting to be the weekly se crawl/crash syndrome. Was told by support SE's crawling our sites literally crashed the server. There were no other traffic spikes at the moment.
Just looking for a more robust option if available.
Yeah, our server crashing last night. Its getting to be the weekly se crawl/crash syndrome. Was told by support SE's crawling our sites literally crashed the server. There were no other traffic spikes at the moment.
Just looking for a more robust option if available.
Server crash because a spider crawls it's sites...? wow... just wow.
Yeah, our server crashing last night. Its getting to be the weekly se crawl/crash syndrome. Was told by support SE's crawling our sites literally crashed the server. There were no other traffic spikes at the moment.
Just looking for a more robust option if available.
hmmm this is highly unusual unless it is vps with 32mb of RAM.
Anyway by any chance you are running WP sites on that server? And must be one at least on the server, who dosen't have it nowdays. The real culprit could be xmlrpc.php in root of wp site. I have seen recently many attacks on this file and one Quad Core server which usual load is about 2-3 constantly, suddently go high up to 40 because of those attacks from many bad bots trying to hacks wp site thru xmlrpc.
Run this against access.log (apache or nginx) to see if there are many access attempts to xmlrpc.php
hmmm this is highly unusual unless it is vps with 32mb of RAM.
Anyway by any chance you are running WP sites on that server? And must be one at least on the server, who dosen't have it nowdays. The real culprit could be xmlrpc.php in root of wp site. I have seen recently many attacks on this file and one Quad Core server which usual load is about 2-3 constantly, suddently go high up to 40 because of those attacks from many bad bots trying to hacks wp site thru xmlrpc.
Run this against access.log (apache or nginx) to see if there are many access attempts to xmlrpc.php
Server crash because a spider crawls it's sites...? wow... just wow.
Yeah, must be good server.
Though I am not sure do I believe this shit. If it is Google they can for starters limit the crawling intensity from webmaster tools, but site crashing because of SE spiders.. come on..
Thats pretty helpful. Thanks a lot. Yes there are some WP sites and bots have been hitting certain files, usually search or jpg related.
Make sure you have something setup to block wp-login.php bruteforce attacks, those can really cause cpu load to skyrocket. I run a lot of of Wordpress blogs on LEMP Stacks and have it set to only allow my IP through to wp-login.php and then I have Fail2Ban block any user who attempted to try to bruteforce (along with banning anyone who searches for Wordpress exploits). You could also install the popular Wordpress plugin "Limit Login Attempts" which will help against bruteforce attacks as well but I prefer the only allowing my ip method.
What are the specs of the VPS and how many wordpress sites do you have on it ?
I will get that to you. It does turn out we were being hit by UA and ID IPs at the same time the SE's crawled. I am guessing the bot hits were timed. They've been doing it off and on since late July.
ID - 202.67.41.51 ++ others
UA - 91.200.12.71 ++ others
Toss in HK, VN, CN and KR from time to time. We've had to add more than 70 block rules on the cloud service and add more each day.
We ramped up security on the wp log ins. Now they are shut off after 2-3 attempts.
Server crash because a spider crawls it's sites...? wow... just wow.
Ya Bad spiders!
I've been on all different kinds and sizes of servers on AmeriNoc for over 12 yrs and love the company.. Hence I list them down there in that area.. But as others said look for the real problem before jumping ship or you will be in the same boat on a different server
Maybe that you go bash some else instead of me. If you have a problem with me than talk to me directly do not go about and talk shit about our service. I have had plenty of clients coming from the big guys who deliver shitty service. Those three people who are not happy are not representing our service.
I've been on all different kinds and sizes of servers on AmeriNoc for over 12 yrs and love the company.. Hence I list them down there in that area.. But as others said look for the real problem before jumping ship or you will be in the same boat on a different server
Love the guys at Amerinoc. Great guys, helpful, supportive and knowledgeable. We've discovered that if you use Amerinoc and decide to add cloudflare you must sign up to cloudflare on your own to get full DNS protection.
Amerinoc did this as a favor to us because we are not tech heads (signing us up). However we are learning we have to do it ourselves and grunt it out learning the tech of it.
Love the guys at Amerinoc. Great guys, helpful, supportive and knowledgeable. We've discovered that if you use Amerinoc and decide to add cloudflare you must sign up to cloudflare on your own to get full DNS protection.
Amerinoc did this as a favor to us because we are not tech heads (signing us up). However we are learning we have to do it ourselves and grunt it out learning the tech of it.
But you haven't made the switch to Cloudflare yet ? Shoot me another PM and I will invite you into one my my Slack channels and help you get this fixed.
Comment