Looking for Host Not Crushed by Search Engines

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • JuicyBunny
    So Fucking Banned
    • Jun 2010
    • 2145

    #1

    Business Looking for Host Not Crushed by Search Engines

    Amerinoc is apparently crushed server wise on VPS packages when Search Engines crawl sites.
    Can any host make an offer to move?
  • Harmon
    ( ͡ʘ╭͜ʖ╮͡ʘ)
    • Mar 2004
    • 20012

    #2
    Originally posted by JuicyBunny
    Amerinoc is apparently crushed server wise on VPS packages when Search Engines crawl sites.
    Can any host make an offer to move?
    What makes you say this? I am seriously curious. I have seen somme site ranking go down over the past few months (mainstream).

    Anything to back it up?
    [email protected]

    Comment

    • JuicyBunny
      So Fucking Banned
      • Jun 2010
      • 2145

      #3
      Originally posted by Harmon
      What makes you say this? I am seriously curious. I have seen somme site ranking go down over the past few months (mainstream).

      Anything to back it up?
      Yeah, our server crashing last night. Its getting to be the weekly se crawl/crash syndrome. Was told by support SE's crawling our sites literally crashed the server. There were no other traffic spikes at the moment.

      Just looking for a more robust option if available.

      Comment

      • Markul
        Likes Pie
        • Dec 2007
        • 12403

        #4
        Originally posted by JuicyBunny
        Yeah, our server crashing last night. Its getting to be the weekly se crawl/crash syndrome. Was told by support SE's crawling our sites literally crashed the server. There were no other traffic spikes at the moment.

        Just looking for a more robust option if available.
        Server crash because a spider crawls it's sites...? wow... just wow.
        But.... I pulled out...

        Comment

        • JuicyBunny
          So Fucking Banned
          • Jun 2010
          • 2145

          #5
          Originally posted by Markul
          Server crash because a spider crawls it's sites...? wow... just wow.
          Yes, very sad. Funeral is at 3pm

          Comment

          • AdultKing
            Raise Your Weapon
            • Jun 2003
            • 15601

            #6
            This doesn't sound right.

            Have they provided you any other information ?

            Comment

            • k0nr4d
              Confirmed User
              • Aug 2006
              • 9231

              #7
              Agreed - that doesn't sound right. Sounds like you might have a script issue or misconfiguration in apache somewhere.

              Webmaster tools lets you throttle the indexing, by the way...
              Mechanical Bunny Media
              Mechbunny Tube Script | Mechbunny Webcam Aggregator Script | Custom Web Development

              Comment

              • Zverka
                Confirmed User
                • Jun 2005
                • 307

                #8
                Originally posted by JuicyBunny
                Yeah, our server crashing last night. Its getting to be the weekly se crawl/crash syndrome. Was told by support SE's crawling our sites literally crashed the server. There were no other traffic spikes at the moment.

                Just looking for a more robust option if available.
                hmmm this is highly unusual unless it is vps with 32mb of RAM.

                Anyway by any chance you are running WP sites on that server? And must be one at least on the server, who dosen't have it nowdays. The real culprit could be xmlrpc.php in root of wp site. I have seen recently many attacks on this file and one Quad Core server which usual load is about 2-3 constantly, suddently go high up to 40 because of those attacks from many bad bots trying to hacks wp site thru xmlrpc.

                Run this against access.log (apache or nginx) to see if there are many access attempts to xmlrpc.php

                grep xmlrpc access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn | head -n8

                Those attacks are usually frequent on weekends when "hackers" are out from school and they are hacking sites from their free amazone instances ;)

                I forbid access to xmlrpc.php in nginx and they are gone; since then server is cool.

                More info here Disable XML-RPC in WordPress to Prevent DDoS Attack - BlogAid

                Hope this will help someone

                Comment

                • Denny
                  Too lazy to set a custom title
                  • Feb 2005
                  • 17393

                  #9
                  Originally posted by Markul
                  Server crash because a spider crawls it's sites...? wow... just wow.

                  Comment

                  • JuicyBunny
                    So Fucking Banned
                    • Jun 2010
                    • 2145

                    #10
                    Originally posted by Zverka
                    hmmm this is highly unusual unless it is vps with 32mb of RAM.

                    Anyway by any chance you are running WP sites on that server? And must be one at least on the server, who dosen't have it nowdays. The real culprit could be xmlrpc.php in root of wp site. I have seen recently many attacks on this file and one Quad Core server which usual load is about 2-3 constantly, suddently go high up to 40 because of those attacks from many bad bots trying to hacks wp site thru xmlrpc.

                    Run this against access.log (apache or nginx) to see if there are many access attempts to xmlrpc.php

                    grep xmlrpc access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn | head -n8

                    Those attacks are usually frequent on weekends when "hackers" are out from school and they are hacking sites from their free amazone instances ;)

                    I forbid access to xmlrpc.php in nginx and they are gone; since then server is cool.

                    More info here Disable XML-RPC in WordPress to Prevent DDoS Attack - BlogAid

                    Hope this will help someone
                    Thats pretty helpful. Thanks a lot. Yes there are some WP sites and bots have been hitting certain files, usually search or jpg related.

                    Comment

                    • AdultKing
                      Raise Your Weapon
                      • Jun 2003
                      • 15601

                      #11
                      Originally posted by JuicyBunny
                      Thats pretty helpful. Thanks a lot. Yes there are some WP sites and bots have been hitting certain files, usually search or jpg related.
                      What are the specs of the VPS and how many wordpress sites do you have on it ?

                      Comment

                      • dirtymind
                        Confirmed User
                        • May 2008
                        • 2348

                        #12
                        come to talk to us, i am sure we can make you a good offer for a stable server for you. We have never had a server crash because of crawlers.
                        skype: codercarlos

                        Comment

                        • AdultKing
                          Raise Your Weapon
                          • Jun 2003
                          • 15601

                          #13
                          Originally posted by dirtymind
                          come to talk to us, i am sure we can make you a good offer for a stable server for you. We have never had a server crash because of crawlers.
                          That may be true, but there are more than enough shitty reviews of your services. So I would suggest the OP decline your offer.

                          Comment

                          • aka123
                            Confirmed User
                            • Jul 2014
                            • 4450

                            #14
                            Originally posted by Markul
                            Server crash because a spider crawls it's sites...? wow... just wow.
                            Yeah, must be good server.

                            Though I am not sure do I believe this shit. If it is Google they can for starters limit the crawling intensity from webmaster tools, but site crashing because of SE spiders.. come on..

                            Comment

                            • anexsia
                              Confirmed User
                              • May 2010
                              • 5735

                              #15
                              Originally posted by JuicyBunny
                              Thats pretty helpful. Thanks a lot. Yes there are some WP sites and bots have been hitting certain files, usually search or jpg related.
                              Make sure you have something setup to block wp-login.php bruteforce attacks, those can really cause cpu load to skyrocket. I run a lot of of Wordpress blogs on LEMP Stacks and have it set to only allow my IP through to wp-login.php and then I have Fail2Ban block any user who attempted to try to bruteforce (along with banning anyone who searches for Wordpress exploits). You could also install the popular Wordpress plugin "Limit Login Attempts" which will help against bruteforce attacks as well but I prefer the only allowing my ip method.

                              Comment

                              • PornSEO
                                Confirmed User
                                • Dec 2011
                                • 398

                                #16
                                Originally posted by JuicyBunny
                                Amerinoc is apparently crushed server wise on VPS packages when Search Engines crawl sites.
                                Can any host make an offer to move?
                                Is it a managed VPS?
                                FREE SEO HELP DESK

                                If you have any questions or need any advice related to SEO of porn or mainstream websites, PM them to me. I will answer them in my spare time.

                                Comment

                                • JuicyBunny
                                  So Fucking Banned
                                  • Jun 2010
                                  • 2145

                                  #17
                                  Originally posted by AdultKing
                                  What are the specs of the VPS and how many wordpress sites do you have on it ?
                                  I will get that to you. It does turn out we were being hit by UA and ID IPs at the same time the SE's crawled. I am guessing the bot hits were timed. They've been doing it off and on since late July.

                                  ID - 202.67.41.51 ++ others
                                  UA - 91.200.12.71 ++ others

                                  Toss in HK, VN, CN and KR from time to time. We've had to add more than 70 block rules on the cloud service and add more each day.

                                  We ramped up security on the wp log ins. Now they are shut off after 2-3 attempts.

                                  Comment

                                  • JuicyBunny
                                    So Fucking Banned
                                    • Jun 2010
                                    • 2145

                                    #18
                                    Sent you a PM, AK.

                                    Ukraine keeps pounding our site. Now its, 91.200.12.116 - which is being blocked in cloudflare but still gets through.

                                    Comment

                                    • SBJ
                                      So Fucking Fabulous
                                      • Apr 2003
                                      • 11387

                                      #19
                                      Originally posted by Markul
                                      Server crash because a spider crawls it's sites...? wow... just wow.
                                      Ya Bad spiders!

                                      I've been on all different kinds and sizes of servers on AmeriNoc for over 12 yrs and love the company.. Hence I list them down there in that area.. But as others said look for the real problem before jumping ship or you will be in the same boat on a different server

                                      Comment

                                      • AdultKing
                                        Raise Your Weapon
                                        • Jun 2003
                                        • 15601

                                        #20
                                        Originally posted by JuicyBunny
                                        Sent you a PM, AK.
                                        Replied. I'm sure this can be mitigated.

                                        Comment

                                        • dirtymind
                                          Confirmed User
                                          • May 2008
                                          • 2348

                                          #21
                                          Originally posted by AdultKing
                                          Replied. I'm sure this can be mitigated.
                                          Maybe that you go bash some else instead of me. If you have a problem with me than talk to me directly do not go about and talk shit about our service. I have had plenty of clients coming from the big guys who deliver shitty service. Those three people who are not happy are not representing our service.

                                          So take your petty shit else where.
                                          skype: codercarlos

                                          Comment

                                          • JuicyBunny
                                            So Fucking Banned
                                            • Jun 2010
                                            • 2145

                                            #22
                                            Originally posted by SBJ
                                            Ya Bad spiders!

                                            I've been on all different kinds and sizes of servers on AmeriNoc for over 12 yrs and love the company.. Hence I list them down there in that area.. But as others said look for the real problem before jumping ship or you will be in the same boat on a different server
                                            Love the guys at Amerinoc. Great guys, helpful, supportive and knowledgeable. We've discovered that if you use Amerinoc and decide to add cloudflare you must sign up to cloudflare on your own to get full DNS protection.

                                            Amerinoc did this as a favor to us because we are not tech heads (signing us up). However we are learning we have to do it ourselves and grunt it out learning the tech of it.

                                            Comment

                                            • AdultKing
                                              Raise Your Weapon
                                              • Jun 2003
                                              • 15601

                                              #23
                                              Originally posted by JuicyBunny
                                              Love the guys at Amerinoc. Great guys, helpful, supportive and knowledgeable. We've discovered that if you use Amerinoc and decide to add cloudflare you must sign up to cloudflare on your own to get full DNS protection.

                                              Amerinoc did this as a favor to us because we are not tech heads (signing us up). However we are learning we have to do it ourselves and grunt it out learning the tech of it.
                                              But you haven't made the switch to Cloudflare yet ? Shoot me another PM and I will invite you into one my my Slack channels and help you get this fixed.

                                              Comment

                                              Working...