Hosting Gods: How do you keep a site up and running normally when performing a backup?

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Marcus
    Confirmed User
    • Jul 2001
    • 3472
    #1

    Hosting Gods: How do you keep a site up and running normally when performing a backup?

    I have a site with a good amount of users on it.

    Everyday when my site does a backup it creates a dump of the database. It takes about a minute to dump that database, so the tables can not be written to for that time.

    It bothers me when this happens because the site is down for 30-60 seconds.
    My members think my site sucks, tell each other it sucks, and they leave to go to my competitors. It happens everyday. That's bad for my business.

    Is this lock normal? Or can something be done about it?
    My competitors sites are much bigger than mine and I dont see them crash. So I dont understand how they do it.

    Tags: None
    • borked
      Totally Borked
      • Feb 2005
      • 6284
      #2
      Use mysql replication.... then backup from the slave. That way no mysql downtime at all.

      For coding work - hit me up on andy // borkedcoder // com
      (consider figuring out the email as test #1)


      All models are wrong, but some are useful. George E.P. Box. p202

        Comment

        • blackmonsters
          Making PHP work
          • Nov 2002
          • 20999
          #3
          Originally posted by Marcus
          I have a site with a good amount of users on it.

          Everyday when my site does a backup it creates a dump of the database. It takes about a minute to dump that database, so the tables can not be written to for that time.

          It bothers me when this happens because the site is down for 30-60 seconds.
          My members think my site sucks, tell each other it sucks, and they leave to go to my competitors. It happens everyday. That's bad for my business.

          Is this lock normal? Or can something be done about it?
          My competitors sites are much bigger than mine and I dont see them crash. So I dont understand how they do it.

          I wondered about this too, although I don't have the problem myself.
          Your neighbor will murder you with frogs
          Click here

            Comment

            • pristine
              So Fucking Banned
              • Dec 2010
              • 1176
              #4
              Originally posted by borked
              Use mysql replication.... then backup from the slave. That way no mysql downtime at all.
              end of thread.

                Comment

                • Marcus
                  Confirmed User
                  • Jul 2001
                  • 3472
                  #5
                  Originally posted by borked
                  Use mysql replication.... then backup from the slave. That way no mysql downtime at all.
                  Do you need 2 servers for this? Im not technically savvy so dont know what the setup is for this.

                    Comment

                    • Klen
                      • Aug 2006
                      • 32234
                      #6
                      I use rsync to avoid downtime.

                        Comment

                        • blackmonsters
                          Making PHP work
                          • Nov 2002
                          • 20999
                          #7
                          LOL!

                          I don't use SQL.
                          Your neighbor will murder you with frogs
                          Click here

                            Comment

                            • borked
                              Totally Borked
                              • Feb 2005
                              • 6284
                              #8
                              Originally posted by Marcus
                              Do you need 2 servers for this? Im not technically savvy so dont know what the setup is for this.
                              well, technically I think you can run two instances of mysql on different ports, but by far the simplest is to run the slave on your backup server or a cheap dedi or even cloud setup. I run mine on a 256MB cloud instance that costs me 1c/hr and have no problems on it. Then that gets backed up to my remote 100GB backup space. Mysql (my most valuable data) is then spread across 3 servers... Backup of sites/configs is spread across 2 servers.

                              Another option would be to take a cheap 250GB dedi box at ?15/month, run the mysql slave there and use that too for backups, that way everything is spread across 3 drives, but I like the cloud availability (100%) for a replication server...

                              For coding work - hit me up on andy // borkedcoder // com
                              (consider figuring out the email as test #1)


                              All models are wrong, but some are useful. George E.P. Box. p202

                                Comment

                                • borked
                                  Totally Borked
                                  • Feb 2005
                                  • 6284
                                  #9
                                  Originally posted by Marcus
                                  Im not technically savvy so dont know what the setup is for this.
                                  you need at least 2 mysql server instances, running on different ports if on the same server. Like I said above, best (and simplest) is to run mysql on another server.

                                  For technical how to - follow this tuto. It's for centos, but effectively shows all the mysql configs which is distro-independent.

                                  For coding work - hit me up on andy // borkedcoder // com
                                  (consider figuring out the email as test #1)


                                  All models are wrong, but some are useful. George E.P. Box. p202

                                    Comment

                                    • webair
                                      Confirmed User
                                      • Feb 2002
                                      • 8531
                                      #10
                                      1. Having a secondary MySQL server as a slave and backing up from that is the best option
                                      2. Rsyncing the binary files, while not recommended, can be "safe" for INNODB based tables, however you cannot guarantee consistency and may need to check/repair the table once restored. This is not recommended at all for InnoDB tables as you will also need to restore the binary log data, which may cause unforseen issues with other data in the database.
                                      3. For InnoDB you can either use InnoDB hot copy, or use mysqldump with the "?single-transaction" option enabled.

                                      #1 and 3 are the preferred methods


                                      ~ Webair Dedicated Cloud Servers™ ~ WEBAIR VSYS™ Virtual Hosting Platform ~ Superior CDN Network ~
                                      ~ Managed Dedicated hosting Specialists ~ DISCOUNT DOMAIN NAMES! ~ WEBAIR FUSION IO MANAGED CLOUD SERVERS! ~

                                      ICQ: 243116321 - TWITTER - @WEBAIRINC - E-Mail: [email protected]

                                        Comment

                                        • chaze
                                          Confirmed User
                                          • Aug 2002
                                          • 9774
                                          #11
                                          No it's not normal.

                                          The site should not be effected by the backups. I assume they are not using rcync and doing a full backup each time which is overloading your server. Mysql can be rsynced just like everything else.
                                          Last edited by chaze; 02-21-2011, 11:06 AM.
                                          Like the desert needs the rain
                                          We do fully manged WordPress, VPS, and Servers. Adult Host Pro https://adulthostpro.com/ Since 2001

                                            Comment

                                            • BestXXXPorn
                                              Confirmed User
                                              • Jun 2009
                                              • 2277
                                              #12
                                              borked is right... Best way is to back up from a slave however...

                                              In the interim and if your site's traffic doesn't justify a separate slave box... There are a plethora of flags/options to set in mysqldump to make sure you aren't tanking your MySQL instance during a backup ;)

                                              EDIT: Check out mysqlhotcopy
                                              Last edited by BestXXXPorn; 02-21-2011, 11:42 AM.
                                              ICQ: 258-202-811 | Email: eric{at}bestxxxporn.com

                                                Comment

                                                • borked
                                                  Totally Borked
                                                  • Feb 2005
                                                  • 6284
                                                  #13
                                                  Originally posted by chaze
                                                  Mysql can be rsynced just like everything else.
                                                  of course it can, but don't expect a simple restore, or any for that matter....

                                                  simply copying the /var/lib/mysql/mysql directory is asking for trouble

                                                  For coding work - hit me up on andy // borkedcoder // com
                                                  (consider figuring out the email as test #1)


                                                  All models are wrong, but some are useful. George E.P. Box. p202

                                                    Comment

                                                    • CYF
                                                      Coupon Guru
                                                      • Mar 2009
                                                      • 10973
                                                      #14
                                                      Originally posted by borked
                                                      of course it can, but don't expect a simple restore, or any for that matter....

                                                      simply copying the /var/lib/mysql/mysql directory is asking for trouble
                                                      agreed
                                                      Webmaster Coupons Coupons and discounts for hosting, domains, SSL Certs, and more!
                                                      AmeriNOC Coupons | Certified Hosting Coupons | Hosting Coupons | Domain Name Coupons

                                                        Comment

                                                        • plsureking
                                                          bored
                                                          • Aug 2003
                                                          • 4909
                                                          #15
                                                          well the best answer is of course never mentioned - stop using mysql.

                                                          flat files are the future. well they'll been my present for half a decade. they are fast, easy to backup, and easy to transfer.

                                                          look into it.
                                                          PornCMS / low cost paysite management with hosting

                                                            Comment

                                                            • borked
                                                              Totally Borked
                                                              • Feb 2005
                                                              • 6284
                                                              #16
                                                              Originally posted by plsureking
                                                              well the best answer is of course never mentioned - stop using mysql.

                                                              flat files are the future. well they'll been my present for half a decade. they are fast, easy to backup, and easy to transfer.

                                                              look into it.
                                                              any chance of geospatial index support in your flat files?

                                                              For coding work - hit me up on andy // borkedcoder // com
                                                              (consider figuring out the email as test #1)


                                                              All models are wrong, but some are useful. George E.P. Box. p202

                                                                Comment

                                                                • chaze
                                                                  Confirmed User
                                                                  • Aug 2002
                                                                  • 9774
                                                                  #17
                                                                  Originally posted by plsureking
                                                                  well the best answer is of course never mentioned - stop using mysql.

                                                                  flat files are the future. well they'll been my present for half a decade. they are fast, easy to backup, and easy to transfer.

                                                                  look into it.
                                                                  I agree, databases are getting outdated but we still have a good 4-5 years before they are obsolete. MySQL it's simply a cpu hog and not as reliable as flat files.
                                                                  Like the desert needs the rain
                                                                  We do fully manged WordPress, VPS, and Servers. Adult Host Pro https://adulthostpro.com/ Since 2001

                                                                    Comment

                                                                    • plsureking
                                                                      bored
                                                                      • Aug 2003
                                                                      • 4909
                                                                      #18
                                                                      Originally posted by borked
                                                                      any chance of geospatial index support in your flat files?
                                                                      lol ya there's a lot of that going on in porn. nerd.
                                                                      PornCMS / low cost paysite management with hosting

                                                                        Comment

                                                                        • borked
                                                                          Totally Borked
                                                                          • Feb 2005
                                                                          • 6284
                                                                          #19
                                                                          Originally posted by chaze
                                                                          I agree, databases are getting outdated but we still have a good 4-5 years before they are obsolete. MySQL it's simply a cpu hog and not as reliable as flat files.
                                                                          This is such a fallacy - I have around 14 million mysql db hits and ~1million of those are on very complex joins involving very non-ordinary indexes. Not even taking into account the load added by apache for all those hits, my server load averages only 0.05. And I can back that up with stats to anyone who wants a private argument off the board over email...

                                                                          What the problem is often frequently craply designed databases. Anyone saying mysql is a hog should run something like mysqltuner on their server and see if all green lights come up. Mine do, and my most used db is 6.5GB in size, and yet mysql is only set to use 4GB of RAM.

                                                                          phooey.

                                                                          For coding work - hit me up on andy // borkedcoder // com
                                                                          (consider figuring out the email as test #1)


                                                                          All models are wrong, but some are useful. George E.P. Box. p202

                                                                            Comment

                                                                            • borked
                                                                              Totally Borked
                                                                              • Feb 2005
                                                                              • 6284
                                                                              #20
                                                                              Originally posted by plsureking
                                                                              lol ya there's a lot of that going on in porn. nerd.
                                                                              ehm, there are exactly 2-3 million daily hits using just those. Nerd

                                                                              For coding work - hit me up on andy // borkedcoder // com
                                                                              (consider figuring out the email as test #1)


                                                                              All models are wrong, but some are useful. George E.P. Box. p202

                                                                                Comment

                                                                                • webair
                                                                                  Confirmed User
                                                                                  • Feb 2002
                                                                                  • 8531
                                                                                  #21
                                                                                  Originally posted by borked
                                                                                  This is such a fallacy - I have around 14 million mysql db hits and ~1million of those are on very complex joins involving very non-ordinary indexes. Not even taking into account the load added by apache for all those hits, my server load averages only 0.05. And I can back that up with stats to anyone who wants a private argument off the board over email...

                                                                                  What the problem is often frequently craply designed databases. Anyone saying mysql is a hog should run something like mysqltuner on their server and see if all green lights come up. Mine do, and my most used db is 6.5GB in size, and yet mysql is only set to use 4GB of RAM.

                                                                                  phooey.
                                                                                  My fav part:

                                                                                  "What the problem is often frequently craply designed databases. Anyone saying mysql is a hog should run something like mysqltuner on their server and see if all green lights come up. Mine do, and my most used db is 6.5GB in size, and yet mysql is only set to use 4GB of RAM."

                                                                                  Well said, & well done!


                                                                                  ~ Webair Dedicated Cloud Servers™ ~ WEBAIR VSYS™ Virtual Hosting Platform ~ Superior CDN Network ~
                                                                                  ~ Managed Dedicated hosting Specialists ~ DISCOUNT DOMAIN NAMES! ~ WEBAIR FUSION IO MANAGED CLOUD SERVERS! ~

                                                                                  ICQ: 243116321 - TWITTER - @WEBAIRINC - E-Mail: [email protected]

                                                                                    Comment

                                                                                    • signupdamnit
                                                                                      Confirmed User
                                                                                      • Aug 2007
                                                                                      • 6697
                                                                                      #22
                                                                                      Most of the time just doing a simple mysqldump of one database then the sites actual files will work fine without any downtime. As long as you do site1db, site1files, site2db, site2files, etc (instead of dumping all dbs then all files) and the databases aren't massive it will be within sync enough. It's even less of a big deal if you keep multiple backups instead of relying on one. Sure to be technical you should lock it all before the backup then unlock it but it usually isn't a big deal for a normal site or small group of them on a server.
                                                                                      Last edited by signupdamnit; 02-21-2011, 01:13 PM.

                                                                                      You don't like my posts? Put me on ignore or fuck right off. I'll say what I want.

                                                                                        Comment

                                                                                        • woj
                                                                                          <&(©¿©)&>
                                                                                          • Jul 2002
                                                                                          • 47880
                                                                                          #23
                                                                                          Originally posted by chaze
                                                                                          I agree, databases are getting outdated but we still have a good 4-5 years before they are obsolete. MySQL it's simply a cpu hog and not as reliable as flat files.
                                                                                          you are just trolling, right?
                                                                                          Custom Software Development, email: woj#at#wojfun#.#com to discuss details or skype: wojl2000 or gchat: wojfun or telegram: wojl2000
                                                                                          Affiliate program tools: Hosted Galleries Manager Banner Manager Video Manager
                                                                                          Wordpress Affiliate Plugin Pic/Movie of the Day Fansign Generator Zip Manager

                                                                                            Comment

                                                                                            • Alky
                                                                                              Confirmed User
                                                                                              • Apr 2002
                                                                                              • 5651
                                                                                              #24
                                                                                              Originally posted by signupdamnit
                                                                                              Most of the time just doing a simple mysqldump of one database then the sites actual files will work fine without any downtime. As long as you do site1db, site1files, site2db, site2files, etc (instead of dumping all dbs then all files) and the databases aren't massive it will be within sync enough. It's even less of a big deal if you keep multiple backups instead of relying on one. Sure to be technical you should lock it all before the backup then unlock it but it usually isn't a big deal for a normal site or small group of them on a server.
                                                                                              mysqldump locks the tables

                                                                                                Comment

                                                                                                • wdsguy
                                                                                                  Ryde or Die
                                                                                                  • Dec 2002
                                                                                                  • 19568
                                                                                                  #25
                                                                                                  Originally posted by borked
                                                                                                  well, technically I think you can run two instances of mysql on different ports, but by far the simplest is to run the slave on your backup server or a cheap dedi or even cloud setup. I run mine on a 256MB cloud instance that costs me 1c/hr and have no problems on it. Then that gets backed up to my remote 100GB backup space. Mysql (my most valuable data) is then spread across 3 servers... Backup of sites/configs is spread across 2 servers.

                                                                                                  Another option would be to take a cheap 250GB dedi box at ?15/month, run the mysql slave there and use that too for backups, that way everything is spread across 3 drives, but I like the cloud availability (100%) for a replication server...

                                                                                                  Curious who you use for the cloud service?

                                                                                                    Comment

                                                                                                    • signupdamnit
                                                                                                      Confirmed User
                                                                                                      • Aug 2007
                                                                                                      • 6697
                                                                                                      #26
                                                                                                      Originally posted by Alky
                                                                                                      mysqldump locks the tables
                                                                                                      You can turn that off though with different options.

                                                                                                      You don't like my posts? Put me on ignore or fuck right off. I'll say what I want.

                                                                                                        Comment

                                                                                                        • Ron Bennett
                                                                                                          Confirmed User
                                                                                                          • Oct 2003
                                                                                                          • 1653
                                                                                                          #27
                                                                                                          Originally posted by Marcus
                                                                                                          Everyday when my site does a backup it creates a dump of the database. It takes about a minute to dump that database, so the tables can not be written to for that time.

                                                                                                          It bothers me when this happens because the site is down for 30-60 seconds.
                                                                                                          My members think my site sucks, tell each other it sucks, and they leave to go to my competitors. It happens everyday. That's bad for my business.
                                                                                                          A minute or so of slowness / non-response per day isn't a biggie for most sites if that occurs at an odd time when relatively few visitors are on.

                                                                                                          Have you tried scheduling the backup for a very slow time when there are the fewest users on? -if no, check your logs to see when that usually is and change the backup time to match.

                                                                                                          Many people do maintenance between 12AM Midnight to 3 AM, but that can often be a bad time when factoring in the location of the server and time zone of most visitors. So don't be surprised if the slowest, more ideal time ends up being somewhat later, such as around 6 AM.

                                                                                                          Ron
                                                                                                          Domagon - Website Management and Domain Name Sales

                                                                                                            Comment

                                                                                                            • Alky
                                                                                                              Confirmed User
                                                                                                              • Apr 2002
                                                                                                              • 5651
                                                                                                              #28
                                                                                                              Originally posted by signupdamnit
                                                                                                              You can turn that off though with different options.
                                                                                                              didn't know that... i guess we never cared to look since we are just backing up from our slave and it doesnt effect us.

                                                                                                                Comment

                                                                                                                • chaze
                                                                                                                  Confirmed User
                                                                                                                  • Aug 2002
                                                                                                                  • 9774
                                                                                                                  #29
                                                                                                                  Originally posted by woj
                                                                                                                  you are just trolling, right?
                                                                                                                  And you are just wrong again right?
                                                                                                                  Like the desert needs the rain
                                                                                                                  We do fully manged WordPress, VPS, and Servers. Adult Host Pro https://adulthostpro.com/ Since 2001

                                                                                                                    Comment

                                                                                                                    • blackmonsters
                                                                                                                      Making PHP work
                                                                                                                      • Nov 2002
                                                                                                                      • 20999
                                                                                                                      #30
                                                                                                                      Originally posted by plsureking
                                                                                                                      well the best answer is of course never mentioned - stop using mysql.
                                                                                                                      Please see post number 7.

                                                                                                                      Your neighbor will murder you with frogs
                                                                                                                      Click here

                                                                                                                        Comment

                                                                                                                        • pristine
                                                                                                                          So Fucking Banned
                                                                                                                          • Dec 2010
                                                                                                                          • 1176
                                                                                                                          #31
                                                                                                                          chaze aka "DWHS" is a complete newbie, pretty obvious from his posts and hosting operation.

                                                                                                                            Comment

                                                                                                                            • Sly
                                                                                                                              Let's do some business!
                                                                                                                              • Sep 2004
                                                                                                                              • 31378
                                                                                                                              #32
                                                                                                                              Your members run for the hills over 60 seconds of downtime? I have trouble with both Google and Yahoo every day, have never considered going somewhere else? Temporary glitch.
                                                                                                                              Vacares - Web Hosting, Domains, O365, Security & More - Paxum and BTC Accepted

                                                                                                                              Windows VPS now available
                                                                                                                              Great for TSS, Nifty Stats, remote work, virtual assistants, etc.
                                                                                                                              Click here for more details.

                                                                                                                                Comment

                                                                                                                                • Marcus
                                                                                                                                  Confirmed User
                                                                                                                                  • Jul 2001
                                                                                                                                  • 3472
                                                                                                                                  #33
                                                                                                                                  Originally posted by Sly
                                                                                                                                  Your members run for the hills over 60 seconds of downtime? I have trouble with both Google and Yahoo every day, have never considered going somewhere else? Temporary glitch.
                                                                                                                                  when it happens again and again they get irritated and angry. Ive had cancellations and increasingly angry emails.

                                                                                                                                    Comment

                                                                                                                                    • Supz
                                                                                                                                      Arthur Flegenheimer
                                                                                                                                      • Jul 2006
                                                                                                                                      • 11057
                                                                                                                                      #34
                                                                                                                                      Do you backup when you know you have the least amount of people on your site?

                                                                                                                                        Comment

                                                                                                                                        • borked
                                                                                                                                          Totally Borked
                                                                                                                                          • Feb 2005
                                                                                                                                          • 6284
                                                                                                                                          #35
                                                                                                                                          Originally posted by wdsguy
                                                                                                                                          Curious who you use for the cloud service?
                                                                                                                                          miniCloud - very useful for this kind of stuff or for automation
                                                                                                                                          http://www.ovh.ie/cloud/

                                                                                                                                          For coding work - hit me up on andy // borkedcoder // com
                                                                                                                                          (consider figuring out the email as test #1)


                                                                                                                                          All models are wrong, but some are useful. George E.P. Box. p202

                                                                                                                                            Comment

                                                                                                                                            • plsureking
                                                                                                                                              bored
                                                                                                                                              • Aug 2003
                                                                                                                                              • 4909
                                                                                                                                              #36
                                                                                                                                              Originally posted by blackmonsters
                                                                                                                                              Please see post number 7.

                                                                                                                                              haha ya i saw it too late.
                                                                                                                                              PornCMS / low cost paysite management with hosting

                                                                                                                                                Comment

                                                                                                                                                Previous template Next
                                                                                                                                                Working...