Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums.

You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today!

If you have any problems with the registration process or your account login, please contact us.

Post New Thread Reply

Register GFY Rules Calendar
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed.

 
Thread Tools
Old 02-21-2011, 11:10 AM   #1
Marcus
Confirmed User
 
Industry Role:
Join Date: Jul 2001
Posts: 3,472
Hosting Gods: How do you keep a site up and running normally when performing a backup?

I have a site with a good amount of users on it.

Everyday when my site does a backup it creates a dump of the database. It takes about a minute to dump that database, so the tables can not be written to for that time.

It bothers me when this happens because the site is down for 30-60 seconds.
My members think my site sucks, tell each other it sucks, and they leave to go to my competitors. It happens everyday. That's bad for my business.

Is this lock normal? Or can something be done about it?
My competitors sites are much bigger than mine and I dont see them crash. So I dont understand how they do it.

Marcus is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 11:19 AM   #2
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Use mysql replication.... then backup from the slave. That way no mysql downtime at all.
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)



All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 11:25 AM   #3
blackmonsters
Making PHP work
 
blackmonsters's Avatar
 
Industry Role:
Join Date: Nov 2002
Location: 🌎🌅🌈🌇
Posts: 20,484
Quote:
Originally Posted by Marcus View Post
I have a site with a good amount of users on it.

Everyday when my site does a backup it creates a dump of the database. It takes about a minute to dump that database, so the tables can not be written to for that time.

It bothers me when this happens because the site is down for 30-60 seconds.
My members think my site sucks, tell each other it sucks, and they leave to go to my competitors. It happens everyday. That's bad for my business.

Is this lock normal? Or can something be done about it?
My competitors sites are much bigger than mine and I dont see them crash. So I dont understand how they do it.

I wondered about this too, although I don't have the problem myself.
__________________
Make Money with Porn
blackmonsters is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 11:38 AM   #4
pristine
So Fucking Banned
 
Industry Role:
Join Date: Dec 2010
Posts: 1,176
Quote:
Originally Posted by borked View Post
Use mysql replication.... then backup from the slave. That way no mysql downtime at all.
end of thread.
pristine is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 11:42 AM   #5
Marcus
Confirmed User
 
Industry Role:
Join Date: Jul 2001
Posts: 3,472
Quote:
Originally Posted by borked View Post
Use mysql replication.... then backup from the slave. That way no mysql downtime at all.
Do you need 2 servers for this? Im not technically savvy so dont know what the setup is for this.
Marcus is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 11:47 AM   #6
Klen
 
Klen's Avatar
 
Industry Role:
Join Date: Aug 2006
Location: Little Vienna
Posts: 32,235
I use rsync to avoid downtime.
Klen is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 11:52 AM   #7
blackmonsters
Making PHP work
 
blackmonsters's Avatar
 
Industry Role:
Join Date: Nov 2002
Location: 🌎🌅🌈🌇
Posts: 20,484
LOL!

I don't use SQL.
__________________
Make Money with Porn
blackmonsters is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 12:05 PM   #8
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Quote:
Originally Posted by Marcus View Post
Do you need 2 servers for this? Im not technically savvy so dont know what the setup is for this.
well, technically I think you can run two instances of mysql on different ports, but by far the simplest is to run the slave on your backup server or a cheap dedi or even cloud setup. I run mine on a 256MB cloud instance that costs me 1c/hr and have no problems on it. Then that gets backed up to my remote 100GB backup space. Mysql (my most valuable data) is then spread across 3 servers... Backup of sites/configs is spread across 2 servers.

Another option would be to take a cheap 250GB dedi box at ?15/month, run the mysql slave there and use that too for backups, that way everything is spread across 3 drives, but I like the cloud availability (100%) for a replication server...
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)



All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 12:26 PM   #9
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Quote:
Originally Posted by Marcus View Post
Im not technically savvy so dont know what the setup is for this.
you need at least 2 mysql server instances, running on different ports if on the same server. Like I said above, best (and simplest) is to run mysql on another server.

For technical how to - follow this tuto. It's for centos, but effectively shows all the mysql configs which is distro-independent.
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)



All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 12:34 PM   #10
webair
Confirmed User
 
webair's Avatar
 
Industry Role:
Join Date: Feb 2002
Location: NYC, NY
Posts: 8,531
1. Having a secondary MySQL server as a slave and backing up from that is the best option
2. Rsyncing the binary files, while not recommended, can be "safe" for INNODB based tables, however you cannot guarantee consistency and may need to check/repair the table once restored. This is not recommended at all for InnoDB tables as you will also need to restore the binary log data, which may cause unforseen issues with other data in the database.
3. For InnoDB you can either use InnoDB hot copy, or use mysqldump with the "?single-transaction" option enabled.

#1 and 3 are the preferred methods
webair is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 01:05 PM   #11
chaze
Confirmed User
 
Industry Role:
Join Date: Aug 2002
Posts: 9,752
No it's not normal.

The site should not be effected by the backups. I assume they are not using rcync and doing a full backup each time which is overloading your server. Mysql can be rsynced just like everything else.

Last edited by chaze; 02-21-2011 at 01:06 PM..
chaze is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 01:40 PM   #12
BestXXXPorn
Confirmed User
 
BestXXXPorn's Avatar
 
Join Date: Jun 2009
Location: Asheville, NC
Posts: 2,277
borked is right... Best way is to back up from a slave however...

In the interim and if your site's traffic doesn't justify a separate slave box... There are a plethora of flags/options to set in mysqldump to make sure you aren't tanking your MySQL instance during a backup ;)

EDIT: Check out mysqlhotcopy
__________________
ICQ: 258-202-811 | Email: eric{at}bestxxxporn.com

Last edited by BestXXXPorn; 02-21-2011 at 01:42 PM..
BestXXXPorn is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 02:20 PM   #13
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Quote:
Originally Posted by chaze View Post
Mysql can be rsynced just like everything else.
of course it can, but don't expect a simple restore, or any for that matter....

simply copying the /var/lib/mysql/mysql directory is asking for trouble
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)



All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 02:23 PM   #14
CYF
Coupon Guru
 
CYF's Avatar
 
Industry Role:
Join Date: Mar 2009
Location: Minneapolis
Posts: 10,973
Quote:
Originally Posted by borked View Post
of course it can, but don't expect a simple restore, or any for that matter....

simply copying the /var/lib/mysql/mysql directory is asking for trouble
agreed
__________________
Webmaster Coupons Coupons and discounts for hosting, domains, SSL Certs, and more!
AmeriNOC Coupons | Certified Hosting Coupons | Hosting Coupons | Domain Name Coupons

CYF is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 02:37 PM   #15
plsureking
bored
 
plsureking's Avatar
 
Industry Role:
Join Date: Aug 2003
Location: PORNCMS.com
Posts: 4,720
well the best answer is of course never mentioned - stop using mysql.

flat files are the future. well they'll been my present for half a decade. they are fast, easy to backup, and easy to transfer.

look into it.
__________________
#
plsureking is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 02:49 PM   #16
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Quote:
Originally Posted by plsureking View Post
well the best answer is of course never mentioned - stop using mysql.

flat files are the future. well they'll been my present for half a decade. they are fast, easy to backup, and easy to transfer.

look into it.
any chance of geospatial index support in your flat files?
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)



All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 02:52 PM   #17
chaze
Confirmed User
 
Industry Role:
Join Date: Aug 2002
Posts: 9,752
Quote:
Originally Posted by plsureking View Post
well the best answer is of course never mentioned - stop using mysql.

flat files are the future. well they'll been my present for half a decade. they are fast, easy to backup, and easy to transfer.

look into it.
I agree, databases are getting outdated but we still have a good 4-5 years before they are obsolete. MySQL it's simply a cpu hog and not as reliable as flat files.
chaze is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 02:58 PM   #18
plsureking
bored
 
plsureking's Avatar
 
Industry Role:
Join Date: Aug 2003
Location: PORNCMS.com
Posts: 4,720
Quote:
Originally Posted by borked View Post
any chance of geospatial index support in your flat files?
lol ya there's a lot of that going on in porn. nerd.
__________________
#
plsureking is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:00 PM   #19
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Quote:
Originally Posted by chaze View Post
I agree, databases are getting outdated but we still have a good 4-5 years before they are obsolete. MySQL it's simply a cpu hog and not as reliable as flat files.
This is such a fallacy - I have around 14 million mysql db hits and ~1million of those are on very complex joins involving very non-ordinary indexes. Not even taking into account the load added by apache for all those hits, my server load averages only 0.05. And I can back that up with stats to anyone who wants a private argument off the board over email...

What the problem is often frequently craply designed databases. Anyone saying mysql is a hog should run something like mysqltuner on their server and see if all green lights come up. Mine do, and my most used db is 6.5GB in size, and yet mysql is only set to use 4GB of RAM.

phooey.
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)



All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:01 PM   #20
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Quote:
Originally Posted by plsureking View Post
lol ya there's a lot of that going on in porn. nerd.
ehm, there are exactly 2-3 million daily hits using just those. Nerd
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)



All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:02 PM   #21
webair
Confirmed User
 
webair's Avatar
 
Industry Role:
Join Date: Feb 2002
Location: NYC, NY
Posts: 8,531
Quote:
Originally Posted by borked View Post
This is such a fallacy - I have around 14 million mysql db hits and ~1million of those are on very complex joins involving very non-ordinary indexes. Not even taking into account the load added by apache for all those hits, my server load averages only 0.05. And I can back that up with stats to anyone who wants a private argument off the board over email...

What the problem is often frequently craply designed databases. Anyone saying mysql is a hog should run something like mysqltuner on their server and see if all green lights come up. Mine do, and my most used db is 6.5GB in size, and yet mysql is only set to use 4GB of RAM.

phooey.
My fav part:

"What the problem is often frequently craply designed databases. Anyone saying mysql is a hog should run something like mysqltuner on their server and see if all green lights come up. Mine do, and my most used db is 6.5GB in size, and yet mysql is only set to use 4GB of RAM."

Well said, & well done!
webair is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:12 PM   #22
signupdamnit
Confirmed User
 
signupdamnit's Avatar
 
Industry Role:
Join Date: Aug 2007
Posts: 6,697
Most of the time just doing a simple mysqldump of one database then the sites actual files will work fine without any downtime. As long as you do site1db, site1files, site2db, site2files, etc (instead of dumping all dbs then all files) and the databases aren't massive it will be within sync enough. It's even less of a big deal if you keep multiple backups instead of relying on one. Sure to be technical you should lock it all before the backup then unlock it but it usually isn't a big deal for a normal site or small group of them on a server.

Last edited by signupdamnit; 02-21-2011 at 03:13 PM..
signupdamnit is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:20 PM   #23
woj
<&(©¿©)&>
 
woj's Avatar
 
Industry Role:
Join Date: Jul 2002
Location: Chicago
Posts: 47,882
Quote:
Originally Posted by chaze View Post
I agree, databases are getting outdated but we still have a good 4-5 years before they are obsolete. MySQL it's simply a cpu hog and not as reliable as flat files.
you are just trolling, right?
__________________
Custom Software Development, email: woj#at#wojfun#.#com to discuss details or skype: wojl2000 or gchat: wojfun or telegram: wojl2000
Affiliate program tools: Hosted Galleries Manager Banner Manager Video Manager
Wordpress Affiliate Plugin Pic/Movie of the Day Fansign Generator Zip Manager
woj is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:39 PM   #24
Alky
Confirmed User
 
Alky's Avatar
 
Join Date: Apr 2002
Location: Houston
Posts: 5,651
Quote:
Originally Posted by signupdamnit View Post
Most of the time just doing a simple mysqldump of one database then the sites actual files will work fine without any downtime. As long as you do site1db, site1files, site2db, site2files, etc (instead of dumping all dbs then all files) and the databases aren't massive it will be within sync enough. It's even less of a big deal if you keep multiple backups instead of relying on one. Sure to be technical you should lock it all before the backup then unlock it but it usually isn't a big deal for a normal site or small group of them on a server.
mysqldump locks the tables
Alky is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:42 PM   #25
wdsguy
Ryde or Die
 
Industry Role:
Join Date: Dec 2002
Location: California-Shanghai
Posts: 19,568
Quote:
Originally Posted by borked View Post
well, technically I think you can run two instances of mysql on different ports, but by far the simplest is to run the slave on your backup server or a cheap dedi or even cloud setup. I run mine on a 256MB cloud instance that costs me 1c/hr and have no problems on it. Then that gets backed up to my remote 100GB backup space. Mysql (my most valuable data) is then spread across 3 servers... Backup of sites/configs is spread across 2 servers.

Another option would be to take a cheap 250GB dedi box at ?15/month, run the mysql slave there and use that too for backups, that way everything is spread across 3 drives, but I like the cloud availability (100%) for a replication server...

Curious who you use for the cloud service?
wdsguy is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:47 PM   #26
signupdamnit
Confirmed User
 
signupdamnit's Avatar
 
Industry Role:
Join Date: Aug 2007
Posts: 6,697
Quote:
Originally Posted by Alky View Post
mysqldump locks the tables
You can turn that off though with different options.
signupdamnit is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:57 PM   #27
Ron Bennett
Confirmed User
 
Join Date: Oct 2003
Posts: 1,653
Quote:
Originally Posted by Marcus View Post
Everyday when my site does a backup it creates a dump of the database. It takes about a minute to dump that database, so the tables can not be written to for that time.

It bothers me when this happens because the site is down for 30-60 seconds.
My members think my site sucks, tell each other it sucks, and they leave to go to my competitors. It happens everyday. That's bad for my business.
A minute or so of slowness / non-response per day isn't a biggie for most sites if that occurs at an odd time when relatively few visitors are on.

Have you tried scheduling the backup for a very slow time when there are the fewest users on? -if no, check your logs to see when that usually is and change the backup time to match.

Many people do maintenance between 12AM Midnight to 3 AM, but that can often be a bad time when factoring in the location of the server and time zone of most visitors. So don't be surprised if the slowest, more ideal time ends up being somewhat later, such as around 6 AM.

Ron
__________________
Domagon - Website Management and Domain Name Sales
Ron Bennett is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 03:57 PM   #28
Alky
Confirmed User
 
Alky's Avatar
 
Join Date: Apr 2002
Location: Houston
Posts: 5,651
Quote:
Originally Posted by signupdamnit View Post
You can turn that off though with different options.
didn't know that... i guess we never cared to look since we are just backing up from our slave and it doesnt effect us.
Alky is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 07:25 PM   #29
chaze
Confirmed User
 
Industry Role:
Join Date: Aug 2002
Posts: 9,752
Quote:
Originally Posted by woj View Post
you are just trolling, right?
And you are just wrong again right?
chaze is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 07:52 PM   #30
blackmonsters
Making PHP work
 
blackmonsters's Avatar
 
Industry Role:
Join Date: Nov 2002
Location: 🌎🌅🌈🌇
Posts: 20,484
Quote:
Originally Posted by plsureking View Post
well the best answer is of course never mentioned - stop using mysql.
Please see post number 7.

__________________
Make Money with Porn
blackmonsters is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 08:00 PM   #31
pristine
So Fucking Banned
 
Industry Role:
Join Date: Dec 2010
Posts: 1,176
chaze aka "DWHS" is a complete newbie, pretty obvious from his posts and hosting operation.
pristine is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 08:04 PM   #32
Sly
Let's do some business!
 
Sly's Avatar
 
Industry Role:
Join Date: Sep 2004
Location: Austin, TX
Posts: 31,323
Your members run for the hills over 60 seconds of downtime? I have trouble with both Google and Yahoo every day, have never considered going somewhere else? Temporary glitch.
Sly is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 09:26 PM   #33
Marcus
Confirmed User
 
Industry Role:
Join Date: Jul 2001
Posts: 3,472
Quote:
Originally Posted by Sly View Post
Your members run for the hills over 60 seconds of downtime? I have trouble with both Google and Yahoo every day, have never considered going somewhere else? Temporary glitch.
when it happens again and again they get irritated and angry. Ive had cancellations and increasingly angry emails.
Marcus is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 09:29 PM   #34
Supz
Arthur Flegenheimer
 
Supz's Avatar
 
Industry Role:
Join Date: Jul 2006
Location: New York City
Posts: 11,056
Do you backup when you know you have the least amount of people on your site?
Supz is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-21-2011, 11:47 PM   #35
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Quote:
Originally Posted by wdsguy View Post
Curious who you use for the cloud service?
miniCloud - very useful for this kind of stuff or for automation
http://www.ovh.ie/cloud/
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)



All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 02-22-2011, 12:15 AM   #36
plsureking
bored
 
plsureking's Avatar
 
Industry Role:
Join Date: Aug 2003
Location: PORNCMS.com
Posts: 4,720
Quote:
Originally Posted by blackmonsters View Post
Please see post number 7.

haha ya i saw it too late.
__________________
#
plsureking is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Post New Thread Reply
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >

Bookmarks



Advertising inquiries - marketing at gfy dot com

Contact Admin - Advertise - GFY Rules - Top

©2000-, AI Media Network Inc



Powered by vBulletin
Copyright © 2000- Jelsoft Enterprises Limited.