MySQL querys traffic and load - GoFuckYourself.com

pipp · 06-03-2002, 09:31 AM

If I have a page that connects to a MySQL db and prints results to the page everytime the page is accessed instead off running crontab and uptate the page 2 time's aday so the page becomes static html. How much traffic would this page handle before the mysql php printing becomes slow?

salsbury · 06-03-2002, 09:37 AM

impossible to say exactly without knowing what queries you're doing and knowing about the dataset, but think of it this way. sending a static page costs practically 0 cpu. processing and sending a dynamic page costs X cpu. X * hits-per-second * 120-seconds is going to be a lot more than 0.

however, if you have a small site (20 hits per second or whatever) you might be OK. the biggest thing to watch out for there is concurrent queries. if you have a page that takes 1/5th a CPU second to process, and it's being accessed 20 times per second, you're obviously going to have issues.

realistically, any time you can change a dynamic page to a static page that is updated periodically, do it - any CPU you save now you can use later, and it's easier to make changes now than later when your site is busier.

btw: most programming languages come with methods to determine CPU usage (in seconds). in C and PHP check out getrusage(), in perl check out times(), in ASP check your head you may have sprung a leak.

raymo · 06-03-2002, 09:44 AM

bah, ASP rocks your world. vbscript is the shit.

WTF? · 06-03-2002, 09:59 AM

Anybody knows where to find good free ASP scripts ??? Thanx!

Babaganoosh · 06-03-2002, 09:59 AM

Quote:

Originally posted by raymo
bah, ASP rocks your world. vbscript is the shit.

pipp · 06-03-2002, 10:01 AM

Quote:

Originally posted by raymo
bah, ASP rocks your world. vbscript is the shit.

Your not allowed to speak !

AdultWire · 06-03-2002, 10:01 AM

Database shit also gets exponentially slower with write queries. So if your table is written to on a frequent basis, don't try to use it for high volume reads at the same time.

Brujah · 06-03-2002, 10:24 AM

Quote:

Originally posted by WTF?
Anybody knows where to find good free ASP scripts ??? Thanx!

http://www.hotscripts.com has free scripts in numerous categories. You can almost find anything you want there.

a1ka1ine · 06-03-2002, 10:36 AM

anyone know where i can find an overview of using crontab? i currently have a tgp which calls a php database on every hit and want to optimse! =)

btw, ill shoot anyone who says google

pipp · 06-03-2002, 10:40 AM

Quote:

Originally posted by a1ka1ine
anyone know where i can find an overview of using crontab? i currently have a tgp which calls a php database on every hit and want to optimse! =)

btw, ill shoot anyone who says google

is it php you want to execute ? http://www.planet-source-code.com/vb...d=388&lngWId=8

salsbury · 06-03-2002, 10:46 AM

note that lynx is actually pretty crappy for use in cron. it's one of the worst text browsers, sadly, because it's just so bloated (if it had some cool features included with the bloat, that'd be one thing.. but i'm straying from topic). in cron, it's worse, because it gets confused and sometimes doesn't even request the page you asked for.

if you're on freebsd, check out its built in 'fetch' command. no fuss, no muss. 'fetch -q -o /dev/null url'. on linux, perhaps 'wget' would work, tho its probably overkill

(just not as much as lynx is.. ugh)

Brujah · 06-03-2002, 10:57 AM

I think he wants a static page generated.

If you want cron info, try:
http://www.ualberta.ca/CNS/HELP/unix/crontab.1.html

If you just needed a daily index.html page generated
You could create a shell script like dailyfetch.sh
chmod 755 dailyfetch.sh

move it into your /etc/cron.daily directory if you have one. maybe even /etc/cron.hourly or /etc/cron.half-hourly, etc.

-- dailyfetch.sh --

Code:

#!/bin/sh

/usr/bin/wget -O saved.html  http://urltosave.com/urltosave.php

hope that turned out right.

salsbury · 06-03-2002, 11:02 AM

yeah, that's good - just one last thing i'd suggest (i promise, i'm not trying to spam the thread.

)

wget -O saved.html.new http://url.com/

then something like

if [ -s saved.html.new ]
then
mv saved.html.new saved.html
fi

immediately afterwards. this isn't perfect, but what it'll do is two things:

1) it'll write to a second file instead of the primary one - this is important because if someone hits your page while the wget process is running, they'd potentially get an incomplete page, or worse, a blank page.. and..

2) the -s part will check if the page is more than zero bytes before moving it in to place. if it's not, it'll just ignore it. you could probably have it echo an error if it isn't if you wanted. last thing you'd want is an error popping up at 3AM while you're in bed, causing your pages to come up blank for who knows how many hours.

this is also one of the advantages to static pages.

boldy · 06-03-2002, 11:11 AM

Quote:

Originally posted by raymo
bah, ASP rocks your world. vbscript is the shit.

ban this guy...

mike503 · 06-03-2002, 11:12 AM

Quote:

Originally posted by raymo
bah, ASP rocks your world. vbscript is the shit.

i hope this is a joke..

mike503 · 06-03-2002, 11:13 AM

actually you could use lynx -dump $url > file for the crontab idea...

raymo · 06-04-2002, 06:32 AM

Quote:

Originally posted by boldy

ban this guy...

ha. Horses for courses eh.

Milton · 06-04-2002, 07:07 AM

06-03-2002, 09:31 AM	#1
pipp Confirmed User Join Date: Oct 2001 Location: Where the sun don't shine Posts: 1,185	MySQL querys traffic and load If I have a page that connects to a MySQL db and prints results to the page everytime the page is accessed instead off running crontab and uptate the page 2 time's aday so the page becomes static html. How much traffic would this page handle before the mysql php printing becomes slow?

06-03-2002, 09:37 AM	#2
salsbury Confirmed User Join Date: Feb 2002 Location: Seattle Posts: 1,070	impossible to say exactly without knowing what queries you're doing and knowing about the dataset, but think of it this way. sending a static page costs practically 0 cpu. processing and sending a dynamic page costs X cpu. X * hits-per-second * 120-seconds is going to be a lot more than 0. however, if you have a small site (20 hits per second or whatever) you might be OK. the biggest thing to watch out for there is concurrent queries. if you have a page that takes 1/5th a CPU second to process, and it's being accessed 20 times per second, you're obviously going to have issues. realistically, any time you can change a dynamic page to a static page that is updated periodically, do it - any CPU you save now you can use later, and it's easier to make changes now than later when your site is busier. btw: most programming languages come with methods to determine CPU usage (in seconds). in C and PHP check out getrusage(), in perl check out times(), in ASP check your head you may have sprung a leak. __________________ Last edited by salsbury; 06-03-2002 at 09:40 AM..

06-03-2002, 09:44 AM	#3
raymo Registered User Join Date: May 2002 Location: AU Posts: 96	bah, ASP rocks your world. vbscript is the shit. __________________ <br>my weeds have sprouted

06-03-2002, 09:59 AM	#4
WTF? Confirmed User Join Date: May 2002 Posts: 264	Anybody knows where to find good free ASP scripts ??? Thanx! __________________ ..

06-03-2002, 10:01 AM	#7
AdultWire Confirmed User Join Date: Feb 2002 Location: Toronto, ON Posts: 962	Database shit also gets exponentially slower with write queries. So if your table is written to on a frequent basis, don't try to use it for high volume reads at the same time. __________________ SIG TOO BIG! Maximum 120x60 button and no more than 3 text lines of DEFAULT SIZE and COLOR. Unless your sig is for a GFY top banner sponsor, then you may use a 624x80 instead of a 120x60.

06-03-2002, 10:36 AM	#9
a1ka1ine Confirmed User Industry Role: Join Date: Apr 2002 Posts: 3,387	anyone know where i can find an overview of using crontab? i currently have a tgp which calls a php database on every hit and want to optimse! =) btw, ill shoot anyone who says google

06-03-2002, 10:46 AM	#11
salsbury Confirmed User Join Date: Feb 2002 Location: Seattle Posts: 1,070	note that lynx is actually pretty crappy for use in cron. it's one of the worst text browsers, sadly, because it's just so bloated (if it had some cool features included with the bloat, that'd be one thing.. but i'm straying from topic). in cron, it's worse, because it gets confused and sometimes doesn't even request the page you asked for. if you're on freebsd, check out its built in 'fetch' command. no fuss, no muss. 'fetch -q -o /dev/null url'. on linux, perhaps 'wget' would work, tho its probably overkill (just not as much as lynx is.. ugh) __________________

06-03-2002, 10:57 AM	#12
Brujah Beer Money Baron Industry Role: Join Date: Jan 2001 Location: brujah / gmail Posts: 22,157	I think he wants a static page generated. If you want cron info, try: http://www.ualberta.ca/CNS/HELP/unix/crontab.1.html If you just needed a daily index.html page generated You could create a shell script like dailyfetch.sh chmod 755 dailyfetch.sh move it into your /etc/cron.daily directory if you have one. maybe even /etc/cron.hourly or /etc/cron.half-hourly, etc. -- dailyfetch.sh -- Code: #!/bin/sh /usr/bin/wget -O saved.html http://urltosave.com/urltosave.php hope that turned out right. __________________ Free Premium Domain Lists and Tools at Clickmojo.com For Sale: Obscenity.com

06-03-2002, 11:02 AM	#13
salsbury Confirmed User Join Date: Feb 2002 Location: Seattle Posts: 1,070	yeah, that's good - just one last thing i'd suggest (i promise, i'm not trying to spam the thread. ) wget -O saved.html.new http://url.com/ then something like if [ -s saved.html.new ] then mv saved.html.new saved.html fi immediately afterwards. this isn't perfect, but what it'll do is two things: 1) it'll write to a second file instead of the primary one - this is important because if someone hits your page while the wget process is running, they'd potentially get an incomplete page, or worse, a blank page.. and.. 2) the -s part will check if the page is more than zero bytes before moving it in to place. if it's not, it'll just ignore it. you could probably have it echo an error if it isn't if you wanted. last thing you'd want is an error popping up at 3AM while you're in bed, causing your pages to come up blank for who knows how many hours. this is also one of the advantages to static pages. __________________

06-03-2002, 11:13 AM	#16
mike503 Confirmed User Industry Role: Join Date: May 2002 Location: oregon. Posts: 2,243	actually you could use lynx -dump $url > file for the crontab idea... __________________ php/mysql guru. hosting, coding, all that jazz.

06-04-2002, 07:07 AM	#18
Milton Confirmed User Join Date: Apr 2002 Location: Mauritius Posts: 1,118	haha