TGP-MGP OWNERS Using Multi-Catagory Gallery Scripts Please Contact me.
 

TGP-MGP OWNERS Using Multi-Catagory Gallery Scripts Please Contact me.. Or developers of them contact me.

Id like to talk about thier effects on your server with the creation and checking processes and the effects of out cgi links performance for surfers.

Dont contact me if you have under 200k in traffic to your sites.

I need to consult with and pick the brain of some established tgp owners with traffic and multi catagory scripts.

icq 66883099

great:disgust

:321GFY

Server 2.4GHz Celeron 512MB (should have 1Gig)
Gallery spider server load: 0.84 (3k galleries in a few hours)
CPU usage while rebuilding pages: 100% for a few minutes.

I don't get it, isn't that the task of the trade script?

I did anyway :upsidedow

xApster will know boneprone!

Boneprone is my lover

Yeah it is. But it seems the other scripts are bogging the server down so the links out are stalling and im taking a traffic hit since sufers are clicking and they are having to wait for either the gallery or trade to load.

I no doubt have super huge loads when the spider script gets and creates, and checks, but im also having swap run and loads even when the cron jpb is not running.

I run ucj, tmanager, and a custom spider and creation, check script. That custom one for sure loads shit and id love to replace it with something better, but even when that thing is not running seems im having link out issues and load.

I know there have to be some people using these same scripts as me, and do more traffic. How are you configuring or dealing with your loads? Putting the ucj and thumb script on a seprate server or what?

I have 2 gigs of Memory so I know memory isnt the issue. Im racked up there! ANd bandwith is not the issue, nor is cpu. It seems my scripts conbined with the traffic is just really putting some loads on and im wondering how some of you deal with it.

Like I said I know a lot of you run these very typical tgp scripts and even do more traffic than I. How the hell you do it?

Can we run ucj, spiders, and thumbmangers on a seprate server than our site?

He runs different scripts.

Has the same trade script, and yeah he has a thumb script, but he doesnt have one that creates gallereis as i do.

check out www.socalmovies.com at the bottom where the multi catagory text links are.

My script is making like 200 new thumb gallereis 4 times a day.

And way I say its making em, i mean its grabing em, checking em, and croping the images off the gallerey to present on the tgp catagory page.

Its doing a lot of shit, and does it pretty fast considering all it has to do. But fuck it loads and stalls things on the sites when it does it.

I know many run similar sites, but Im wondering if its best to have it do these acts on another server or if a dual server is needed? However like I said I have 2 gig memory already so a dual wont really help.

Just wanting some ideas or options here. Im growing and growing but i think as i grow my performance suffers. And the script that is doing the duty isnt necessarly doing all the load since I have it only do it on one site, and then all my other sites share its creations.

last pid: 70390; load averages: 1.71, 1.99, 1.85 up 5+01:23:49 16:01:07
849 processes: 12 running, 780 sleeping, 57 zombie
CPU states: 70.4% user, 0.0% nice, 24.9% system, 4.7% interrupt, 0.0% idle
Mem: 1042M Active, 485M Inact, 374M Wired, 103M Cache, 199M Buf, 7848K Free
Swap: 2048M Total, 67M Used, 1980M Free, 3% Inuse

my scripts really bog down things.

And this is without the cron opperating.

I need someone who has a spider and thumb script for sale to contact me. Similar to the one I have on socalmovies.com

This thread is a good example why i will never start a TGP.
I would need to spend 6 months learning some tech knowledge before i jump into it :1orglaugh

But at least i know some HTML :thumbsup

boneprone why dont you start 10.000 tgps at once?

:winkwink:

Yeah its driving me ape shit....

Getting fucking tired.

Programers blaming my server, and host techs blaming the codes..

Meanwhile I have no fucking clue.

I want to try a more efficient script if there is one.

This one is written in perl.

In addition to this script in question i run UCJ and and thumbmanger and do about 650k uniques. Not sure if UCJ and Thummanger can cause these loads, but I know for damn sure when the perl script runs it kills things!!

But im just wondering why im struggling even when its not running. Im getting a shitload of clicks though..

Anyhow if there are any coders who can make something like what I have on socalmovies.com at the bottom of the site that spiders multi-catagory movie thumbs, rips em and crops em and checks em in bulk with low loads hit me up..

Or does that even exist?

I like to play with HTML

You can create nice sites without using graphics.

Id start 100 if i could.

But im having a hard time with 6.

I need to get things more effificent here.

Maybe have the scripts run on a seprate server or something.
But not sure how to do that, or if these scripts can do that?

Never heard of ucj or thummanger running on a seprate server??

And if i split this and put 3 sites on one server and 3 on the other, the script doing the loading will still be on each server and probably creating the same problem on em both.

Im just lost for words.

boneprone.com has lost soooo much traffic its not funny.

But i need this script that does the loads for modelsgroup.com socalmovies.com and digitalmpegs.com

if i remove it and bring the load down, then those sites die, and then my text sites will grow since the link out click speed will shoot back up for the surfers again.

graphics arent the problem here.

Its the programing and the load and effects of them working.

I dont..

But I know people who do!

:Graucho

its a custom script as well so its not like i can talk to other webmasters as to how they deal with the loads as i normally do with ucj or thumb manager issues.

But like i said, it seems i have problems even when its not running so thats strange as well.

how much traffic can ucj and thumbmanger handle on a server?

if you didn't need the thumb previews Thumbnail Gallery Bush would be a nice script for you... but it only dose text unfortunately.

With the '' old '' way you have problems with running 6 TGP's yes. In first place you should think of having lets say 5/10k tgps with all about 5/10k traffic that has much more effect then having 6 big tgps with 100/200k traffic

I have such script that can do this but i'm not ready to publish it yet but i'm seriously thinking of publishing it. It's text links only so far but its working pretty basic. It has automic updates, it filters traffic on niche, also there is no '' skim '' needed since it's all your own network. The surfer wont even notice he's in your network only.

I'm still betatesting my script but it's turning out good for now. I can update all my sponsor codes in 1 click, ta da.

You'll need to put the spider on a different server otherwise your stuff will periodically slow down. Once you get beyond 200k or so it takes a hell of a lot of cpu power. Its really your only answer.

Maybe its time for a real server :Graucho

http://www.sun.com/servers/midrange/...rev1280_lg.jpg

ok ok .... first off, it MIGHT be the amount of traffic you have on that server thats causing the loads.

second, it may also be the tremendous amount of galleries you have in the database using the custom script. i heard its somewhere between 600k-700k galleries ...

third one ... this ive heard from a couple of webmasters. andy has had problems with his tmanager ... if my memory serves me right, it has something to do with a flaw in the script which had been there since the start. ask slick, the bros at paradisenudes/fantasticnudes about it ... they've had their share of problems with tmanager.


about those processes even if your scripts arent doing anything, it might be because of some memory leaks with the script ... possibly with the custom script(which i will have installed too in a few days and im kinda worried lol) or the tmanager ... but im pretty positive that its not the trade scripts causing the problems. been using UCJ since 1999 and never had problems with it.

ive also had my problems with memory leaks but the tech guys from my host were able to solve em or got around them.

I do not run spiders but have about 12 sites running ucj or other trade scripts/autogallery SQL/banner rotators ect. I had a single processor server and it was always loaded. like in the 2.0-3.0 range. then they move me to another server (duel Xeon 2.4 GhZ 1 Gig ram) and that solved it. loaded went from the 3.0 to like 0.60-1.0. You can't beat a duel processor server.

Shane

hope it helps :) i wont be around for today but ill be watching this trade over the day :)

good luck buddy

But how come I have these loads even when the spider script isnt running?

Of course when it does run its almost dehabiliting but as you see with the numbers i have above it seems high even when its not running..

Can this be ucj with 600k traffic? I dont know if it is? Can it be?

Hmmm..

IVe talked to slick and he's running about 800k or so on his tmanger and its running smooth isnt it??? Last i heard it was..

is that 200k lifetime of the site?

yeah i have the text on boneprone and soft17

runs fine.

The thing here is that somehow the thumbs one of the scipts or ucj or AT2 or maybe just the mere fact of heavy traffic alone is causing load.

IO just need to figure out how to tacle it and what some of you others have done.

As far as simular scripts

Tony at Adult devil makes one here: http://www.adultdevil.com/scripts/directory.shtml (and its on sale hehe)

XP has one here: http://www.tgptools.com/scripts.html


Crystal makes one called Super Directory (Crysty at ICQ: 41313171)

I do not work for any of these people so if that is concidered spam please delete it ;-)

Shane

I see on Socal Movies you're using the new version of the script.
Maybe you could temporary try to use the old version without thumbs and see if this improve things.
I don't think UCJ is the problem, I'm using at2/ucj combination on 2 of my servers. The only difference is I'm still using the old version. Do you have 3 spiders running or one for all 3 sites?

Very simple solution. Really not sure why no one ese has mentioned it.

Load balanced servers. Have your host set it up. I think you can have about 256 servers load-balanced; identical data on each server and then a router / switch determines what server load is the least and sends them to it.

GFY is in a cluster I believe.

I should add the load-balancing is seamless; your surfers won't know or see it.

Cheers,
Matt

Hmmm..
Ill look into that.
I just put in a request to my host and let you know how it goes.

NEvertheless im still in the market for more efficient scripts.

Definitely not necessary. That's overkill. If you're not serving completely dynamic content, load balancing is a waste. Sounds like more of a script issue to me.

...assuming you aren't doing a completely ridiculous amount of traffic. :winkwink:

Get another server to load thumbs upgrade to tmanager gold and host thunbs on a second server the loading of thumbs is what is bogging down apache probably $400 for a 10mbs box to unload the thumbs to and your all good.

You know you're a redneck if you don't know the difference between "your" and "you're."

Hey boneprone. We can develop a software suite for you that will be as efficient and lightweight as possible. There's several options for keeping the load down such as the script running smaller batches, using less resources, spread out over a longer period of time (instead of all crammed at once.) Please contact me if you have the chance at one of my contact methods below. I've got several ideas for you, and hopefully I can help you get this issue resolved.

Thanks,

Matt Madigan

www.imadigan.com
AOL/Yahoo: avanteguard2004
MSN/Email: [email protected]
ICQ: 314-942-262

Ok.. As assumed I can exclude the UCJ script from the list of suspects giving me my problems.. It may create a shitload of zombies on the server especially running it on 6 sites, but it has no real bearing.


"A single UCJ site was running close to 2 million raw hits per day a few years ago on a server of the day with minimal overall load

Other servers run 10+ UCJ installations with suspected traffic levels well over that amount.

The IO routines on UCJ are in C and take the equivialent server load of displaying a couple of additonal thumbnails. You can negate the addition of UCJ to a site by removing 3 thumbnails displayed on your main page."

Tim - UCJ

Ofcourse I don't know your exact server configuration, but mine gets slow as shit as soon as it is going to swap memory. You have only 8 MB Free and 67 MB swapping... You should ask Jupiter to have a look at your configuration files, maybe that can free up some memory/optimize SQL etc. I'm sure they can speed it up a little.

So just put the thumb images on the seprate server?

Or the thumbs script too?

I dont think its the actual images bogging down apache is it? Its the clicks and the processing of the program itself, and the php updates its does on the network of sites that does it isnt it?

1. That load is nothing, start to worrie when it goes over 20
2. Use only script written in c++
3. If you use mysql use mysql 4, much faster
4. You are out of memory, insert more
5. If you don't already, use freebsd, not linux

YEah, Im on swap all the time now, and I can actually see for myself that things are slow when i click on my links. ive been loosing traffic casue links out have been stalling have been moving slow and the surfers are just leaving rather than clicking.

Prod is way down. So much so that boneprone.com is 50% less traffic than it should be. The only reason the over all number to my sites is doing ok is becasue of the latest additions of new sites.

But what im trying to figure out here, is if these loads are normal for someone doing this much traffic with these scripts or abnormal? As far as I know not many people either run these scripts AND do this much traffic, or if they do it seems they have them spread over a few servers. Im just wondering what needs to be done to fix this. I dont want to have to utilize another server when its not needed and a simple solution is avialble, but it seems there just arent many people to cosult with who know, or do my kind of traffic with these scripts.

Everyone has an idea or two, but Id really like to talk to someone who has been or has dealt with what im going through.

As of now, my guesses or the guesses of my host techs are just that. Guesses. Neither of us have really dealt much with TGP script load issues.. But I know many of you have.

Im suprised Im not getting much more insite here.
Ive sent this thread to a bunch of people on icq, and only a small hand full of replies.

What the fuck?

Hit me up when you get a chance. I have been through this numerous time.

Are you a hosting company?

shit sorry bro.
My bad.

Just processed im my head who you were.

Yeah ill hit ya up.
Id like to pick your brain.

Seems the only people hitting me up and trying to help me have been hosting companies.. Kinda irratating. Im not in the market of changing. Im just trying to gather some more info so I can make a educated suggestion and refrences for my hosting tech so he can have a better idea of whats going on..

This is all new to him too. Jupiter hosts many TGPs im sure, but there are only a few sites or networks like mine with the combination of traffic and similar scripts running, and I dont think any of em are on Jupiter.. And those that do have waht I have going on arent exactly reaching out a lending hand to help me or tell me how they configured or deal with these issues.

Again... FUCKERS.

Tmanager gold supports thumb images on a secondary server
dont tell me the family hasent got an update yet :(

Go suck a dick.

Dude, you sure just simply hosting the image thumbs on a seprate server is gunna relieve this?

Its not a bandwith issue, it seems to be a php, mysql, apache, CPU and memory load issue.

Putting the thumbs on another server will only speed up the surfers download time wont it and not the script performance.

Or will it?

YEah I have Gold on socal and digitalmpegs, but not modelsgroup.

Should I get it? And is thummanger casuing this? Or is it my traffic amounts to the scripts causing it?

Did you have loads with your sites?

thumbs on a seperate server tmanger stays on your domain.

5 min's of work per site to do it and see if load goes down.