Web log analysis and load balanced servers - GoFuckYourself.com

BRISK · 01-19-2006, 10:18 AM

Does anyone have suggestions for the best way to view visitor stats for a website that uses multiple servers in a load balanced environment?

Is it possible to get a program like AWStats to look at the log files from multiple servers and report it as if it was just one server?

SinisterStudios · 01-19-2006, 10:20 AM

Quote:

Originally Posted by BRISK

Does anyone have suggestions for the best way to view visitor stats for a website that uses multiple servers in a load balanced environment?

Is it possible to get a program like AWStats to look at the log files from multiple servers and report it as if it was just one server?

Depends on how your load balancing them, do you have a director and children under that or are you doing round robin style balancing?

BRISK · 01-19-2006, 10:34 AM

Quote:

Originally Posted by SinisterStudios

Depends on how your load balancing them, do you have a director and children under that or are you doing round robin style balancing?

Right now using round robin, but thinking of using a proper hardware load balancer in the future. So I'd be interested in hearing about any methods of doing this.

BRISK · 01-23-2006, 08:39 PM

????????

aflex · 01-23-2006, 08:43 PM

Quote:

Originally Posted by BRISK

Right now using round robin, but thinking of using a proper hardware load balancer in the future. So I'd be interested in hearing about any methods of doing this.

F5 BIG IP 9.x <--the hardware you need

9.x has a few traffic analysis tools but not as deep as browser types, reffering url, etc that you would find with like webalizer logs. It also uses iRules so that you can shape your traffic and send them to different servers/pages upon setting a variable.

if you have specific q's about BIG IP, i'm on ICQ

i'm not sure how to add web logs from diff servers to make them viewable as youre requesting but i'm sure theres a way

disregard · 01-23-2006, 08:47 PM

It is possible regardless of how you load balance. Hit me on msn if you want someone to implement it for you.

someblackdude at hotmail dot com

zagi · 01-23-2006, 08:49 PM

After you collect your logs cat them into 1 big file on a central log processing server

cat /server-file-1 >> /total
cat /server-file-2 >> /total
...

Then run the web analyzer on /total file

simple as pie

01-19-2006, 10:18 AM	#1
BRISK Too lazy to set a custom title Join Date: Feb 2003 Posts: 12,240	Web log analysis and load balanced servers Does anyone have suggestions for the best way to view visitor stats for a website that uses multiple servers in a load balanced environment? Is it possible to get a program like AWStats to look at the log files from multiple servers and report it as if it was just one server? __________________ I post on GFY so that when people ask me what I do, I can tell them that I work with the mentally retarded.

01-23-2006, 08:39 PM	#4
BRISK Too lazy to set a custom title Join Date: Feb 2003 Posts: 12,240	???????? __________________ I post on GFY so that when people ask me what I do, I can tell them that I work with the mentally retarded.

01-23-2006, 08:47 PM	#6
disregard Confirmed User Join Date: May 2004 Posts: 118	It is possible regardless of how you load balance. Hit me on msn if you want someone to implement it for you. someblackdude at hotmail dot com __________________ house o' fart

01-23-2006, 08:49 PM	#7
zagi Confirmed User Join Date: Jan 2004 Posts: 1,238	After you collect your logs cat them into 1 big file on a central log processing server cat /server-file-1 >> /total cat /server-file-2 >> /total ... Then run the web analyzer on /total file simple as pie __________________ Managed US/NL Hosting [ [Reality Check Network ] Dell XEON Servers + 1/2/3 TB Packages ICQ: 4-930-562