you probably know about this, but if you don't:
http://www.plentyoffish.com
making around $400k a month with adsense
probably even more with other affiliate programs
up to 5 million pageviews an hour
and amazingly all runs on four servers.
here's some technical info of how he runs it, direct from the site owner - pretty amazing stuff:
quotes might be a little out of context as copied from another forum
http://www.plentyoffish.com
making around $400k a month with adsense
probably even more with other affiliate programs
up to 5 million pageviews an hour
and amazingly all runs on four servers.
here's some technical info of how he runs it, direct from the site owner - pretty amazing stuff:
I have 4 servers.
1. DB server
2. Web server, handles 1 million pageviews an hour at peak. No static pages at all, way to slow. All pages are Gzipped on the fly.
3. Mail Server. Handles 1 million emails/day and also has a webserver that handles a Instant messager. That translates to 4-5 million polling pageviews/hour at peak.
4. Image server, Like all major sites it serves images to a massive content distribution system/cache.
5. Outbound traffic is 70 to 100mb/sec If it was uncompressed it would probably run at 140mb/sec
I have a sort of a AI, that i built that handles the site for me. When you've got 2 years of steady growth you can build something super fancy to automate problems as they come up. I don't have a single competitor with under 200 servers and 200 staff.
1. DB server
2. Web server, handles 1 million pageviews an hour at peak. No static pages at all, way to slow. All pages are Gzipped on the fly.
3. Mail Server. Handles 1 million emails/day and also has a webserver that handles a Instant messager. That translates to 4-5 million polling pageviews/hour at peak.
4. Image server, Like all major sites it serves images to a massive content distribution system/cache.
5. Outbound traffic is 70 to 100mb/sec If it was uncompressed it would probably run at 140mb/sec
I have a sort of a AI, that i built that handles the site for me. When you've got 2 years of steady growth you can build something super fancy to automate problems as they come up. I don't have a single competitor with under 200 servers and 200 staff.
I had a hybrid of asp and asp.net for a while. The more i learned asp.net the more pages i ported over and all new stuff was in .net.. Took me a while
In public filings by match, americansingles, myspace, friendster etc they all have the same pattern. They all need huge numbers of servers to accomplish anything.
Taken from a public filing of matchnet PLC Jan 2003, when they had sub 9 million pageviews/day.
"we currently own approximately 300 web servers, 40 database servers and 12 file servers"
The fact that my site has 10 to 20 times the technical complexity of their sites due to location based searches on every pageview and 14 million pageviews a day gives you an idea of what i've accomplished. Not to mentional all those above mentioned companies have 200 to 600 employees each.
At the end of the day, its all about algorithms and AI and not about platforms/languages..
In public filings by match, americansingles, myspace, friendster etc they all have the same pattern. They all need huge numbers of servers to accomplish anything.
Taken from a public filing of matchnet PLC Jan 2003, when they had sub 9 million pageviews/day.
"we currently own approximately 300 web servers, 40 database servers and 12 file servers"
The fact that my site has 10 to 20 times the technical complexity of their sites due to location based searches on every pageview and 14 million pageviews a day gives you an idea of what i've accomplished. Not to mentional all those above mentioned companies have 200 to 600 employees each.
At the end of the day, its all about algorithms and AI and not about platforms/languages..
I know its hard to believe, whenever i'm exposed to new things it takes me a while to believe 
My db is quad dual core opteron with 32GB/ram. Unlike the other sites i don't have a bunch of 8 way servers. I've got the db CPU & disk bound! (you don't see that often)
My other servers are just ~$3k machines with 2 CPU's. Nothing special. Image server is just a server deadicated to storing and serving images.
All the stuff on optimization i learned on my own. I spent 2 months rewriting a billing system for one of the worlds largest companies. They had a 32 CPU db maxed... I got the run time from 45 days down to 2 minutes. I had to deal with impossible fun stuff like select statements causing deadlocks because there was so much traffic.
From 1990 to 1995 some professors used several super computers and hundreds of computers to find a string of prime numbers. I spent half a year writing a program that made the search thousands of times faster by storing numbers in multipul dimensions and then scanning all dimensions at the same time via a wheel factorization sieve. I found a bigger string of numbers in under 2 weeks on a single machine. Its all public.
Those 2 things taught me how to optimize.... I don't want to get into 2 many tech details because competitors would give anything to know what i'm doing. A few have even asked to lisence my software.
As for wikipedia & forums serving, those are extremely trival you could do 12-60 million pageviews a day off nearly any server as long as you have the drives to handle IO, or some kind of html cache. Also last i read Webmasterworld had 10 or so servers?
My db is quad dual core opteron with 32GB/ram. Unlike the other sites i don't have a bunch of 8 way servers. I've got the db CPU & disk bound! (you don't see that often)
My other servers are just ~$3k machines with 2 CPU's. Nothing special. Image server is just a server deadicated to storing and serving images.
All the stuff on optimization i learned on my own. I spent 2 months rewriting a billing system for one of the worlds largest companies. They had a 32 CPU db maxed... I got the run time from 45 days down to 2 minutes. I had to deal with impossible fun stuff like select statements causing deadlocks because there was so much traffic.
From 1990 to 1995 some professors used several super computers and hundreds of computers to find a string of prime numbers. I spent half a year writing a program that made the search thousands of times faster by storing numbers in multipul dimensions and then scanning all dimensions at the same time via a wheel factorization sieve. I found a bigger string of numbers in under 2 weeks on a single machine. Its all public.
Those 2 things taught me how to optimize.... I don't want to get into 2 many tech details because competitors would give anything to know what i'm doing. A few have even asked to lisence my software.
As for wikipedia & forums serving, those are extremely trival you could do 12-60 million pageviews a day off nearly any server as long as you have the drives to handle IO, or some kind of html cache. Also last i read Webmasterworld had 10 or so servers?
Keeping my site fast and running is one of my smallest issues.
1. Approving and editing 20,000 images/day
2. BLocking 1000 nigerian russian scams, escorts etc per day.
3. Blocking fake accounts, trouble makers etc.
4. 100k+ edits/modications/day
5 to 10% of yahoo's and the industries total signups are scams escorts etc. I'd guess these guys steal on the order of $100-400 million per year from the industry.
1. Approving and editing 20,000 images/day
2. BLocking 1000 nigerian russian scams, escorts etc per day.
3. Blocking fake accounts, trouble makers etc.
4. 100k+ edits/modications/day
5 to 10% of yahoo's and the industries total signups are scams escorts etc. I'd guess these guys steal on the order of $100-400 million per year from the industry.
99% is done by the site AI.
I don't use a load balancer, i've only got one web server serving up the site. I figure in the future round robin DNS will allow me to scale further and still use session data.
My Gzip is turned on, when I turn it off my bandwidth doubles.
As for why it doesn't work for you I don't know what to tell you. I went into the metabase and added aspx and html to the compression settings and then just followed the rest of the tutorial for turning on compression on microsofts site. Don't try and compress images on the fly?
My Gzip is turned on, when I turn it off my bandwidth doubles.
As for why it doesn't work for you I don't know what to tell you. I went into the metabase and added aspx and html to the compression settings and then just followed the rest of the tutorial for turning on compression on microsofts site. Don't try and compress images on the fly?
Comment