NATNET Clients and Friends...Outage Report - GoFuckYourself.com

SweetT · 05-27-2004, 12:02 AM

Hey Guys....

Just a quick note to let you know that we are aware of the issues and are working to resolve them now.

All that we know at this point is that simultaneoulsy something affected our entire HVAC infrastructure. This caused the temperature to rise in our Data Center and is causing heat loads on several peices of equipment. This is why some servers are affected and others are not.

We have HVAC people on site working on the situation, and we have called in for portable Air Conditioning that should be setup within an hour or so in case the normal systems are not repaired by then. Either way, we dont anticipate the sporradic outages that you are seeing to last much longer.

Thanks for your continued trust and confidence, and rest assured we will get this completed ASAP.

Tony Morgan
President
NationalNet, Inc.

Global Dialers · 05-27-2004, 12:07 AM

you guys rock Tony

p.s... buy my sig ;)

Lensman · 05-27-2004, 12:12 AM

Outages can happen to anyone, sorry to hear that T.

I'm sure they'll get things up soon.

Fletch XXX · 05-27-2004, 12:15 AM

Professional.

As any good host, address the issue on GFY within time.

THAt speaks further than a fucking server special.

Ill put my money with Nat Net or Webair because of service like this.

excellent support.

I didnt see my NatNet tour go down but others did see downtime, awesome to know its all good and bumping.

cheers all.

StuartD · 05-27-2004, 12:15 AM

Quote:

Originally posted by Lensman
Outages can happen to anyone, sorry to hear that T.

I'm sure they'll get things up soon.

yup, that's why no one can claim 100% uptime... these things happen, completely beyond anyone's control.

Good of you to take the time to come and make an announcement on the situation yourself Tony, very admirable of you

Brujah · 05-27-2004, 12:16 AM

the servers that are down, are overheating ?

the Shemp · 05-27-2004, 12:21 AM

Nice to see the Boss up at 03:00

SweetT · 05-27-2004, 12:38 AM

Thanks for the kind words everbody.

Yes, this type of thing can happen to anyone. It is why we always hate hearing about any hosting company having issues because we know it could just as easily be us.

Brujah....it is not exactly that the servers are overheating...it is the electrical systems that are suffering the effects of the heat. UPS's and Power Strips do not work well in the heat, so they kick before anything dramatic would ever affect the server. I hope that answers your question.

Shemp....Well, I am sure that anyone in my position would do the same thing. We actually have a very good "Crisis Team" made up of 3rd Level System Administrators, Network Engineers, and Senior Management that can be assembled in less than 30 minutes for occassions such as this. Of course, tonight I would rather have a team of Air Conditioning mechanics on staff!! LOL!

We are working to power down all non-essential, non-customer equipment that is running in order to bring back up more of the sites that are down but we will not be able to be back in full production until the Portable Air arrives or until the HVAC Infrastructure is repaired.

I will keep you posted.

--T

makefuckingmoney · 05-27-2004, 01:06 AM

open some windows

ill fedex a big fan for my server

On a serious note, I have all confidence in the nat net people..

pradaboy · 05-27-2004, 01:09 AM

thnx for letting us know what's up Tony... I'm sure you guys will get it solved ASAP.

EviLGuY · 05-27-2004, 01:18 AM

Seems to be a bad week for adult hosting companies. NatNet is top drawer though, so if you have a server there you wont be down a second longer then absolutely neccesary.

SweetT · 05-27-2004, 01:21 AM

A quick update....

The first set of HVAC units are coming back online as I type this and the redundant systems should be online soon after. The portable units are being installed anyway, as a fail safe, until we can determine what happened and put together a plan so that this is never repeated.

Once the HVAC system and the portable units are all running it will take a little while for the Data Center to cool back down to optimal temperature so that we can begin to reboot all of the devices that are still down.

The good news is that we are on the downhill side of this now....thanks for your continued trust and support!

--T

SleazyDream · 05-27-2004, 01:25 AM

i pity the poor techs that were on duty when this happened.

TheDoc · 05-27-2004, 01:25 AM

Thanks for letting us know Tony, very professional....

Calvinguy · 05-27-2004, 01:26 AM

For how many hours has this been going on?

makefuckingmoney · 05-27-2004, 01:33 AM

I think in my close to 2 years at nat net..this is my first downtime I am aware of.

SweetT · 05-27-2004, 01:54 AM

Almost there....

All of the HVAC systems are up and operational and the temp is dropping faster than expected. We hope to be able to have everything back online in just a little while.

I will update again when we are done.

--T

SweetT · 05-27-2004, 04:09 AM

OK....last update....we are at the finish line

All of the repairs to the HVAC Infrastructure are now completed and the systems are running in normal mode. However, the heat caused us to lose one phase of power throughout our data center....it is the equivalent of "tripping a breaker" in your home. In order to resolve this we are going to be forced to power cycle the entire center. This will take place in the next little while and will create a very short outage (somewhere between 2 and 10 minutes). Once everything comes back online this entire experience will be behind us.

If you are a NatNet client and have any issues after the power cycle is complete, please contact us through http://www.mynatnet.com or by calling the Data Center toll free at 1.877.471.9075 Option 1.

Rest assured that we are going to learn from this and we have already put measures in place to be certain that this is never an issue again for our valued clients.

As always, thanks for your continued trust and business.

--T

justsexxx · 05-27-2004, 04:17 AM

Good job...Very professional..Good luck with your business

Andre

Phoenix · 05-27-2004, 04:21 AM

Quote:

Originally posted by SweetT
OK....last update....we are at the finish line

All of the repairs to the HVAC Infrastructure are now completed and the systems are running in normal mode. However, the heat caused us to lose one phase of power throughout our data center....it is the equivalent of "tripping a breaker" in your home. In order to resolve this we are going to be forced to power cycle the entire center. This will take place in the next little while and will create a very short outage (somewhere between 2 and 10 minutes). Once everything comes back online this entire experience will be behind us.

If you are a NatNet client and have any issues after the power cycle is complete, please contact us through http://www.mynatnet.com or by calling the Data Center toll free at 1.877.471.9075 Option 1.

Rest assured that we are going to learn from this and we have already put measures in place to be certain that this is never an issue again for our valued clients.

As always, thanks for your continued trust and business.

--T

calling now thanks

JFK · 05-27-2004, 04:26 AM

Quote:

Originally posted by SweetT
A quick update....

The first set of HVAC units are coming back online as I type this and the redundant systems should be online soon after. The portable units are being installed anyway, as a fail safe, until we can determine what happened and put together a plan so that this is never repeated.

Once the HVAC system and the portable units are all running it will take a little while for the Data Center to cool back down to optimal temperature so that we can begin to reboot all of the devices that are still down.

The good news is that we are on the downhill side of this now....thanks for your continued trust and support!

--T

Thanks For the update T.......its communication like this at a time of crisis that makes you guys Tops

Calvinguy · 05-27-2004, 04:31 AM

My server is still down

pradaboy · 05-27-2004, 07:10 AM

everything's up for me again, good work Tony

SpeakEasy · 05-27-2004, 07:31 AM

Quote:

Originally posted by Calvinguy
My server is still down

Can't you read sweet's post above??

dantheman · 05-27-2004, 07:31 AM

tony and natnet would be my 2nd choice for hosting. topnotch guys and service.

notjoe · 05-27-2004, 07:40 AM

Quote:

Originally posted by MaskedMan
yup, that's why no one can claim 100% uptime... these things happen, completely beyond anyone's control.

Good of you to take the time to come and make an announcement on the situation yourself Tony, very admirable of you

There could have always been a secondary system to cool the place down incase something like this should happen

Phoenix · 05-27-2004, 07:43 AM

Quote:

Originally posted by notjoe
There could have always been a secondary system to cool the place down incase something like this should happen

i thought redundancy would have been mandatory considering the volume running through

Mike AI · 05-27-2004, 07:44 AM

ATL without AC - I am suprised anything is working!

NAT NET has always been top notch, something's are beyond anyone's control!

gregtx · 05-27-2004, 08:10 AM

good to see everything is being resolved... great response Tony...

I"m going to use this thread to copy and paste to the webmasters who were whining about stats being down.... b/c they were afraid they might miss their one sale a month ;)

JimW · 05-27-2004, 08:44 AM

pick up the phone and give them a ring for fastest service.

Quote:

Originally posted by SpeakEasy
Can't you read sweet's post above??

05-27-2004, 12:02 AM	#1
SweetT Shank-A-Potamus Join Date: Jan 2002 Location: Atlanta, Georgia USA Posts: 1,756	NATNET Clients and Friends...Outage Report Hey Guys.... Just a quick note to let you know that we are aware of the issues and are working to resolve them now. All that we know at this point is that simultaneoulsy something affected our entire HVAC infrastructure. This caused the temperature to rise in our Data Center and is causing heat loads on several peices of equipment. This is why some servers are affected and others are not. We have HVAC people on site working on the situation, and we have called in for portable Air Conditioning that should be setup within an hour or so in case the normal systems are not repaired by then. Either way, we dont anticipate the sporradic outages that you are seeing to last much longer. Thanks for your continued trust and confidence, and rest assured we will get this completed ASAP. Tony Morgan President NationalNet, Inc.

05-27-2004, 12:15 AM	#4
Fletch XXX GFY HALL OF FAME DAMMIT!!! Join Date: Jan 2002 Location: that 504 Posts: 60,840	Professional. As any good host, address the issue on GFY within time. THAt speaks further than a fucking server special. Ill put my money with Nat Net or Webair because of service like this. excellent support. I didnt see my NatNet tour go down but others did see downtime, awesome to know its all good and bumping. cheers all. __________________ Want an Android App for your tube, membership, or free site? Need banners or promo material? Hit us up (ICQ Fletch: 148841377) or email me fletchxxx at gmail.com - recent work - About me

05-27-2004, 12:16 AM	#6
Brujah Beer Money Baron Industry Role: Join Date: Jan 2001 Location: brujah / gmail Posts: 22,157	the servers that are down, are overheating ? __________________ Free Premium Domain Lists and Tools at Clickmojo.com For Sale: Obscenity.com

05-27-2004, 12:21 AM	#7
the Shemp congrats to the winners Industry Role: Join Date: Nov 2001 Location: Echo Beach Posts: 10,891	Nice to see the Boss up at 03:00 __________________ i use Vacares...so should you Submit your picture galleries to my site...Outlaw TGP

05-27-2004, 01:09 AM	#10
pradaboy sell me your banners Industry Role: Join Date: Dec 2003 Location: on the tubes Posts: 12,931	thnx for letting us know what's up Tony... I'm sure you guys will get it solved ASAP. __________________ Media Buyer - Sell me your traffic! FREE to register domains... Better than 99% of the crap sold here!

05-27-2004, 12:07 AM	#2
Global Dialers Confirmed User Join Date: May 2004 Location: Spain Posts: 1,349	you guys rock Tony p.s... buy my sig ;)

05-27-2004, 12:12 AM	#3
Lensman GFY Chaperone Join Date: Jan 2001 Location: Adult.com Posts: 9,846	Outages can happen to anyone, sorry to hear that T. I'm sure they'll get things up soon.

05-27-2004, 12:38 AM	#8
SweetT Shank-A-Potamus Join Date: Jan 2002 Location: Atlanta, Georgia USA Posts: 1,756	Thanks for the kind words everbody. Yes, this type of thing can happen to anyone. It is why we always hate hearing about any hosting company having issues because we know it could just as easily be us. Brujah....it is not exactly that the servers are overheating...it is the electrical systems that are suffering the effects of the heat. UPS's and Power Strips do not work well in the heat, so they kick before anything dramatic would ever affect the server. I hope that answers your question. Shemp....Well, I am sure that anyone in my position would do the same thing. We actually have a very good "Crisis Team" made up of 3rd Level System Administrators, Network Engineers, and Senior Management that can be assembled in less than 30 minutes for occassions such as this. Of course, tonight I would rather have a team of Air Conditioning mechanics on staff!! LOL! We are working to power down all non-essential, non-customer equipment that is running in order to bring back up more of the sites that are down but we will not be able to be back in full production until the Portable Air arrives or until the HVAC Infrastructure is repaired. I will keep you posted. --T

05-27-2004, 01:06 AM	#9
makefuckingmoney Confirmed User Join Date: Oct 2003 Posts: 3,277	open some windows ill fedex a big fan for my server On a serious note, I have all confidence in the nat net people..

05-27-2004, 01:18 AM	#11
EviLGuY So Fucking Banned Join Date: Apr 2003 Location: malta Posts: 12,745	Seems to be a bad week for adult hosting companies. NatNet is top drawer though, so if you have a server there you wont be down a second longer then absolutely neccesary.

05-27-2004, 01:21 AM	#12
SweetT Shank-A-Potamus Join Date: Jan 2002 Location: Atlanta, Georgia USA Posts: 1,756	A quick update.... The first set of HVAC units are coming back online as I type this and the redundant systems should be online soon after. The portable units are being installed anyway, as a fail safe, until we can determine what happened and put together a plan so that this is never repeated. Once the HVAC system and the portable units are all running it will take a little while for the Data Center to cool back down to optimal temperature so that we can begin to reboot all of the devices that are still down. The good news is that we are on the downhill side of this now....thanks for your continued trust and support! --T

05-27-2004, 01:25 AM	#13
SleazyDream I'm here for SPORT Industry Role: Join Date: Jul 2001 Location: Phone # (401) 285-0696 Posts: 41,470	i pity the poor techs that were on duty when this happened. __________________ This dog, is dog, a dog, good dog, way dog, to dog, keep dog, an dog, idiot dog, busy dog, for dog, 20 dog, seconds dog! Now read without the word dog.

05-27-2004, 01:25 AM	#14
TheDoc Too lazy to set a custom title Industry Role: Join Date: Jul 2001 Location: Currently Incognito Posts: 13,827	Thanks for letting us know Tony, very professional.... __________________ ~TheDoc - ICQ7765825 It's all disambiguation

05-27-2004, 01:26 AM	#15
Calvinguy Confirmed User Join Date: Oct 2002 Location: European Union Posts: 1,752	For how many hours has this been going on?

05-27-2004, 01:33 AM	#16
makefuckingmoney Confirmed User Join Date: Oct 2003 Posts: 3,277	I think in my close to 2 years at nat net..this is my first downtime I am aware of.

05-27-2004, 01:54 AM	#17
SweetT Shank-A-Potamus Join Date: Jan 2002 Location: Atlanta, Georgia USA Posts: 1,756	Almost there.... All of the HVAC systems are up and operational and the temp is dropping faster than expected. We hope to be able to have everything back online in just a little while. I will update again when we are done. --T

05-27-2004, 04:09 AM	#18
SweetT Shank-A-Potamus Join Date: Jan 2002 Location: Atlanta, Georgia USA Posts: 1,756	OK....last update....we are at the finish line All of the repairs to the HVAC Infrastructure are now completed and the systems are running in normal mode. However, the heat caused us to lose one phase of power throughout our data center....it is the equivalent of "tripping a breaker" in your home. In order to resolve this we are going to be forced to power cycle the entire center. This will take place in the next little while and will create a very short outage (somewhere between 2 and 10 minutes). Once everything comes back online this entire experience will be behind us. If you are a NatNet client and have any issues after the power cycle is complete, please contact us through http://www.mynatnet.com or by calling the Data Center toll free at 1.877.471.9075 Option 1. Rest assured that we are going to learn from this and we have already put measures in place to be certain that this is never an issue again for our valued clients. As always, thanks for your continued trust and business. --T

05-27-2004, 04:17 AM	#19
justsexxx Too lazy to set a custom title Join Date: Aug 2001 Location: The Netherlands Posts: 13,723	Good job...Very professional..Good luck with your business Andre __________________ Questions? ICQ: 125184542

05-27-2004, 04:31 AM	#22
Calvinguy Confirmed User Join Date: Oct 2002 Location: European Union Posts: 1,752	My server is still down

05-27-2004, 07:10 AM	#23
pradaboy sell me your banners Industry Role: Join Date: Dec 2003 Location: on the tubes Posts: 12,931	everything's up for me again, good work Tony __________________ Media Buyer - Sell me your traffic! FREE to register domains... Better than 99% of the crap sold here!

05-27-2004, 07:31 AM	#25
dantheman OG Industry Role: Join Date: May 2002 Posts: 3,308	tony and natnet would be my 2nd choice for hosting. topnotch guys and service. __________________ M3server.com VPS>Get your 2nd month free Ded>$100 off your 2nd month since 1996 icq-25135623 dannyh at~m3server DOT com

05-27-2004, 07:44 AM	#28
Mike AI Confirmed User Join Date: Jan 2001 Location: Elysian Fields Posts: 3,624	ATL without AC - I am suprised anything is working! NAT NET has always been top notch, something's are beyond anyone's control! __________________ Make big money on your Domains! Why wait 40 days to get paid with the other guys? Parked.com pays the most for your traffic, and cuts checks twice a month!

05-27-2004, 08:10 AM	#29
gregtx Confirmed User Industry Role: Join Date: Jan 2002 Location: Dallas, TX Posts: 1,929	good to see everything is being resolved... great response Tony... I"m going to use this thread to copy and paste to the webmasters who were whining about stats being down.... b/c they were afraid they might miss their one sale a month ;) __________________ Processing Partners http://www.processingpartners.com