Major Hosting Problems - Help Please
 

Ok, so I have 4 boxes at a popular host here, I won't name them now because I'm not trying to start a witch hunt or anything but here's my concern.

Since starting with this host, I can count at least 3 times that our boxes have been rebooted due to power loss, in the span of about 4 months.

I only ever get a shrug and a sorry, but it's seriously causing problems.

This last time our storage box came back up, but its raid mount was unmounted, and needed to be FSCK'd (again) which takes a few hours.

We also lost our NFS mounts on all the other machines that access this storage box, and they had to be removed and re-added manually. Our boxes are managed at this host just to be on the safe side, and their tech was experienced in BSD, fixed the raid, fixed the NFS mounts, and even helped us speed up one of the dual processor machines. I've been very happy with their service. I'm semi-needy, and they always help me out, and have yet to charge me extra for anything I've asked for, even things I would expect them to charge for.

However.. Boxes rebooting in the middle of the night, not cool. We don't have startup scripts setup on all the boxes quite yet, it's a real pain in the ass.

So to be clear, does this happen at all hosts? Should my boxes EVER lose power? Seems crazy to me.

PS: This last time we had no clue the boxes even went down, except that shit wasn't working right.. They came back up, but things weren't right, as I mentioned above.. No phone call, no e-mail, we had to find out ourselves and wonder WTF is going on.. When I saw /raid/ wasn't there I almost swallowed my head.

No it shouldnt happen.. I have 5 boxes all over different hosting companies, Let me ask this though does the hosting company ya talking about start with "rig" ? if so lets chat :)
they are the newest for me and had a few ups and downs so far.

Are you sure it's power loss, and not an OS panic?

That should not happen - it appears the hardware in your box(es) might be faulty.
New servers might really be in order...

BTW, if it's really a true power loss and ALL your boxes were affected then that's a problem on the host's side. In that case switching to a different host might be in order.

Hey everyone, all power loss.. Two times electrical problems, another time was a "scheduled" reboot for some kind of port audit or something.

It's the electrical problems I'm concerned about. Not sure why our boxes should be turned off for ANY reason. Even if there are electrical issues that aren't immediate, shouldn't a good host be able to re-route our power before they mess with something?

Has this happened at other big adult hosts? We've been hosting websites for years and I've never heard of electrical problems that I can think of ever.

Have they offered a credit? Do they have an SLA?

I had electrical problems with one host and they ended up crediting me back half my monthly fee, even though their SLA technically said they only needed to credit me something like 15%. In that instance it was due to a faulty power controller. No such problems since, or prior for that matter. That's the only time in ~7 years of having dedicated boxes sprinkled around the globe that I have experienced issues with power.

Re-reading your first msg, you need to find out what monitoring and escalation procedures they have in place. A good host should be the one telling you that your servers are down, not the other way around.

You should also look at getting your startup in order so that a reboot will not require human attention (assuming the fsck doesn't require human intervention)

Its power, breaker going out, probably. I would doubt its the entire data center, the charging of fees reallly depends on the agreement you made. Is it fully managed, sounds like thats what you need. As a rule I do managed on an multiserver environment. Start up scripts, that should have beent aken care before, if not at least after the first power problem. Clients don't always let you know what they know, so the first one is excusable, but on the the first one then the host knows what they need to do.
someone will invariably come in here and shout a lot of BS about how its all supposed to be this or that, and it will all be BS. What I am telling you is realistic. 12years in the isp/hosting business realistic. You can pay lots to some and get fully managed to the keystroke and that can be a cool way to go, you can get fully managed where you have all the help you need, you can get dedicated where generally they charge you by incident.
I am 99 percent sure I know the company you are talking about, and I won't out them either.
Good luck either way, If you want some real help, my icq number is below.

You shouldn't have any power failure at any good data center. One of the main obligations of hosting companies is to have very redundant electricity. Never had a problem myself but I don't do any hosting at a "adult host". If you do a lot of business with your hosting co, a cool thing to do is to take a tour of their data centers and operations.

"However.. Boxes rebooting in the middle of the night, not cool. We don't have startup scripts setup on all the boxes quite yet, it's a real pain in the ass."

-Like other said, before putting a box in production, you should make sure that if it reboots everything is fine and all the scripts load starting whatever services you need if it happens to reboot.

"PS: This last time we had no clue the boxes even went down, except that shit wasn't working right.. They came back up, but things weren't right, as I mentioned above.. No phone call, no e-mail, we had to find out ourselves and wonder WTF is going on.. When I saw /raid/ wasn't there I almost swallowed my head."

You should have notification traps setup, so that when any service or box is unreacheable you get an email, or you can even do some more funky custom scripts stuff like getting a call with nagios for example.


Best of luck! Get your startup scripts and notification traps set up ASAP :)

I think i know who you be talking about, I have had this problem and not even been there a month yet and somedays I wake up and ALL machines rebooted...

I am going to inquire

This is not good PR press :p

Absolutely not shopuld you ever have a power loss, and especially a few of them??? Not a solid host if you ask me.:2 cents:

Assuming the link in Socks' sig is his site, it's not that hard to figure out the host. :Graucho

fail rowan, not the same host :)

no, you should never have those sort of power problems....

Who is the host I wanna know....

what caza said... i bet its a breaker... ask your host how many amps in your cab and how many servers are in the cab and ask them if they have done a power audit in your cab to make sure you arent riding aroun d the threshold...

Names!

10 char

3 times in 4 months is unacceptable. I can imagine happening once every couple of years and under very certain circumstances.

This is why I host with National Net. They are top notch all the way around and have the best support in the industry.

I have been happy as can be since switching over to Nat Net :thumbsup