Techiemedia down 9+ hours!

[rowan]

Ok, normally I'd keep my mouth shut for a minor outage, but it's going past 9 hours now that my server has been unreachable... WTF...

[MaDalton]

must be yours - mine are working fine

[Barefootsies]

Yowsa. That is a long time.

Hopefully you get it worked out sire.

[rowan]

must be yours - mine are working fine

Did you go down at all? My MRTG graphs have been stuck at ~4Mbit the past 9 hours (a sure sign of a router prob, since MRTG can't collect new data)

[Spudstr]

Did you go down at all? My MRTG graphs have been stuck at ~4Mbit the past 9 hours (a sure sign of a router prob, since MRTG can't collect new data)

He could have equipment in a different datacenter. Or off a different piece of network equipment.

[desus]

Down for me as well

[Helix]

I hope you contacted them....Techiemedia was always very responsive whenever I needed anything.

[pussyluver]

Down here as well. An update on estimated time to things working would be welcome. Hate to bug them with questions when they are in the crisis mode. An email reply to my trouble ticket said a router was down.

If anyone has more info, please share. My sites have been up without a hitch since the Comus thing....

[rowan]

I hope you contacted them....Techiemedia was always very responsive whenever I needed anything.

Yes, contacted them literally minutes after my sites died to make sure it was a widespread issue and not just something on my server. Given that I couldn't reach my dedicated KVM I figured it was probably widespread.

[MaDalton]

Did you go down at all? My MRTG graphs have been stuck at ~4Mbit the past 9 hours (a sure sign of a router prob, since MRTG can't collect new data)

dunno - click my sig - everything seems to work fast as usual

[BIGTYMER]

9 hours is not good... I hope they get it fixed soon for you all.

[rowan]

I have backups in my home office, TGP thumbs cached on another server, and I control my own DNS, so I managed to put together a temporary TGP to point the dead domains at...

This outage started just as I was about to go to bed, I decided to wait just in case I needed to do anything once it came back up, 2 1/2 hours later I gave up and went to bed, figured it would be fixed while I slept. Got up 6 1/2 hours later to find it's still down. Wish I'd just gone to bed rather than waiting, I could have put this temporary site into action sooner. :-\

[pussyluver]

Can't get to http://support.techiemedia.net/ -- anyone else?

[desus]

Can't get to http://support.techiemedia.net/ -- anyone else?

Thats down for me as well.

[pussyluver]

They must be facing some kinda bad sheeeeet. Techie has been A+++++++ for a long time.

[hypedough]

Their support panel is down, but my sites are fine as of now.

[pussyluver]

did a tracert - get all the way to techie then time outs

[desus]

They must be facing some kinda bad sheeeeet. Techie has been A+++++++ for a long time.

This is my first outage with them. Been very happy so far, their support is best and I've had sites on most of the majors.

[rowan]

This is my first outage with them. Been very happy so far, their support is best and I've had sites on most of the majors.

Second extended outage here

Coincidentally the first was also router related.

[pussyluver]

Just talked to 'em. Expect a post on the boards soon. Actually appears to be a fiber issue is the short version that was explained to me. I'm sure a detailed one will follow from Techiemedia.

[JD]

Just talked to 'em. Expect a post on the boards soon. Actually appears to be a fiber issue is the short version that was explained to me. I'm sure a detailed one will follow from Techiemedia.

did they happen to mention how long they expect the down time to last?

[BBOL]

Same 4me...

I got a call that my site was down about 11 something am contacted techie and they said router problem.. I asked how long until fixed and they said a few mins.... a few hrs went pass and i hit them again and they said unfortunately they couldn't give me a estimated time, so please be patience... I'm hurting ova here....

[mmcfadden]

my site is working... support link is not

[pussyluver]

did they happen to mention how long they expect the down time to last?

They still expect a fairly quick fix is what I was told and I think they expected that all along... They have auto backups to hardware.... I can't speak for Techiemedia on a detailed explination.

[pussyluver]

Support link up now for me

[fatfoo]

It sucks that the server is unreachable and the site is down.

[rowan]

How come no one else started an "is techiemedia down?" thread? Were we all waiting for someone to do it? :D (I checked the front page a few times for any threads, left it alone since I thought only a small number of people were affected...)

Coming up to 12 hours shortly.

[desus]

Anyone's site back up yet?

[LiveDose]

It sucks that the server is unreachable and the site is down.

Pure genius.

[NemesisEnforcer]

How come no one else started an "is techiemedia down?" thread? Were we all waiting for someone to do it? :D (I checked the front page a few times for any threads, left it alone since I thought only a small number of people were affected...)

Coming up to 12 hours shortly.

Those are usually started when you're not in the know and checking to see if it is only you. We're down too but they told us what the problem was right away, thus, no need to ask a third party about it.

[rowan]

Those are usually started when you're not in the know and checking to see if it is only you. We're down too but they told us what the problem was right away, thus, no need to ask a third party about it.

Yeah, but after a couple of hours you start wondering whether the problem has been fixed but there's some other issue with your own specific equipment.

There's also no point them telling me "it's a problem with a router" a few mins after I open a ticket if I hear nothing further in the next 12 hours!

[NemesisEnforcer]

Yeah, but after a couple of hours you start wondering whether the problem has been fixed but there's some other issue with your own specific equipment.

There's also no point them telling me "it's a problem with a router" a few mins after I open a ticket if I hear nothing further in the next 12 hours!

This is true, we haven't heard anything since 1:00 PM ET.

[BBOL]

This is some BULLSHIT....!!!

[JD]

still down here too

[rowan]

Just sidetracking for a moment, and this has nothing specifically to do with Techiemedia...

How many of you have complete backups - or at least enough to get you going again - to recover from a catastrophic event such as fire physically destroying your server, or an extended outage?

Or something similarly devastating such as a host quietly going out of biz, and by the time you hear of it your server has been trucked off to who knows where to have its HD erased and given to a new customer?

(And if you have no backups - then something as simple as a HD failure could completely wipe you out...)

Just something to think about.

[webair]

Just sidetracking for a moment, and this has nothing specifically to do with Techiemedia...

How many of you have complete backups - or at least enough to get you going again - to recover from a catastrophic event such as fire physically destroying your server, or an extended outage?

Or something similarly devastating such as a host quietly going out of biz, and by the time you hear of it your server has been trucked off to who knows where to have its HD erased and given to a new customer?

(And if you have no backups - then something as simple as a HD failure could completely wipe you out...

Just something to think about.

Great post!!! Finally! At the very least you should have a backup of all your sites locally! Hard drive failures happen on a regular basis. If you do not have a redundancy plan in place, or are not paying for backups, you should have something locally in place.

Additionally, when you are shopping for hosting look into the technical aspects of the companies you are considering. How redundant are they???

Do they have redundant connectivity into their facilities?
What providers do they offer?
Do they have an adequate power backup plan in place?
Do they have redundant power circuits?
Consider ordering an additional power supply with your server
Inquire as to their network architecture (this should be readily available)
What tier is the facility rates? Tier 1-5 (5 being the highest level datacenter)
Do they have more than one datacenter location?
Tour the facilities!
How long have they been in business?

These are just a few things you should be asking (off the top of my head) when selecting a good host. Mind you I am not bashing techie, I think they are a very reputable company that care for their clients immensely. I'm sure they will post shortly as to what is happening, and a cure. This post is intended to stress the face that when choosing a hosting company, the lowest price is not always the best choice & backup backup backup!

IMHO

[Chosen]

Hope they get back up soon...

[Davy]

Down for me, too. 12+ hours now.
I contacted their support right when it happened, too.
Told me it was some problem with the router. And when you run a traceroute, you see that you can't reach their network.
Haven't had a support ticket update in the backend for over 12 hours, which upsets me quite a bit.

[BBOL]

Down for me, too. 12+ hours now.
I contacted their support right when it happened, too.
Told me it was some problem with the router. And when you run a traceroute, you see that you can't reach their network.
Haven't had a support ticket update in the backend for over 12 hours, which upsets me quite a bit.

This is really starting to worry me...

[MrDeiz]

there's no reports at webhosting talk atm

[Cyandin]

Router problems can be a nightmare to fix.

Best of luck to the TM crew.

[rowan]

Set another box to ping my techiemedia one every 5 secs... it just started dinging.

16 hours.

[desus]

Back in business.

[TrainWreckContent]

Set another box to ping my techiemedia one every 5 secs... it just started dinging.

16 hours.

16 hours damn i hopethey get it resolved quickly for you

[NemesisEnforcer]

This is really starting to worry me...

Yeah, me too. No news as to when this will be fixed.

[desus]

Set another box to ping my techiemedia one every 5 secs... it just started dinging.

16 hours.

Are your sites back up?

[rowan]

Yes, mine are up... are you guys still off the air??!!?



Here's the 16 hour "freeze" on my MRTG graphs...

[SGS]

Down for 16 hours?????

[Techie Media]

The problem started at 10am EST when we immediately noticed that our links were down to our second datacenter. We saw link but no activity on our 2 sets of fiber that connect our 5th floor datacenter with our 8th floor datacenter.

We run a layer-3 switch setup with load balanced hot spare connected to our core network between floors to light our second facility. Our immediate thought was that the primary switch failed in some way and the secondary did not take over like it should. This lead to a very long trouble shooting process with our tech even involving the switch vendor who had us update IOS and just about every test they could thing of with no result since the switches were identical in every way we had to be sure. We then decided to put a cold spare switch online to take their place which was a different brand and we had the same results, link and no activity.

The next step was to have the fiber company that handles our dark fiber test everything and also the the building meet me room check everything, they both told us the problem must be with our equipment.

We went back to more trouble shooting and actually brought the primary switch from the second datacenter all the way up into the main datacenter and connected it to the core on short fiber and it was working fine. It just did not work when connected between floors.

We then went back and demanded that the fiber company take a second look. This time they DID find and issue and corrected it and our second datacenter was back online.

This is one of those times we did everything we possibly could, worked tirelessly with all vendors that were involved and the problem was still caused and then prolonged by something we could not control as the hours of the day just seemed to fly by. So, to sum it up we got hit in a soft spot and it hurt.

We have been in constant contact with our customer and many have been told what we just posted by phone and ICQ.

The situation has been 100% resolved.

[LoveSandra]

seriously sucks