| newbreed |
10-03-2005 05:42 PM |
*** Announcement - What happened at ArikaAmes.com and what comes next. ***
OK everyone, it has been a painful last few days, but it's time to come out and let everyone know exactly what happened and what is being done at this point with the site.
As some of you may remember, we suffered a server hard disk failure about 4 days ago. I first noticed problems with the site's performance around 11:00p.m. the night before the crash. After talking to Tech Support I was told several times the server was doing some disk check procedure. This went on for several hours, and the site's performance got weaker and weaker. I continued to have Tech Support look into why the server was acting up and was finally told a few hours later the hard drive failed, and it was a total loss (the disk), not responding to any kind of diagnostics, etc.
I was initially told that the hard drive would need to be replaced, and the Techs did get a new drive in the box within a couple of hours, and we were able to start replacing content and other critical files. I was also told there may be a chance that some data was recoverable, but there were no garauntees. I decided not to wait for the drive to get looked at, we needed to get stuff uploaded and back online asap to protect our affiliates and members.
Now, this is where it gets bad....
I was under the assumption (foolishly it now seems) that 1, we had active and current backups and 2, our mysql DB server was not actually sitting on our machine (I thought it was virtual and on a different box, as I have been told this before). The reality of the situation however, as I was told later that day, is we did NOT have a backup plan, nor were the DB's on another server. I was further told an e-mail was sent out sometime over the summer stating that our host no longer included backups of our data, that the backup service was now a paid option. I NEVER recieved ANY such notification, and as I told the Tech guys, the only mail I ever get from my host is the bill every month. In talking to a few other people who have had a similar (although not as severe problem like this) occurance, they never recieved any notification of the change either. I thought we were backed up, I had no reason to ever think we were not. Never in my wildest imagination would I attempt to run a business requiring this kind of stability and not have certain basic measures in place. Same thing with the DB server. I had always been under the impression the DB server worked like the mail server, it was virtual and not sitting on our box. With a location of "REMOVEDBUTMYSERVERNAME.webair.com" it's easy to understand why after i had been told this, then looked at the DB location, I would come to this conclusion.
So what does this mean? It means that everything we had online is gone, with little chance of being recovered. When I say everything, I mean EVERYTHING. All the content (which we do have local backups of and have restored to the site), all the sales and affiliate data, all the marketing tools, everything, is gone. Also about 2 years of design and script work and other programming I had done for other people. We have been told the disk can be sent to a data recovery specialist, and that is the plan for a lot of what we do not have local copies of.
So where do we go from here? We've rebuilt 90+% of the Member Area, and fortunately have not lost a great deal of members. Every indication we have is that member retention has not been nor will be affected by this catastophe. A message was immediately posted on the site and so far we have had NO complaints or hickups that I know of with members. That's the good news.
The bad news, and the most devastating, is we have lost all of our affiliate database, including all of our sales and rebill information. I am not real sure what to say about it, it got real bad when that reality set in. Our only hope is the affiliate database and other information can be recovered by the disk doctor people.
So then, we have had to start from scratch, and are doing ALL we can as FAST as we can. Everyone of our affiliates who has contacted us by e-mail or ICQ has been very understanding and supportive, and we really appreciate that from all of you!!!
We are working every channel we have access to to get everything back up and running or recovered as quickly as possible. The wonderful people at CCBill and NATS had us all set back up the next day (aside from no database to reconnect). The only things we are waiting for right now is hearing back from the recovery people and getting the affiliate program content all back in place and set back up. There is a chance that affiliates may need to re-signup, as I really do not know what all will be recovered from the bad disk.
All I can do is personally apologise to all of you as I know most of you put in as much work as we did to make ArikaAmes.com successful thus far. Without you we would be nowhere, and we cannot thank you enough for sticking with us through this ordeal and everything in the past.
Mistakes were made on my part, me personally, not Arika. I should have done a lot more followup on some things to be sure my assumptions were correct should anything like this ever happen. I did the best I could with the information I had at the time, and in this case it was not good enough.
We have some plans and ideas to make things up to affiliates who may have lost any sales or rebill income, but we will not know for sure the extent of all of that for a few more days.
In the meantime, anyone who has any questions, please feel free to post here or hit me up on ICQ (346-940-591) and I will do the best I can to field info as I get it back in.
This about all I can say for now, I know it's long, and I may have missed things, but i will be here for a while in case anything else comes up.
Bill
|
