Redhallowians,
Before I begin, I want to be completely transparant with this post and let you know the full details. We hope that you can understand the severity of this issue and stay with us as we get through this.
This is a post I was hoping to never have to write and is difficult for me to do so. Honestly, it is just embarrassing, but please read the entire post. Around 5:30 cst this evening our server crashed. We have all experienced these and commonly the server would just restart. This time it didn’t.
Once this crash was brought to my attention I quickly wrote our server host about this issue and they responded with this.
Hello Steven,
We have some good news and some bad news for you, and I’m not going to try to “sugar coat” it and make it sound better than it really is. We’re not one of those companies.
Unfortunately, the node you are on (USA Node 1) had a catastrophic Multicraft control panel malfunction that caused it to irrecoverably delete all customer data on that node.
There’s 13 other customers in the same place as you right now.
Because of this, we’re offering 2 free months of service, if you choose to stay with us.
If you choose to not stay with us, I completely understand.We’re implementing a backup system very soon to prevent this from ever happening again.
I am sure that your reactions were about the same as mine when I read that. From this email it appears that ALL of our data from the last 9 months is gone.
Out of anger and frustration, I responded with this message:
Devin
Let me begin this email to say that I am royally pissed.
Okay. How in the world can you not have a backup system for customer files? This is completely irresponsible for a hosting company, as it should be basic systems installed.
We have well over 100 members spending hours working on projects that they have invested time into and are proud of the work. How am I supposed to tell them that all that is just GONE??
I would be considering changing hosts, but at this point I can as I have just recently spent all the work changing over. I may have to if this is the kind of hosting company you guys are! Two months free is it for losing nearly 9 months of work?!
Is all the data corrupted? Is there anything that can be recovered? I have finals this week and have no time to deal with fixing this, especially since we have many users who have donated and expect good uptime. I would be contacting Multicraft and finding out what happened, letting them know of this.
Please let me know more information soon so I can piss off all of our members.
We can not say sorry enough as a staff about this, and I personally wished this would never happen. I know there is nothing I can say to make to make this situation better.
But…
Now, here is where we will pray we have your help with this. At this time, our staff has not made a decision as to how we will move forward with this, but we do want your input. I can tell you that we do have an entire server backup from March 16, 2012. This would lose about 1 months worth of work, and we know that that time was well spent. We want your opinion on what you think we should do to move forward. I will be moderating the comments in this post closely, so PLEASE DO NOT POST WITH MASSIVE ANGER. We want you to post your concerns about this situation and let us know your suggestions. Mainly we have two choices.
- Upload backup from March 16, 2012 and begin reconstruction.
- Start fresh with an entirely new world.
My guess would be that the 2nd of these may take more time, but some may want to do that.
Also, for donators. I know you have personally invested funds into this server, as you know it does cost money to run this. If you want a refund, please email me and we will discuss that. (admin@redhallow.com) I can not promise a full refund, but I will do what I can to get a good majority of it. (this depends on when you donated) I can promise you that we will keep this server running though and with whatever decision we make, we will be up and running (hopefully) within a week or so. (could be much sooner) Uploading from a backup would be a quicker turn around time, but some may want to consider a new map.
Either way, our staff will make the final decision and will be letting you know the decision we make.
We will keep you up to date on this situation here. Pray that somehow our data can be recovered from the server malfunction.
Update 8:45pm
Hey Steven,
Yes, we understand completely as we all come from hosting popular Minecraft servers ourselves.
We had already begun working on a new control panel and backup system before this ever happened in an effort to make sure it never did happen, but obviously we weren’t able to get it done fast enough.We have already spoken with the creator of Multicraft and he’s aware of the issue and assured us that in the latest version, 1.6, which has just recently been released, the problem is fixed.
We had already made plans for upgrading to 1.6 tonight or tomorrow night, so the possibility of this ever happening again is literally nil.In addition, as Devin has said, we are currently getting a remote backup system in place which again, we had already planned to do before this happened.
As for the data being recovered, we’re working on it right now but more than likely, we won’t be able to get much if anything back.
When files are deleted on linux, it’s pretty difficult, if not impossible, to get them back, especially hours after it has happened.Also, if you have that many active members and have put that amount of work into your server, you could have created/downloaded backups of your map at any time.
I’m sure your loyal members will understand and stick with you; there’s simply nothing more we can do at this point other than apologize, give you free service, and keep trying to recover the data.If you are deeply concerned about this happening again and decide to stay with us, simply wait until we roll out our central backup service which should be out within a week.
Update 10:15pm cst:
There have been several messages between our host and myself. I can say that the work they are putting into this situation is phonomal and I am happy with how they are treating this situation. I will let you know of any major developments that arise in our conversations. I know that their owner will hopefully be calling me here within an hour or so with a personal update on this situation.
Update 11:59pm cst
I just got off the phone with our hostâ??s owner. Here are a few notes.
-Attempted recovery has not begun as they are waiting for things from the data center.
-He would be able to set us up a temporary server for the time being on another node to allow us to reconfigure things. (may be good if we restart fresh and I have to redo all our plugins). Thankfully I have those backed up from 3/15.
-He really wants to help us out as he is a crafter himself. This guy knows his stuff and knows how big if a problem this is.
-There were a few other things but am tired an cant remember. Those are the more important notes.
Again I will keep you up to date here on what the latest is.
No matter what I am sticking with the server, weather we start over new or not. I will be donating to you in the future because I have met some interesting people through Red Hallow and I would like to continue to do so. Behind you all the way with w.e you choose.
We thank you so much for those kind words. We are at a loss of words right now and are unsure what to do. We are truly thankful for users like you. You all make Red Hallow what it is!
This truly does suck. Majorly. As everyone did work on major projects. But in my opinion i feel we should just restart completely over again. Like we did in the 1.0 update. Now i know and understand how some people wont like that idea, and some people may be surprised that i am suggesting this as i spent nearly 4 months on Vai. But, starting fresh is one of the best ways to move forward when terrible, frustrating things happen. Im sure many people wished they could have built things better, or in different ways, well, this is one of those chances. If the world was lost, then yes, it does suck. Majorly. To know that my best creation in Minecraft that took 4 months to create, that wasnt even close to done, was erased in a matter of minutes, yes, that angers me. But there were many things i would love to change about that creation. Nows the chance. I hope all of you great, members understand where im coming from, and dont lose your temper about this. We as a community have been through countless problems from hosts, and now this community is like a family. Which sets this awesome server apart from every single other server out there. So lets move forward from this everyone. In the way that is most comfortable for the majority.
Rally seeing that you, who (no offense to anyone) has probably built the best things and Vai, wanting to restart really surprises me but i think it would be a great idea. It would be perfect bonding for some of us haha and we could really pull some amazing things out of the dust.
I really do like Rally’s opinion and will consider it in my recommendation for what our staff will decide on.
What kind of hosting website doesn’t backup their files? I mean that’s just common sense.
Anyway, I’d say go with the backup. A lot of people have a pretty insane amount of time on the stuff in the new world. It’d be a shame to lose things like Vai and everfree. If people really want a blank slate I suppose a new world isn’t out of the question?
And as long as everything is back up eventually, I fully intend to stay with you guys and I support however you choose to go about this. I know it must royaly suck to be in your shoes about now. :/
Anyway, good luck with this. And send that host an extra pissy letter just for me.
Brandon,
That is pretty much what I said… Who doesn’t backup their server files, and then blames me for not doing it.
I can tell you that at some point we will be back up. This won’t stop us, we will keep pushing on!
I don’t have time to read all of this post at the moment, have a date with my boyfriend in a few, but I’m not surprised something like this happened. I vote for starting fresh. Especially since there was some item abuse in the past, it would be nice for a fresh start.
Just to add to what I already said:
after reading Travis’s post, I can totally see going for a new world, and thats fine with me, but I’m worried about all the new guys we’ve gotten since the 1.0 update. Since they dont see us as ‘their’ community yet, and they haven’t donated and don’t have any attachment to us, we might lose quite a few of them if we change everything up suddenly.
Rock and a hard place, mate. :/
Speaking as a new person: I joined up not too long ago, trying to find a community of Minecrafters that I could become a part of. I almost immediately realized that so much had been done that I was having a hard time finding anywhere to build and any way to be unique, and that a community had already been formed that did not seem open to new members (my experience only). I stopped playing, actually, but I never unsubscribed from the email list so that’s why I’m here now. I hate that this has happened to you all and it makes me pretty angry for you. Just think: if they had lost a bunch of important bank data or a school’s enrollment records (no idea if they would even have that stuff, doesn’t matter), would they be sending those clients those kind of emails and offering two free months of service, or do you think they’d be flipping the fuck out and bending over backwards to make it right? The fact is, they lost YOUR data, it was THEIR fault, and it is NOT their place to decide how important it is. So, in my opinion, you should demand that they do better than two free months of service, no matter how nice the guy is being. Anyway, the whole reason I’m here commenting is to say (looking on the bright side) that if you started fresh with a new world, I would be much more likely to play again and I think people might be more interested in working together to get it back to it’s former glory. It could really strengthen the community and retain new players. On the other hand, I would hate to have lost any of my builds on my other server, so I understand if you don’t want to do that too. Either way, try not to let it keep you down and best of luck to you all.
Tiffany
I am sorry you felt separated from the server. I hope that with whatever decision we make, you can find a way onto the server and a place for you to enjoy our community.
Pissed doesn’t even begin to describe how I feel about this… Really, it doesn’t. Mostly I feel bad for the members with a lot more time and creativity than me that lost such epic builds like the cities. My vote is for….
A whole new world
A new fantastic point of view
No one to tell us no
Or where to go
Or say we’re only dreaming
A whole new world
A dazzling place I never knew
But when I’m way up here
It’s crystal clear
That now I’m in a whole new world with you
Now I’m in a whole new world with you
Unbelievable sights
Indescribable feeling
Soaring, tumbling, freewheeling
Through an endless diamond sky
A whole new world
Don’t you dare close your eyes
A hundred thousand things to see
Hold your breath – it gets better
I’m like a shooting star
I’ve come so far
I can’t go back to where I used to be
Yep, that just happened. Seriously though, I would prefer to just start over.
Thanks,
Chris
Btw, good luck on finals!
Thanks for the poem man!
Aladdin helped me with the majority of it… j/k completely plagiarized! my english teacher would beat me with a ruler if she knew [^o^]
Yea, fail for me. Totally didn’t realize that last night
I havent been on much recently, but I think that we should start over. I’d like to help build some of the “public” places aswell. I would like to redo my base and I’m sure there are a few others out there that would like to aswell. Plus if we start fresh all the half finished an abandoned projects will be gone. On the other hand it does suck that we lost almost everything. I do have a bit of the server backed up, WorldDownloader mod. Not sure how up to date it is or how much I have, but if it’d help out let me know. I’m pretty sure I have the main “public” areas.
Im not gona pretend to not be mad about this. Not for my sake but more for the sake of others. Well all i can say is there is no way ill ever leave this server i love it and all the people to much. I understand this must be hard on you steve and the staff and all the players but we can pull through it. Although i really love the server atm i think maybe it would be a good experience to restart. I mean is there anyway to pull up the March backup and copy some things onto the new server such as some of the nicer things? But i think that it would be great for us as a server and team to build up again. I don’t care which option happens I’m with it. I plan on never leaving Redhallow till its gone so lets look for ward to the future and what good can come out of this.
Come on guys i know this blows but theres a lot of potential from this!
Keep your heads up guys and hang in there Steve,
Matty
I thought about pulling stuff in from our march backup into a new world, but I think if we are going to start new, we start new. I know it sucks royally but it may be the best if we were to go fresh.
Ya ur right it sounds like the best idea!
Just something small to add. Does anyone have a server they could run for a week so we could discuss things and talk about the proceedings? i think it would be a good idea seeing the opportunity we need some plans. Just and idea that might be helpful so we can present some ideas to Steve when he’s back from finals.
Good luck with them Steve!
I don’t think it would be good for us to start building on another server, but I wouldn’t mind opening up our voice server to hold a meeting possibly to discuss this. I will think about this.
Sorry for the confusion i just meant something we could talk on and maybe build a building or 2 to show some designs but just to plan and stuff but ur idea is better.
This may be possible, for some of us to get a spawn town done at least
Hey all, I will get around to responding to you all but I appreciate that you all are consideriate during this. You all could have the right to be pissed off and leave, but this shows why I love having you all around!
Cool! We get to restart EVERYTHING from before I even got in the server.
We have not decided on this yet, but it does seem that is the consensus so far. Our staff will be making a decision in the near future.
Im with ya mull
Well was not on the server for about a couple months but this past week I invested about 20 hours on finishing my house and this is quite upsetting. I would go with the backup, because of the time everyone has spent on the server. I was planning to donate soon but this has made me a little skeptic about the server. But the months I’ve spent on the server makes me feel strongly about it.
Robert
I wish I could apologize enough about this. I hope we can earn your trust back over these months as we get back up from under out feet.
I have a feeling that Red Hallow can only get stronger from this incident! I’m very excited for the future Robert! We’d love for you to be a part of that
I have thought of this topic overnight and have concluded that I will vote for a fresh new start on the server. I believe that you will make this server better with a new start, my brother and I wanted to build a town but there was not much room before. Now we can build a town with no limits!
Hey Robert,
I’ve seen you on quite a bit and the server definetly needs you as part of our community! Please don’t let this screw up on the host’s part discourage you from Red Hallow.
I feel that a new start will only be better. But thanks for the feedback.
This obstacle will not stand in the way of the Iron Warriors! We will continue our grand crusade into the heavens! I don’t mind if you restart the server or regain the last save, as long as everyone keeps their ranks. In Mortis Est Gloriam!
Everyone’s ranks will stay the same for sure.
When rebuilding will all ranks be allowed on or only certain ranks? And if coppers and low donors are allowed will they abstain creative or bunch of supplies from staff for comp to rebuild?
These decisions will be made by the staff when we make a decision.
This is so unprofessional on the company’s part, I mean how could they let all this data go to waste in a second? Its ridiculous. I have just started on this server, so if we do go back to the old data, then I have to restart all my legit building (Not to mention a little over 8 hours of work put in). Even if we lost all of that amazing world, but, I think it would be good to start fresh again. I am totally willing to help out rebuilding the server.
Thank you for the input
Glad to have your support!
This is outright ridiculous on the company’s part. I don’t see how they couldn’t have a backup of the maps of their customers. Customers pay THEM to keep their servers intact. It is so unprofessional for them to have such a big mistake like this, then blame the CUSTOMER for there mistake? Downright unprofessional, its this kind of bullcrap that just pisses me off, as well as everyone who has also gone through this same situation, im sure. But in reality, we will have to deal with the fact that we have to rebuild, I vote to make a brand new fresh server. One better then before,bigger more complex. That would be great. Thanks everyone!
I understand your frustrations. I want to just clarify that they didn’t directly blame me. I understood their point in saying I could have made backups, it just could have been worded differently.
Oh ok, that would have been even worse if they blamed you. But still this is utterly unprofessional on their part.
That which does not kill us, only makes us stronger
Except polio.
Update 10:15pm cst:
There have been several messages between our host and myself. I can say that the work they are putting into this situation is phonomal and I am happy with how they are treating this situation. I will let you know of any major developments that arise in our conversations. I know that their owner will hopefully be calling me here within an hour or so with a personal update on this situation.
This news is definitely very good, thank you for the update. How much damage are we talking? I think starting afresh would be a good thing.
“Also, if you have that many active members and have put that amount of work into your server, you could have created/downloaded backups of your map at any time.”
That almost sounds like their blaming you. That would piss me off more than anything.
Anyway, sounds like we’ll be getting a new server! This will be so much fun
.
And I’ll definetly join in on the voice server this weekend.
I understand how they wrote that, but if you read my next comment I think you will understand.
All
Here’s another update.
I just got off the phone with our host’s owner. Here are a few notes.
-Attempted recovery has not begun as they are waiting for things from the data center.Â
-He would be able to set us up a temporary server for the time being on another node to allow us to reconfigure things. (may be good if we restart fresh and I have to redo all our plugins). Thankfully I have those backed up from 3/15.Â
-He really wants to help us out as he is a crafter himself. This guy knows his stuff and knows how big if a problem this is.Â
-There were a few other things but am tired an cant remember. Those are the more important notes.
Again I will keep you up to date here on what the latest is.
Goodnight.
Im VERY happy and proud to see how everyone is dealing with this. Our beloved server has grown not only into a great community, but like i said earlier, a great, close-knit family that provides for eachother and looks for everyone involved. Im so greatful to know every single one of you and thank you all for being so helpful to not only myself, but the staff, and above all, our awesome one of a kind owner Steve.
I know I’m one of the newer members at RH and I’ve only been here about half a year or so, but I vote for starting fresh. It would give us new land to explore, build on, and hopefully some of the more senior members would create cities together instead of one or two people building the entire thing. Maybe it will create a more community feeling? That’s my two cents on the choices. As for refunds or leaving the server, I’ve dealt with worse, honestly, from CAL teams disbanding on CS to donations to servers and having the server close weeks after with no refund. I feel that offering a refund in this situation is very professional on Steven’s part. You can keep my donation, Steve, for growing up with legos and 2 siblings, I’ve learned to recover from months of work being lost. I’ll stay on the server. Now that I’ve said my two cents, you’ll have to excuse me, I have a final exam in 8 hours.
p.s. I never post because I’ve forgotten my forum information so if you could be kind enough to e-mail that to me, Steven, I would be grateful.
Tophat, thank you for the kind words!
As for your account I don’t have access to your passwords and don’t know what it is, but I would try using the “Forgot password” section. If you need more help, email me.
I have a feeling RH is going to be better than ever!
Hey Steve, I was told to put an input in for VanillaIsle (nanaki658, bilbothemonkey, bahamut686, mycaquin, chuanfa, and myself katcorcoran).
We all appreciate all the work that is going into fixing this “natural minecraft disaster” as it could be called.
We all will be staying with the server, as we love the people here and the freedom we have to build vanilla.
As we are all new since easter…. we vote for a fresh start, since for us, no matter what, it is a fresh start.
Hope it is up and running soon…
Thanks
Kat
(VanillaIsla)
Thank you so much for the support!
I am more than happy for the world to start over again. As one of the newer members it can be easy to get isolated in such a large world. Starting over from scratch would help people such as myself get involved from day one
I vote for starting afresh. We’ve all lost a crazy amount of work and It’ll be nice for a fresh start with all the plugins/nether working properly.
Hopefully we can get some new plugins that’ll help prevent the overspawning issues too.
I guess every cloud has a silver lining.
(PS Lord_Cadmium is going to KILL me when she finds out; this was just after the backup. it’s all legit except the circle stone XD [img]http://i200.photobucket.com/albums/aa197/ZeBerrys/hoard8-4-12.jpg[/img]
She’s gotten another few chests since then.)
Rally- you wanna team up and build a city together??
A Travis/John city? I can’t imagine the awesomeness
As for Lord_Cadmium, she’s a boss for getting that legit, and will probably be quite mad haha
That’s pretty amazing to get that legit. That makes me think though. What if some people would rather have their items spawned back instead of starting fresh?
That would raise too many difficulties.
If we are to start fresh, we start fresh.
Saying that, giving everyone a one-off kit of stone tools would help things.
My thoughts:
Wow. Just….Wow. I never expected it to be gone in a blink of an eye.
I say we start completely fresh. We need to start a whole new world. It will clean up all of the unwanted random homes and houses that banned people have made, but will be a whole new experience for all players, from Coppers to donaters to even for you Steven.
Another thought:
We should help the coppers who worked extremely hard to get supplies and make their homes and little villages. We need to get them back on their feet. They make up the majority of the server. It’s the least we can do as a server to help. Just my thoughts though.
Remember: “An eye for an eye makes the whole world blind.” Gandhi
“I do not believe in a fate that falls on men however they act; but I do believe in a fate that falls on them unless they act.”
Buddha
“In prayer it is better to have a heart without words than words without a heart.”
Gandhi
Quoting famous people, eh? well how about this one:
Victory is achieved through mettle;
Glory is achieved through metal.
guys cool it with the quotes here. a great man once said “bitch i fucking kill you!” but that is totally irrelevant.
A great man once said, “Testicles,” and it is completely relevant
Guys I think we’ve all voted for a fresh start so lets do it! Theres just so many possibilities that we can do. First off i think we should add some more detail and amazingness to the whole spawn town not just initial spawn. And we can some great cities but not only that we have so much possibilities we can make adventures and quests or have some fun games on the server such as CTF and mob arena. Theres just os much that this allows us to do.
“Sometimes you’re not afraid of letting go, you’re just afraid of accepting the fact that its gone.”
“No, this is not the beginning of a new chapter for our server; this is the beginning of a new book! That first book is already closed, ended, and tossed into the seas; this new book is newly opened, has just begun! Look, it is the first page! And it is a beautiful one!”
Yeah uh… we’re starting fresh. It ain’t the second coming of Christ.
So Steve, I know you said that the backup would take less time, but it looks like we’re going for a fresh start. How long would it take to delete everything and create a new world? Not trying to rush anything, just wondering.
Also, I think it would be a good idea to open the server up to only staff and high level donators, with a few exceptions for great builders (Lord C?), just until the immediate spawn is done.
Thats what i was thinking tab but i didn’t really want to say it. glad u did haha. But ya i think that would be the best of options. Plat and higher cuz they have creative
Personally, I don’t mind, but I think it may be a little unfair because im sure people who were excited about starting fresh also wanted to contribute to the rebuild.
Thats why I say ‘immediate’ spawn. As in JUST the part that you spawn in. The rest of spawn town I would say open it up for all to help.
oh ok. That sounds cool.
Agree with you both
I think we will make the decision Brandon
. WE should be able to announce some things hopefully tomorrow.
How long of an estimate before we can go on and test some stuff and maybe start to build?
I can’t give any sort of time frame at this point. The staff is going to try to have a meeting tonight and make some decisions.
Oh yeah I hear ya, just posting my ideas.
Is there a temporary server yet?
Anyone know a good server for the meantime? If so, lemme know please.
If we do reset completely, I know a nice little seed with mountains far away enough that they won’t impact on spawn. The seed is 1381538624208156757
Any other seeds for us to check out??
Ill look for some, ill email you them.
I can tell you that progress is being made and we hope to make some announcements tomorrow!