Gandi Bar - Disk problems - Comments

Disk problems - Redsauce

Redsauce — Mon, 19 May 2008 12:15:32 +0200

I am still having problems with my site, and the support department does not answer my emails (I don't know why I continue trying, only 15% of my emails to them are answered)

Problems means that the total amount of RAM assigned to it is so low that the services can't start. I am just using 3 shares. Tried rebooting the server but nothing changed.

Could you (Gandi) please update me about the status of the recovering? My clients are getting nervous

Disk problems - Ryan

Ryan — Sun, 18 May 2008 07:17:31 +0200

Welp, it's been 3 days now and I still can't access my site. Good luck with everything but I'm moving on.

Disk problems - Ryan

Ryan — Sun, 18 May 2008 02:10:16 +0200

I should have checked here before I sent another support ticket. Looks like I'm not the only one who still cant create or install anything.

Disk problems - Vergo

Vergo — Sat, 17 May 2008 22:02:51 +0200

@Chris Boyle, I had that also before the disk troubles but it wasn't that severe. Before the forced reboot I had about 80% of 256MB usable and now it's down to 54%. That 80% was still enough to run the services I need but 54% starts to limit quite much. I asked about that memory issue two weeks ago on #gandi-hosting (@freenode) and got this reply from aegiap (Nicolas Chipaux) of Gandi:

"It's related to Xen memory page. We are investigating to fix this problem."

So that would indicate that they at least know about the problem but it would be nice to have it fixed since I don't see much point for me to buy another share to get more memory when the current share isn't having the memory it should.

Disk problems - Vergo

Vergo — Sat, 17 May 2008 20:57:24 +0200

I suggest that those who have their server up again check their system memory usage. At least my server seems to now have some kind of problem on that side after all servers had the forced reboot. Although with one share the system can see 256MB total ram, there's actually only 138MB usable and after that the system starts swapping... and that isn't nice.

Disk problems - terii

terii — Sat, 17 May 2008 19:21:06 +0200

They might not have any spare drives sitting around and needs to order them or whatnot. Ever rebuild TB arrays before (and fsck)? It can take hours and hours...

Disk problems - Desi

Desi — Sat, 17 May 2008 18:27:59 +0200

anyhow, is there anyway if i want to move my server to another filler/cluster/place or whatever you may call it?
--------------
One of my servers stopped Thursday at 19pm +- and is down till now, I was trying to to delete the affected server to create a new one, but gandi as also blocked the creation of new servers so we are stucked.

My server is now in a new state "blocked" can you tell me what is happening.

One thing that is missing from you part (gandi) is an estimation of how long it will take to solve the problem of the affected servers. And please give us the worse scenario and not the vague ASAP

Good luck!

Disk problems - john

john — Sat, 17 May 2008 17:30:59 +0200

36hours...still nothing
too bad i'm with those 20% server that gandi still not restart
i'm developing a website that has deadline, i need my server to be up or else i will be wasting my time .. i can't do anything

anyhow, is there anyway if i want to move my server to another filler/cluster/place or whatever you may call it?

Disk problems - Desi

Desi — Sat, 17 May 2008 17:00:20 +0200

I agree with Chris, I'm constructing a server with 10shares and 100Gb of data disk occupied with 300 clients data, that I pretend to lunch when you reach final phase, it will be impossible to have an effective remote backup policy with that amount of data, and will create unnecessary traffic that it will penalize more your infrastructure than my server. And if the data of this server is lost my business will be ruined and I will have 300 mad persons after me.

And for 10€ a share (10x10 = 100€) if something like this happened and I lost data, you can be sure I would terminate the service with you in the next minute, because I could find similar prices in the market, for the price as it is now (6x1060€) I probably took more time deciding If I left or not.

I can add that still now I'm a little scared to go on this with you seeing that one filler sent you all hosting infrastructure down creating a down time of almost 6h for everyone and lost of data for others and a down time of plus 60h and counting.

With your hosting approach with a solid backup policy and with prices a little shorter then the ones that you are planning for the final phase, and I don't have droughts that you will become a would reference in hosting.

Best,

Disk problems - Nomine Mutato

Nomine Mutato — Sat, 17 May 2008 13:29:27 +0200

Someone can you update me about situation stage? I have a 1-share VM with Debian, but from crash to now it's still inaccessible through SSH. I tried to reboot from web panel (where the VM appears running), but after 3 hours it's pending yet.

Disk problems - john

john — Sat, 17 May 2008 11:22:13 +0200

24hours passed.. nothing happened =_="

Disk problems - Chris

Chris — Sat, 17 May 2008 09:37:21 +0200

1) Can somebody explain how a problem on 1 filer (filer 13) can affect the whole platform? Does that mean that every time a filer goes down (hopefully not ever happening again) all servers on Gandi Hosting are affected?

2) I have had 2 server drives fail a few minutes from each other about 10 years ago. They were on raid 1. All data was lost. Ever since then on mission critical project I've been using 6+1. I think this is what you are trying to accomplish as well. Important however is to have a backup tool. Hard disk storage comes very cheap these days. I think it is imperative that Gandi allows for an easy backup and restore tool. If not, just to safeguard against possible occurences like today. Backing up off-site is nice but will result in heavy daily traffic at your data centers as well (unless this is a way to increase profit from data transfers). Since you have multiple data centers I think it should be quite do-able to have an on-site and off-site backup that synchs every day.

Right now you are profiling yourself as a great hosting alternative at an unbeatable price. I would hope that you keep that unbeatable edge by keeping the price the same AND providing a free backup. If you do that instead of increasing the price, I am convinced that your hosting success will be enormeous as people worldwide will swamp to your company.

It may lose you money in the short term, building these extra backup pools but it will make you money on a long-term basis as customers once they move to your company, they stay. Good price, safe data. Clean and simple. If you build your company on that, Gandi will soon be a brand recognized worldwide for excellence in Hosting.

Good luck and thank you for providing us with honest information.

Best regards,
Chris

Disk problems - Nicolas (gandi)

Nicolas (gandi) — Fri, 16 May 2008 22:25:17 +0200

Sorry, new disk and server creations have been put on hold until those affected by filer 13 are back online; We are therefore optimizing our efforts to do so before launching the new server creations.

Disk problems - desi

desi — Fri, 16 May 2008 22:13:35 +0200

Hi,

Is the creation of new data disks also with problems? I'm trying to create a new disk but the operation is taking too long than from the last times.

I'm also trying to stop my server 92.243.5.252 so that I can delete him and create a new one, but it is pending operation and I cannot delete it.

Is it possible also to create new servers at this stage ?

Best

Disk problems - Nicolas (gandi)

Nicolas (gandi) — Fri, 16 May 2008 21:13:59 +0200

Servers not directly impacted directly by filer 13 and the data loss should all be up and running now...the filer 13 replacement process is still ongoing

Disk problems - Ryan

Ryan — Fri, 16 May 2008 20:47:23 +0200

My server is still unresponsive as it has been since about 9am AZ time yesterday. It isn't a problem for me because I'm still developing my site, so that is at least fortunate. I'm just ansy to work on my project and I'm sure I'm not the only one. At least we are being kept updated though.

Disk problems - Sameisco

Sameisco — Fri, 16 May 2008 20:40:44 +0200

I can assure you guys that Gandi working hard on it:
All my server are back on track!

Disk problems - Notus

Notus — Fri, 16 May 2008 20:28:47 +0200

28. On Friday 16 May 2008, 19:05 by Nicolas (Gandi)

full reboot done for 80% of total servers.
Notus: ssh will come, os is starting

----------

It looks like you are having troubles other then just rebooting, I still can not access the ssh and one hour lookes like more than enought for a OS start.

Continuing the transparency policy you have been having today, can you tell us what problems are you steel facing, why the reboot is taking so long, and if you think more disks that the initial mentioned might be also with problems (even the people that didn't receive the disaster email)

And please be honest if the problem will be only solved monday (hope not) please tell us now and not in one hour intervals (is almost solved) until monday.

I prefer to tell my clients we are down until monday than it is up in one hour and the i is not and they call me again, as I'm writing you again

Keep the good work and thanks for the suport

Disk problems - Wouter Van Hemel

Wouter Van Hemel — Fri, 16 May 2008 19:24:57 +0200

Well, this is why it's beta testing. At least Gandi is honest about the problems and trying to find a solution.

This "disk loss" and "temporary freezing" – is it a problem with RAID 6 or some specific hardware issues? Sounds like a substantial problem for hardware that's supposed to protect against data loss.

Disk problems - Nicolas (Gandi)

Nicolas (Gandi) — Fri, 16 May 2008 19:05:32 +0200

full reboot done for 80% of total servers.
Notus: ssh will come, os is starting