Over the weekend players flocked to the brand new Diablo-like action-RPG Wolcen: Lords Of Mayhem (formerly known as Umbra), so many players in fact, that the servers couldn't handle it and started deleting peoples' stuff. Over Saturday and Sunday the developers worked to get it all back in working order, reverting to a previous save so players could get back characters and quest progression that just started disappearing.
The servers are back online now, but what on earth went wrong?
Well, Wolcen: Lords Of Mayhem is just too popular.
The new hack and slash left early access on Thursday, and almost straight away started suffering from the influx of players. At first, the developers released an update that seemed pretty jovial, they were thankful for the interest people had and apologised for the inconvenience.
The next update came on Friday listing a lot of the known issues - including characters disappearing, items placed in your chest disappearing online, and quest progression… well, disappearing. A hotfix was applied later that day, but unfortunately the issues didn't end there, and the game stayed offline for nearly 20 hours.
In their most recent post, they explain in more detail what went wrong, and how they went about getting it sorted:
"This issue was caused by a too large amount of requests leading the database to drop. We contacted our partners and looked for a solution with them, but that implied rolling back to a previous save of the database shortly before the hotfix.
"Since then, we are working with our partners to restore the database and merge it with the current one, while fixing the issues that caused these incidents.
"Unfortunately, other complications appeared in the process that led us to continuously push back the ETAs we provided for the resolution of the situation, and we're still working hard to fix these issues and make sure that the game works properly Online, which can still take some time.
"We want to sincerely apologise for the inconvenience, and to assure you that we're working around the clock to fix everything and ensure that you will have a pleasant experience when the Online mode comes back."
The online servers are back up, for now, though the devs are still looking into some issues on the Asia Pacific servers. It's worth noting that even when the online mode isn't working you should still be able to play offline.
So, Wolcen's looking to be a pretty popular one then. While they may not quite have the servers to handle it, the devs already have a very large number of players invested in their game. When the servers went back online yesterday, they immediately hit their all time peak of concurrent players of 127,542 (back in early access their peak was only 1934).