This has to be my least favourite type of post. Thanks to some tweets from you, I learned EBSQ was down before I even got into the office. A check of our logs shows we’d been down since 4:17am EST and a quick call to our Data Center confirmed they were having ongoing general network issues affecting all of their accounts.
We’re now back up, but we wanted to apologize for the unplanned downtime all the same. Thanks for saving a spot for us and enjoy the rest of your snow day with EBSQ!
Part of yesterday’s downtime was due to a scheduled maintenance window at our Data Center. Part of it was due to a wide-spread network outage at the same Data Center during the aforementioned scheduled maintenance. According to the last tech I spoke with there, they’re still having some network issues so we may still yo-yo a bit. But the biggest current issue is our email. While it’s not technically “down” right now, because of the system-wide problem at our data center, all of their email is essentially stuck. We have 100K messages in the spooler waiting for their turn in the queue to be delivered (can’t wait to get all that spam, lol!)
What this means for you:
If you’re waiting for a response from us, it’s likely we haven’t gotten your message yet. We’re unable to send mail as well, so we can’t even respond to the messages we do have downloaded from our server. If you want/need technical assistance, your best bet is to go straight to the EBSQ Member Forums. You can also give us a tweet.
We’ve been assured that our mail *should* start showing up in droves sometime later today. Once it arrives and is sorted, we’ll get to your questions and requests as soon as we can. Thanks for your patience, folks!
EBSQ was experiencing brief outages around 7 PM and is now completely down. We’ve learned this is part of a larger issue at our data center. We hope to have this issue resolved ASAP and we’ll update this post as we learn more.
Hopefully you’re off enjoying a lovely Saturday evening and will be reading this post in retrospect.
Thanks for your patience!
Edited 10:39 pm EST We’re back up! Again, thanks for your patience.
Edited 11:42 pm EST Our Data Center is doing maintenance so there may be some continued yo-yo connectivity into the wee hours. We hope to be back up to full steam tomorrow morning. See you on the other side!
EBSQ is doing scheduled maintenance on our servers. The site is now in read-only mode and the forums are currently unavailable. This means no art uploads/COA generation and periods of brief downtime as a result. We should be back up to full-speed around noon Eastern Time, and we’ll let you know if this time estimate changes.
Thanks for your patience; we’ll see you on the other side!
Update 9:17 am EST: EBSQ forums are back up, we’re now working to correct our RSS feeds, most of which are showing in reverse!
Update 9:32 am EST: Site search, uploads, and COA management is still unavailable. We’re working to have that back online ASAP.
Update 10:28 am EST : Art uploads are enabled again, but not cycling through the Freshest Art RSS feed. If it’s important to you that your work be seen on the front page and your upload isn’t time sensitive, you might want to wait. Otherwise, upload away!
Update 10:43 am EST: Whoops, I misspoke. NOW the forums are back up. They were back online but somebody (and I’m not admitting its actually me) kinda forgot to turn them back on so that the rest of you can post. Yeah, that was my bad. Sorry, guys!
Update 2:00 pm EST: All systems *should* be functioning normally. Please drop us a line if you find anything not working as expected and we’ll send in an exterminator to squash the bugs. Thanks all!
EBSQ will be migrating to shiny new servers on Saturday, 26 December 2009 starting at Noon Eastern Standard Time. The current estimated total downtime is between 1 & 2 hours. We picked a weekend we hoped you all would have other plans, so hopefully we chose well! You can stay posted on our progress by following us on Twitter: http://twitter.com/EBSQ
And one more service-related note: Customer Service and Tech Support will be limited on Christmas Eve and closed on Christmas Day so our staff can spend the day with their families (and try not to eat all of the Christmas cookies!). If you have a general question, you’re always welcome to stop by our member forums. It’s likely some clueful person will be about to help if they can!
From all of us here at EBSQ, we wish you a safe and Happy Holiday Season and an Artful New Year!
We’ve been experiencing connectivity issues since yesterday and full-blown downtime for all of our servers since ~7am EST today. We have just learned this is the result of a network outage at our ISP. We have been given no ETA on when we’ll be back up. Sorry for the rotten start to your week, folks. Hope to see you back online soon!
During the evening of Friday, January 23rd starting at 8:00pm Eastern Time through the early morning of Saturday, January 24th ending at 8:00 am Eastern Time we will be performing system wide maintenance.
This will involve upgrading and patching servers which will require the occasional restart of each server. Web and email services may be unavailable for a few minutes at a time during a server restart during this maintenance period.
We’re not anticipating any significant downtime. Thanks for your patience!