Website server crash

Hi FlexSim Community Members,

You may have noticed last weekend that we had a server outage that lasted from roughly noon on Saturday, until Monday night (MST: UTC -7:00). Then again Early Tuesday morning for several hours.

First off - our apologies for such long down times. Its inconvenient when you need to check your licenses, submit a support request, post on the forum, etc. Its also bad for our business as people can't discover and learn more about FlexSim or sign up for a guest account, etc.

Our server host has told us that there was a catastrophic hard drive failure on Saturday, resulting in the complete loss of data on their server. Starting Saturday afternoon they were restoring sites from backup (we were on a shared server that hosted many sites in addition to flexsim.com).

Unfortunately there were problems in restoring our particular site. While I don't think it is a particularly large or complex site, it apparently dwarfs the other sites that were hosted on that server, which made it a particular challenge to restore. We communicated back and forth with them from Saturday until Monday morning.

When our site still wasn't up on Monday morning, we made the decision to try to relaunch the site ourselves on a different hosting provider. This we have done, and so far we're very happy with the move. We have provisioned a faster and more powerful server. The site feels faster and more responsive to us in our office, and we hope it does for you around the world as well. If you want to share your experience, good or bad, with the site speed on this new server, we'd love to hear it.

I'll make another post shortly detailing some other repercussions from the server crash.

The server crashed on Nov 1, around noon. The last database backup occurred on Oct 30. Any data submitted to the site in the gap between Oct 30 and Nov 1 is lost. This includes forum posts, guest account signups, account profile changes, licenses added to accounts, etc.

If any of this affects you, your profile, or your licenses, please contact [email protected] to describe your problem. We will work to manually restore these sorts of losses.

Another unfortunate affect of this crash is that all (well, at least most) user submitted data is lost. This includes any files uploaded to our ftp sites, or submitted as forum post attachments .

We have backups of all the files that WE upload to the site, but we weren't actively archiving uploads done by others. Shame on us. We were relying on our host's backups for this content. We have worked closely with them on this matter, but it appears that all backups they have for our site are either corrupted, or the necessary directories are completely missing (I believe these were the major problems with their restoration of our site in the first place).

Having learned a valuable lesson (keep your own backups - multiple backups), on our new server we are planning to set up a much more robust backup solution. Our new hosting provider has an option to keep backups, and we are taking advantage of that as a first step. However, it will not be our only defense as it has hitherto been.

We also plan to include a real-time data sync to our home office, as well as to a 2nd web server that we can failover to in the event of a new server crash. This should minimize down time and maximize redundancy.

Another part of the backup plan is for our local backup to be versioned. This will allow us to "go back in time", so that corruption/problems that have gone undetected (and backed up!) for some time can be rolled back to when files were still good and problems didn't exist.

We're not a server company. Web sites and servers are not our main focus. Wherever possible we contract with service providers who are experts at what they do, so that we can put our focus on our own expertise - simulation. However, these days everyone has to make sure that a robust backup solution is in place, and unfortunately for all of us, we dropped the ball.

Again, please accept our apologies. If you have attachments available to previous posts you've made, please re-upload those.

And we promise to do better on our own backups.

Thanks -

In case you missed it:

If you have attachments available to previous posts you've made, please re-upload those.

Apologies and thanks once again -

Hello Ben,

I can re-upload some models, but is it possible to edit the original post / replace the upload? I can't seem to edit my original post.

Furthermore I get an error popup when I try to go to the FlexSim Online Content, is this related to the server crash as well or would this be some other issue?

Esther

The in-software online-content error was related to the server crash and should now be fixed.

I will look into the forum-post-editing issue.

Thanks -

We can't download the attachment file and model in the forum posting.
Has it not been fixed yet?

We can't download the attachment file and model in the forum posting.
Has it not been fixed yet?

Unfortunately post attachments that were uploaded prior to the Nov 1 server crash are irretrievably lost.

I'm currently in the process of notifying users who posted attachments. Its possible that post authors may be able to upload their attachments again. I'm sure there are some attachments that will never be restored, but hopefully post authors will be able to re-upload many of them.

Thanks -

Originally Posted by Esther Bennett

I can re-upload some models, but is it possible to edit the original post / replace the upload? I can't seem to edit my original post.

Esther (and anyone else who couldn't edit your old posts to add missing attachments) - I changed some settings that should allow you to edit your posts and add the missing attachments.

Thanks -

Found the new site...thanks anyway.