2017 into 2018
Oh where to begin, lets rewind back to September 2017 to get a full appreciation of what has happened over the past 12 months. It all started about September 30th 2017 if I remember correctly. On that day which was a Saturday, I was enjoying a nice breakfast with my wife and daughter. I came home and opened my mailbox and had a letter from my bank. I open that letter and inside is a picture of my paycheck and a notice that it was declined due to insufficient funds. I immediately text the owner of the company I work for (there were only 6 employees), and he says he will look into it. Well without getting too deep into this story, He took forever to get me a replacement check, that replacement also bounced, and by October 17 I was behind about 3 paychecks or 6 weeks worth of work. So I issued an ultimatum to the owner, get current with my pay or I am not doing any further work.
Well he paid up, but told me that it was obvious that I did not have faith in his company any longer, and that I had 4 weeks to find a new job. I turned in my immediate resignation that night, I was not about to work any longer and deal with this bullshit. So that meant I had to run around from October 17th to October 31st to get any doctor/dentist appointments completed before my coverage expired at the end of October. November I took it easy, as this had stressed me out more than anything, I went as far as canceling out any fluff bills that I could find. This meant stuff like my cell phone, Netflix, Usenet and even the newspaper were canceled.
I went on job interviews here and there in the months of November and December, but those are really tough months to get hired in. With Thanksgiving, Christmas and New Years, no one wants to hire employees and then give them all those days off, plus many of the people that will train you are taking off full weeks in these months.
Well the crap show didn't stop with me losing my job, on December 26th 2017 the dedicated server that Retromags sits on had a RAID failure, and over the weeks of being unemployed, the backups had not been working correctly. Luckily LiquidWeb was able to restore what felt like everything, but then we started to notice missing images in the Gallery. We deleted these as we found them and started to get back on our feet. It was also on December 26th that I accepted a new job, same title, but with more pay. My first day was going to be January 2nd 2018. I am a Systems Engineer by day, which is a fancy level Helpdesk person. I basically support around 25 different companies IT needs in the Northern Virginia area. I have support a wide array of clients, pizza shops, restaurants, law offices, fertility clinics, non profits, accountants, etc.
Well when you are dealing with such different clients and 25+ of them, you really need to rely on documentation. The unfortunate part of this is that my predecessors in my position, really didn't document down how things were configured, or why they were set up a certain way. So from January until now I have been untangling the mess that the previous guys left me. Also I have been trying to untangle the mess at Retromags. When this website was restored, something felt off.....I could not place my finger on exactly what was wrong, but some things that had previously worked just fine, now started giving us issues. One of these issues was the ability to upload multi page spreads to the Gallery, which was pointed out by @kitsunebi77
LiquidWeb was charging me $150 per month to host Retromags and their support started playing the blame game. LiquidWeb would tell me the server was working just fine...and that the problem was with the Invision Power Board software. So I made a copy of Retromags and put it over on a different server at DreamHost....guess what? The problems didn't happen! So now I had to start looking for a new host to move us to. I didnt want to go to a $20 a month host like DreamHost as we are sharing resources with other websites. I also didnt want to go back to a dedicated host, because donations had stop coming in to offset this cost, and now we had moved our downloads off to a seedbox server.
With the 200GB worth of downloads on a seedbox, we could scale back to running Retromags off of a VPS, which is a step above a shared host, but a step below a dedicated server. Anyhow, while I was getting up to speed on my job that helps me afford keeping Retromags running, I was also trying to get all of our releases onto the seedbox, provide torrents of all our releases and migrate us to a new server.I found a new hosting provider named KnownHost and I have migrated Retromags over to this new VPS server. When we did this some of the Gallery images were not showing up again. Long story short, the RAID crash from 2017 was causing issues again, I had to export all the records from the gallery and write a script to compare these to files that exist. Out of 20,000 Gallery records, 20 files were missing the original images, which is needed for the rebuild tool to not freak out.
Well that was just fixed a few days ago, and it seems as though all is right in the world again. I have had Retromags running since 2005 and I thought I knew best about backups. I got my ass handed to me and the lesson has been learned. So now the following happens....
Everyday @ 7AM - The Retromags website syncs to the Retromags QNAP at my home
Everyday @ 7PM - The daily sync on the Retromags QNAP, does a versioning backup to an external drive
Everyday @ 7AM - Our Seedbox backs up to my Microsoft OneDrive, which then also syncs to my QNAP
Everynight @ Midnight - The overall Database for Retromags backs up and syncs down to the QNAP and my OneDrive
Every 6 hours - The Magazine Database backs up and syncs to my One Drive and QNAP
With the disaster plan above, we should never lose more than 24 hours worth of work on this site.
EDIT: Forgot that in January or February, my medical coverage was crap and I ended up in the hospital for 12 hours violently ill and that resulted in a $5500 bill that I am still paying off