Oops, our bad :(
Providing customer support is a top priority for us, be it through our support mailbox, online forums, phone or direct email. We actively try to make our support services available, responsive and of the highest quality.
So what happened?
However, our efforts hit a road bump in the last couple of weeks, when we had an internal problem with one of our smtp services. This mail service is in charge of relaying mail going out of our support system to our customers, and unknown to us it stopped working. This caused our support emails – including follow up on issues, checking up that problems have been solved and sending patches – did not go out.
Finding out was hard
Turns out this can go on for a while before finding out. We were churning along, providing support for the issues that kept coming in, sending out emails to customers and changing the ticket status to ‘waiting’, all the time oblivious to the problem. Eventually, we had a day where a support case email was cc’d to another team member for follow up, and when he complained he didn’t get it we found out about the problem. Upon further investigation the faulty mail service was identified and fixed. It also turned out we had no information about how long we were offline.
Trying to make up for lost time
We immediately went back to our email archive and looked for the last customer acknoledgement of a support email. This was a couple of weeks in the past so we went ahead and re-sent all the emails we were trying to get out in these weeks.
So what did we learn?
Several things, really. First, that a lull in support cases can be a sign the end of the year is coming and things are slowing down, but it can also be a sign something is not working well. Another thing is that we need to harden our IT systems and get an early indication of a system failure before it impacts our customers. We already set out to implement a self test that will run daily to make sure our support system is working as it should.
To summarize, I apologize for all of you who didn’t get the support you deserve during that time; if you feel we might have missed your case, or if it’s otherwise urgent, please write me at firstname.lastname@example.org, and I promise you’ll get an answer fast.