Response from Heart Internet
As of Friday 9th November, Heart Internet posted a new blog article in response to the days outage that happened on Monday earlier in the week. Although they seemed to explain what happened in more detail, there still doesn’t seem to be any reassurance over how it won’t happen again, or how it could have been prevented.
Actually, saying that, it could have been prevented as they do admit they were deciding to upgrade their hardware. However, they also admit that the hardware they were replacing was still functional. So perhaps, this is a lesson learnt for them?
Hopefully whatever updates they are making will make things a bit more robust in future, and maybe this might entice customers to return. I’m sure there have been many customers who have now left Heart, I am one of them.
You can read the blog response here at https://www.heartinternet.uk/blog/status-of-heart-internet/.
So yesterday was, as was for many other users, the last straw in an increasing run of issues and unreliability from Heart Internet, who are deemed to be one of the top 5 web hosts in the UK.
With an explanation still up in the air, and no official word from the host themselves yet, I’ve already started to make the switch to a new web host (which this website is actually hosted on). It shouldn’t take too long to get everything migrated, and all future websites and projects hosted on the new platform either.
Let me just give you a rough summary of how the day went.
Heart first seem to acknowledge something was wrong by posting the following on their status page:
We are currently experiencing some network connectivity issues in our data centre. Our most senior networks team is currently investigating this as a top priority, and we will update you here again as soon as possible. Apologies for any inconvenience caused.
Customers of Heart will recognise this as the standard initial response to show they are aware of something. The ‘most senior networks team’ line is a bit of a canned response, but at least it gives customers some reassurance that they’re taking it seriously and are trying to get things fixed as quickly as possible.
The worrying thing is that this status remained like this, unchanged, for over 6 hours. By this time most of their customers (including myself) were wondering what on earth was going on, with no access to websites, control panels, email, FTP or even Heart’s own support system. But then spirits started to be lifted as Heart finally, after what seemed like an eternity, post a status update.
Work to resolve this issue is ongoing and fixes and changes to network routing are being rolled out currently, we will update this page again as soon as we have any more to report. We’ll provide further updates as soon as possible.
Needless to say, this wasn’t the update we were looking for.
Now into the start of the working day, still with no website access, emails, control panels or system support, clients are waking up to find their websites are not working, and demanding to know why and when it will be fixed.
With still no word from Heart, and with notions of deja vu running back to the massive power outage of 2016, things were beginning to look a little familiar. This was going to be ‘one of those days’.
Okay, so we’re now over 9 hours of outage and into the afternoon. Surely things should be identified and recovery actions put into place. The truth is, this is only half the story.
Having identified a hardware issue as part of the root cause we now have an engineer making the required replacement, we will update you as soon as there is any significant change to report.
Pray for small miracles, and you shall receive. I suppose we should be thankful that this seems like progress, right? At least Heart Internet have managed to identify what the issue is, and can now start tackling at getting things back to how they should be. A huge company such as this is bound to have backup plans to put into place when something goes wrong.
Unfortunately, this didn’t seem to happen as quickly as some of us would have liked. Although this update came in, this is where things stayed for an even longer period of time.
By this time, social media was starting to light up with increasing numbers of angry customers who couldn’t believe that there was no information being fed to them. Rightly so, as they were having their own customers biting at their ankles for answers, answers that they weren’t able to give them.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue, apologies for any inconvenience caused.
Things have now gone beyond a joke. The working day is over, nothing has changed, and I would imagine most customers have had all of their loyal years and memories with Heart Internet ruined and discarded. We’re now over 12 hours since the outage began, with lack of update and with the growing feeling that even Heart Internet themselves have no idea what has happened and how to fix it. This isn’t going to end well.
Then, almost as if out of nowhere, the first real update of the whole situation arises and we are actually given a useful piece of information from Heart Internet, when they post this status update.
We’ve identified an internal network issue is the cause of the service disruption. We are working to isolate the issue. This is strictly an internal network issue and has not impacted customer data.
I’m not sure how others really felt about this, I’m sure some were pleased that there was actually some progress being made now. For me, however, I started to become a little sceptical of this message. Was there something that Heart Internet just weren’t telling us so we wouldn’t ask too many questions? Could this have been an attack disguised as something else?
It’s 18 hours of outage now. Whatever decent reputation Heart Internet had as one of the UK’s top 5 web hosting providers, rest assured that is now well and truly tarnished. I had already made my mind up. I’m not sticking around.
Finally, we receive the status update that I think everyone had been waiting for.
All services have been restored. If you are still experiencing issues, please contact customer support.
But if you were looking for an explanation or detailed report of what happened, don’t count on it. We weren’t even given an apology. However, Heart Internet have said that every customer affected would be compensated, but only after an official statement had been made.
This hasn’t materialised yet, even after the end of the day after the event.
I won’t be chasing compensation myself, I don’t run online e-commerce websites or have daily income generated through them. However, I do have to explain why some people’s websites were offline for an entire day, even though I don’t know myself.
So where from here? Well, like I said already, I’ve made my decision to ditch Heart Internet. It’s a shame as I was recommended to them years ago by a dear friend, and their level of service and reputation was pretty damn high at the time.
I don’t know whether Heart will actually release a detailed report of what happened, but I’ll be waiting for it the next few days.
Adios Heart Internet.