Several nodes lost power

[Howard]

Due to a circuit breaker trip, the following nodes are being booted up now:

VZ102
VZ105
VZ93
VZ95
VZ92
VZD1
HSPC1
VZ110
VZD31
VZD39
VZ107
MSVS9
VZD27
VZD28
VZ101
VZ109
VZ104
VZ97
VZ96
VZ99
VZ94
VZ4
VZ14
HSPCD12
HSPCD18
VZ106
HSPC14

We will update this thread when we have more information.

Thank you,

[RyanK]

At this time, only a few nodes are not back up and are currently being worked on. A majority of services either have been restored, or are currently in the process of being restored, and will be available shortly.

We'll continue to update this thread. Thanks.

[Daniel]

As a note to this we accidentally sent a blast for VZ93 to our entire customer base. This was human error.

As such, many customers getting the email pointing to this thread did not actually go down or have any service interruption at all.

To check if your server is or was affected by this, please run a traceroute on your computer using a command prompt.

The second to last hop should look something like the following:

eqashhspc30.iad.keepitsecure.net (74.200.88.130) 0.886 ms 0.884 ms 0.873 ms

In this case the node is HSPC30. If there is no hspc in the hostname, it is a VZ node.

For example:


eqashhspc30.iad.keepitsecure.net is hspc30

eqash30.iad.keepitsecure.net is vz30

Thank you for you understanding and patience. We are working hard to reply to all ticket regarding this (we have hundreds, thousands if you count our internal alarms) and will get a reply out to everyone.

At this time all nodes except for hspc1 which has forced a fsck.

Technicians are looking at each node to make sure all VPSes come back online.

[Howard]

At this time, all nodes have recovered. If you still have an issue submit a normal support request and we will take a look at it for you.

Thank you