My site is hacked today, support says we have no backup

[infogate]

About 10 hrs back, I felt something strange on one of my sites, as some php scripts stoppted working. I tried to reboot, and the server went down, I contacted PowerVPS Support, and they took it back online.

It was night in India, so I went to sleep, When I woke up, the entire site was hacked ( www.bhojpuria.com (http://www.bhojpuria.com) ).

I asked support to restore it from a backup and they replied

--
Hi there,

I've just checked, and we don't seem to have any backups for your VPS on our backup nodes -- my apologies.

Best Regards,
Phil
Support Technician III
Defender Technologies Group, LLC./PowerVPS Support
--

We never kept any backup, as I have Cpanel VPS Plan II, and I was told that they will take daily backups.

The website is a very famous community portal, and it is hurting my business reputation, when the site is down.

What should I do ?? Why they dont have any backups ??

Kumar

[airoid]

Ouch...let us know what kind of response you get from defender...

[infogate]

--
Hi there,

I've just checked, and we don't seem to have any backups for your VPS on our backup nodes -- my apologies.

Best Regards,
Phil
Support Technician III
Defender Technologies Group, LLC./PowerVPS Support
--

This is the last one they sent, when I asked for a restore/clarification again, they have stopped responding.

Kumar

[infogate]

I tried calling the Critical Support line for at least 5-6 times, each time the IVRS says - I am uable to connect the party....

[DeanClinton]

Reply to the ticket with URGENT in the subject.

Also, isnt there an e-Mail address that says "how am I doing? email soandso about it"

[infogate]

I removed that one...

[nadzri]

To what extent was your site hacked? Can you still access your vps? I went to the link and your site is up. Problem solved?

If it is a dynamic web site and most of the files and database are still good, you can try reconstructing it (your site). Even if only the db is good, you can re-install your web app and use the current db.

[infogate]

We got it back up, from our local backup...

Powervps support said, the backup is automated, and if there is no backup, they are not responsible for that.

Kumar

[DeanClinton]

Wow - that isn't good practice is it?

Makes you wonder if its happened to anyone else without us knowing?

[airoid]

There have been other cases that I've heard on the forums where a backup has been unavailable or very old. In fact, when I changed nodes a few months ago I contacted support to verify the backups were taking place as I didn't see them in the VZ panel. I was told they had in fact not been setup, although I only went a week or so without backup.

It is true that they aren't responsible for damages from missing or old backups, but they are still responsible to provide daily backups as it is included on their product pages on powervps.com.

I'm not sure if all nodes are on the new backup system yet, but the new system does not allow for us to verify backups in the VZ panel anymore. To verify backups we have to open a support ticket. I suppose I will need to open a ticket once a week or so to verify backup...

Oh, and if I every received this one sentence response to a problem like this I would be very angry:
I've just checked, and we don't seem to have any backups for your VPS on our backup nodes -- my apologies.

[StingRay]

I just checked my vps power panel and it says "No Backups are stored for this VPS"

Does this mean powervps is not making daily/weekly backups? Or is this separate? I assume this is separate because I thought the backups powervps makes are located on a different box entirely..

Some one please clarify whether or not I should be concerned.

[airoid]

You are probably on the new backup system which does not show backups in the power panel. You will want to contact support to verify backups are taking place.

[DavidP]

Hey everyone I just wanted to clear a few things up, and provide some information so that no one is taken by surprise with the new backup system.

1. Your VZPP interfaces with the old Virtuozzo Backup systems, but does not interface with the new solution we are using. We are looking into ways to allow our users to check that their backups exist, but at this point if you're worried, open a support ticket and have us check for backups. If they are NOT working/you do NOT have backups, we will have engineering fix whatever issues have caused this and get backups for you. If you DO have backups, we'll let you know what dates they are available for.

2. Daily backups: We keep backups for nodes/VPSes in case of hardware failure on our side, so we can get you back up and running if your data is lost. We offer you the ability to request that support restore backups/files from backup as a free-of-charge service when you purchase a VPS from us, as a courtesy. However, as has been said before, the backup processes are automatic, so we can't guarantee that we will always have non-corrupt, up-to-date backups for each VPS on our network, as there are thousands. Manual audits of the backups take a great deal of time, so we sometimes miss that a VPS is not being backed up for several days, but we are putting procedures into place to reduce the chance of us missing that, and to deal with it more proactively.

With all of this in mind, it is ok to rely on our backups for convenience, but please be sure that if your data is mission-critical, and irreplaceable, that you have backups of that important data in multiple locations. Remember, our backup solutions can still suffer from hardware failure, just as the node on which your VPS resides. It is alway a good idea, and an industry standard, to keep your own backups.

[Shammyh]

Is this the new R1 CDP system? And out of curiosity any reasons to moving to it versus the standard Virtuozzo backup system?

Thanks.

- Sam

[dexus]

This is realy bad. I think that those backups are very important and such mistake shouldn't happend. Unfortunately software or human can sometimes make a mistake, but problem is that with this new backup sistem we don't have a possibility to check backups of our VPS, like we were able before.

I was before checking backups of all my VE's every week and if I don't see them for some VPS in VZPP, I contact support and they fix it. This way I can just hope that my VPS will have a backup... :(

So question for staff is, are you planing to gave us some possibility to just see our backups somehow... :)

[DeanClinton]

Unfortunately, this isn't the first time this issue has cropped up.... it's all sounding too familiar.

http://forums.powervps.com/showthread.php?t=2298