PowerVPS To Upgrade to SLM on VPS Packages [Archive]

Robert

01-22-2007, 11:16 AM

PowerVPS is pleased to announce that after extensive testing and very positive results, we are in the process of implementing SWsoft's next generation memory management tool for our VPS customers. This new and improved method of memory management is known as Service Level Management, or SLM.

One of the primary reasons for upgrading to the new and improved SLM memory management tool is to make resource allocations easier to understand for our customers. One of the top complaints we hear from customers is that it is very difficult to manage and understand the memory usage inside their server and what limits they may have been exceeding.

With the existing memory management tool (known as "User Bean Counters" or UBCs), users have over 20 different parameters related to memory, such as privvmpages, kmemsize, and dcachesize. In addition, each individual VPS reported memory usage of the entire node instead of from their own VPS.

SLM solves these problems, and more! SLM eliminates all of the individual memory parameters and combines them into one simple, easy to understand number shown in megabytes and shows memory for just YOUR VPS. In addition, it introduces a new intelligent process killer. If your VPS exceeds its memory limit, SLM will first attempt to delay the execution of new tasks in order for it to wait for memory to become available. If necessary, it will attempt to send a signal to processes about memory restrictions and give it a chance to operate with less resources or exit before just ending processes. Previously if a VPS was out of memory, the hardware node would just deny memory to whichever application tried to access the server after the limit was reached.

Some of the benefits of SLM:

- Ensures each VPS does not exceed its allocated memory.
- Gives users one main memory parameter they can monitor to see their memory usage.
- Gives users the ability to see the memory of their individual VPS rather than the entire hardware node.
- Is more application-friendly regarding how it signals processes regarding memory usage.

As we roll-out SLM, we continue to update older nodes to the newest release of Virtuzzo 3.0 with Linux Service Pack 1. You can read more about this exciting release here:

http://www.swsoft.com/en/products/virtuozzo/new/

We will have an ongoing thread on our forums to discuss the upgrades. In addition, we will be sending email notifications before the upgrade takes place. As always, for scheduled maintenance where a node needs to reboot to load the latest stable kernel, we will provide 48 to 72 hours notice, and schedule the reboots for late evening Eastern time.

Thank you for your your continued business.

DeanClinton

01-22-2007, 11:51 AM

Great News!
So does this mean, that when we use a tool like phpSysInfo, it will display the amount of memory allocated to our vps, rather than the whole server?

TomK

01-22-2007, 12:10 PM

Great News!
So does this mean, that when we use a tool like phpSysInfo, it will display the amount of memory allocated to our vps, rather than the whole server?

Yes, and so much more! :o

DeanClinton

01-22-2007, 12:13 PM

Great! I can't wait.... I recently had a few emails backwards and forwards to technical support, as I couldn't grasp the concept of why it was showing memory usage at 99% with 30odd MB free.... in the end, I made out I understood, but I actually didn't :-p

Looking forward to the upgrades - fingers crossed it all goes smooth!

Just out of curiosity - are there any contingency plans in place, incase the upgrades go belly up?

winger

01-22-2007, 12:26 PM

hello,

will be possible to upgrade the RAM? like add 128MB or 256MB?

regards,
winger.

RCCI

01-22-2007, 12:28 PM

Thank You, thank you, thank you.

How will we be able to access it?

Wojtek

01-22-2007, 12:30 PM

Excellent News! :)

DeanClinton

01-22-2007, 12:37 PM

I don't think you will be able to access it any other way than viewing the figures inside the PowerPanel

RCCI

01-22-2007, 12:39 PM

I don't think you will be able to access it any other way than viewing the figures inside the PowerPanel

By "PowerPanel", are you referring to cPanel or WHM?

cyberhostuk

01-22-2007, 12:47 PM

By "PowerPanel", are you referring to cPanel or WHM?

No PowerPanel is a Virtuozzo Control Panel that comes with every VPS. It has several advanced features such as being able to reinstall your VPS from scratch etc.

It can be accessed at

https://www.your-domain.ext:4643/vz/cp

If you don't know your password, refer to your PVPS welcome email. Your username is 'root'.

Hope this helps :)

T

sdjl

01-22-2007, 01:58 PM

Will memory usage and availability for each VPS, rather than node, be available via top now?

Edit
Just a note to this. Logging in via Virtuozzo's control panel I see 100% system usage, but n/a under slmmemorylimit in extended resources.

mmmm...

bpm

01-22-2007, 02:12 PM

My VPS is not sending email and I have no access.

The root access to WHM, VPP and SSH is blocked?

69.65.99.234

Tony

01-22-2007, 02:25 PM

will be possible to upgrade the RAM? like add 128MB or 256MB?

We already have two memory only upgrades available - Power1-Plus and Power2-Plus. These plans give you the memory resources for the next plan up:

i.e. Power1-Plus gets 512mb ram, and Power2-Plus gets 768mb ram :)

Great! I can't wait.... I recently had a few emails backwards and forwards to technical support, as I couldn't grasp the concept of why it was showing memory usage at 99% with 30odd MB free.... in the end, I made out I understood, but I actually didn't :-p

Yeah, that's SLM - to me, one of the best parts about it, is that you can actually SEE for yourself exactly what your resource consumption is. There was a lot of confusion about UBCs.

Looking forward to the upgrades - fingers crossed it all goes smooth!

Me too! Preliminary results look brilliant.

Just out of curiosity - are there any contingency plans in place, incase the upgrades go belly up?

Unfortunately, the only contingency plan available in the event of this happening, is to switch back to UBCs. This is extremely easy to perform in the event that we see issues; however, we've been testing SLM extensively over the last two weeks, and have seen nothing but good things, so I don't expect we'll have to turn it off.

Will memory usage and availability for each VPS, rather than node, be available via top now?

Yes! That's one of the best features of SLM. No more confusing UBCs! As a sidenote to this, /proc/user_beancounters is now defunct with SLM.

Edit
Just a note to this. Logging in via Virtuozzo's control panel I see 100% system usage, but n/a under slmmemorylimit in extended resources.

mmmm...

PM me either your VEID, or your primary IP address, I'd like to see whats going on here.

My VPS is not sending email and I have no access.

The root access to WHM, VPP and SSH is blocked?

69.65.99.234

Unfortunately, this isn't the right place for tickets, please go ahead and drop a ticket to support AT powervps DOT com, and the support guys will take care of you. In this case, I've asked them to look at your VPS, but please, drop a ticket, or if you must post, open a NEW post, as Tom, Rob and I are the only ones likely to be replying to this!

Robert

01-22-2007, 03:16 PM

Regarding seeing your memory:

a) You can use a PHP script that gives you the details
b) You can check via your control panel (like cPanel, Plesk, DA)
c) You can check via top, or using the "free -m" command

When you login to a Power-1 VPS and run "free -m", you might see something like:

-bash-3.00# free -m
total used free shared buffers cached
Mem: 1024 149 824 0 0 0
-/+ buffers/cache: 149 824
Swap: 0 0 0

This would tell you you're using 149MB of your 256MB quota. It shows 1024MB TOTAL because 1024MB = 1GB (which is what the BURST is). So 256MB - 149MB= 107MB free in terms of guaranteed memory. If that number was higher than 256MB, you would be using burst resources.

Dru

01-22-2007, 04:11 PM

We already have two memory only upgrades available - Power1-Plus and Power2-Plus. These plans give you the memory resources for the next plan up:

i.e. Power1-Plus gets 512mb ram, and Power2-Plus gets 768mb ram :)

I had no idea these were available! How much more is the power2-plus than the power2?

I'm eagerly awaiting my node to be upgraded, this SLM sounds great! :)

mag

01-22-2007, 04:23 PM

Maybe I'm being dense, but I'm not seeing anywhere where it explains what's involved in the SLM upgrade as opposed to the VZ3 upgrade. Is there going to be downtime involved in this?

Martin

01-22-2007, 04:25 PM

hello,

will be possible to upgrade the RAM? like add 128MB or 256MB?

regards,
winger.

You must ask them about the "Plus" addons.

(example: upgrade from a Power-1 to a Power-1+ (plus))

This will give to your Power1 the ram of the Power2.

EDIT: Sorry this question was already answered :(

DeanClinton

01-22-2007, 04:25 PM

We will have an ongoing thread on our forums to discuss the upgrades. In addition, we will be sending email notifications before the upgrade takes place. As always, for scheduled maintenance where a node needs to reboot to load the latest stable kernel, we will provide 48 to 72 hours notice, and schedule the reboots for late evening Eastern time.
(Said at the top of the thread)

mag

01-22-2007, 04:26 PM

That's for the VZ3... right?
Because I just went through that for VZ3 and SLM is not on there.

winger

01-22-2007, 05:27 PM

You must ask them about the "Plus" addons.

(example: upgrade from a Power-1 to a Power-1+ (plus))

This will give to your Power1 the ram of the Power2.

EDIT: Sorry this question was already answered :(

Thank you!

I didn´t know that.

regards,
winger.

airoid

01-22-2007, 05:38 PM

That's for the VZ3... right?
Because I just went through that for VZ3 and SLM is not on there.

No, this has nothing to do with the VZ 3.0 upgrades. If you have already received the VZ 3.0 upgrade, you will also experience a few minutes of downtown for the SLM upgrade.

This is awesome news! I can't wait until this is rolled out.

Robert

01-22-2007, 05:56 PM

If you're on vz3 already, it does not mean you're necessarily on SLM. SLM has to be specifically enabled and configured. RIGHT NOW there are around 6 hardware nodes totally running on SLM. There are about 40-60 nodes that just need to have the individual VPS restarted for SLM to kick into effect. The rest of the nodes will need a physical reboot.

This Thursday morning at 12:01 (Wednesday night/Thurs morning), nodes that are 100% ready for SLM that just need individual VPS restarted will be done. Boxes that need physical reboots will start happening the next day. I'm working on list of boxes to be done Thurs night/Friday morning to have them notified tonight or early tomorrow.

It's going to be a busy next couple of days for me. :D

winger

01-22-2007, 07:17 PM

hello,

we will not able to have our own swap inside the VPS yet isn´t?

do you belive that a Cpanel VPS will work fine with only 256MB?

I already use one with SLM and 384MB and works fine, but I never tried one with just 256MB...:confused:

Robert

01-22-2007, 08:01 PM

No Virtuozzo-based VPS has ever had swap ability.

The hardware node itself can swap as needed, but indiviudal VPS cannot swap, as it could potentially crush the IO on the box for everyone else. Swap is MUCH slower than actual physical ram.

256MB has been fine for many customers over the years. :) We have thousands running on it fine. The nice part about SLM is that when someone does use more memory than they're allocated on a regular basis, the system can log what processes were killed and when. In addition, we can show you your exact usage. Before, we would have to say something like "you're hitting your privvmpages limit. You can see this usage in your PowerPanel..." and folks would have no clue what that really was or how it effected them.

Nothing is really changing here majorally. It's just a new way of enforcing what's already been done while improving reporting to the end user.

winger

01-23-2007, 04:29 AM

thank you! :D

dexus

01-24-2007, 12:49 PM

Can you please tell us the prices for those memory upgrades.

Thanks.

madfiddler

01-25-2007, 05:55 AM

please ignore me - post deleted

Milovan

01-25-2007, 11:46 AM

Can you please tell us the prices for those memory upgrades.

Thanks.

It would be best to contact the support department to receive additional information about these memory upgrade plans.

robb montgomery

01-26-2007, 01:59 AM

Well this upgrade has seen my server down for most of the day.

Had several techs optimize Apache, and we had to kill Spam Assassin, optimize FTP IMAPd and well it has been a headache and a half.

The new scheme just shuts down pretty much all of your services any time you you hit the ceiling. They do not come back 'automagicly.'

I have logged into virtuozzo a dozen times to restart the VPS - only then does my Web server come back on. No fun.

I am now upgrading to Power1Plus - a new option I guess for $69.99 to allocate more RAM. Even after all the grooming today my mem usage is still around 220mb and that's not much headroom to get to the 256MB hard top.

I don't mean to sound cynical it's just that this is not an improvement from where I was operating until yesterday.

Maybe there was a memory problem all along and I did not see it. Maybe. Who knows?

But this is a sad way to sell me on upgrading resources.

I have generally been very happy with the VPS and the tech suport is swift.

I have learned a lot more about SSH and Linux than I ever cared to but the techs are great.

Annyone else here now upgrading their RAM? (I also am paying for the monitoring service - I have to sleep knowing my server will not be down all night llike it was last night.

Robb

ashokjp

01-26-2007, 02:09 AM

I was very happy with PowerVPS.
But the SLM upgrade did put the same on me

right from the time i shifted to slm my server has gone instable.
cpanel, httpd all getting killed and worst thing is even when usage comes low these services require manual restart, sometimes even a reboot is necessary.

Support informed that when i exceed the allotted ram, this happens.
Burstable ram of 1gb or 2gb is not of much use when slm is there.

SLM is like forcing us to have a upgrade.

256MB has been fine for many customers over the years. :)
Rob's statement is going to be false after a while.

I am sure other users will have the same opinion once they are also shifted to this.

sdjl

01-26-2007, 06:20 AM

I know the majority of people who have posted have had issues, so I thought i'd be different.
I've had no problems since SLM was rolled out. I'm on a Power2 plan, so maybe that's why - I have more RAM.

I think in the long run and once SLM has been operational for a while, everyone will appreciate it more. After all, if I need access to some burst for a few seconds and can't because someone else is hogging it, my services are impaired.
This way, everyone gets equal treatment and you know where you stand with regards to memory usage.

Well done PowerVPS for rolling out the new technology.

David

DeanClinton

01-26-2007, 06:24 AM

Guys, I've just had a thought - what happens if SLM kicks in, whilst cPanel is updating, knocks off the cPanel service (and others), and renders the box useless?

Dru

01-26-2007, 07:20 AM

Good point, I must admit I'm a little more apprehensive about SLM now after fully considering it's effects. Not to say that I don't appreciate the many benefits it will bring in the long term of course.

I'm hopeful that I won't have any problems on my power-2 when it gets updated though as it's fairly well tuned.

Robert

01-26-2007, 09:56 AM

Regarding cPanel updates... here's a couple of scenarios:

#1) Your VPS is teetering at around 245 of your 256MB limit. cPanel update kicks off and you're server is trying to go to another 100MB of ram. Because you're average usage is so close to your limit, you would only be able to burst for a short period of time before the kernel starts signaling that ya're "walkin' da line".

#2) Your VPS is using about 150MB of your 256MB limit. cPanel update kicks off and it's doing perl updates, updating Clam, etc... it's decided to pull alot of memory during the perl updates and your total memory spikes to 300MB. (Wow!)

In this case, you would be able to burst for much longer as your average usage would take much longer to get up there. You would be using burst resources temporarily and should have plenty of time for it to finish what it's doing, then give back the goodies. :)

DeanClinton

01-26-2007, 11:04 AM

That still doesn't say if it's gonna brick the vps if it fails during an update.... :)

Dru

01-26-2007, 11:47 AM

I suppose the only thing to do to guarantee a cpanel update goes smoothly is to halt the processes on your VPS before commencing the update to free up ram. However that's not much use if you have automatic cpanel updates on.

I saw it mentioned somewhere that certain processes are supposed to be safe or less likely to be killed in the event that high memory usage is sustained over a long period, might it be possible in someway to restrict SLM from killing cPanel update processes and instead attempt to reduce the memory usage of other processes that aren't as critical during an update?

As I run with automatic updates enabled I'm quite concerned like DeanClinton that if my server is under heavy load SLM might kill the update process part way through.

Robert

01-26-2007, 12:58 PM

cpanel updates don't just "break" a VPS. Now... if you're hovering RIGHT at the line of your limit... and a cpanel update starts... it's going to most likely want more memory (thus burst). It will be allowed to burst for a short amount of time before being told to reduce usage or be forced to give up the extra memory.

The majority of customers have not had any problem with this. :) Almost half of our hardware nodes are now running SLM and very few people have had issues. But just like anything else... if you're already at your limits and try to add more work... then yes, you'll have problems.

DeanClinton

01-26-2007, 01:08 PM

Hi Robert,
cpanel updates don't just "break" a VPS.

Technically, this is inaccurate.
Here's a scenario.. cPanel issues security fixes via automatic cP updates... so lets say (for example), your close to your memory limit.. cP starts to update. Included in this update is an Apache (replace with mysql, bind and so on) upgrade. The upgrade starts, and the process has already been given it's "warning".. it continues, and the upgrade gets about half way through.
SLM then gives the kill command on the cP service, killing the update.

httpd fails (either str8 away, or its next restart... or whatever service it was updating), bricking it - it's possible you won't be able to reinstall the broken components.

That's what I'm worried about happening.. I've had a cP box have to be refomatted (thankfully not with you guys) before because of updates gone wrong.. this is why i'm concerned - i'm worried enough about cP, never mind SLM possibly screwing it up!

Where are those cookies!

DavidP

01-26-2007, 01:13 PM

Well, what if upcp was running and you hit your kmemsize limit on UBCs? The same thing would happen.

DeanClinton

01-26-2007, 01:17 PM

Yes, but then it would seem that people were hitting those limits before, whereas now with SLM, services are being killed.

mag

01-26-2007, 01:19 PM

Well, what if upcp was running and you hit your kmemsize limit on UBCs? The same thing would happen.
...Gracefully. If it determined there was not enough memory available it wouldn't start in the first place. SLM is killing things after the fact during the operation. It's just a matter of time before there is a dataloss or worse that happens during one of these ops.

winger

01-26-2007, 01:19 PM

I already have a VPS in other host that use SLM, with 384MB I can say that all cpanel updates was made so easy (with spamd and clamav enable).

mag

01-26-2007, 01:20 PM

With 384, sure, but not with 256.

ashokjp

01-26-2007, 02:17 PM

So you guys have kept this fixed... :(

SLM is not at all friendly..

It seems a way to force upgrades :(

Robert

01-26-2007, 02:46 PM

SLM does not FORCE you to upgrade. It forces you not to exceed what you're paying for. :)

If you're spending the MAJORITY of your time in BURST resources, you really should be upgrading or finding a way to optimize your resources.

You should be spending the majority of your time in your GUARANTEED memory. Remember... burst resources are not something guaranteed to be there. They're designed to help you deal with short bursts of extra memory as needed. Not to be something to LIVE off of. :)

Dru

01-26-2007, 03:06 PM

Damn, I really need to refresh and check the thread before I post :o

bpm

01-26-2007, 06:11 PM

My VPS worked fine for months and months without any downtime.

But this week after that upgrade I spent entire daytimes trying to make it up again without success. My cell phone rings every 5 minutes.

Upgrading to 512 Mb memory didn´t solve the problem. The server is overloaded.

I´m VERY unhappy and really don´t know what to do.

CPU Usage
Resource Capacity
CPU The resource is in Red Zone
100.0%
Load Average 182.25, 181.15, 173.92

System Usage
Resource Capacity
System The resource is in Black Zone
99.39%

Robert

01-26-2007, 06:25 PM

bpm,

Have you opened a support ticket?

If you're running out of resources, support can show you what is using it and make suggestions for optimization.

sdjl

01-26-2007, 08:24 PM

CPU usage" shouldn't" have anything to do with SLM. It's purely a RAM tool that I'm aware of.

Tony

01-26-2007, 09:31 PM

...Gracefully. If it determined there was not enough memory available it wouldn't start in the first place. SLM is killing things after the fact during the operation. It's just a matter of time before there is a dataloss or worse that happens during one of these ops.

I really have to argue with that statement. UBCs are a hell of a lot less kind about resource usage, and would SIGKILL anything at all if you hit your resource limits. The only major difference is that under UBCs, VEs could burst indefinately, unless all available RAM on the node was used, at which point, the VPS would switch to its RAM guarantee. To be perfectly honest, UBCs are an extremely poor method of resource control, and cause overall performance degredation.

To go back to the original point, you see just the same things happening under UBCs when you hit your limits as you will with SLM - the only difference, is SLM is a little kinder about requesting processes free up memory.

For all interested, I made an in depth post about how SLM works Here (http://forums.deftechgroup.com/showthread.php?p=15788#post15788).

Tony

01-26-2007, 09:32 PM

CPU usage" shouldn't" have anything to do with SLM. It's purely a RAM tool that I'm aware of.

Indeed, SLM has absolutely zero to do with CPU

jasonstx

01-26-2007, 09:52 PM

Here is what I was afraid of. I was bumping a VPS from mysql 4.0 to 4.1 and monitoring it at the same time:

It killed named as well as a few other services and never restarted them. I did go thru and clean up alot of processes and trimmed the fat off of apache somewhat so i now use 100MB of ram. However, at the time of this screenshot, I was using 180MB average on a Power 1 plan.

As I said, I was going from 4.0 to 4.1 and was disconnected.

http://www.mydietsucks.com/WTF.JPG

Now, the thing that scares me is this. The upgrade script was running for less than 8 seconds before it killed my services. Now, the script finished successfully. (thankfully or I would have been rebuilding something)

I mean fine, kill sshd, named and crond... take down httpd as well.... let my whm script update run though and then when the memory usage drops like a rock again, restart them por favor.

Although, thanks for forcing my hand and making me finally optimize my VPS

Tony

01-26-2007, 09:57 PM

According to that screenshot, you were bursting when it killed your ssh session...

jasonstx

01-27-2007, 12:36 AM

yes, I was bursting, thats for sure. My concern was that a) it killed ssh after 8 seconds and b) it didn't restart it.

How exactly could you even use the 1GB of burst available for some temporary need (like an upgrade) when it kills things that quickly?

Is it all because of the average memory I used (close to 200MB before the burst)? If it was lower (say 100MB) would it have let me stay connected longer?

nadzri

01-27-2007, 03:38 AM

If I may chime in...
yes, I was bursting, thats for sure. My concern was that a) it killed ssh after 8 seconds and b) it didn't restart it.
SLM is in the business of killing, not giving life. :) I don't know about cpanel but DA has a list of services that should be running and if any is not, it will restart it. Maybe you can get support to install SIM for you, if not already installed.

How exactly could you even use the 1GB of burst available for some temporary need (like an upgrade) when it kills things that quickly?
1GB burst is the maximum, doesn't mean that you will be able to burst that high.

Is it all because of the average memory I used (close to 200MB before the burst)? If it was lower (say 100MB) would it have let me stay connected longer?
I'd say yes. From my understanding, the reason why you can burst longer when your average use is lower is because it takes longer to reach the average limit.

Robert

01-27-2007, 09:16 AM

For a nice explanation about how bursting works exactly, please see:

http://forums.deftechgroup.com/showpost.php?p=15788&postcount=42

DeanClinton

01-27-2007, 11:04 AM

Am I the only one getting a headache trying to figure it all out? I'm nowhere near the limits (i think), so I really shouldn't be worrying about it just yet, but getting prepared is giving me a bad head!

:cookies: :again:

I'm gonna have to stop eating cookies - someone, please take them off me!

BTW - How do I know if I am using SLM?
Server 1 (http://phpsysinfo.etohost.com/) Server 2 (http://phpsysinfo2.etohost.com/)
I am asuming Server 2 is not, as it's never changed from that...

Robert

01-27-2007, 12:29 PM

OK.... easy way to tell...

If you type free -m and you see more memory than your burst is supposed to allow, then you're NOT running SLM.

1024MB = burst of Power-1 for example. If you see 8192.... that's most likely memory for the node as you're not gunna get 8GB of memory to yaself. (Sorry, lol)

Server 2's pic shows it is not running SLM at this time.

Regarding trying to figure it all out... it's honestly pretty simple. :)

free -m

If your usage is MORE than your guaranteed memory limit, you're not in good shape. You don't want to keep at that point for long periods of time, as SLM will say NO MORE. You can't LIVE in burst.

If your usage is LESS than that you're fine. The less average memory usage you have, the longer you can burst.

gmantakis

01-27-2007, 04:12 PM

Hi to all

I currently have a LINUX PLESK POWER-2 with PowerVPS and i am VERY impressed with their service and support so far. But i am a bit confused since they upgraded to SLM.

Logging in to my VPS via SSH and executing the command "free -m" i get the following results:

---------------------RESULTS BEGIN----------------------
[root@host ~]# free -m
total used free shared buffers cached
Mem: 1843 126 1716 0 0 0
-/+ buffers/cache: 126 1716
Swap: 0 0 0
[root@host ~]#
--------------------RESULTS END--------------------------

Also from within VIRTUOZZO i get the follwing output for the SLM Parameters:

---------------------VIRTUOZZO OUTPUT--------------------
ID Current use Soft Limit Hard Limit Units
slmmemorylimit n/a 536,870,912 1,932,525,568 bytes (which is 1.80GB)
---------------------VIRTUOZZO OUPUT-------------------------

From within Plesk i get:

----------------------PLESK STATISTICS---------------------
Memory usage
Total Used Free
1.80 GB 123.98 MB 1.68 GB
--------------------PLESK STATISTICS----------------------

Does this mean that i have now 1.8GB instead of 2GB in burstable RAM? Every output that i try, is it now SSH, VIRTUOZZO, Plesk shows me that i have only 1.80GB in burstable RAM. Since i am on a LINUX PLESK POWER-2 isnt it that i must see in these values 2.0GB??? What do others users see in their SLM outputs?

Sincerely

George

dexus

01-27-2007, 04:23 PM

Yes, it looks like they change that. We all have 1.8GB to burst, but don't worry about that because SLM will probably kill your processes that are using most of the memory before you reach 700MB.

StingRay

01-27-2007, 04:42 PM

Here's an example for others to be aware of.

Today I upgraded MailScanner to the latest version, in vain hopes that it might use less memory...

My VPS normally hovers around 440 usage.
I ran the Mailscanner upgrade from configserver, option 1 (which checks all the perl modules etc).

My memory usage spiked to 670 and didn't go below 512 for 5 minutes until the upgrade finished.

If I was on a Power-2 (512 limit) would it have killed my upgrade? I don't know but I assume so because the last time it kicked in on me it just nuked everything in sight.

So the info to learn here today is that an upgrade of MailScanner will eat up over 200 of your memory so make sure you have it before upgrading. Kill off any process you can before starting.

I'm just lucky i ask for a Power-3 upgrade before I did this or my server would have barfed.

Anyone have any idea on how much memory an apache rebuild takes? :p

dexus

01-27-2007, 05:52 PM

Yes, I also realised that, I was planning cPanel updates on my VE's for this weekend and I realised that there is no whay that cPanel can finish upgrade when my VE's are hovering around 400MB.

On just one small burst today to 530MB for 5-10 sec. my apache, named and exim was killed. :) So I just realized that I will never upgrade cPanel or Apache again :) Isn't that just great?

airoid

01-27-2007, 06:03 PM

On just one small burst today to 530MB for 5-10 sec. my apache, named and exim was killed. :)

Just out of curiousity, how do you know you were only bursting 5-10 sec?

dexus

01-28-2007, 04:28 AM

Just out of curiousity, how do you know you were only bursting 5-10 sec?

Because I was looking in that moment (actualy I was looking entire day :)) at my simple php script that is showing me some info about load and memory usage.

I was looking exactly a minute before that, and I saw that it was hovering around 450MB, and I started to refresh that page constanly, and in next moment it started to increase, I was still constatnlty refreshing page, and it was going to 480-500-510-520-530... and next refresh "Page not found"... cool, Apache is dead. :)

All this happend in less than 20 seconds, about 10 seconds below limit (480-512MB) and less than 10 seconds in the burst (512-530MB), and my conclusion is that this bursting just have no purpose now.

nadzri

01-28-2007, 06:04 AM

I was looking exactly a minute before that, and I saw that it was hovering around 450MB, and I started to refresh that page constanly, and in next moment it started to increase, I was still constatnlty refreshing page, and it was going to 480-500-510-520-530... and next refresh "Page not found"... cool, Apache is dead. :)

LOL! :D

Have you optimised your KeepAlive and Timeout values? And maybe offload unneeded modules to trim Apache down a little bit.

dexus

01-28-2007, 06:18 AM

Yes, it was optimized, but I had to optimize it more now, and it's now hovering around 300-350 and will (I hope) have enough space to increase if it needs to. But now mysql is working almost without cache :D and still I have a feling that everithing is still just on the edge.

So I have to use 300MB on a VPS of 512MB to have a space to be able to burst inside my guarantied RAM if it needs to, because I can't count on the burst outside of that. :)

So we now have a sytem that is running in the edge, it doesn't have a swap memory, and now it doesn't have a burst that was there instead of swap because Virtozo doesn't have a capability of swapping inside the VPS. That is why the bursting was there at the first place just to replace a swap, and now we have a small edge walking toy server. :D

We will just need to have more memory, and to always hover at least 30-40% below the limit and I think that should work ok. We just need to learn to live with that and it will probably be fine. :)

ikaruz

01-28-2007, 11:28 AM

Cpanel server with sql and apache optimized will use more than 256Mb. The following are the best values we've ever reached in the less busy VPS we run (we have 3 Power-2). So Power-1 should not be suitable to run Cpanel under SLM. And also imagine this can be extrapolated to every other panels.

Apache = 40Mb
Mysql = 120Mb
Bind = 54Mb
Exim = 32Mb
Cpanel Services = 50Mb
Mailman = 90Mb
Other services (ftp, ssh, courier, etc. ) = 40Mb

Total = 426MB

Please note that this doesn't include spamd nor MailScanner (Maybe you can leave out mailman and run MailScanner ad we did)

MySql (Almost without any cache) = 130Mb
mysql 1548 0.0 0.3 130524 28008 ? Sl Jan26 0:25 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/lib/mysql/sigma.modulserver.net.pid --skip-external-locking

MailScanner (1 child) = 100Mb (For each additional thread created when scannin for spam one 80Mb thead created, just for the scanning time)
mailnull 7536 0.0 0.9 79368 76584 ? S 08:36 0:06 MailScanner: waiting for messages
mailnull 1894 0.0 0.2 22756 18756 ? Ss Jan10 0:00 MailScanner: starting child

Bind (DNS server, empty no zones added) = 54Mb
named 1792 0.0 0.0 54024 2336 ? Ssl Jan10 1:45 /usr/sbin/named -u named
Bind (DNS server, in use around 350 zones) = 70Mb
named 1792 0.0 0.0 71728 7012 ? Ssl Jan10 1:45 /usr/sbin/named -u named

Mailman = 90Mb
mailman 3560 0.0 0.0 11328 8068 ? S Jan10 0:49 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s
mailman 3557 0.0 0.1 11376 8256 ? S Jan10 0:49 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=ArchRunner:0:1 -s
mailman 3558 0.0 0.0 11180 7792 ? S Jan10 0:48 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=BounceRunner:0:1 -s
mailman 3563 0.0 0.0 11220 8032 ? S Jan10 0:50 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s
mailman 3565 0.0 0.0 11156 7780 ? S Jan10 0:48 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=VirginRunner:0:1 -s
mailman 3538 0.0 0.0 10272 5004 ? Ss Jan10 0:00 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/mailmanctl -s start
mailman 3566 0.0 0.0 10060 6684 ? S Jan10 0:00 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=RetryRunner:0:1 -s
mailman 3562 0.0 0.0 10044 6712 ? S Jan10 0:49 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=NewsRunner:0:1 -s
mailman 3559 0.0 0.0 10036 6696 ? S Jan10 0:49 /usr/local/bin/python2.4 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=CommandRunner:0:1 -s

Cpanel services = 50Mb
root 3402 0.0 0.1 12156 8996 ? SN Jan10 1:14 cpanellogd - sleeping for logs
root 1915 0.0 0.1 11308 9172 ? S Jan10 1:28 chkservd
mailnull 3273 0.0 0.0 9272 6248 ? S Jan10 0:28 eximstats
root 3401 0.0 0.0 8328 6024 ? S Jan10 0:05 cpbandwd
root 32057 0.0 0.1 17344 8368 ? S 02:18 0:00 cpsrvd - waiting for connections

root 15366 0.0 0.0 7048 4372 ? S 00:22 0:00 /usr/bin/perl /usr/local/cpanel/bin/leechprotect

Exim (just waiting) = 32 Mb
mailnull 3077 0.0 0.0 8436 2048 ? Ss Jan10 0:06 /usr/sbin/exim -bd
mailnull 3085 0.0 0.0 8316 2024 ? Ss Jan10 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -q60m
mailnull 2046 0.0 0.0 8196 2044 ? Ss Jan10 0:02 /usr/sbin/exim -bd -oX 587
mailnull 3090 0.0 0.0 7408 2036 ? Ss Jan10 0:00 /usr/sbin/exim -tls-on-connect -bd -oX 465

root 7468 0.0 0.1 9880 8480 ? S Jan24 0:04 /etc/authlib/authProg
root 27736 0.0 0.1 9880 8480 ? S Jan24 0:03 /etc/authlib/authProg
root 9873 0.0 0.1 9880 8480 ? S Jan24 0:04 /etc/authlib/authProg
root 22441 0.0 0.1 9880 8480 ? S Jan21 0:08 /etc/authlib/authProg
root 22347 0.0 0.1 9880 8480 ? S Jan21 0:09 /etc/authlib/authProg

Apache (Optimized, max 5 threads) = 40Mb
nobody 15368 0.0 0.0 6192 2968 ? S 00:22 0:00 /usr/local/apache/bin/httpd -DSSL
nobody 11686 0.0 0.0 6052 2816 ? S 00:35 0:00 /usr/local/apache/bin/httpd -DSSL
nobody 15394 0.0 0.0 6048 2812 ? S 00:22 0:00 /usr/local/apache/bin/httpd -DSSL
nobody 22456 0.0 0.0 5916 2760 ? S 00:28 0:00 /usr/local/apache/bin/httpd -DSSL
nobody 24499 0.0 0.0 5880 2700 ? S 08:42 0:00 /usr/local/apache/bin/httpd -DSSL
root 24439 0.0 0.0 5872 2948 ? Ss Jan12 0:43 /usr/local/apache/bin/httpd -DSSL

All other services = Aprox 40Mb
root 7871 0.0 0.0 2472 928 ? Ss Jan10 0:04 crond
root 3100 0.0 0.0 3836 1460 ? S Jan10 0:58 antirelayd
root 11812 0.0 0.0 2300 804 pts/0 R+ 10:58 0:00 ps -auxh --sort=size
root 3493 0.0 0.0 3828 1384 ? Ss Jan10 0:01 pure-ftpd (SERVER)
root 11676 0.0 0.0 6592 1924 ? Ss 10:58 0:00 sshd: root@pts/0
root 1815 0.0 0.0 4004 1032 ? Ss Jan10 0:08 /usr/sbin/sshd
root 30663 0.0 0.0 1796 368 ? S Jan10 0:00 /usr/libexec/courier-authlib/authdaemond
root 30662 0.0 0.0 1796 368 ? S Jan10 0:00 /usr/libexec/courier-authlib/authdaemond
root 30661 0.0 0.0 1796 368 ? S Jan10 0:00 /usr/libexec/courier-authlib/authdaemond
....

We might be wrong, but this are the conclusion we arrived ... :(

ozgreg

01-28-2007, 09:13 PM

"So Power-1 should not be suitable to run Cpanel under SLM. And also imagine this can be extrapolated to every other panels."

Actually No that depends on the panel, if you are using a all in one cpanel then yes I would not disagree with this, however a panel like DirectAdmin does not have anywhere near the resource requirements of cpanel and will run in a power-1 account..

However, personally I feel the power-1 accounts should never be sold, fullstop, 256MB of ram is about as tight as it gets, and considering the number of connections you be limited to, you be better off going to a shared hosting account, however that is something the hosting industry as a whole needs to come to terms with, not just limited to PowerVPS.

esc

01-29-2007, 03:54 AM

In my opinion the above conclusion is not correct, because all binaries which are shared between the different VEs on a host (e.g. those which are used unaltered from the Virtuozzo templates and perhaps others) are not counted fully for your RAM usage but only to there aliquote part.

I for example am on a Power-1 plan. After recent optimization of MySQL and Apache and switching off Mailman at all, as non of our accounts uses mailing lists, I see this memory usage:
root@mandrel [~]# free -m
total used free shared buffers cached
Mem: 1024 153 870 0 0 0
-/+ buffers/cache: 153 870
Swap: 0 0 0
So I’m fully in the green zone with my account. And since the upgrade to Virtuozzo 3.0 it’s a lot more responsive than before (we are running serveral Typo3 installations here). I appreciate this very much.

On the other hand, if I do a ‘top –b –n1’ and sum up all numbers in the VIRT column, I get about 409 MB which is obviously not counted fully into my own memory consumption by Virtuozzo.

Beside this I have to admit that the templates for new accounts should be changed anyway to set them up with really low memory consumption especially now with SLM in place. And then probably the setup of the termination mechanism should be reworked as I noticed (before optimizing my VPS) that services were terminated which should have been running anyway (like SIM and named) and others like mailman have remained untouched.

Erich

F.N

01-29-2007, 04:22 AM

ermm, how do one tell know if one's running on SLM? I logged into VZPP and did not notice anything new; SSH in and it's still showing 8GB as total memory and uptime was 40+ days (ie no reboot since migration)

(Received notifcation email saying that the migration was completed.)

esc

01-29-2007, 04:48 AM

SSH into your box and issue a 'free -m' from the command line. If you see just the maximum burstable RAM (1 MB for Power-1) under 'total', SLM is running . If you see the whole RAM of the server you are either still under UBC or you have a somewhat larger plan than me :)

And you can see it on the user_beancounters. Here 'barrier' and 'limit' show 2^31 = 2147483647

root@mandrel [~]# cat /proc/user_beancounters
Version: 2.5
uid resource held maxheld barrier limit failcnt
968: kmemsize 4507433 4529294 2147483647 2147483647 0
lockedpages 0 0 2147483647 2147483647 0
privvmpages 59825 59825 2147483647 2147483647 0
shmpages 731 731 2147483647 2147483647 0
dummy 0 0 2147483647 2147483647 0
numproc 48 48 2147483647 2147483647 0
physpages 38518 38590 2147483647 2147483647 0
vmguarpages 0 0 2147483647 2147483647 0
oomguarpages 38518 38590 2147483647 2147483647 0
numtcpsock 24 24 2147483647 2147483647 0
numflock 2 2 2147483647 2147483647 0
numpty 2 2 2147483647 2147483647 0
numsiginfo 0 0 2147483647 2147483647 0
tcpsndbuf 214656 205712 2147483647 2147483647 0
tcprcvbuf 393216 376832 2147483647 2147483647 0
othersockbuf 24152 24152 2147483647 2147483647 0
dgramrcvbuf 0 0 2147483647 2147483647 0
numothersock 43 43 2147483647 2147483647 0
dcachesize 0 0 2147483647 2147483647 0
numfile 2393 2410 2147483647 2147483647 0
dummy 0 0 0 0 0
dummy 0 0 0 0 0
dummy 0 0 0 0 0
numiptent 341 341 2147483647 2147483647 0

Erich
-- Edit: added user_beancounters

ozgreg

01-29-2007, 05:24 PM

I for example am on a Power-1 plan. After recent optimization of MySQL and Apache and switching off Mailman at all, as non of our accounts uses mailing lists, I see this memory usage:
root@mandrel [~]# free -m
total used free shared buffers cached
Mem: 1024 153 870 0 0 0
-/+ buffers/cache: 153 870
Swap: 0 0 0
So I’m fully in the green zone with my account. And since the upgrade to Virtuozzo 3.0 it’s a lot more responsive than before (we are running serveral Typo3 installations here). I appreciate this very much.

Just to make sure here, you are using 153MB out of your 256MB limit so yes you are using roughly 60% of your memory resource limit and in the green. When checking your memory also check the number of connections. If you are running a dynamic content site then the number of connections will have a impact on how much memory is available..

DavidP

01-29-2007, 07:29 PM

I've been going through and optimizing several vpses throughout the day and I've noticed that I can get a cpanel power-1 running with about 80mb of memory consumption with no connections and no spamd. With a light load it was sitting at about 99mb, and with a light load + spamd it was using 130-134 mb constantly.

Now here's a simple test to put things in perspective:

On your home computer or office computer press ctrl + alt + delete. Open up the task manager (if you're not taken directly to it), click on the Performance tab and check your pagefile use.

How close to 100% are you? With everything running, I'm sitting at about 40% use (and I run a lot of stuff) at this time. I will now open a bunch of other programs that I shouldn't have open at work....

I am now near 100% of my physical memory use, my cpu use is still good, but I'm starting to notice things like frames redrawing more slowly as my system waits for memory to use/swaps memory. On a webserver, this would equate to lag as pages loaded.

Now admittedly, there are several factors playing into this, including the fact that the system was swapping to the disk (windows virtual memory), cpu use was higher than it should have been (around 40%) and my video card probably didn't like what I was doing, so the amount of slowness I saw was amplified by these. I reduced the strain on cpu and the video card by minimizing everything but my web browser, but I'm sure they still played a good part.

That aside, the high memory use also played a part as the system had to wait for memory to be freed before it could allocate more to the programs requesting it. Had I reached maximum memory use and tried to open another program, the system would have told me I was out of memory and the execution of that program would have failed. If the system was out of memory and one of these processes had requested more, that process would have died (and on windows it probably would have been a very ugly exit, if not a system crash).

On a linux system, if a core process had requested more memory at that point, it could have dropped into zombie state or core-dumped and terminated the process tree.

Sure, there's swap space, but once that's full and memory is full there isn't much you can do.

There is some speculation as to why we've removed swap from VPSes for this reason. The answer: We haven't. VPSes on the virtuozzo platform have never had swap at all.

The main virtuozzo kernel handles all swapping, and individual vpses can't swap. The node swaps when it needs to, but virtuozzo vpses don't and never have.

But the whole point of this thread is simple: If you were using 80-90% of your memory on your home/office computer, the performance would be HORRIBLE, so why would you do that to a production hosting server? Only the fact that the node controls memory and you're not actually hitting physical limits on your VPS keeps it running.

I once got so fed up with a friend's computer (it took 10 minutes to log into windows xp) that I bought 512mb more RAM and stuck it in the box. The result? An INCREDIBLE performance increase (windows loaded in less than than a minute).

If someone were running on actual dedicated hardware with 256mb of ram, used 200+mb constantly, and they needed 1GB to run an update, the update would core dump and never run. With the use of 200mb of memory constantly the performance of the server would be horrible, the amount of lag you saw when loading pages/checking mail would be huge.

The only difference between Virtuozzo terminating services when it's determined you've used too much memory on average for too long, and a dedicated box running out of memory is that Virtuozzo terminates the processes gracefully, allowing such things as mysql to handle the exit. On a dedicated system everything would just core dump and die.

Now with that said, we ARE working on telling SLM which processes to kill first, we are optimizing VPSes to the best of our ability at each customer's request, and we are listening to what you're saying. We've created a new package to help those of you on Power-1s who are teetering on the edge.

We do not want this to be a bad experience for you! Our initial testing with SLM shows better performance and more intelligent handling of memory (how many of you remember Virtuozzo reaping your processes because you hit your kmemsize limit, or privvmpages limit).

Work with us, keep giving us suggestions, tell us where SLM is hurting you and where it's helping, and we will do our very best to help each and every one of you. We are trying to make our products better and improve our service to you, but we need your help, your input, your suggestions, your compliments, your complaints, and we need to hear from all of you.

In the end, PowerVPS is about you, and we want every hosting solution we offer to be the best in the business, backed by the best support in the business.

Charlie

01-29-2007, 08:10 PM

DavidP, thanks for your explanation. It puts things into perspective a little better. I haven't had the upgrade yet, so have been watching this thread with interest.

Mailscanner and mailman are important parts of my web presence so memory useage is a factor. I tried running on a power1 and kmemsize was a big problem. Upgraded to a power2 and have had no problems for a year.

I use loadavg to keep track of daily load averages. I used this tool to slowly optimize over the last year (trying to see what really works). I average about 370 mb per day. Low seems to be 290 - 310, Peaks are usually about 410 - 470 several times each day, but last less than a minute.

What has me worried is
1. A couple times Cpanel updates have taken me up to 560 for a few minutes in the evening.
2. For a while this fall I was seeing 10,000 plus emails a day being scanned by mailscanner, some sort of spam attack that lasted for a couple weeks and just died off, but I was seeing 660mb several times each day for a minute or so.
Normal traffic is 4000 - 5000 per day. I have reduced the mailscanner from 2 to 1 process since then.

Going to a power3 isn't interesting me very much since I haven't seen a load over 1.8 and selom goes over 1, I use 38% of my space and a fraction of my bandwidth.

So I will continue to watch this thread with interest and try out some of the tips and hints. I appreciate posts like David's that show they want to help with this transition.

ikaruz

01-29-2007, 08:22 PM

Thank you David
Some PowerVPS staff post was needed :)
Now there is something I don't understand, how could you set upt a VPS with less than 100Mb?
Which services are running?
I triple checked what I posted, and compared with powerpanel numbers..
Could you please share that info with us, and forget about that SLM ghost once for all!!!!

drobbins

01-29-2007, 09:00 PM

...If you were using 80-90% of your memory on your home/office computer, the performance would be HORRIBLE, so why would you do that to a production hosting server? ...

I think this is, at best, an analogy. Memory management on a Windows machine is fundamentally different than on Linux. On Linux it is normal to see 80-90% memory usage, even on an near-idle machine (but I'm sure you know that). For example, right now on my old betsy here at home:

[me@home:~] free -m
total used free shared buffers cached
Mem: 503 474 28 0 152 272
-/+ buffers/cache: 49 453
Swap: 1027 0 1027

What needs some discussion is (IMO) how memory management differs between a dedicated machine and a VPS.

F.N

01-29-2007, 11:31 PM

SSH into your box and issue a 'free -m' from the command line. If you see just the maximum burstable RAM (1 MB for Power-1) under 'total', SLM is running . If you see the whole RAM of the server you are either still under UBC or you have a somewhat larger plan than me :)

OK, I'm NOT on SLM. It shows 8GB total. What do I need to do? Migration is 100% completed?

UID = 1633

total used free shared buffers cached
Mem: 8070 5353 2716 0 1320 2604
-/+ buffers/cache: 1428 6641
Swap: 8001 211 7789

Dru

01-30-2007, 04:33 AM

There's nothing you can do to get SML at the moment as far as I know. They're optimizing and tuning a few things from the current batch of migrated servers and I believe they're resuming the migrations later this week.

dexus

01-30-2007, 04:34 AM

There is some speculation as to why we've removed swap from VPSes for this reason. The answer: We haven't. VPSes on the virtuozzo platform have never had swap at all.
Nobody said that you removed swap, you just removed a burst that was there instead of swap. Swap is something that is necessary to keep system stable after using all physical memory, and because Virtuozzo VPS doesn't have swaping capability sw-soft gave it capability of bursting instead (that was probably easier to implement), and now when we don't have bursting instead of swaping and slowing down we have a killing and dieing instead. :) That can't compare to swaping slowdowns.

You are just talking about a slow windows without memory, but here is not problem slowness, problem is killing of processes, and that just can't compare to slowness.

So we now have to keep our memory usage at 50-60% and to pray that some little burst wan't go over the limit for more than 10 seconds. And that is not system for hosting a web site, when you have to pray that it won't just die because of too much web site visits at some point.

DeanClinton

01-30-2007, 05:45 AM

I don't know if it's a coincidence, but has anyone noticed that their Server Load has decreased? I know mine seems to have done...

dexus

01-30-2007, 05:54 AM

No, that is not a coincidence... I posted that already in other thread, here:

http://forums.deftechgroup.com/showpost.php?p=15808&postcount=47

That is indeed a good thing with SLM, because it will not allow every VPS to use as much as it want's, and will instead kill it's processes and that will reduce load on entire node, and on every VPS too, because high load of the node is also affecting all VE's load.

ozgreg

01-30-2007, 05:17 PM

I will backup DavidP here, you certainly can run Cpanel (I run DirectAdmin but same thing) in a Power-1 without any SPAM filtering tools in my case with a number of dynamic sites under 128MB.. The real issue is the lack of spam filtering, with all the issues of MF I still miss it as an option, you just cannot run spamd on 256 it just eats your memory resources and leaves you with a slim margin to handle any spikes..

Dru

01-30-2007, 06:33 PM

I'm investigating ASSP for spam filtering at the moment, so far it's much better than SA in terms of resources. It's peaked at 40mb and that was while it was rebuilding the spam db, normally it's less than that.

esc

01-31-2007, 02:07 AM

paulcook

01-31-2007, 02:35 AM

Hmm, interesting reading! I suppose I'm another person in the category of those pleased that SLM will ensure that my VPS is not crushed by other badly-managed VPS's continuously bursting; but I'm also a little worried about rapid process killing if I need to burst.

It occurs to me that one possible solution would be to extend the baseline for average usage calculation by, say, a factor of at least ten. I have no idea if this is possible! But it would increase the burst time allowed from the 10 seconds we've been hearing about, to more like 100 seconds. In the end this is still going to keep people within the guaranteed memory limit, but will make the burst capability more useful for things like updates.

Lorio

01-31-2007, 02:57 AM

So we now have to keep our memory usage at 50-60% and to pray that some little burst wan't go over the limit for more than 10 seconds. And that is not system for hosting a web site, when you have to pray that it won't just die because of too much web site visits at some point.

In other words Burst RAM is nearly irrelevant with SLM? The question is if that if that can be fixed with updates by SWSoft and with finetuning over time. Are there many Pleskuser on SLM with trouble?

What are the main issues without SLM (UBC):
-higer load
-the risk of two vps bursting at the same time
-no flexibility of using your dedicated memory (e.g. more iptables entries)

Not sure if offering VPS both with and without SLM will ease the situation because then more VPS with bursting needs are on the same node.

esc

01-31-2007, 03:43 AM

Is this a new city legend coming up, that one can only burst for 10 seconds when SLM is running?

If I remember a recent post by Robert correctly SLM uses a decaying logarithmic average to calculate the mean current memory consumption. That means it looks within a gliding time window on your baseline plus your bursts. If you have a low baseline you will have more reserves to burst. If this calculated average exceeds your guaranteed RAM-usage SLM kicks in. What it will do depends on the availability of the overall RAM on the system. If there is a shortage, first new memory allocation requests will be delayed which will make your system run slower. Ultimately when this is not sufficient it will start to terminate processes. Here it should use an intelligent and by the Virtuozzo administrator configurable algorithm which should terminate not vitally important childe processes first which will render the system still more slower. This will last as long as the mean current memory consumption is over the plan’s limit or the memory shortage of the system remains.

The process is similar to jogging or biking. If you’re in good shape (you are well beneath your body’s limit) you might go uphill (burst) without any slump until your aerobic energy depots are emptied (the average exceeds your guaranteed RAM). Then your body’s SLM will switch into the anaerobic domain and you will to have to calm down or even stop a while, if you exaggerate too much.

The difference in this picture between the new SLM and the old UBC system is that in SLM everybody is running on one’s own while with UBC people tended to run on there neighbours costs instead of upgrading their plan. It’s like sitting on a tandem and having your partner struggle if you want to go uphill with a speed over your body’s ability.

This at least is my view of those matters.

Erich

StingRay

01-31-2007, 03:40 PM

ikaruz

01-31-2007, 08:01 PM

I have to admit that with SLM things are totally different ....

I was playing with a new VPS under SLM and the amount of memory reported has nothing to do with primvpages reported at power panel under UBC.

The very same config running under UBC says privvmpages 112,824 (450Mb) and under SLM is reported with free as 180Mb (including mysql with 1M cache on each option, 5 Apache childs, mailman and 1 Mailscanner children with clamav and spamassasin).

I imagine the big confusion was because UBC and SLM uses totally different parameters to calculate memory usage, and I based all my calculations on UBC for the previous post (I didn't have any VPS with SLM yet).

Regarding the updates and upgrades, I tested upgrading and downgrading cpanel several times, and only saw an extra use of 60Mb the most, also compiled apache and used about 80Mb of extra memory for a couple of minutes.

But I only notice this since compare side by side two VPS with almost same config. This might clarify some points and misunderstandings regarding this new resource limiter.

The only thing left is a better way to select which process will be killed, and as I know PowerVPS staff, they are working with SWSoft technicians to obtain a better way to kill processe to release memory!!!!

ashokjp

01-31-2007, 08:40 PM

esc

02-01-2007, 03:56 AM

can you post your httpd.conf(excluding user domain parameters) and my.cnf
According Apache: I followed some of the recommendations of MichaelS’s HOWTO (http://forums.deftechgroup.com/showthread.php?t=2524). Especially I reset the following values in httpd.conf:

Timeout 180
MinSpareServers 1
MaxSpareServers 5
StartServers 1
MaxClients 50
MaxRequestsPerChild 300

According MySQL: MySQL does not fork off several child processes but uses only threads to do the work. If you see them with ‘pstree –c’ or not depends on the Linux kernel and the libraries to which it was linked at compilation time. Older libraries show these threads as processes, newer don’t.

My VPS showed them until last week (version 4.0). Now after upgrading MySQL I’m on version 4.1.27 and I see only one process as in my above listing. So this does say nothing about the number of running threads. In this case you can see your threads with ‘ps aux –M | grep mysql’.

For optimization of my.cnf I used a mixture of the values seen in my-medium.cnf which is a sample configuration which comes with the distribution and some of the recommendations of the above mentioned HOWTO. Especially I’ve set the wait_timeout to 15 seconds. As the load on my dynamic sites is not very high and my VPS memory dropped into the green zone, I did no further optimization. Perhaps I will go through all those buffer sizes when I find time to check the actual use of them with some appropriate tools.

[mysqld]
skip-networking
skip-innodb
log-slow-queries
skip-locking

datadir = /var/lib/mysql
socket = /var/lib/mysql/mysql.sock
user = mysql

max_connections = 250
table_cache = 256
max_delayed_threads = 5
max_tmp_tables = 32
key_buffer = 16M
query_cache_type = 1
query_cache_size = 4M
thread_cache_size = 4
thread_concurrency = 4
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
wait_timeout = 15

[safe_mysqld]
err-log = /var/log/mysqld.log
pid-file = /var/run/mysqld/mysqld.pid

Erich

DeanClinton

02-01-2007, 07:54 AM

Wow! I disabled SA, and my memory usage went down from 127MB to 84MB - that's a lot! (even with the tweaks applied from other threads)

Dru

02-01-2007, 08:05 AM

Yeah SA is pretty heavy memory-wise. Having done some testing elsewhere for a few days I installed ASSP on my VPS this morning. It's free but I decided to pay $15 to get a special cPanel script that gives individual accounts access to an interface they can control their spam with.

It's in training currently but it's already doing a pretty good job with just 14mb memory usage, maxing out at about 40mb when rebuilding the spam database which takes about 4 minutes or less once a day.

It's definitely a good alternative to SA if you're having memory usage problems. :)

ashokjp

02-01-2007, 08:11 AM

Yeah SA is pretty heavy memory-wise. Having done some testing elsewhere for a few days I installed ASSP on my VPS this morning. It's free but I decided to pay $15 to get a special cPanel script that gives individual accounts access to an interface they can control their spam with.

It's in training currently but it's already doing a pretty good job with just 14mb memory usage, maxing out at about 40mb when rebuilding the spam database which takes about 4 minutes or less once a day.

It's definitely a good alternative to SA if you're having memory usage problems. :)

Can you share the frontend cpanel script (show a demo if possible to see how it is)

DeanClinton

02-01-2007, 08:23 AM

Yeah, it'd be nice to see it (not the source, as that wouldn't be legal)... just a screen shot?

I'm contemplating this, as my clients really need a spam filter, but where do I start with ASSP ?

ashokjp

02-01-2007, 08:38 AM

Yeah SA is pretty heavy memory-wise. Having done some testing elsewhere for a few days I installed ASSP on my VPS this morning. It's free but I decided to pay $15 to get a special cPanel script that gives individual accounts access to an interface they can control their spam with.

It's in training currently but it's already doing a pretty good job with just 14mb memory usage, maxing out at about 40mb when rebuilding the spam database which takes about 4 minutes or less once a day.

It's definitely a good alternative to SA if you're having memory usage problems. :)

I dint mean you reveal the source, but can we also use the frontend you are using..

Dru

02-01-2007, 09:29 AM

Well the script I bought to add the user interfaces is from http://www.grscripts.com/ there's screenshots and everything there. The interface isn't the prettiest, but it is functional and sports full documentation so that your users can see how to manage their spam.

ASSP which, by itself is free, is available from here http://assp.sourceforge.net/

Whether you use the cPanel script with ASSP or not the installation is quite simple and if you decide you don't like it removing it takes a matter of minutes. :)

ashokjp

02-01-2007, 09:47 AM

Well the script I bought to add the user interfaces is from http://www.grscripts.com/ there's screenshots and everything there. The interface isn't the prettiest, but it is functional and sports full documentation so that your users can see how to manage their spam.

ASSP which, by itself is free, is available from here http://assp.sourceforge.net/

Whether you use the cPanel script with ASSP or not the installation is quite simple and if you decide you don't like it removing it takes a matter of minutes. :)

Is it monthly 15$ or one time... i couldnt get it exactly.

Dru

02-01-2007, 10:04 AM

It's $15 one off and that gets you the script and a year of updates. After that it's $8 a year for updates, which I don't think is too bad. He keeps the script up to date with the latest versions of cPanel and it has an auto update script which I'm going to try out later.

Wojtek

02-11-2007, 11:13 PM

Any news when the migration of the rest of the nodes will resume?

TomK

02-12-2007, 06:03 PM

Any news when the migration of the rest of the nodes will resume?

This week, we are doing more nodes. I think Rob sent out the maint notices already.

DeanClinton

02-15-2007, 07:25 PM

Hi Guys,
Do you have an ETA on when the node that my VPS is on will be "upgraded"?
ID: 5001

Kind Regards,
Dean

TomK

02-15-2007, 11:57 PM

Hi Guys,
Do you have an ETA on when the node that my VPS is on will be "upgraded"?
ID: 5001

Kind Regards,
Dean

Dean,

We're pushing back the upgrades, as we tweak and tune the SLM configs a bit more, or try to, we have some ideas we are working with SWSoft on currently, so I cannot honestly give you a new ETA yet, stay tuned.

Tom

ashokjp

02-19-2007, 09:25 AM

Dean,

We're pushing back the upgrades, as we tweak and tune the SLM configs a bit more, or try to, we have some ideas we are working with SWSoft on currently, so I cannot honestly give you a new ETA yet, stay tuned.

Tom

What do you mean by pushing back the upgrades ?
Does it mean slm is taken back ?

TomK

02-19-2007, 10:20 AM

What do you mean by pushing back the upgrades ?
Does it mean slm is taken back ?

I mean we are holding off upgrading any new nodes to SLM at this time as we work on existing SLM nodes with SWSoft trying to improve the kill process.

We are 100% committed to SLM and all nodes will be running SLM in the future. Overall it is a great improvement for 99% of our customers, and every single node that has SLM enabled is running much faster and more stable overall.

albatroz

02-19-2007, 05:13 PM

Any ideas for tweaking (or disabling) SA in Cpanel VPS. My new VPS is really unusable right now. All the services are dropping all time.....

albatroz

02-19-2007, 09:45 PM

Unless you have a POWER-2 Plan or greater, you may not want your node changed to SLM. I have the impression that 256MB is not enough memory for running a Cpanel installation in the SLM model.

ashokjp

02-20-2007, 06:51 AM

Any ideas for tweaking (or disabling) SA in Cpanel VPS. My new VPS is really unusable right now. All the services are dropping all time.....

Maybe you wanna try this
http://forums.deftechgroup.com/showthread.php?t=2595