Anti-Spam Testing Round 2

[TomK]

Folks,

We are going to do a brief and limited round 2 of anti-spam testing again.

We have updated the Mailfoundry to the latest code, and so far it looks much better then it did when it was using their "beta" MIQ technology.

We have a few slots available for beta testers, however I need to put some requirements around it this time:

We need customers with high-volume domains, I'd like to see some testers on there that can push between 5-20K emails per day on up to 5 domains. If you have this volume, post here and we'll get in contact with you to setup the beta.

Thanks!

[wbennet]

I have some customers with many and few e-mails daily. I would like to participate of this test. This question Spam became very critical for my customers.

thanks.

[FFR]

I have 3.000-6.000 e-mails daily summary.
Spam - big problem my customers. Spam contains 60-80% from all e-mails.
I too want to participate this test.

[Charlie]

I am surprised more people don't want to take part.
I would be happy to take part in the test if you feel the following would make me a useful tester.

I am very pleased with my Mailscanner system and how it handles spam/viruses. The amount of memory it uses is another story.

Here are some examples from my Mailscanner daily summary
I also have loadavg installed and have several months of graphic displays of memory useage so we could do some comparisons.
I was also part of the original mailfoundery test.

3-1-2007
MailScanner Status:
5806 messages Scanned by MailScanner
106.9 Total MB
4709 Spam messages detected by MailScanner

2-28-2007
MailScanner Status:
6190 messages Scanned by MailScanner
275.8 Total MB
4943 Spam messages detected by MailScanner


2-8-2007
MailScanner Status:
4778 messages Scanned by MailScanner
98.6 Total MB
3758 Spam messages detected by MailScanner

As far as by domain
Here are the three top domains for
March 1st email activity
#1 1259 emails for 1 domain (this is the one that pounds my mailscanner)
#2 567
#3 396

The other 30 or so domains do the remaining 2500 or so emails per day.

[nadzri]

I am surprised more people don't want to take part.
I would be happy to take part in the test if you feel the following would make me a useful tester.


I would like to take part but I don't have 5k emails a day... maybe around 500 to 1000/day and that too most of them have gone to Postini. ;)

Any external spam filter will be very useful as it will reduce email traffic by 50% to 80% since that seems to be the usual spam vs ham ratio.

[TomK]

For those folks interested in taking part, even those with less the 5K emails a day, please send the following to beta (at) powervps.com:


Your full name
Login and password you would like
A list of the domains you want protected


We'll get you setup in the system and get you started. I am looking for the beta to last about 30-45 days MAX.

Thanks!

[TomK]

Those requested as of the date of this post have been processed.

Please remember to post feedback and questions here, support via the helpdesk is not available at this time.

Thanks for the help with testing.

[TomK]

Folks, A part was missing for the instruction email:

After changing your MX record, you need to add an SMTP route to the antispam server:

You need to add an 'SMTP Route' that tells the antispam server where to
send the mail to once it "cleans" it, typically this is mail.yourmain.com.

Click on 'SMTP Settings' on the top, then "SMTP Routes" on the left hand
nav side, and then add your mail server address.

[nadzri]

Hi Tom,

Thanks for the quick setup.

While setting up various options in MessageIQ and SMTP Settings, there is a dropdown box "Processing Steps for Domain:" on the top part. It would make things much quicker and easier if there is an "-All Domains-" option in there otherwise we'd have set everything individually, especially in the initial stage.

The interface hasn't changed much and it's good to be able to set the address where the digests are coming from.

Hope this will be better than the last time. :)

Btw, I tried accessing via https, the certificate still says "mailfoundry1.defenderhosting.com".

[TomK]

Hi Tom,

Thanks for the quick setup.

While setting up various options in MessageIQ and SMTP Settings, there is a dropdown box "Processing Steps for Domain:" on the top part. It would make things much quicker and easier if there is an "-All Domains-" option in there otherwise we'd have set everything individually, especially in the initial stage.


There is as from the systems admin option, and on the reports you can choose 'all domains' but not as a domain user from what I can see.

Let me submit a feature request.

[orenatsmart]

After almost 2 days back on mailfoundry here are the initial stats:

Domains: 3
Total Messages: 1,460
Allowed Messages: 398
Allowed: 27.26%
Blocked Messages: 1,062
Blocked: 72.74%

So far there have been no false positives and maybe 10-20 false negatives. I'm adding a 4th domain in today.

So far so good.

Tom,
Do you have any guidance as to pricing?

Oren

[TomK]

Tom,
Do you have any guidance as to pricing?

Oren

Oren,

The thought process is as follows:

Pricing will be on a per domain basis with an allotment of messages allowed per domain.

Example:

1 domain, 100K total messages.
1 domain, 500k total messages.

Etc.

Doing this on a per user/account basis (johndoe@somedomain.com and susiedoe@somedomain.com) is too hard to track and maintain.

Pricing isn't finalized yet, however the starting price might be in the $7-$10 for a domain per month.

Thoughts?

[orenatsmart]

Pricing will be on a per domain basis with an allotment of messages allowed per domain.

Example:

1 domain, 100K total messages.
1 domain, 500k total messages.


Would you consider something along the lines of

1 login (1-5 domains), 100K messages
1 login (5-10 domains), 500K messages
1 login (11+ domains), 1000K messages

As an example, I currently have 4 domains in this round of testing which I see under a single login to the MailFoundry. One account has 40+ email address where there other 3 have less than 5. Most of the domains on my VPS follow this pattern (few with a large number of email addresses and most with just a couple addresses).

A single login should be easier for PowerVPS to manage while bundling the domains into a single cost makes it easier for me to offer to smaller domains.

Oren

[orenatsmart]

There is an Outlook plugin available [1] to tag email as span/not spam. This should help with feedback - especially for spam that makes it through the MailFoundry.

Oren

[1] The help page [http://mx1.powerxmail.com/support.phtml] includes a link to the Outlook Plugin [http://www.mailfoundry.com/support/downloads/]

[Charlie]

I have one domain set up and here are day 1 results
I am using the stock mailfoundry setup
I have Mailscanner on my server set to remove anything that scores over 9

536 total emails processed
514 tagged by Mailscanner as spam
277 tagged by MailFoundry as spam
All MailFoundry spam was also caught by Mailscanner
No false positives by either Mailscanner or Mailfoundry

So I received 22 good emails today
With MailFoundry, as configured, I would have received 22 good emails and 237 junk mails.

Charlie

[Fred]

i'm not sure i understand charlie...

So Mailfoundry failed ?
If i understand correctly it found about 60% of the spam mailscanner did found ?
Or did i miss something ?

[nadzri]

Spammers also send directly to your email servers. So although MF caught only about 60% of what MS did, it doesn't mean MF is not as good. One has to look at the headers of those spam emails that made it through and see whether they went through MF or not.

Putting the MF device as the sole MX record won't cut direct spam 100%. Spammers now send to the web IP address assuming it shares the same machine as the email server, and even if you separate them you might still get spam. In the first MF trial months ago, it took 4 to 8 weeks for direct spam to disappear after putting MF as the sole MX record.

MS or SpamAssassin is needed as the 2nd layer of anti-spam. If you set MF to quarantine spam, what it does is reduce the emails that your server has to process, in Charlie's example above, by 60%.

:)

[Fred]

your points are good ... Putting things in perspective is always good :)

[TomK]

I've also seen these scumbags cache/store MX records, so even after changing your MX records, months later they are still sending to old MX listings.

You can firewall off your IP to only allow inbound from the MF device and your secondary MX device, if you have one, and force everything though the antispam appliances.

So far, I am very happy with the newest Mailfoundry release. We've also been testing the CanIT-Pro antispam solution, and it is very good as well, just much more complicated then the MF device for the end user, although it scales much much more then any of the MF products.

[Charlie]

All the above points are valid along with - the Baysian database gets fed regularly by me for Mailscanner, Whitelists and Black lists are used. These increase the efficiency of mailscanner over a basic Mailfoundry set up.

Right now I just want to see the initial results with no modifications to MailFoundry.

[Charlie]

To cover nadzri's and Tom's points about bypassing MailFoundry.

I ran a mailwatch data query on March 9 results looking for Mail tagged as spam by Mailscanner that DID NOT CONTAIN mx1.power in the headers
Results was 212, much higher than I expected. But its early in the test .

Shows a great deal of spam is bypassing Mailfoundry and going directly to the server.

I'll include that figure in future reports and see how (or if) it changes as time goes on.

[nadzri]

Is it possible to

1) change the name of the Quarantine Digest sender? Presently it is "PowerXMail."

2) put our own logo, or none at all, in the Digests?

Also, will there be some kind of redundancy later on? I.e. 2 or more MX records.

[TomK]

Is it possible to

1) change the name of the Quarantine Digest sender? Presently it is "PowerXMail."

2) put our own logo, or none at all, in the Digests?

Also, will there be some kind of redundancy later on? I.e. 2 or more MX records.

1. Not the send, not that I am aware of.

2. No. Send me a copy of the digest email you received and I'll take a look at it.

3. No redundancy on MX yet, until MF supports clustering. We'll have a cold-standby MF device incase the one of them in the farm fails however.

[nadzri]

3. No redundancy on MX yet, until MF supports clustering. We'll have a cold-standby MF device incase the one of them in the farm fails however.
Not sure what you mean here. You're saying no redundancy, but also "one of them in the farm"?

Would the cold-standby device contain the necessary info of the failed one, e.g. domains, smtp routes etc., so as to be able to just be plugged in and run?

Sent you a PM regarding #2. Nothing special in the digest other than a logo of Defender's, which resellers might like to have or not.

[orenatsmart]

A quick update - after processing almost 10,000 emails and blocking over 7,000, I've found my first false positive.

There are still a small number (within acceptable range) of false negatives (spam that makes it through), although I have not checked to see if they were sent directly to my machine or came through MailFoundry.

Overall I've VERY happy with the performance of the filter.

Oren

[Charlie]

So far it seems to work well. No false positives, some false negatives.
Eliminating all of my MX records except the MailFoundry device should eliminate the Bypassed items.

A Mailscanner with a years worth of Baysian tuning, whitelists, blacklists, and score tuning SHOULD be more effective than a plain unmodified Mail Foundry.

I think my testing so far shows just how good the device works right out of the box.

I have two back up mailservers and a few of the bypassed emails are coming from them.

Date: 3/15/2007
Total emails: 491
Tagged by Mailscanner: 477
Tagged by Mail Foundry: 377
Bypassed Mail Foundry: 70

WERE scanned by Mail Foundry
False negatives by Mail Foundry 37
==============================
Date: 3/17/2007
Total emails: 476
Tagged by Mailscanner: 421
Tagged by Mail Foundry: 373
Bypassed Mail Foundry: 67

WERE scanned by Mail Foundry
False negatives by Mail Foundry 23
========================
Date: 3/19/2007
Total emails: 575
Tagged by Mailscanner: 540
Tagged by Mail Foundry: 390
Bypassed Mail Foundry: 97

WERE scanned by Mail Foundry
False negatives by Mail Foundry 51
==============================

[nadzri]

From what I see in the statistics, MF has blocked 70% to 90% of my domains' emails, all of which seems to be spam because I've not had a report of any false positives yet. I have had a few false negatives but considering it has reduced about 90 to 95% of the spam, I'd say MF has done a good job.

I've also not seen any real delay in delivering emails; what's the volume the MF unit currently processes per day? One feature that I like is you can easily and quickly change the destination SMTP server per domain so if your server is down and you have a backup (mirror) server, you can easily redirect emails without having to wait for dns propagation. What would be better is if you can have the option of changing values for multiple domains in one go.

I think the only issue left, as far as I'm concerned, is pricing. It's always that, isn't it? :D

Pricing of course is entirely up to the provider, however I do hope you can consider a staggered number of messages per MF account model e.g. $5 for 30,000 msgs/mo, $10 for 80,000 msgs/mo, $20 for 200,000 and so on (just an example). Ultimately it's really about the number of messages that goes through the device and the number of spam that gets filtered, isn't it? Not domains, not addresses....

:)

[Charlie]

While this antispam solution works well and has lots of potential, Defender Hosting has killed it for me. The additional memory that has just been offered will allow my MailScanner to work with room to spare.

MF is easier to set up, maintain and works very well. I believe with a little work it would work nearly as well as MailScanner.

But for me, I have a couple of years of Mailscanner experience and a fair amount of time and money invested in it. The reporting and searches provided by Mailwatch when integrated with MailScanner are very useful to me.

While oomguaranteed memory USUALLY tops out around 490 or so, it has occasionally touched 560 for a few minutes. With SLM on a Power2, that would have been disaster. Thats why using MF as a Preprocessor to remove only the most flagrant spam intriguedme. Now with 640 standard, I will be able to use my MailScanner/ Mailwatch without worrying about a memory issue.

[blueface]

I'd like to participate in this beta test.

My question is will it work with email piping?
I have a domain with kayako esupport and would like to use the email piping to parse emails to tickets.

[Robert]

I'd like to participate in this beta test.

My question is will it work with email piping?
I have a domain with kayako esupport and would like to use the email piping to parse emails to tickets.

Basically what happens is you change your domain's MX to point to the MF. The mail foundry receives all of your email... cleans it etc. Then takes the cleaned results and sends it to the VPS to handle.

So it does not change anything at all in terms of piping, etc. Your mail server will still handle the ultimate delivery. It's just a middle man to stop stuff before it gets to your server.

[Robert]

(By the way, we use Cerb and Kayako with piping and have had no problems with MF interfering.)

[blueface]

That's good to hear Rob.
I just sent email to support to sign up for beta testing.

[caucus]

Hello Charlie

Are you saying I do not have to pay the extra $20 on a power 2 so I can have MailScanner work and not trash server?

I sure would like to save $20!

I am on a Power2 that should give me ??? with the upgrade?

[Charlie]

Check the Announcements section - Power-1 and Power-2 free memory upgrades!

Power 2 will get 640 guaranteed. My accounts have around 150 email addresses and I see 4-6 thousand emails daily being processed.

Using LoadAvg graphing, Average memory is about 390 with Max of about 460 - 490. Every few months I'll see a spike of 540 for a minute or so. Having 640 should work FOR ME with room to spare. Since SLM was announced with its special problems, I have carefully monitored and tweaked to get everything as optimized as possible.

Which brings me back on topic. I was interested in using MF as a Preprocessor for my most heavily used client domains to reduce the amount of work Mailscanner would need to do, and I am sure it would work fine for that, as would going to a Power 2 + like you have with Mailscanner.

But now Power VPS has presented another solution that will work for me. It may not be the solution for you. Only months of monitoring and tweaking will tell you what your best solution is.

I may be entirely wrong and the whole VPS will fry every twenty minutes when they turn on SLM for me :eek:

[nadzri]

You not wrong at all - I'm sure 640MB is more than enough.

On the other hand I don't look at MF merely as an antispam solution. It also acts as a backup spool for times your server is offline, and you can easily and quickly redirect email to another server without having to edit your MX record.

[blueface]

I'm happy with the performance of MF so far.
It's blocked 92% of spams I usually got on one domain.

Problem is quarantine doesn't seem to work for me. Do I need to enable digest?

[NETRIX]

Oren,

The thought process is as follows:

Pricing will be on a per domain basis with an allotment of messages allowed per domain.

Example:

1 domain, 100K total messages.
1 domain, 500k total messages.

Etc.

Doing this on a per user/account basis (johndoe@somedomain.com and susiedoe@somedomain.com) is too hard to track and maintain.

Pricing isn't finalized yet, however the starting price might be in the $7-$10 for a domain per month.

Thoughts?

When are you going to launch this service?

[Atomm]

Are there any beta slots open? Any ideas on product launch?

Mailscanner is killing me, even with the extra memory and only one process.

[nadzri]

Something happened to the MF service?

I had set my anti-spam setting to Quarantine but a couple of days ago I started getting many messages marked with "***Spam***". After logging in I found out the setting for all domains was "Rewrite subject with ***Spam***".

[albatroz]

Any news on the beta testing or this new product?

[NETRIX]

What is the current status? No update for long time. :confused: (http://geography.about.com/library/faq/blqztibet.htm)

[albatroz]

There is currently a third party company that is providing a service
like the one PowerVPS was planning to offer.
Maybe they dropped the idea.

[NETRIX]

There is currently a third party company that is providing a service
like the one PowerVPS was planning to offer.
Maybe they dropped the idea.

Which company? can you please post the Url.

Thanks

[Robert]

We're still using MailFoundy internally. However before it becomes an official paid product, we're waiting on a few more pieces from MF, including an API to be able to automate billing/provisioning.

Without these pieces, it's impossible to scale the product. (Can you imagine the amount of time it would take in tracking everything, from billing to account creations, etc without automation when dealing with thousands of accounts?)

[albatroz]

Which company? can you please post the Url.

Thanks

Easyantispam is the company, and this is the URL http://www.easyantispam.com/service/

I understand all the complications explained by Rob in his post. Maybe PowerVPS could do some kind of arrangement with EasyAntispam and arrive to a sort of partnership and produce a super new product.

[TomK]

WOW! $30/month? for 500k emails? And most of you guys made it clear that $10/month was wayyyyyyyyyy to expensive for our offering.

:rolleyes:

Easyantispam is the company, and this is the URL http://www.easyantispam.com/service/

I understand all the complications explained by Rob in his post. Maybe PowerVPS could do some kind of arrangement with EasyAntispam and arrive to a sort of partnership and produce a super new product.

[albatroz]

One thing I still don't have clear is how to calculate how many
email messages my VPS receive in a month. One difference of the
Easyantispam offer vs the PowerVPS one is the fact that you can
cover all your domains with one payment (as long as you don't surpass
the quota of course).

[nadzri]

WOW! $30/month? for 500k emails? And most of you guys made it clear that $10/month was wayyyyyyyyyy to expensive for our offering.

Tom, I believe their offering is slightly different with your intended one. That $30/mo covers unlimited domains on 1 server whereas PVps' idea was $10/domain/mo.

I am not more inclined towards them because I have less than 60k emails per month and I want the flexibility of changing destination email servers.

[TomK]

That was sort of my point - not so much about the differences in the offerings, just about the price point - I fully understand the differences etc.

Review the thread, just about everyone (if not everyone) had issues with a $10 month fee for spam protection, and I chuckled when I saw the post for a $30/month offering. Yes, I understand they are different - and $30 for unlimited domains and 500k emails - as long as they go to one server/vps only.

Not very flexible is it? Or is it?

Just struck me as funny to see an alternative for $30 a month (yes yes different plan) when everyone had issues about $10 month.

We are most likely going to offer it on a per-domain basis, $10/month for a domain with a pricing structure based on the # of emails since more emails means more processing and more equipment. Anything under $10/month just really isn't attractive to us at this point to have to deploy/offer support on/etc for this type of service.

Tom


Tom, I believe their offering is slightly different with your intended one. That $30/mo covers unlimited domains on 1 server whereas PVps' idea was $10/domain/mo.

I am not more inclined towards them because I have less than 60k emails per month and I want the flexibility of changing destination email servers.

[NETRIX]

When are you going to launch this service? :confused:

Many of our clients have shown interest in this service. Please launch it soon

[NETRIX]

Any update?

[TomK]

Not yet, sorry..

Any update?

[sudhakar]

Hi Tom

Any update on this? Please letus know the schedule release, if any, of Anti-Spam/MF Service. Thanks.

[Robert]

There is still no movement from MF regarding tools to automate billing/provisioning, etc.

Until they come out with something, there is not much we can do. We're interested in the technology, but until we can scale it.... we can't sell it. :)