The purpose of this project is to assess the effectiveness of a set of five full AV/AM applications and two AM/AS applications in detecting 1000 mixed samples from the last month in an on demand scan and their effectiveness in detecting and removing fifteen live infections from a system.

On Demand Scan test

Methodology used in the on demand test:
1. Windows XP Professional Service Pack 3 is installed and updated with all important updates.

2. An image of the Operating System is created with internet access.

3. A clone of the Imaged system is made for each program to be used in the test.

4. An individual program is installed with default settings on each of the Cloned systems.

5. Any real time protection is disabled.

6. On each Cloned system the folder containing the samples of malware is placed.

7. All the programs are fully updated.

8. Real Time protection and other default methods of detection/prevention used by the applications are turned on prior to the start of the test.

9. The test is conducted by performing a right click scan of the folder containing the samples and allowing the application to delete / quarantine any samples detected.*

* Prevx is limited to detecting and cleaning 256 malicious samples at a time, therefore, we performed as many scans as was required to clean all the samples it was able to detect.

The applications tested were as follows:

a-squared Anti-Malware 4.5.0.27

AntiVir Premium 9.0.0.452

Bluepoint Security 1.0.0.83

Hitman Pro 3.5.3 Build 80

Malwarebytes' Anti-Malware 1.42

Prevx 3.0.5.23

SUPERAntiSpyware Professional 4.31.1000


The results were as follows:




Infected System Rescue test

Methodology used in this test:
1. Windows XP Professional Service Pack 3 is installed and updated with all important updates.
An image of the Operating System is created with internet access.
2. A clone of the Imaged system is made for each program to be used in the test.
3. An individual program is installed with default settings on each of the Cloned systems.
4. A Snapshot is taken of each cloned system.
5. Any real time protection is disabled.
6. On each Cloned system the folder containing the fifteen samples of malware is placed.
7. All the programs are fully updated.
8. Each malware sample is executed individually, with the system being rebooted after each execution, until all fifteen samples have been executed.
9. A second snapshot of the cloned system is taken, allowing us to know all changes / infections.
10. All differences between the first and second snapshots are noted.
11. Real Time protection and other default methods of detection/prevention used by the applications are turned on.
12. The test is conducted by performing a full system scan and allowing the application to perform its detection and removal activities.
13. Once the application finds no malware / reports a clean system, the cloned system is compared to the first snapshot so an assessment of cleanup effectiveness can be made.

The applications tested were as follows:

a-squared Anti-Malware 4.5.0.27
AntiVir Premium 9.0.0.452
Bluepoint Security 1.0.0.83
Hitman Pro 3.5.3 Build 80
Malwarebytes' Anti-Malware 1.42
Prevx 3.0.5.23
SUPERAntiSpyware Professional 4.31.1000

List of malware samples used:

AdWare.Win32.Agent.pwl
Backdoor.Win32.Hupigon.iyzf
Email.Worm.Win32.Iksmas.fva
P2P.Worm.Win32.Palevo.keh
Trojan.BAT.Qhost.gx
Trojan.Downloader.Win32.Agent.ctrh
Trojan.Downloader.Win32.Genome.zng
Trojan.Dropper.Win32.Agent.bhrg
Trojan.Dropper.Win32.Mudrop.fgp
Trojan.Spy.Win32.Zbot.acyk
Trojan.Win32.Buzus.cmsb
Trojan.Win32.FraudPack.zdf
Trojan.Win32.Inject.admx
Trojan.Win32.Kreeper.hf
Trojan.Win32.Refroso.scn

The results were as follows:

a-squared Anti-Malware: System Rescued
(2 harmless traces left in the registry)

AntiVir Premium: Fail. Failed to remove the following samples;
Backdoor.Win32.Hupigon.iyzf
Trojan.Win32.Refroso.scn

Bluepoint Security: Fail. Failed to remove the following samples;
Trojan.Spy.Win32.Zbot.acyk
Trojan.Win32.FraudPack.zdf

Hitman Pro: System Rescued

Malwarebytes' Anti-Malware: Fail. Failed to remove the following samples;
Backdoor.Win32.Hupigon.iyzf
Trojan.Win32.Buzus.cmsb

Prevx: Fail. Failed to remove the following samples;
Trojan.Win32.FraudPack.zdf
Trojan.Win32.Buzus.cmsb
Trojan.Win32.Inject.admx,
Trojan.Spy.Win32.Zbot.acyk
Trojan.Dropper.Win32.Agent.bhrg

SUPERAntiSpyware Professional: Fail. Failed to remove the following samples;
Trojan.Spy.Win32.Zbot.acyk
Backdoor.Win32.Hupigon.iyzf
P2P.Worm.Win32.Palevo.keh
Trojan.Win32.Buzus.cmsb
Trojan.Downloader.Win32.Agent.ctrh
Trojan.Win32.Refroso.scn
Trojan.Win32.FraudPack.zdf
Trojan.Dropper.Win32.Mudrop.fgp
Trojan.Downloader.Win32.Agent.ctrh
Trojan.Downloader.Win32.Genome.zng

This test is property of Malware Research Group, any unauthorized reproduction of this test is strictly forbidden.

I am surprised to see SuperAntiSpyware score so low on that test

Hi computersplus.

Whilst SAS has detected only about 45%, we must remember that it is designed to be used in support of a full AV/AM.

For us, the shock result is MBAM out performing Prevx. Like SAS, MBAM is designed to complement a full AV/AM and consequently, has a very small number of signatures compared to these.

The fact it detected nearly 82% of the samples is testament to how good the Malwarebytes’ team is at keeping on top of current threats and making sure their database is lean but effective.

Best regards,

Chris

Very true Chris I agree hats off to Malwarebytes team like I said I was just shocked SAS score was so low I have always found the on demand scanner to do a good job and I use it a lot I just expected more that's why I love the testing you guys do nobody else compares the products you do and in the way you do and hats off to all of you as well keep up the good work.

Infected system rescue test is in its final stages, so far plenty of surprises and shocks. I hope that we will be able to publish the results by tonight (fingers crossed)

Once again hats off to the testing team!
PrevX seems more and more to be a nice interface without that much inside or is it just me?

Woodrow

Infected System Rescue test results added to the On Demand Scan test ones.

I have been loving Hitman Pro, I have used it several times lately. My only concern with it is it's being dependent on an internet connection, how many infected PC's end up with a hosed TCP/IP stack.

Yes, that is the downside of using cloud technology, on the other hand the removal capabilities of Hitman Pro are just superb.

I have to say that Malwarebytes performed very well in this test, it is starting to show very good results in all the latest testing.

If you check Early Warning Scoring (EWS) under Settings, Hitman Pro will scan your computer WITHOUT an internet connection. This mode lists all suspiciously behaving files using the Behavioral Scoring model that is a core part of Hitman Pro.

A rootkit blocking your internet connection will certainly show up in EWS. After cleanup, your internet connection is most likely restored and you can perform a cloud consulting scan to complete your system cleanup.

In EWS mode you can also remove the threats without a license. But please note that in EWS mode you must know what you are doing as non-malware files might get listed. So EWS is for advanced users only.