MRG Archive
Malware Research Group Project 024
Project number: 024
Project Details: Browser Security / Financial Malware test
Operating System used: Windows XP Professional Service Pack 3
Number of applications used: 27
List of applications used:
1. AVG Internet Security
2. Avira Premium Security Suite
3. BufferZone Pro
4. CA Internet Security Suite
5. DefenseWall HIPS
6. ESET Smart Security
7. F-Secure Internet Security
8. G DATA Internet Security
9. GeSWall Professional Edition
10. Kaspersky Internet Security
11. McAfee Internet Security
12. Norton Internet Security
13. Online Armor ++
14. OutpostPro Security Suite
15. PC Tools Internet Security
16. Prevx SafeOnline
17. SafeCentral
18. SandboxIE
19. SentryBay Data Protection Suite
20. SpyCop Cloak
21. SpyShelter
22. Trend Micro Internet Security
23. Trust Defender
24. Trusteer Rapport Emerald
25. Vipre Antivirus Premium
26. Zemana AntiLogger
27. ZoneAlarm Internet Security
You can download the test report here ->MRG Online Banking Browser Security Project
Malware Research Group Project 023
Project number: 023
Project Details: On Demand Scan Test
Operating System used: Windows XP Professional Service Pack 3
Number of applications used: 15
Number of malware samples used: 259.694
List of applications used:
1. A-Squared Anti-Malware 4.5.0.29
2. avast Antivirus Professional 5.0.462
3. AVG Anti-Virus Professional 9.0.801
4. Avira AntiVir Premium 10.0.0.597
5. BitDefender Antivirus 13.0.20.347
6. COMODO Internet Security 4.0.138377.779
7. ESET Nod32 Antivirus 4.0.474.0
8. F-Secure Antivirus 9.22 build 15450
9. G DATA Antivirus 20.2.4.1
10. Kaspersky Anti-Virus 9.0.0.736
11. McAfee AntiIVrus Plus 14.0.306
12. Microsoft Security Essentials 1.0.1961.0
13. Norton AntiVirus 17.6.0.32
14. Online Armor++ 4.0.0.35
15. VIPRE Antivirus Premium 4.0.3248
Detailed Test report is available for download here -> MRG On Demand Scan Test april 2010
MRG Online Banking Browser Security Test – March 2010
Project number: 022
Project Details: Online Banking Browser Security Test
Operating System used: Windows XP Professional Service Pack 3
Number of applications used: 10
Number of simulation tools used: 6
List of applications used:
Spydex, Advanced Anti Keylogger 3.7
Global Information Technology (UK), Anti-keylogger 9.2.1
Zemana, AntiLogger 1.9.2.172
SoftSphere Technologies, DefenseWall 2.56
QFX Software, KeyScrambler Professional 2.6.0.2
EMSI Software, Mamutu 2.0.0.22
Prevx Ltd. Prevx 3.0.5.91
Trusteer Ltd, Rapport 3.5.912.25
Soft Media Publishing Inc. SpyCop Cloak
SpyShelter, SpyShelter 3.0
Detailed Test report is available for download here -> MRG Online Banking Security Test Mar 2010
Rogue Software Infection Prevention test
Project details: Rogue Software Infection Prevention test
Operating System used in this test: Windows XP Professional Service Pack 3
Total number of programs used in this test: 24
Programs divided into two groups: Complementary Anti-Malware Applications & Full Featured Anti-Malware Applications/Internet Security Suites
Amount of samples used in this test: 30
The “complementary” antimalware applications tested were:
• Ad-Aware Free 8.1.4
• Corbitek Antimalware 2 Beta
• Immunet Protect 1.0.24-32*
• IObit Security 360 v1.40
• Malwarebytes’ Anti-Malware 1.44
• ParetoLogic Anti-Spyware 5.7
• PC Tools Spyware Doctor 7.0.0.514
• Prevx 3.0.5.50*
• Sunbelt CounterSpy 3.1.2848
• SUPERAntiSpyware 4.33.1000
• TrojanHunter 5.2
The full antimalware / internet security applications were:
• a-squared Anti-Malware 4.5.0.29
• avast! Antivirus 5.0.396
• Avira AntiVir Premium 9.0.0.452
• BluePoint Security 2010 1.0.98
• COMODO Internet Security 3.14.129887.586
• G Data AntiVirus 2010 20.2.4.1
• Kaspersky Internet Security 2010 9.0.0.736
• Microsoft Security Essentials 1.0.1611.0
• NANO Antivirus 0.6.0.6 Beta
• NOD32 Antivirus 4.0.474
• Online Armor ++ v4.0.0.15
• Panda Cloud Antivirus 1.0
• Trend Micro Internet Security Pro 17.50 Build 1366
Additional information:
All programs tested using their default settings.
Online Armor ++ enables HIPS by default
COMODO Internet Security enables Defense+ by default
Trend Micro Internet Security enables Proactive Intrusion Blocking by default
Kaspersky Internet Security enables Proactive Defense by default
Programs that manage to block installation of all 30 samples will receive MRG System Protected Award
Full report of Rogue Software Infection Prevention test is available for download in PDF format.
Jan 2010 Rogue Test
MRG On Demand and System Rescue test
The purpose of this project is to assess the effectiveness of a set of five full AV/AM applications and two AM/AS applications against 1000 mixed samples on demand and their effectiveness in detecting and removing fifteen live infections from a system.
On Demand Scan test
Methodology used in the on demand test:
1. Windows XP Professional Service Pack 3 is installed and updated with all important updates.
2. An image of the Operating System is created with internet access.
3. A clone of the Imaged system is made for each program to be used in the test.
4. An individual program is installed with default settings on each of the Cloned systems.
5. Any real time protection is disabled.
6. On each Cloned system the folder containing the samples of malware is placed.
7. All the programs are fully updated.
8. Real Time protection and other default methods of detection/prevention used by the applications are turned on prior to the start of the test.
9. The test is conducted by performing a right click scan of the folder containing the samples and allowing the application to delete / quarantine any samples detected.*
* Prevx is limited to detecting and cleaning 256 malicious samples at a time, therefore, we performed as many scans as was required to clean all the samples it was able to detect.
The applications tested were as follows:
a-squared Anti-Malware 4.5.0.27
AntiVir Premium 9.0.0.452
Bluepoint Security 1.0.0.83
Hitman Pro 3.5.3 Build 80
Malwarebytes’ Anti-Malware 1.42
Prevx 3.0.5.23
SUPERAntiSpyware Professional 4.31.1000
We used 1000 samples of malware, the samples up to one month old , only Trojans, Backdoors, Worms, Rogues, Spyware and Viruses were used.Results:
| Rank | Program | Detected Samples |
| 1 | Hitman | 984 |
| 2 | A-Squared | 983 |
| 3 | BluePoint | 982 |
| 4 | AVIRA | 959 |
| 5 | Malwarebytes | 817 |
| 6 | Prevx | 728 |
| 7 | SUPERAntiSpyware | 448 |
Infected System Rescue Test
Methodology used in this test:
1. Windows XP Professional Service Pack 3 is installed and updated with all important updates.
An image of the Operating System is created with internet access.
2. A clone of the Imaged system is made for each program to be used in the test.
3. An individual program is installed with default settings on each of the Cloned systems.
4. A Snapshot is taken of each cloned system.
5. Any real time protection is disabled.
6. On each Cloned system the folder containing the fifteen samples of malware is placed.
7. All the programs are fully updated.
8. Each malware sample is executed individually, with the system being rebooted after each execution, until all fifteen samples have been executed.
9. A second snapshot of the cloned system is taken, allowing us to know all changes / infections.
10. All differences between the first and second snapshots are noted.
11. Real Time protection and other default methods of detection/prevention used by the applications are turned on.
12. The test is conducted by performing a full system scan and allowing the application to perform its detection and removal activities.
13. Once the application finds no malware / reports a clean system, the cloned system is compared to the first snapshot so an assessment of cleanup effectiveness can be made.
The applications tested were as follows:
a-squared Anti-Malware 4.5.0.27
AntiVir Premium 9.0.0.452
Bluepoint Security 1.0.0.83
Hitman Pro 3.5.3 Build 80
Malwarebytes’ Anti-Malware 1.42
Prevx 3.0.5.23
SUPERAntiSpyware Professional 4.31.1000
List of malware samples used:
AdWare.Win32.Agent.pwl
Backdoor.Win32.Hupigon.iyzf
Email.Worm.Win32.Iksmas.fva
P2P.Worm.Win32.Palevo.keh
Trojan.BAT.Qhost.gx
Trojan.Downloader.Win32.Agent.ctrh
Trojan.Downloader.Win32.Genome.zng
Trojan.Dropper.Win32.Agent.bhrg
Trojan.Dropper.Win32.Mudrop.fgp
Trojan.Spy.Win32.Zbot.acyk
Trojan.Win32.Buzus.cmsb
Trojan.Win32.FraudPack.zdf
Trojan.Win32.Inject.admx
Trojan.Win32.Kreeper.hf
Trojan.Win32.Refroso.scn
Results:
| Program | Result |
| A-SQUARED | System Rescued |
| Hitman Pro | System Rescued |
| AVIRA | Failed |
| BluePoint | Failed |
| Malwarebytes | Failed |
| Prevx | Failed |
| SUPERAntiSpyware | Failed |
List of samples which were not successfully removed from the system, for each program separately:
Avira:
Backdoor.Win32.Hupigon.iyzf
Trojan.Win32.Refroso.scn
BluePoint:
Trojan.Spy.Win32.Zbot.acyk
Trojan.Win32.FraudPack.zdf
Malwarebytes:
Backdoor.Win32.Hupigon.iyzf
Trojan.Win32.Buzus.cmsb
Prevx:
Trojan.Win32.FraudPack.zdf
Trojan.Win32.Buzus.cmsb
Trojan.Win32.Inject.admx,
Trojan.Spy.Win32.Zbot.acyk
Trojan.Dropper.Win32.Agent.bhrg
SUPERAntiSpyware:
Trojan.Spy.Win32.Zbot.acyk
Backdoor.Win32.Hupigon.iyzf
P2P.Worm.Win32.Palevo.keh
Trojan.Win32.Buzus.cmsb
Trojan.Downloader.Win32.Agent.ctrh
Trojan.Win32.Refroso.scn
Trojan.Win32.FraudPack.zdf
Trojan.Dropper.Win32.Mudrop.fgp
Trojan.Downloader.Win32.Agent.ctrh
Trojan.Downloader.Win32.Genome.zng
This test is property of Malware Research Group, any unauthorized reproduction of this test is strictly forbidden.
Malware Research Group Project 021
Project details: On Demand Scan test
Operating System used in this test: Windows XP Professional Service Pack 3
Total number of programs used in this test: 21
Amount of samples used in this test: 554.891
Malware categories used in this test and the amount of samples in each category:
Trojans/Backdoors- 398.951
Windows Viruses- 8.864
Worms- 61.928
Adware/Spyware- 48.552
Rootkits/Exploits- 10.736
Other Malware- 25.860
List of programs used in P#21 and their program versions:
a-squared Anti-Malware 4.5.0.27
avast! Professional Edition 4.8.1356
AVG Anti-Virus 9.0.663 Build 1703
Avira AntiVir Premium 9.0.0.447
BitDefender AntiVirus 13.0.15.297
COMODO Internet Security 3.12.111745.560
eScan Antivirus 10.0.997.491
ESET Nod32 Antivirus 4.0.467
F-Secure Antivirus 10.00.246
F-Prot Antivirus 6.0.9.3
Ikarus Virus Utilities 1.0.97
G DATA Antivirus 20.0.1.1
Kaspersky Anti-Virus 9.0.0.463
McAfee VirusScan Plus 13.11.102
Norman Antivirus & Anti-Spyware 7.10.02
Norton AntiVirus 17.0.0.136
Online Armor++ 3.5.0.50
Panda Antivirus 9.00.00
Twister Anti-TrojanVirus 7.32
Sophos Anti-Virus 7.6.10
Spy Emergency 7.0.195.0
Methodology used in this test:
1.Windows XP Professional Service Pack 3 is installed and updated with all the important updates.
2. Image of the Operating System is being created.
3. Clones of the Imaged system have been made in the amount of programs used in the test.
4. On each of the Cloned systems a separate program is being installed.
5. All the programs used in this test are being updated with the latest databases , the updating process is finished within 60 minutes for all programs. When the updating procedure is finished and the successful program updates have been verified, internet is disconnected.
6. Malware package that was prepaid earlier is being placed into every PC scheduled for testing.
7. All All programs were tested using their default (out of the box) settings..
8. After each program finishes the test, another scan is being performed on the undetected items.
9. When each of the programs completes the second scan, the samples missed are being counted and stored into the external storage unit.
10. The final results are presented and show the amount of samples that were detected and removed..
Additional information:
McAfee VirusScan Plus enables Artemis by default therefore we tested McAfee VirusScan Plus with active internet connection at the same time the other oprograms were being updated with their latest databases.
The table shows the program tested, the amount of malware samples (all of the categories above) that were detected and removed.
| Program | Detection Rate (%) |
| a-squared | 99.8% |
| Online Armor ++ | 99.8% |
| G Data | 99.6% |
| Avira | 99.5% |
| Ikarus | 99.4% |
| Panda | 98.9% |
| Norton | 98.8% |
| Avast | 98.7%* |
| McAfee | 98.7% |
| BitDefender | 98.6%* |
| eScan | 98.6% |
| F-Secure | 98.5% |
| Nod32 | 98.3% |
| Kaspersky | 98.2% |
| Comodo | 98.1% |
| AVG | 97.4% |
| F-Prot | 95.7% |
| Twister | 94.6% |
| Sophos | 94.4% |
| Norman | 93.2% |
| Spy Emergency | 66.5% |
This test is property of Malware Research Group, any unauthorized reproduction of this test is strictly forbidden.
If you have any questions regarding this test, please visit our forums
Malware Research Group
Malware Research Group Project #20
Project details: RWS Real Time test
Operating System used in this test: Windows XP Professional Service Pack 3
Total number of programs used in this test: 25
List of programs used:
a-squared Anti-Malware 4.5.0.22
avast! Professional Edition 4.8.1351
AVG Anti-Virus 8.5.409 Build 1634
Avira AntiVir Premium 9.0.0.446
BitDefender AntiVirus 2010 Build 13.0.15.297
COMODO Internet Security 3.11.108364.552
Dr.Web 5.00.1.08170
eScan Antivirus Edition 10
F-Prot Antivirus 6.0.9.2
F-Secure InternetSecurity 2010 10.00 Build 246
G DATA InternetSecurity 2010 20.0.3.0
Ikarus Virus Utilities 1.0.97
Kaspersky Anti-Virus 2010 9.0.0.463
McAfee VirusScan Plus 2009 13.15.101
Microsoft Security Essentials 1.0.407.0 (BETA)
NOD32 Antivirus 4.0.437
Norman Virus Control 5.99 R14
Norton AntiVirus 2009 16.5.0.134
Online Armor ++ 3.5.0.32
Panda Antivirus Pro 2010 9.00.00
Panda Cloud Antivirus 0.08.82
Prevx 3.0.1.65
Spy Emergency 2009 6.0.605
Twister Anti-TrojanVirus V7 R3(7.32)
VIPRE® Antivirus + Antispyware 3.1.2775
Amount of malware samples used in this test: 60
We used the following samples of malware:
Adware.Win32.AdMedia.ed
Adware.Win32.Iebar.w
Backdoor.Win32.Bifrose.bksm
Backdoor.Win32.Kbot.tg
Backdoor.Win32.NewRest.an
Backdoor.Win32.NewRest.ao
Backdoor.Win32.Poison.anpg
Backdoor.Win32.Small.ejx
Backdoor.Win32.Wuca.ee
Backdoor.Win32.Wuca.ek
Email.Worm.Win32.Joleee.dbe
Net.Worm.Win32.Kolab.cnx
Net.Worm.Win32.Koobface.bjc
Net.Worm.Win32.Koobface.bjm
Net.Worm.Win32.Koobface.bjs
Net.Worm.Win32.Koobface.bju
Rootkit.Win32.Bezopi.a
Trojan.Win32.Agent.ctap
Trojan.Win32.BHO.xsv
Trojan.Win32.Crot.v
Trojan.Win32.Inject.ahhq
Trojan.Win32.Inject.ahte
Trojan.Win32.Pakes.now
Trojan.Win32.Refroso.cpj
Trojan.Win32.Smardf.fuz
Trojan.Win32.TDSS.aeaf
Trojan.Win32.Vaklik.fsi
Trojan.Win32.Vaklik.ftt
Trojan.Downloader.Win32.Agent.cmcq
Trojan.Downloader.Win32.Agent.cndd
Trojan.Downloader.Win32.Dadobra.dbd
Trojan.Downloader.Win32.FraudLoad.eyw
Trojan.Downloader.Win32.FraudLoad.wooi
Trojan.Downloader.Win32.Small.ambd
Trojan.Downloader.Win32.Small.ambv
Trojan.Dropper.Win32.Agent.apfr
Trojan.Dropper.Win32.Agent.aven
Trojan.Dropper.Win32.Agent.awwv
Trojan.Dropper.Win32.Agent.ayqa
Trojan.Dropper.Win32.Agent.ayzr
Trojan.Dropper.Win32.Agent.azhd
Trojan.Dropper.Win32.Agent.baoo
Trojan.PSW.Win32.LdPinch.dis
Trojan.PSW.Win32.LdPinch.gxo
Trojan.PSW.Win32.VB.akp
Trojan.Ransom.Win32.SMSer.in
Trojan.Spy.Win32.Agent.azmu
Trojan.Spy.Win32.Goldun.cnx
Trojan.Spy.Win32.KeyLogger.cly
Trojan.Spy.Win32.Zbot.aacf
Trojan.Spy.Win32.Zbot.aaha
Trojan.Spy.Win32.Zbot.aaim
Trojan.Spy.Win32.Zbot.gen
Trojan.Spy.Win32.Zbot.zte
Win32.AdvancedAntivirus.ib
Win32.OnLineGames.bkzf
Win32.OnLineGames.vjmz
Worm.Win32.AutoRun.afcb
Worm.Win32.AutoRun.auku
Worm.Win32.Bezopi.be
All samples of malware used in this test came from infected machines, samples were collected from 05/08/2009 – 20/05/2009. The testing was conducted from 27/08/2009 – 31/08/2009.
Methodology used in this test:
1.Windows XP Professional Service Pack 3 is installed and updated with all important updates.
2. An image of the Operating System is created.
3. A clone of the Imaged system is made for each program to be used in the test.
4. An individual program is installed on each of the Cloned systems.
5. On each Cloned system the package containing 60 samples of malware is placed.
6. All the programs are fully updated.
7. Real Time protection/On Access scanners as well as all other methods of detection/prevention used by various Security Applications are turned on prior to the start of the test.
8. The test is conducted by trying to execute each of the 60 malware samples.
9. In this test goal is to block the execution of each of the 60 malware samples, therefore we allowed various categories of Security Applications to be used in the same test, we tested mostly Antivirus, Internet Security Suite applications with their default settings (out of the box).
10. After each program used in this test is tested on against all 60 malware samples, the system is checked for any traces of active malware.
11. We will show the list of missed malware for each of the programs which failed to block all 60 of the samples.
12. The results will be presented separately for Antivirus and Internet Security Suite applications.
The tables shows: Program tested, Amount of samples blocked, Amount of Samples missed, Passed or Failed the test.
|
Additional information:
Using the same engine doesn’t mean that the result is going to be the same, if the two product don’t share the same features in real time protection (heuristics, behavior monitoring…) and the samples are not covered by signature database, they results will differ.
F-Secure, Microsoft Security Essentials, Prevx and Panda Cloud require a live internet connection in order to function properly. These four AVs were tested on VMs with live connections within 45 minutes of the traditional AVs Images being finalized in order to ensure they had no measurable advantage over them in terms of signature age.
For all other information, please visit our forums.
System Protection Award winners:
a-squared Anti-Malware
Avira AntiVir PE Premium
COMODO Internet Security
F-Secure Internet Security
G DATA Internet Security
Ikarus Virus Utilities
Kaspersky Anti-Virus
Online Armor ++
Panda Antivirus Pro
Panda Cloud Antivirus
Prevx
VIPRE® Antivirus + Antispyware
Programs that failed this test and the samples that were not blocked:
Avast
Trojan.Downloader.Win32.Agent.cmcq
Trojan.Dropper.Win32.Agent.apfr
AVG
Trojan.Spy.Win32.KeyLogger.cly
BitDefender
Backdoor.Win32.Kbot.tg
Backdoor.Win32.NewRest.an
Trojan.Win32.Refroso.cpj
Trojan.Dropper.Win32.Agent.aven
Trojan.Dropper.Win32.Agent.ayzr.
Trojan.Ransom.Win32.SMSer.in
Trojan.Spy.Win32.Zbot.gen
Dr.Web
Backdoor.Win32.Poison.anpg
Trojan.Downloader.Win32.Agent.cmcq
Trojan.Dropper.Win32.Agent.apfr
eScan
Backdoor.Win32.NewRest.an
Backdoor.Win32.Poison.anpg
Trojan.Win32.Refroso.cpj
Trojan.Win32.Vaklik.ftt
Trojan.Dropper.Win32.Agent.aven
Trojan.Dropper.Win32.Agent.ayzr
Trojan.Ransom.Win32.SMSer.in
Trojan.Spy.Win32.Zbot.gen
F-Prot
Backdoor.Win32.NewRest.an
Trojan.Win32.Inject.ahte
Trojan.Win32.Vaklik.ftt
Trojan.Downloader.Win32.Agent.cndd
Trojan.Downloader.Win32.FraudLoad.wooi
Trojan.Dropper.Win32.Agent.apfr
Trojan.Dropper.Win32.Agent.aven
Trojan.Dropper.Win32.Agent.baoo
Trojan.PSW.Win32.LdPinch.gxo
Trojan.PSW.Win32.VB.akp
Trojan.Ransom.Win32.SMSer.in
Trojan.Spy.Win32.Zbot.aaha
Trojan.Spy.Win32.Zbot.aaim
Trojan.Spy.Win32.Zbot.gen
McAfee
Trojan.Win32.Inject.ahhq
Microsoft (BETA)
Trojan.Win32.Inject.ahhq
Trojan.PSW.Win32.VB.akp
Trojan.Spy.Win32.Agent.azmu
NOD32
Trojan.Win32.Inject.ahhq
Trojan.Ransom.Win32.SMSer.in
Norman
Backdoor.Win32.Poison.anpg
Rootkit.Win32.Bezopi.a
Trojan.Win32.Inject.ahte
Trojan.Win32.Smardf.fuz
Trojan.Win32.Vaklik.fsi
Trojan.Dropper.Win32.Agent.apfr
Trojan.Dropper.Win32.Agent.ayzr
Trojan.PSW.Win32.LdPinch.gxo
Trojan.Ransom.Win32.SMSer.in
Trojan.Spy.Win32.Agent.azmu
Norton
Trojan.Win32.Inject.ahhq
Trojan.Downloader.Win32.Agent.cndd
Twister
Trojan.Win32.Vaklik.ftt
Trojan.Dropper.Win32.Agent.ayzr
Spy Emergency
Backdoor.Win32.Kbot.tg
Backdoor.Win32.NewRest.an
Backdoor.Win32.NewRest.ao
Backdoor.Win32.Poison.anpg
Rootkit.Win32.Bezopi.a
Trojan.Win32.Agent.ctap
Trojan.Win32.Vaklik.fsi
Trojan.Win32.Vaklik.ftt
Trojan.Downloader.Win32.Agent.cmcq
Trojan.Dropper.Win32.Agent.apfr
Trojan.Dropper.Win32.Agent.ayzr
Trojan.PSW.Win32.LdPinch.gxo
Trojan.Spy.Win32.Zbot.aacf
Trojan.Spy.Win32.Zbot.aaha
Trojan.Spy.Win32.Zbot.gen
Trojan.Spy.Win32.Zbot.zte
- No comments yet.
Polls
Loading ...Translator
