Archive

Archive for the ‘Heuristics’ Category

Q2 2011 Test Results of Security Suites

July 20th, 2011 5 comments

Recently both AV-Test.org and AV-Comparatives.org have announced respective results for their dynamic real-world or whole-product tests. Basically these AV tests try to replicate user experience by introducing malware to the test machine in pretty much the same way a regular user would encounter malware and get infected. We are very proud of the results of Panda Internet Security 2011/2012 as it shows consistency in providing top quality detection and protection, on top of better known security vendors such as Symantec, Avast, AVG, ESET, Trend Micro, Microsoft, Webroot, etc.

AV-TEST REAL-WORLD TEST – Q2 2011 RESULTS

In this real-world test results for Q2 2011 Panda was one of only 4 vendors to achieve a score higher than 15 points.

AV-COMPARATIVES WHOLE-PRODUCT TEST – JUNE 2011

In the June 2011 test Panda Internet Security achieved the first place in “blocking” rate without requiring any user interaction along with two other vendors.

Tis the comparative season

April 25th, 2011 Comments off

There’s been a few comparative tests published as of late. In case you’ve missed any of them here’s a quick rundown of the most significant ones.

First on the list are the Q1-2011 quarterly results of the Full Product Test (FPT)  by AV-Test.org. These FPT’s are performed on a monthly basis and are very in-depth, covering pretty much all aspects of a modern security software and testing from a users’ perspective by replicating infection vectors and user experience. The areas tested include real-time blocking of malicious websites, detection of relevant and active malware samples (zoo malware, wildlist malware according to AV-Test.org criteria of wildlist, not the limited WildList.org list), false positive testing, performance testing, disinfection testing, detection and disinfection of active rootkits, behaviour-based dynamic detection, dynamic false positives and packing and archive support. Overall one of the most comprehensive regular tests out there. It’s such a tough test that 5 out of the 22 vendors tested did not obtain the minimum score to achieve certification. Panda Internet Security came out with very good scores and achieved certification. The report on Panda Internet Security can be found here (PDF) and the complete results for all vendors here.

Next in line are a couple of tests by AV-Comparatives.org. The first one is the traditional On-Demand test from February 2011 which also tests false positives and performance of the on-demand scanner. In this test Panda Antivirus Pro achieved the #4 rank in terms of malware detection. We still had 18 false positives that, even though are of low prevalence according to AV-Comparatives.org, prevented us from achieving the Advanced+ certification. We’re doing a lot of work in improving in this area. Panda Antivirus Pro also achieved the #2 rank in the performance test for scanning speeds. The full report can be downloaded from the AV-Comparatives.org website here.

The second test by AV-Comparatives.org that has been published recently is the Whole-Product Test. Similar to the AV-Test.org Full Product Test, this test tries to test user experience by replicating the infection vector. Unlike the AV-Test.org FPT, this one focuses only on malicious websites and behaviour-based dynamic detection. Panda Internet Security scored very good with a 98.8% protection index. More information can be found at the AV-Comparatives.org site here.

If you’re interested in these types of AV tests, make sure to vote on your favourite AV testing outfit in our open poll here. So far both AV-Comparatives and AV-Test are leading the pack.

Panda Antivirus Command Line Scanner 9.5.1.2

February 10th, 2011 5 comments

We have an updated version of Panda Antivirus command-line scanner available, version 9.5.1.2.  It can be downloaded from http://research.pandasecurity.com/blogs/images/pavcl.zip. The package includes a signature file (pav.sig) from today. In order to download updated signature files you can use a current license to any Panda products (except Panda Cloud Antivirus) to access the updates available a http://acs.pandasoftware.com/member/pavsig/pav.zip with the license credentials.


Possible parameters:
-auto Scan without user intervention.
-nob Do not scan boot sectors.
-lis Show virus list
-del Delete infected files.
-cmp Search for viruses into compressed files.
-clv Disinfect the viruses found.
-exc: Use exclusion list
-ext: Use valid extension list
-help Show help
-heu Activate heuristic detection method.
-heu: Activate heuristic detection method with level (1-3).
-onlype Use only PE Heuristic during analysis
-nbr Does not allow interrupting the program with Ctrl-C.
-nomalw Do not detect Malware
-nojoke Do not detect Jokes
-nodial Do not detect Dialers
-nohackt Do not detect Hacking Tools
-nospyw Do not detect Spyware
-nof Do not analyze files
-nocookiesDo not detect Tracking Cookies.
-nor Do not generate result files.
-noscr Do not output to console.
-nos Deactivate sounds.
-nsub Do not scan nested subdirectories.
-path Scan the directories specified in the path environment variable.
-sig: Alternate location for signature files
-ren Rename infected files.
-rto Restore original name for renamed files
-rpt: Report file
-save Saves the parameters to a file for its use the next time it is run.
-esp Change language to SPANISH.
-eng Change language to ENGLISH.
-aex Scan all files, independently of their extension.
-info Show configuration status information.
-no2 Do not perform the second action
-loc Analyze local drives
-all Analyze all drives

Categories: Heuristics, Malware, Utils Tags: ,

AV-Test.org 2010 Test Results

January 31st, 2011 2 comments

The independent AV testing organization AV-Test.org recently released the last results of its monthly “Full Product Tests”. The Full Product Tests are a comprehensive look at anti-malware products’ ability to protect end users in real-life situations. It covers three main areas of each product: Protection, Repair and Usability. Under each area there are multiple sub-tests, such as signature detection, behavioural or dynamic detection, etc. The detailed results are available at www.av-test.org/certifications.

In order to gain certification a product has to achieve a minimum score of 12 or above. The results are very revealing, with many products not reaching the mininum score nor the certification. We are happy to announce that in the 3 quarters that AV-Test.org has conducted these tests, Panda Internet Security has achieved the certification in all cases.

On a related note, AV-Test.org recently surpassed the 50 million unique malicious sample mark. This is aligned with what our Collective Intelligence servers have analyzed and processed automatically, which is up to 146 million files (both good and bad files).

Automated False Positives

June 2nd, 2010 5 comments

I’ve covered the impact that automated detection systems have on false positives in the past. Hispasec, the makers of VirusTotal, also talked about this issue in their blog post aptly named Antivirus Rumorology. More recently Kaspersky conducted an experiment during a press conference and showed a bunch of journalists how these false positives roll over from one vendor engine to the next. Of course being journalists, they only took home the message “AV copies each other and mostly us” as is shown in the articles published covering the event . Even though the objective of the experiment was put under scrutiny, the fact remains that this is an industry-wide problem and no single vendor is immune to its effects, not even Kaspersky as we will see.

As some of the regular readers of this blog will probably remember, in March 2010 we published a “PandaCloudTestFile.exe” binary file to test the connectivity of Panda products with its cloud-scanning component, Collective Intelligence. This “PandaCloudTestFile.exe” is a completely harmless file that only tells the Panda products to query the cloud. Our cloud-scanning servers have been manually configured to detect this file as malicious with the only objective of showing the end user that the cloud-scanning component of his/her product are working correctly.

Initially this file was only detected by Panda as Trj/CI.A (a Collective Intelligence detection) and Symantec’s Insight (noting that this is not a very common file, even though treating reputation alone as “suspicious” is by itself grounds enough for debate — maybe another future post).

Panda 10.0.2.2 2010.03.10 Trj/CI.A
Symantec 20091.2.0.41 2010.03.11 Suspicious.Insight

A few days later came the first problematic detection, this time from Kaspersky, who detected the “PandaCloudTestFile.exe” with a signature, specifically calling it a Bredolab backdoor. I call this detection problematic as it is clearly not a suspicious detection nor a reputation signature. It is also clearly an incorrect detection as the file in itself is not related in any way to Bredolab. Soon we will see why this Kaspersky signature is problematic.

Kaspersky 7.0.0.125 2010.03.20 Backdoor.Win32.Bredolab.djl

In the next few days some other AV scanners started detecting it as well, in many cases with the exact same Bredolab name.

McAfee+Artemis 5930 2010.03.24 Artemis!E01A57998BC1
Fortinet 4.0.14.0 2010.03.26 W32/Bredolab.DJL!tr.bdr
TheHacker 6.5.2.0.245 2010.03.26 Backdoor/Bredolab.dmb
Antiy-AVL 2.0.3.7 2010.03.31 Backdoor/Win32.Bredolab.gen
Jiangmin 13.0.900 2010.03.31 Backdoor/Bredolab.bmr
VBA32 3.12.12.4 2010.03.31 Backdoor.Win32.Bredolab.dmb

In the month that follows (April 2010) a bunch of new engines started detecting it, mostly as the Bredolab name we are now familiar with, although some new names started appearing as well (Backdoor.generic, Monder, Trojan.Generic, etc.).

a-squared 4.5.0.50 2010.04.05 Trojan.Win32.Bredolab!IK
AhnLab-V3 2010.04.30.00 2010.04.30 Backdoor/Win32.Bredolab
AVG 9.0.0.787 2010.04.30 BackDoor.Generic12.BHAD
Ikarus T3.1.1.80.0 2010.04.05 Trojan.Win32.Bredolab
CAT-QuickHeal 10.00 2010.04.12 Backdoor.Bredolab.djl
TrendMicro 9.120.0.1004 2010.04.03 TROJ_MONDER.AET
Sunbelt 6203 2010.04.21 Trojan.Win32.Generic!BT
VBA32 3.12.12.4 2010.04.02 Backdoor.Win32.Bredolab.dmb
VirusBuster 5.0.27.0 2010.04.17 Backdoor.Bredolab.BLU

And to top it all off, during this month of May 2010 the following engines started detecting “PandaCloudTestFile.exe” as well. Here we can also even see a “suspicious” detection, probably the only one out of all of them that could make any sense.

Authentium 5.2.0.5 2010.05.15 W32/Backdoor2.GXIM
F-Prot 4.5.1.85 2010.05.15 W32/Backdoor2.GXIM
McAfee 5.400.0.1158 2010.05.05 Bredolab!j
McAfee-GW-Edition 2010.1 2010.05.05 Bredolab!j
Norman 6.04.12 2010.05.13 W32/Suspicious_Gen3.CUGF
PCTools 7.0.3.5 2010.05.14 Backdoor.Bredolab
TrendMicro-HouseCall 9.120.0.1004 2010.05.05 TROJ_MONDER.AET
ViRobot 2010.5.4.2303 2010.05.05 Backdoor.Win32.Bredolab.40960.K

It is worth noting that consumer products have other technologies included in their products, such as white-listing and digital certificate checks, which could cause the file to not be detected on the consumer endpoint, but the fact that there is a signature for such file is a good indicator that it will probably be detected on the endpoint.

So why am I writing about all this? First of all, to emphasize the point I tried to make in the past that automated systems have to be maintained, monitored, tuned and improved so that more in-depth analysis is done through them and not rely so much on “rumorology”.

Secondly, to show that this is an industry-wide problematic that results from having to deal with tens of thousands of new malware variants per day, and no vendor is immune to it. What matters at the end of the day is that the automated systems are supervised and improved constantly to avoid false positives.

I can certainly understand why vendors point to their signatures being “rolled over” to other AV engines, but these same vendors should also take care so that they do not become the source of these “false positive rumors” in the first place.
 

UPDATE June 3rd, 2010: Reading Larry’s post over at securitywatch, it seems Kaspersky has reacted quickly and has removed their signature for the PandaCloudTestFile.exe file. Thanks Larry & Kaspersky!

Blog Comment Spam Honeypot

January 25th, 2010 6 comments

One of the most common vectors for distributing malware nowadays is spamming blogs with comments pointing to malicious sites that host exploits, malware, rogue antiviruses or other types of scams.

In order to analyze the huge volume of spam comments that come in through our various Panda Blogs (PandaLabs, Panda Research, Panda Cloud Antivirus Blog, etc.) Iker from PandaLabs has developed a “blog comment spam honeypot” which is basically a modified Akismet plugin for WordPress. The honeypot basically posts everything that Akismet detects as spam into an XML which is then processed and all links are followed to detect malware, exploits, drive-by downloads, etc.

If you have a wordpress blog and would like to install the honeypot to send your trapped spam to PandaLabs for analysis, simply download and install the blog comment spam honeypot.

Thanks to Iker for all his work on spam research.

Arguments against cloud-based antivirus

December 1st, 2009 5 comments

With any advance in science and technology there will always be critics and people oppossed to change. This has happened over and over again in the course of history. Antivirus is no different. We saw resistance when we released behavioral analysis in 2004 (which is mainstream technology nowadays) and we have seen it recently with the release of Panda Cloud Antivirus.

In this post I have compiled a list of all arguments against cloud-based antivirus that I was able to find. Let us review these arguments against cloud-based antivirus and see why they are based on either misconceptions or simple lack of understanding and knowledge of how this technology works.

A malware could cripple the Internet connection and render the cloud antivirus useless
Exactly the same thing could happen to the traditional signature based antivirus. If a malware gets through the traditional signature defenses and manages to disable your Internet connection, you will not be able to get signature updates from your AV vendor and therefore will not be protected against the new malware variants, rendering your traditional AV just as useless.

A cloud-based antivirus needs to check everything against the cloud. Takes more time
Actually not everything is checked against the cloud. At least with Panda’s implementation of cloud-scanning there are locally installed technologies (heuristics, cache of cloud-detection, goodware cache, etc.) that are able to detect a good deal of malware threats and known good files. All these files are not checked against the cloud. Think about it, once you install the cloud-based antivirus, how many new programs do you install on your computer every day? Not that many, right? Once installed, only new programs copied or trying to run on your computer are checked against the cloud (if they are not detected first by the local technologies). From our beta testing phase we have seen that on average Panda Cloud Antivirus only consumes a few KB of bandwidth per day, less than the typical traditional signature updates.

It is an invasion of privacy. I do not want my files & documents to leave my computer
This is one of the most common misconceptions, maybe due to some weak implementations of cloud-scanning by some vendors. At least in Panda’s implementation of cloud-scanning when a file is “scanned by the cloud” it doesn’t actually leave your computer, it is not uploaded to our Collective Intelligence servers. What really happens is that Panda Cloud Antivirus creates a really small reverse signature of the file and the signature is what gets checked against the cloud. Also cloud-scanning is only implemented to Portable Executable (PE) files, so your Word/Excel documents, etc. are not checked against the cloud. There is one scenario with PE files where, if it is flagged as suspicious and Collective Intelligence doesn’t already have a copy of the file, then the file is uploaded for further analysis. But even then people can opt-out of participating in the community by simply un-checking this option in the product.

Cloud-based antivirus do not protect while offline
While this might be true of some cloud-based antivirus implementations, in the case of Panda Cloud Antivirus it is not true. Panda Cloud Antivirus has a local cached copy of the Collective Intelligence cloud servers. This local cache is tasked with detecting (even while not connected to the Internet) malware that is in the wild, non-PE malware and other threats. Unlike traditional signature updates, this local cache update is a “moving target” of what the community sees as circulating out there in the wild. Therefore it is able to efficiently protect against the important threats. This local cache does not protect against Win98 or DOS viruses or even malware that is dead or not circulating anymore. That is why the community aspect of Panda Cloud Antivirus is so important as, the more people use it, the better protection it offers.
UPDATE: Panda Cloud Antivirus 1.1 includes 4 additional new layers of offline protection: 2 behavioural engines (blocking & runtime analysis), autorun disabling and USB vaccination.

So that means that it provides lower protection while offline
First let’s take a look at the practical aspect: after running the beta and release of Panda Cloud Antivirus for over 7 months with millions of users, we have not had a single recorded incident of an infected user while not connected to the Internet. There’s a common misconception that protection = detection rates of millions of samples as tested by magazines. This is not really true as those tests include malware that is dead, not circulating anymore or even does not work on your operating system (like old DOS/Win98 viruses). If we define protection as stopping real-life malware that is circulating then the offline protection that is offered by Panda Cloud Antivirus is more than enough.

So if I have some old malware and disconnect from the Internet, can I infect myself?
Yes of course. You can also take a stroll down the worse neighborhood of your city sprouting a gold watch and necklaces and there’s a pretty good chance you will be (at least) mugged. Or you can just drive off a 200 meter cliff hoping your seatbelt and airbag will be enough to save your life. Panda Cloud Antivirus was designed for real people and real-life use. With that in mind you won’t have to worry about these highly unlikely scenarios during your normal computing experience.

I’m worried about latency and response time
This a very valid worry with regards to an AV whose real-time monitor (on-access scanner) works in a synchronous mode against the cloud. Currently we have two “timeouts” in the product, a first one to notify the user of problems with latency and a second one for blocking the execution altogether if no answer is received. However from our measurements these last months in over 98% of the cases the response time of the on-access scanner is below a second. Keep in mind that only a few bytes are sent back and forth when a file is queried, so the real impact is really low.

Cloud-scanning is just the latest marketing buzzword
It seems it is becoming much more a buzzword. But it doesn’t mean there is not benefit behind it. Many different products (not only security-related) are migrating their “intelligence” to the cloud and leaving behind those old, overloaded, slow applications in exchange of faster, always up-to-date clients. There is a clear benefit not only from the perspective of developers who are much less constrained by the limitations of a single PC, but also from the point of view of the user who gets an improved computing experience without all the negative aspects of resource consumption of his/her PC.

Cloud-scanning is just a way for AV vendors to lower their cost of downloading signatures
Yeah right, you should talk to our CFO about this (he stands out as the only one with grey hairs because of how expensive this whole thing has been :) ). Seriously, it would have been waaaaay cheaper to stick to the existing traditional signature download infrastructure approach than to set-up an additional multi-million infrastructure just for cloud-scanning. Not only is there the initial investment but also the continuous maintenance. And of course this does not take into consideration the additional investment in development and QA that’s also needed to develop and maintain this technology in the products.

Cloud-scanning is only good as a second opinion
This might have been true of the first cloud implementations a couple of years ago (online scanner, the first cloud-only products, etc.) but it is not true anymore. At least with Panda’s implementation, Panda Cloud Antivirus is a full replacement of a traditional AV. Panda Cloud Antivirus has the best of both worlds; it includes local protection for offline and the most effective protection while online. While some vendors are adding some cloud-scanning abilities to their existing products’ (as an additional technology in the mix of different technologies), Panda Cloud Antivirus has been developed from scratch to work in real-time in synchronous mode against the cloud. It has been proven as an effective replacement of traditional signature approach.

If you can think of any other argument against this type of technology feel free to let us know! :)

First Independent Test of Panda Internet Security 2010

June 26th, 2009 23 comments

As you may know we released our Panda 2010 products yesterday. In addition to the traditional Panda Antivirus Pro 2010, Panda Internet Security 2010 and Panda Global Protection 2010, this year we've also released a tailor-made product for netbooks and ultra portables called Panda Antivirus for Netbooks.

I just got word from Andreas Marx from AV-Test.org that they've put Panda Internet Security 2010 (PIS 2010) to the test today. Some conclusions from the test can be seen below, using Andreas' own words:

WildList Test.  We started with a detection test against all samples from the most recent WildList 05/2009 and malware from older releases. Our test set includes 3,194 confirmed malicious and widespread samples. We tested the set with the on-demand scanner and on-access guard. In both cases, Panda was able to detect and remove these viruses, worms and bots easily.

Full Collection Test. We were able to test PIS 2010 against a larger set of about 680,000 malware samples, including ad- and spyware, trojan horses and other critters. It detected 99.6% of these files, without flagging any files in our false positive / clean file test set, which is a very good result.

TruPrevent Test.  We have tested the dynamic (behaviour-based) detection with a few recently released malware samples which are not yet detected by heuristics, signatures or the "in the cloud" features and found that Panda warned in about 45% of the cases when we executed the malware sample. However, it only blocked and quarantined just a few of these tested samples. (More testing in this area needs to be performed to report statistically significant results.)

Disinfection Test. The detection and removal of an already infected PC was working properly, all active components were removed during the system repair process and just in some cases, registry keys belonging to the malware were left behind.

Rootkit Test. The detection and removal of actively running rootkits was quite impressive: all rootkits in our test were successfully identified and deleted.

As you may imagine we're very happy about the results of this test and hope other independent tests come along soon that also validate the highest level of quality provided by our most advanced ever anti-malware solutions.

For detailed testing methodology (for rootkit detection and removal, system disinfection, dynamic detection, etc.) I recommend you visit AV-Test.org Papers selection.

Other advanced testing methodologies worth reading up on can also be found at ATMSO's Document Library.

Panda Collective Intelligence and VirusTotal

February 12th, 2009 21 comments

As you know we've been using Panda Collective Intelligence from-the-cloud-scanning technologies since about two years ago, initially in our online scanners ActiveScan and also in our Panda 2009 consumer products. Thanks to Collective Intelligence we are able to use complete automation (community-driven information, threat analysis, multiple technology checks, malware/goodware determination and signature creation) to protect against the newest and most dangerous variants faster than using the traditional signature approach.

I'm happy to report that we've now integrated the Panda Collective Intelligence cloud-scanning technology into the VirusTotal service. You'll notice it by the 10.x version numbering next to the Panda engine.

To see Panda Collective Intelligence in action let's look at a new malware that started spreading a few hours ago (MD5: a0713a3639c9d4901daf774022f4bfd2). It is an Adware/Antivirus2009 rogue antivirus. Let's run it through VirusTotal and see the results as of 02.12.2009 12:35:51 (CET):

 

Check the updated VirusTotal scan result here (search for a0713a3639c9d4901daf774022f4bfd2) to see how other engines add detection progressively.

Fenomen(al) False Positives

May 19th, 2008 15 comments

One of the problems with automation of antivirus signature creation is that if a few AV vendors start detecting something as malicious, even with heuristics, "automagically" soon afterwards other AV vendors start doing the same without even checking if the file in question is in fact malicious or not, even going as far as creating specific signatures for it via automated systems.


An example of such a False Positive (FP) problem with automatic AV signature creation is the case of Fenomen Games (aka Gamecentersolution), by Legacy Interactive. Fenomen is a company that creates and distributes games. They do so via a bunch of "Game Downloaders" which basically allow users to choose and download different games on-the-fly. The problem is that these "Game Downloaders" have very similar characteristics to known "Trojan Downloaders", such as the runtime-packing and their behaviour (connecting to the Internet, downloading something, executing it and then exiting), so they naturally set off heuristic alarms like a christmas tree.

After manual analysis the only thing I found truly suspicious about it is the fact that we have over 200.000 different unique "Game Downloaders" from Fenomen Games in our Collective Intelligence database. The ones I checked are not malicious in any way nor do they do anything different than what they advertise (if you have evidence of the contrary please let me know). Fenomen seems pretty active from a partner/affiliate perspective and this could be the reason for the multitude of unique MD5's.

So let's look at detections by different AV engines. Most of the Fenomen Game Downloaders out of the 200.000 we have checked are detected by anywhere from 4 to almost 20 different AV engines:

The problem with these detections are not the "heuristic" detections but the signature detections. Normally (traditionally that is) a signature detection signifies a "100% known malicious" program. However in today's world where signatures are created automatically based on other criteria, False Positives are amplified and rolled-over to other engines freely.

Some statistics of detections per engine based on the 200.000 Fenomen Games Download samples we have (names have been omitted to protect the "innocent"):
       Scanner A               137.465 detections
       Scanner B               101.061 detections
       Scanner C                96.472 detections
       Scanner D                68.264 detections
       Scanner E                45.602 detections
       Scanner F                38.027 detections
       Scanner G                31.603 detections
       Scanner H                28.152 detections
And so on…

These include both heuristic and signature detections. All of the latter are false positives by very well known AV engines!

The other problem created by these "FPs generated by automated signature systems" is that, once considered malicious, samples of these FPs are included in regular "collection sharing packages" amongst different AV labs and, more importantly, independent research and testing organizations. These type of organizations, which rely on multi-scanners to classify their testbeds, should take good care of not falling into the same mistake. So the next time you see detection rates based on AV signatures published in a magazine or website, you should be asking yourselves "what" is truly being tested.

All in all, automation at the lab is an absolute must for any AV vendor that wants to keep up with the large volume of new incoming malware. However it is critical that these systems are well supervised, finetuned and backed by engineers who oversee the signatures generated automatically to avoid creating "fenomenal" false positive problems.

Categories: Heuristics, Malware Tags: