The future of independent antimalware tests

msft-mmpc 1 Aug 2014 6:00 PM  Our guiding vision at the Microsoft Malware Protection Center (MMPC) is to keep every customer safe from malware. Our research team and machine learning systems, as well as industry engagement teams, function around the clock in an effort to achieve this vision.
As part of these efforts, we are also working with independent antimalware testing organizations towards advancing the relevance of independent testing and reporting. Our goal is to help enable independent antimalware testing organizations to test using malware that has significant customer impact. We have come a long way together, and we can still make significant advances to on-demand file-detection tests.
Current on-demand file-detection tests have some limits. They are typically carried out by first assembling a set of malware samples, and then scanning them with antimalware products. The samples in the testing set that aren’t detected by the products are counted, and then their percentage is calculated. Finally, the undetected percentage is compared to other products to calculate the comparative test results. Some testers use prevalence data to choose their sample set, and some apply curves to the results, but ultimately the fundamental test scheme is the same across the board.
One major issue with the above methodology is that there is no differentiation between samples in the test set. While each sample in the test set has a different impact on customers, in the above methodology, they are weighted equally. This methodology has been of concern to us, as it doesn’t take into account the prevalence-based customer impact.
 
Very interesting Article from Microsoft TechNet Blogs: Full Article
 
And we can understand as to why WSA doesn't get tested because WSA works and protects it's users very much differently than any other solution out there!!
 
Daniel 😉