Source:
https://www.mrg-effitas.com/wp-content/uploads/2017/05/MRG-Effitas-360-Assessment-2017-Q1_wm.pdf
17 Applications Tested
386 In-the-Wild malware samples used.
Certified
(level 1):
No security product was able to defend the system 100%
during this test period.
Certified
(level 2):
avast! Internet Security, Avira Internet Security, AVG Internet Security,
Bitdefender Internet Security, ESET Smart Security,
Kaspersky Internet Security,Panda Internet Security, SurfRight HitmanPro, Symantec Norton Security, ThreatTrack Vipre Internet Security, Trend Micro Maximum Security, Webroot SecureAnywhere.
Daniel
MRG Effitas 360 Assessment & Certification Programme Q 1 2017
Userlevel 7
+56
Page 1 / 2
Userlevel 7
Nice one, Webroot...and thanks for posting, Daniel.
I noted the following in the document:
"MRG Effitas marked these samples as failed, but Kaspersky Lab disputed these samples. Kaspersky Lab agrees that these samples are malicious, but because the sample did not perform any malicious activity during the test timeframe, they argument is that these tests are not valid tests and has to be excluded from the test."
Could this be the 'great' Kaspersky Labs coming around to the Webroot way of thinking...eventually? ;)
I noted the following in the document:
"MRG Effitas marked these samples as failed, but Kaspersky Lab disputed these samples. Kaspersky Lab agrees that these samples are malicious, but because the sample did not perform any malicious activity during the test timeframe, they argument is that these tests are not valid tests and has to be excluded from the test."
Could this be the 'great' Kaspersky Labs coming around to the Webroot way of thinking...eventually? ;)
Yes, I noticed that remark by Kaspersky as well.@ wrote:
I noted the following in the document:
"MRG Effitas marked these samples as failed, but Kaspersky Lab disputed these samples. Kaspersky Lab agrees that these samples are malicious, but because the sample did not perform any malicious activity during the test timeframe, they argument is that these tests are not valid tests and has to be excluded from the test."
Could this be the 'great' Kaspersky Labs coming around to the Webroot way of thinking...eventually? ;)
It does seem somewhat strange that (an) AV testing organisation(s) should mark as fail an AV product that witnesses a malware "not perform any malicious activity" :@
But then who am I, insignificant creature as I am, to question the mysterious ways and unfathomable logic of AV testing organisations' testing processes 😞??
Why hasn't Webroot participated in Q2? Was it something to be shy about? Genuinely interested. Same question concerning the banking certification which is of REAL interest to me, and I am sure many others.
Userlevel 7
MRG & Webroot are keeping tight lipped about the reasons/causes for (a) the poor showings & (b) the lack of participation in Q2. We will have to wait for one or both of them to make a statement or the like.
No point in speculating any further.
No point in speculating any further.
Userlevel 7
+56
We will see what @ has to say on the recent problems with the Testing of WSA and MRG. But all I know there is a problem and can't say anymore then I already did.
MRG Effitas 360 Assessment & Certification Programme Q2 2017: https://www.wilderssecurity.com/threads/webroot-secureanywhere-discussion-update-thread.364655/page-134#post-2702185
MRG Effitas Online Banking Certification Q2 2017: https://www.wilderssecurity.com/threads/mrg-effitas-online-banking-certification-q2-2017.396839/
Thanks,
Daniel
MRG Effitas 360 Assessment & Certification Programme Q2 2017: https://www.wilderssecurity.com/threads/webroot-secureanywhere-discussion-update-thread.364655/page-134#post-2702185
MRG Effitas Online Banking Certification Q2 2017: https://www.wilderssecurity.com/threads/mrg-effitas-online-banking-certification-q2-2017.396839/
Thanks,
Daniel
Thanks for that TH.
I think one of the most interesting points emerging from these various comments not just here but elsewhere is that WRSA may well have detected an issue with Ccleaner already back in June, which if proven, will rightly create another tsunami of concern across the world of IT security about how lax Piriform were, or how skilled the hackers were. I hope Webroot can become transparent about this particular incident and share what happened and why it was not more widely reported, even if that highlights some potential internal failures, let's be honest and learn and improve.
I think one of the most interesting points emerging from these various comments not just here but elsewhere is that WRSA may well have detected an issue with Ccleaner already back in June, which if proven, will rightly create another tsunami of concern across the world of IT security about how lax Piriform were, or how skilled the hackers were. I hope Webroot can become transparent about this particular incident and share what happened and why it was not more widely reported, even if that highlights some potential internal failures, let's be honest and learn and improve.
Well, I missed that one! Can you illuminate??@ wrote:
Thanks for that TH.
I think one of the most interesting points emerging from these various comments not just here but elsewhere is that WRSA may well have detected an issue with Ccleaner already back in June...
Here is the specific blog post:
http://blog.talosintelligence.com/2017/09/avast-distributes-malware.html?showComment=1506026923086#c2724456599402286610
It was referenced in a discussion on Wilders which TH was part of:
https://www.wilderssecurity.com/threads/webroot-secureanywhere-discussion-update-thread.364655/page-135
http://blog.talosintelligence.com/2017/09/avast-distributes-malware.html?showComment=1506026923086#c2724456599402286610
It was referenced in a discussion on Wilders which TH was part of:
https://www.wilderssecurity.com/threads/webroot-secureanywhere-discussion-update-thread.364655/page-135
Wow! Thanks, @ . I hadn't yet caught up on the latest Wilders blog posts.
I guess we'll all be waiting with bated breath now to discover whether this CCleaner malware Webroot had detected way back in June was an FP or the infamous recently announced Trojan.
I guess we'll all be waiting with bated breath now to discover whether this CCleaner malware Webroot had detected way back in June was an FP or the infamous recently announced Trojan.
Userlevel 7
+56
Only Webroot would know so we will see if@ wrote:
Here is the specific blog post:
http://blog.talosintelligence.com/2017/09/avast-distributes-malware.html?showComment=1506026923086#c2724456599402286610
It was referenced in a discussion on Wilders which TH was part of:
https://www.wilderssecurity.com/threads/webroot-secureanywhere-discussion-update-thread.364655/page-135
Paul ComtoisSeptember 21, 2017 at 4:48 PMI manage the Antivirus systems for my Employer's business here at Triella in Canada and I have evidence from Webroot that this started much earlier than August. We have a client record of a blocked CCLeaner.exe detection on June 25th flagged as W32.Hacktool.RpdpatchWe were lucky that I did not whitelist the threat as safe becasue at the time Webroot had a problem with mis-categorizing legitimate software as malware. This threat was found on a server and since we regularly used CCleaner on desktops but not servers, I was suspicious of it and contacted Webroot support about it as well. I am going to be posting an article on our website about this shortly. It doesn't jive, with this version that was released in August: "it appears that the affected version (5.33) was released on August 15, 2017."
Daniel
...unless "the attacker was running limited "trial runs" of the malware to test it out and to determine the effectives of his backdoor code against existing detection by security products." https://www.wilderssecurity.com/threads/security-notification-for-ccleaner-v5-33-6162-and-ccleaner-cloud-v1-07-3191-for-32-bit-windows-users.396778/page-8#post-2708173
Userlevel 7
+35
That appears to be unrelated to the recent issue with CCleaner. Based on the information provided - only a filename and detection name - my guess would be that ccleaner.exe was simply the filename of some malware used by an attacker as part of an RDP-based attack.@ wrote:
Only Webroot would know so we will see if@ wrote:
Here is the specific blog post:
http://blog.talosintelligence.com/2017/09/avast-distributes-malware.html?showComment=1506026923086#c2724456599402286610
It was referenced in a discussion on Wilders which TH was part of:
https://www.wilderssecurity.com/threads/webroot-secureanywhere-discussion-update-thread.364655/page-135
@ can say anything to the fact.
Paul ComtoisSeptember 21, 2017 at 4:48 PMI manage the Antivirus systems for my Employer's business here at Triella in Canada and I have evidence from Webroot that this started much earlier than August. We have a client record of a blocked CCLeaner.exe detection on June 25th flagged as W32.Hacktool.RpdpatchWe were lucky that I did not whitelist the threat as safe becasue at the time Webroot had a problem with mis-categorizing legitimate software as malware. This threat was found on a server and since we regularly used CCleaner on desktops but not servers, I was suspicious of it and contacted Webroot support about it as well. I am going to be posting an article on our website about this shortly. It doesn't jive, with this version that was released in August: "it appears that the affected version (5.33) was released on August 15, 2017."
Daniel
-Dan
Userlevel 7
+56
Thanks Dan it's much appreciated! 😉@ wrote:
That appears to be unrelated to the recent issue with CCleaner. Based on the information provided - only a filename and detection name - my guess would be that ccleaner.exe was simply the filename of some malware used by an attacker as part of an RDP-based attack.@ wrote:
Only Webroot would know so we will see if@ wrote:
Here is the specific blog post:
http://blog.talosintelligence.com/2017/09/avast-distributes-malware.html?showComment=1506026923086#c2724456599402286610
It was referenced in a discussion on Wilders which TH was part of:
https://www.wilderssecurity.com/threads/webroot-secureanywhere-discussion-update-thread.364655/page-135
@ can say anything to the fact.
Paul ComtoisSeptember 21, 2017 at 4:48 PMI manage the Antivirus systems for my Employer's business here at Triella in Canada and I have evidence from Webroot that this started much earlier than August. We have a client record of a blocked CCLeaner.exe detection on June 25th flagged as W32.Hacktool.RpdpatchWe were lucky that I did not whitelist the threat as safe becasue at the time Webroot had a problem with mis-categorizing legitimate software as malware. This threat was found on a server and since we regularly used CCleaner on desktops but not servers, I was suspicious of it and contacted Webroot support about it as well. I am going to be posting an article on our website about this shortly. It doesn't jive, with this version that was released in August: "it appears that the affected version (5.33) was released on August 15, 2017."
Daniel
-Dan
Userlevel 7
+56
@ wrote:
We will see what@ has to say on the recent problems with the Testing of WSA and MRG. But all I know there is a problem and can't say anymore then I already did.
MRG Effitas 360 Assessment & Certification Programme Q2 2017: https://www.wilderssecurity.com/threads/webroot-secureanywhere-discussion-update-thread.364655/page-134#post-2702185
MRG Effitas Online Banking Certification Q2 2017: https://www.wilderssecurity.com/threads/mrg-effitas-online-banking-certification-q2-2017.396839/
Thanks,
Daniel
Okay I got some info from Grayson!
Hi Daniel,
The issue we’ve encountered revolves around WSA and VM snapshots where we’re seeing inconsistent communication between a recently reverted image and WSA connecting to our cloud.
We have provided MRG with instructions to do a fresh install of WSA prior to beginning a round of testing which should solve the testing performance issues while we investigate the agent state issue.
Ultimately, we’ve pulled WSA from the latest MRG report as based on sample data provided by MRG, it was clear our score was not accurate to our true detection.
It is important to note that this issue is specific to situations where a VM image is reverted to over and over again which is not representative of WSA users, but is more common in testing environments.
Either way, we are investigating and fully intend to resolve this issue and resume in normal MRG participation as we have been for the past several years.
Kind regards,
__________________________________________
Grayson Milbourne | Security Intelligence Director
WEBROOT
Userlevel 5
+24
I'd like to echo the concerns with regard to overall efficacy.
Background: I wasn't able to find all the MRG-Effitas 360 Assessments with some cursory googling, but the ones I was able to find (2017 Q1 & Q2, 2016 Q2 & Q3, 2014 Q3) seem to show Webroot trailing overall. I have real concerns about continuing with Webroot, when there are products out there that consistently perform better, have more features (true firewall, encryption offerrings, etc.), integrate into our other management platforms, and are priced about the same. I have reached out directly to our local reps, and have had a couple of lengthy calls with Webroot Engineers to address this concern. I leave out the gory details, but when pressed about the lack of participation in the tests, they expressed that Webroot doesn't participate in those tests because they do not feel that they adequately test or reflect an endpoint's true efficacy. To that I would say, what do you have to lose? Participate. Give us more data to show how well the product works. Worst case scenario is that you find something you could improve upon to make the product better. Reach out to NSS Labs if you don't like AV-Test, etc.
Perception: The perception, at least for me, is that Webroot knows it may not fair well, intentionally avoids them for fear of a poor rating while calling them poor indicators of true performance. It says to me that there isn't much confidence in the produce.
The Bottom Line: Whether or not Webroot belives the testing methodology to be a true test of the product, this is the research (AV-Test, MRG, NSS Labs, Gartner, Forrester, SANS, etc.) that we use to help make a decision on endpoint. Yes, there are other factors (cost, ease of deployment & management, integration points, independent testing of the products, etc.), but those tests help us to narrow the field. By choosing not to participate, you may be eliminating yourself from consideration (or renewal). I did not choose Webroot - it was here when I arrived. I have made a very hearty attempt to embrace it, but I lack confidence in the overall efficacy of the product.
Background: I wasn't able to find all the MRG-Effitas 360 Assessments with some cursory googling, but the ones I was able to find (2017 Q1 & Q2, 2016 Q2 & Q3, 2014 Q3) seem to show Webroot trailing overall. I have real concerns about continuing with Webroot, when there are products out there that consistently perform better, have more features (true firewall, encryption offerrings, etc.), integrate into our other management platforms, and are priced about the same. I have reached out directly to our local reps, and have had a couple of lengthy calls with Webroot Engineers to address this concern. I leave out the gory details, but when pressed about the lack of participation in the tests, they expressed that Webroot doesn't participate in those tests because they do not feel that they adequately test or reflect an endpoint's true efficacy. To that I would say, what do you have to lose? Participate. Give us more data to show how well the product works. Worst case scenario is that you find something you could improve upon to make the product better. Reach out to NSS Labs if you don't like AV-Test, etc.
Perception: The perception, at least for me, is that Webroot knows it may not fair well, intentionally avoids them for fear of a poor rating while calling them poor indicators of true performance. It says to me that there isn't much confidence in the produce.
The Bottom Line: Whether or not Webroot belives the testing methodology to be a true test of the product, this is the research (AV-Test, MRG, NSS Labs, Gartner, Forrester, SANS, etc.) that we use to help make a decision on endpoint. Yes, there are other factors (cost, ease of deployment & management, integration points, independent testing of the products, etc.), but those tests help us to narrow the field. By choosing not to participate, you may be eliminating yourself from consideration (or renewal). I did not choose Webroot - it was here when I arrived. I have made a very hearty attempt to embrace it, but I lack confidence in the overall efficacy of the product.
In a word: no.
Neil Rubenking, official AV tester and reviewer for PC Mag puts it quite well: "(Webroot's) detection style doesn't fit very well with standard antivirus tests, especially those just using static samples." Go see this comment in its context in the section Absent Lab Results of his October review: https://www.pcmag.com/article2/0,2817,2470312,00.asp. Better still, read the whole review.
I have been using Prevx since Oct 2006 and, since Dec 2011, Webroot, which acquired Prevx and rebuilt its entire architecture and technology on that of Prevx. My experience? Before I moved to Prevx/Webroot, I used better known, indeed household, AV names (all of which score better with the traditional AV test organisations), and I have to say with each of them I was all too regularly infected. Since my changeover: I have not been infected, and I am actually still waiting for the first infection to happen! That's 11 years and counting...
My suggestion? Don't trust the testers too much. The best test is how well you are protected on your devices. Also, learn how Webroot works and then learn also how the main AV testing organisations work. You will find this illuminating.
Here, by the way, is an excellent explanation regarding AV tests and Webroot: https://www.wilderssecurity.com/threads/wsa-poor-detection-result.353478/#post-2282246. It may be old, but the remarks are still highly pertinent.
If despite all this you don't feel comfortable with Webroot, then I would agree that the best thing for you to do is to change your AV. Whatever you decide to do, I wish you the best in your search for the best solution for the protection of your devices.
Userlevel 5
+24
Thanks for the thorough response. I have read the articles, etc. that you posted. I have sat through Webroot's technical and sales training, and have a good grasp on how it works. Webroot is incredibly easy to deploy, configure & manage especially when tied to other management tools.
I'm not saying that our decision making process is solely based on any test, but it is something we do look at. Insofar as staying with or moving away from Webroot, it wouldn't be a snap-decision, rather evaluation of endpoint protection is a lengthy process of testing & assessment to generate our own findings, and is multi-faceted (integrations, support, billing, forums, marketing, etc.)
I think if I could sum up my message, it would be: Consumers need solid, independent research & data that we can point to in order to demonstrate that Webroot is a superior product. This is especially true in the MSP space where current and potential clients want hard evidence for product recommendations, etc.
Thanks also, @ . And my apologies that I hadn't picked up, until after I had posted, that you are an MSP manager.
I don't at all like resorting to publicity created by a company to evaluate that company and for very obvious reasons but...have you seen these customer testimonial videos? More important of course, what is your and your clients' realtime experience of the protection that Webroot provides (post-2012 as there are, in fact, two Webroots which are very different beasts: Webroot before its acquisition of Prevx technology, and Webroot since that date) ?? And how does Webroot compare to any previous security solutions you may or may not have used? I, and I expect all of us on this Forum, would be very interested in hearing any responses you can give to those questions (regardless of whether they're positive or negative).
I don't at all like resorting to publicity created by a company to evaluate that company and for very obvious reasons but...have you seen these customer testimonial videos? More important of course, what is your and your clients' realtime experience of the protection that Webroot provides (post-2012 as there are, in fact, two Webroots which are very different beasts: Webroot before its acquisition of Prevx technology, and Webroot since that date) ?? And how does Webroot compare to any previous security solutions you may or may not have used? I, and I expect all of us on this Forum, would be very interested in hearing any responses you can give to those questions (regardless of whether they're positive or negative).
A
in the latest test i can find on there site think its q3?
has level 2 as well
but in one test i find a bit of a downer
summary here
full spectrum test
76.9% got autoblocked
6 failed
16 behavior blocked
seems a bad score to me
financial malware banking test ( maybe not word for word title)
77.5% autoblocked
1 failed
in ransomeware got a good 94%
has level 2 as well
but in one test i find a bit of a downer
summary here
full spectrum test
76.9% got autoblocked
6 failed
16 behavior blocked
seems a bad score to me
financial malware banking test ( maybe not word for word title)
77.5% autoblocked
1 failed
in ransomeware got a good 94%
Userlevel 7
Hi 9wood
Welcome to the Community Forums.
As I have not been through the specific tests that you mention (but will try to look them up later) I cannot comment with any certitude, but suffice to say that many of the 3rd party independent tests are 'flawed' when it comes to WRSA as they do not cater within the timeframe of the testing for the unique way that the application works.
They look at/consider responses to active & non active malware, when we know that WRSA focuses on the former whilst ignoring the latter until it becomes active, etc. And if the determination of whether a file is good or bad cannot be ascertained it is then placed under restriction/monitored & its actioned logged so that if eventually determine as being bad, those actions can be reversed (which most probably accounts as to why the ransomware side of the tests impart a better score).
As I said previously, I am not saying that the testing is 'bad' just that it is most likely not pertinent in terms of handling & fairly judging how WRSA operates/works.
And in all of the years I have used WRSA it has intervened on the right occasion and kept my system safe...which is the most important test for me.
Hope that helps in some way?
Regards,Baldrick
Welcome to the Community Forums.
As I have not been through the specific tests that you mention (but will try to look them up later) I cannot comment with any certitude, but suffice to say that many of the 3rd party independent tests are 'flawed' when it comes to WRSA as they do not cater within the timeframe of the testing for the unique way that the application works.
They look at/consider responses to active & non active malware, when we know that WRSA focuses on the former whilst ignoring the latter until it becomes active, etc. And if the determination of whether a file is good or bad cannot be ascertained it is then placed under restriction/monitored & its actioned logged so that if eventually determine as being bad, those actions can be reversed (which most probably accounts as to why the ransomware side of the tests impart a better score).
As I said previously, I am not saying that the testing is 'bad' just that it is most likely not pertinent in terms of handling & fairly judging how WRSA operates/works.
And in all of the years I have used WRSA it has intervened on the right occasion and kept my system safe...which is the most important test for me.
Hope that helps in some way?
Regards,Baldrick
A
so you are blasting or discrediting a test you haven't seen yet as some " 3rd party" mistake or flawed?
perhaps the test results dont fit your opinion but they are what they are like or dislike
perhaps the test results dont fit your opinion but they are what they are like or dislike
Userlevel 7
+63
A
despite this onesided excuses being made
fact is the test are test and seems that its not being taken serious becuase it dfoesnt fit your agenda in my opinion
i guess the people who had there data stolen ( experian) and yahoo hacks ( which they hid) all had same idea oh no not me never i'm safe , sure
guess if you had a really bad ( and i can name a few) antivirus with many false postives and didnt catch much if any thing your 100% protected as well? or just in your mind?
begs the question are you really safe?
maybe when you get some incident your tune may change but as long as you think your safe its the greatest thing ever right?
hiding behinf webroot detects things different ( which seems to be a them on here ) dfoesnt mean its better than anyone else nor escape the fact a virus, malware or whatever is still that no matter how it gets detected if it does or not
fact is the test are test and seems that its not being taken serious becuase it dfoesnt fit your agenda in my opinion
i guess the people who had there data stolen ( experian) and yahoo hacks ( which they hid) all had same idea oh no not me never i'm safe , sure
guess if you had a really bad ( and i can name a few) antivirus with many false postives and didnt catch much if any thing your 100% protected as well? or just in your mind?
begs the question are you really safe?
maybe when you get some incident your tune may change but as long as you think your safe its the greatest thing ever right?
hiding behinf webroot detects things different ( which seems to be a them on here ) dfoesnt mean its better than anyone else nor escape the fact a virus, malware or whatever is still that no matter how it gets detected if it does or not
A
Page 1 / 2
Reply
Login to the community
No account yet? Create an account
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.