Solved

Webroot False Positive - Update April 25


Userlevel 7
Badge +48
GSM & SMB Customers 
 
Due to a rule error that propagated for 13 minutes yesterday morning at 11:52am MT, good applications were mistakenly categorized as malware. This has created many false positives across the affected systems and has resulted in those applications being quarantined and unable to function.
We recognize that we have not met the expectations of some customers, and are committed to resolving this complex issue as quickly as possible. Webroot is making progress on a resolution and will update you when that’s available. In the meantime:
 
  • Affected customers should not uninstall the product or delete quarantine, as this will make quarantined files unrecoverable.
  • We have rolled back the false positives. Once the fix is deployed, the agent should pick up the re-determinations and perform as normal.
  • Customers should ensure that endpoints are on and connected to the Internet to receive the resolution. Once files have been removed from quarantine, some endpoints may require rebooting.
 
Those who need to address the issue immediately manually should follow the instructions posted on Webroot Support
 
We are conducting a thorough technical review to ensure we have a complete understanding of the root cause.  A summary will be posted in the Webroot Community, and Webroot account representatives will be prepared to discuss the findings in greater detail with you.
icon

Best answer by freydrew 26 April 2017, 18:25

View original

11 replies

Userlevel 7
Badge +48
For those that have not seen this email yet from Mike Malloy, Executive VP Product & Strategy, I wanted to share this with you. We sent this out to all MSP registered admins earlier today. 
 
 


 
Yesterday morning at 11:52 am MT, some good applications were mistakenly categorized as malware. This has created many false positives across the affected systems and has resulted in those applications being quarantined and unable to function. We recognize that we have not met the expectations of our customers, and are committed to resolving this complex issue as quickly as possible.
 
 Webroot is making progress on a resolution, and our entire organization is dedicated to addressing this issue.  We will update you with latest information on our Community and Blog.  In the meantime,
  • Affected customers should not uninstall the product or delete quarantine, as this will make quarantined files unrecoverable.
  • We have corrected the false positives in our backend systems, and we are working on an automated fix to reverse the false positives on endpoints. 
  • Customers should ensure that endpoints are on and connected to the Internet to receive a resolution.  Once files have been removed from quarantine, some endpoints may require rebooting.
Those who wish to address the issue manually should follow the instructions posted on Webroot Support.   We are conducting a thorough technical review to ensure we have a complete understanding of the root cause.  Once our analysis is complete, your Webroot account representatives will discuss the findings in greater detail with you. We apologize for the pain this has caused you and your customers.  Webroot appreciates your business, and our entire team is dedicated to being your most trusted partner.  We did not live up to that in this situation, but we are taking the actions to earn your trust going forward. Mike MalloyExecutive VP Product & Strategy
Userlevel 7
Badge +35
 And another update in case anyone didn't receive it:
 


 
Dear Partner,
 
In an effort to support you in communicating with customers affected by the false positive issue, we would like to offer the attached as a resource.  We want to partner with you in rebuilding any trust that may have been damaged between you and your customers regarding the services you provide.
 
We are committed to keeping you informed as information develops.
 
Thank you for your continued patience in this matter.
 
Yours sincerely,
 
Mike Malloy
Executive VP of Product & Strategy
Userlevel 7
Badge +48
APRIL 25, 2017:  We have a final beta version of the false positive repair utility ready for immediate evaluation. We need five additional customers to participate in our test. If you would like to participate, please call our support team at one of the following numbers:
 
 
Business support phone numbers

US Support (toll free)
1-866-254-8400

Australia Support (toll free)
Australia Support (direct line)
1 800 848 307
+61 (0) 8071 1903

Ireland Support (toll free)
1 800 902 213

UK Support (toll free)
+44 (0) 808 101 7260
Userlevel 7
Badge +48
We continue to make progress on a resolution to our false positive issue.
 
We created a comprehensive repair utility, and have successfully completed QA. We are currently rolling out the utility to a group of beta customers to ensure it works for our broader customer base. We expect to complete that work soon, and then will make it available incrementally to the entire customer base to ensure a successful deployment.
 
Stay here for ongoing updates.
 
Our Support team remains available to those of you who need urgent assistance, and we thank you for working with us through this challenging issue.
Userlevel 7
Badge +48
UPDATE: April 26, 2017
 
In addition to the manual fix issued Monday, April 24, we have now issued a standalone repair utility that provides a streamlined fix for business customers. It will release and restore quarantined applications to working order on the impacted endpoints. 
 
For access to the repair utility, customers should open a support ticket, or reply to your existing support ticket related to this issue.  Please include your phone number within the support ticket.
 
Our sincerest thanks to the MSP beta customers who worked with us to test and validate this repair. We appreciate the support of our customers and thank you for your patience.
Userlevel 7
Badge +35
 Here is an update and letter for anyone who needs to communicate an update to their staff and stakeholders:
 


 
Dear Customer,
 
In an effort to support you in communicating with your stakeholders regarding the false positive issue, we would like to offer https:///webroot/attachments/webroot/ent2/1523/1/Webroot%20Customer%20Response%20Letter.docx.  
 
We are committed to keeping you informed as information develops.
 
Thank you for your continued patience in this matter.
 
Yours sincerely,
 
Mike Malloy
Executive VP of Product & Strategy
Userlevel 7
Badge +48
UPDATE 4/27/17 9:21 a.m. MDT: We have 0 calls in queue on our phone line, and are working through about 100 tickets related to the False Positive repair utility. A good portion of those are simply awaiting customer verification.
 
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
 
Our sincerest thanks to the MSP beta customers who worked with us to further test and validate this repair. We truly appreciate the support of our customers and thank you for your patience.
Userlevel 7
Badge +48
UPDATE 4/27/17 2:47 p.m. MNT: We have 0 calls in queue on our phone line, and are working through about 130 tickets related to the False Positive repair utility. A good portion of those are simply awaiting customer verification.
 
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
 
Our sincerest thanks to the MSP beta customers who worked with us to further test and validate this repair. We truly appreciate the support of our customers and thank you for your patience.
Userlevel 7
Badge +48
For those that have not seen this email yet from Mike Malloy, Executive VP Product & Strategy, I wanted to share this with you. We sent this out earlier today.
 


 
We want to remind you that we have created a repair utility to address a false positive issue that arose on Monday.  
 
On April 24 at 11:52 am MT, some good applications were mistakenly categorized by Webroot as malware. This created false positives across the affected systems and resulted in those applications being quarantined and unable to function. 
 
Our repair utility will release and restore quarantined applications to working order on the affected endpoints.  
 
To obtain the repair utility, please open a support ticket, or reply to your existing support ticket related to this issue. Please include your phone number in the ticket.  
 
We appreciate the support of our customers and partners, and thank you for your patience.
 
Yours sincerely,
 
Mike Malloy
Executive VP of Product & Strategy
Userlevel 7
Badge +48
UPDATE 4/28/17 11:44 a.m. MNT: We have 0 calls in queue on our phone line, and are working through about 80 tickets related to the False Positive repair utility. A good portion of those are simply awaiting customer verification.
 
Please note, the utility was built to address only this specific false positive issue. It will be deactivated in the future. 
 
If applications are operating normally on your systems, you do not need to implement the utility. 
 
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
 
Thank you.
Userlevel 7
Badge +48
For those that have not seen this email yet from Mike Malloy, Executive VP Product & Strategy, I wanted to share this with you. We sent this out earlier today.
 


 
 
As a reminder, the repair utility to address the false positive issue that arose on Monday, April 24, is available. The utility will release and restore quarantined applications to working order on the affected endpoints.
 
Please note, the utility was built to address only this specific false positive issue. It will be deactivated in the future.
 
If applications are operating normally on your systems, you do not need to implement the utility.
 
To obtain the repair utility, open a support ticket, or reply to your existing support ticket related to this issue. Please include your phone number in the ticket.
 
I want to thank each of our customers and partners for their patience during this time, and we are committed to earning your trust going forward. 
 
Yours sincerely,
 
Mike Malloy
Executive VP of Product & Strategy

Reply