Solved

Update on Winlogin 4005 & Terminal Servers - November 22, 2016


Userlevel 7
  • Administrator
  • 239 replies
Important update on new release for customers operating Webroot Secure Anywhere on Terminal Servers in a virtualized environment
 
Issued: 22nd November 2016
 
Overview of known issue:
Webroot has been actively tracking and monitoring an issue that affects some customers operating Terminal Servers in a virtualized environment. During a fault condition, a Winlogin 4005 error occurs where an RDS user is unable to connect to the RDS server, and the event logs display a message indicating that Winlogin has stopped responding.
 
Interim advice issued by Webroot engineering teams recommended setting the WSAB Self-Protection function to ‘Minimum’ on all Server Policies as this significantly reduced the frequency of Winlogin 4005 errors in most cases.
 
Update:
Further to that advice, Webroot engineering teams developed a new, stabilizing agent release to address causes of the 4005 error and made this agent available to a select group of customers to trial from 15th November.
 
This new v9.0.13.75 agent has shown noticeable improvement towards resolution of the 4005 issues with the select group of customers. Given the positive feedback, Webroot made a decision to make this new agent build available to all Webroot customers via ‘Auto updates’ today (Tuesday 22 November). This will eliminate the need for customers to manually install this agent build in in the field and should provide global relief.
 
Webroot engineering teams continue to complete regression and environment testing on a fix. The engineering and QA teams continue to treat this as their highest priority and are predicting that a new build will be targeted for release in December following in depth analysis of the stabilizing agent.
 
Following successful roll out of the fix and a suitable review period, Webroot will remove the recommendation to operate Self-Protection at the minimum setting and restore ‘Maximum’ as the default setting.
 
Webroot Release deployment steps
  • Please log into the Webroot GSM management console and confirm the Webroot agent has been ‘Auto- updated (to v9.0.13.75) and is checking in on all machines.
Next Steps:
  • Webroot will communicate an update to this message on/by Friday 2nd
  • If you face issues with 4005 errors, please contact Webroot Support for assistance at your earliest convenience.
Webroot apologizes for the disruption and inconvenience caused and for the time taken to manage this complex fault.  We also thank you for your continued patience and understanding in this matter.

Kindest regards,
The Webroot Product Management Team
icon

Best answer by JGiffard 1 September 2017, 10:35

Working closely with our customers, Webroot has identified an issue that manifests itself in the form of existing Terminal Server sessions becoming un-responsive with users no longer being able to log in.   The affected Terminal Servers have required restarting in order for normal service to be resumed.   For customers who have experienced this, an update to WSA is available now and Webroot support will provide assistance as required.   The fix will also be rolled into the next general release of WSA which is forecast to be automatically deployed in October.  Forthcoming product bulletins will advise of the exact date.
 
To download and install the update with the fix now, you may run the following executable on any server experiencing these issues:
http://download.webroot.com/9.0.17.32/WRSASME.EXE
If you experience any further issues, please update your support ticket and the escalation team will get back to you promptly.
 
Thank you for your patience whilst we have investigated and developed the update.  It is deeply appreciated by us all at Webroot.
 
If you wish to discuss this matter further or just to talk about the direction of WSA in general, please do contact me directly.
 
 
Regards
 
 
Jonathan.giffard
Senior Product Manager
WSA Business
E: .giffard@webroot.com
 

View original

35 replies

Userlevel 7
Badge +30
We're about two months away from having a release that contains addtional work to help resolve this issue. 
 
I'm expecting a  private beta before then ; if you are interested in trying that out, please contact me directly.
 
Regards
 
Jonathan.giffard
Senior Product Manager
WSA Business
 
Userlevel 7
Badge +30
Working closely with our customers, Webroot has identified an issue that manifests itself in the form of existing Terminal Server sessions becoming un-responsive with users no longer being able to log in.   The affected Terminal Servers have required restarting in order for normal service to be resumed.   For customers who have experienced this, an update to WSA is available now and Webroot support will provide assistance as required.   The fix will also be rolled into the next general release of WSA which is forecast to be automatically deployed in October.  Forthcoming product bulletins will advise of the exact date.
 
To download and install the update with the fix now, you may run the following executable on any server experiencing these issues:
http://download.webroot.com/9.0.17.32/WRSASME.EXE
If you experience any further issues, please update your support ticket and the escalation team will get back to you promptly.
 
Thank you for your patience whilst we have investigated and developed the update.  It is deeply appreciated by us all at Webroot.
 
If you wish to discuss this matter further or just to talk about the direction of WSA in general, please do contact me directly.
 
 
Regards
 
 
Jonathan.giffard
Senior Product Manager
WSA Business
E: .giffard@webroot.com
 
Userlevel 7
Badge +30
There's another agent update, 9.0.15.43, that will be out in the next week or so.  That has additional work to address 4005 errors. 
 
Jonathan.giffard
Senior Product Manager
WSA Business
 
Userlevel 7
Badge +35
@DTMT - those fixes were implemented with 9.0.17.24, but there may be something else going on. Our team is looking at the logs and the defect and will be in touch with you shortly. 
Userlevel 1
That's good. Hopefully we can get something that works. Please email me the beta if you want me to run it on a test server I have. More than happy to help. :)
Userlevel 7
Badge +30
As part of the changes that were made to quickly deal with the BSOD issue in 9.0.15.43, additional fixes for 4005 were removed  as a precautionary measure.   We're working hard on getting these fixes into a forthcoming, quality, release .   When I have some more details to share with you all, and I know that 4005 is causing pain, I will do. 
 
If there are any people who would like to take part in testing the fixes before the official release, please contact me using message system. 
 
 
Regards,
 
Jon.giffard
Senior PM
WSA - Business
 
Userlevel 7
Badge +30
We did indeed have several updates in 9.0.17.28 for various scenarios that could result in 4005 issues. We were aware that there could well be others.
 
Our support and engineering teams are working together on your issue and will be contacting you to obtain further details. 
 
Please feel free to contact me at any point
 
 
Jonathan
 
Jonathan
 
Userlevel 7
Badge +30
We've identified a resource leak in WSA that can lead to this situation.  There is a patch build being created, with a fix for this, that you will be able to get from support by the end of this week. 
 
The fix will be rolled into the next release of WSA which is expected in the fall of this year. More news on that will be posted neared the time.
 
 
Jonathan.giffard
Senior Product Manager
WSA Business
Userlevel 7
Badge +30
winlogon.exe -  I'd avoid doing that. 
 
You're entirely right in saying that this has been going on for a long time.  No matter what we've tried in house, we can't reproduce this.  Some customers have Terminal Servers that rarely, if ever, see this issue.  It's been a pain in arse for everyone concerned, but particulary our customers. 
 
But there's some good news.  We've been given a full crash dump from a server that was experiencing a 4005 event and that's undergoing analysis at the moment.  I'm expecting that will give us direction as to what the underlying root cause is and then we'll be able to do something about it.  Given the past history, it's going to take a number of releases for us to flush out the solution. 
 
Jonathan
 
 
 
 
 
Userlevel 7
Tagging @GeorgeA who may be able to help answer your question. I believe there have been quite a few updates since Nov 22nd. 
Userlevel 7
Badge +35
We have an upcoming agent update which will address the winlogin errors on RDS terminal servers. Please see the post and product bulletins here.
 
Click to see post and bulletin
Userlevel 6
Badge +24
We previously had an RDS server with the 4005 error, which we moved to Silent Audit mode as a result of the problem.  After installing 9.0.17.32, I moved the system back to our normal server policy.  So far no issues.
 
I have since updated all client RDS servers with this version and am monitoring further.
Userlevel 1
Thanks Cat. Hopefully they will be able to shed some light on it. 
Userlevel 1
Akim responded in the other thread. https://community.webroot.com/t5/Product-Questions/Users-unable-to-login-to-terminal-server-with-Webroot-installed/m-p/283312#M4663
Userlevel 7
Badge +30
The new client does not contain fixes for 4005.  We're about two months away from having a release that contains addtional work to help resolve this issue. 
 
I'm expecting a  private beta before then ; if you are interested in trying that out, please contact me directly.
 
Regards
 
Jonathan.giffard
Senior Product Manager
WSA Business
Userlevel 7
Badge +30
Please contact our support team immediately so that we can troubleshoot your issue
 
Regards
 
jonathan 
 
Userlevel 7
Badge +30
The patch addresses a discovered resource leak which was discovered in the current release of WSA v9.17.28
 
Other areas
 
In GSM, create a policy for all of your terminal servers.  Make sure that you copy the settings in the default server policy exactly and then switch off the following:-
 
  • ID Shield
  • Web Threat Shield
  • Firewall
  • System Optimizer
  • Core System Shield
  • Scan Schedule 
 
Once you've done that, assign all of your servers to that policy.
 
This will have the effect of turning down the agent resource usage ( you still have a decent level of protection ).  If there is still an issue, we have a smaller feature area to focus on. 
 
If the everything is stable, start turning each setting back on, leaving a reasonable time gap between each to assess the impact.
 
Jonathan
 
 
 
 
 
Userlevel 7
Badge +30
Hello,
 
Working closely with our customers, Webroot has identified an issue that manifests itself in the form of existing Terminal Server sessions becoming un-responsive with users no longer being able to log in. The affected Terminal Servers have required restarting in order for normal service to be resumed. For customers who have experienced this, an update to WSA is available now and Webroot support will provide assistance as required. The fix will also be rolled into the next general release of WSA which is forecast to be automatically deployed in October. Forthcoming product bulletins will advise of the exact date.
 
Before applying the agent build we have created to address this issue, please ensure that you have applied the below Microsoft patches. These patches were designed specifically to address 4005 errors/RDS connection issues.
 
http://support.microsoft.com/kb/3172614
http://support.microsoft.com/kb/3179574
http://support.microsoft.com/kb/3197875
http://support.microsoft.com/kb/3197874
 
Before installing the following agent build please ensure that you have removed the agent currently installed and ensure that C:Program DataWRData has been removed (if not please delete this folder:
http://download.webroot.com/9.0.17.32/WRSASME.EXE
Please ensure that you reboot the server after applying the above update. If you experience any further issues, please update your support ticket and the escalation team will get back to you promptly.
 
Thank you for your patience whilst we have investigated and developed the update. It is deeply appreciated by us all at Webroot.
 
Regards
 
Webroot Global Escalation Team”
Userlevel 7
@seantheitguy, please reach out to our Support Team to look into this further for you.
 
Business Technical Support: Call 1-866-254-8400 M-F 7:00 AM – 6:00 PM MT
Open a Support Ticket

Userlevel 6
Badge +24
@akim @JGiffard We have a server with 9.0.17.28 that is experiencing the Winlogon 4005 error.  It is a critical error for this client.
 
According to the following thread:
https://community.webroot.com/t5/Product-Releases/Product-Update-Bulletin-Update-for-Winlogon-Issues/td-p/295636
 
there was supposed to be a fix in July.  Can you let me know whether that fix has gone live yet?  I am now concerned about other clients of ours that may be experiencing this error, and need to know if I have to disable Webroot on our clients' terminal servers.
 
I have opened Ticket 106309 for reporting the issue and have uploaded WSALOGS.
Userlevel 6
Badge +24

@dmaunder wrote:

@DTMT wrote:
@akim @JGiffard We have a server with 9.0.17.28 that is experiencing the Winlogon 4005 error.  It is a critical error for this client.
 

What version of Windows Server?

@dmaunder Server 2012 R2.
This patch does not work we still have servers crashing because of this.
Is there something else besides this patch that needs to be done?
Userlevel 1
Any update on this? I have been using .75 for a while now and all seems fine but I am running self protection on 'minimum' on my servers still and wonder if I can switch it back to full? Thanks.
Userlevel 1
I second that request!

Reply

    Cookie policy

    We use cookies to enhance and personalize your experience. If you accept or continue browsing you agree to our cookie policy. Learn more about our cookies.

    Accept cookies Cookie settings