BSOD 0x50: PAGE_FAULT_IN_NONPAGED_AREA

  • 1 February 2017
  • 42 replies
  • 920 views

Badge +3
The latest Webroot release 9.0.15.43 with its updated wrkrn.sys file seems to have caused some BSOD's with 2008r2 after our systems rebooted last night.  I was able to delete the file from a repair command prompt and then reboot into normal operating mode. 
I have a ticket in with Webroot support about this issue, but I thought my issue should be shared.
 
Update:  I have talked with an MSP using Webroot who has had the same problem today.  They also report that if you reboot your machine again after Webroot has a chance to reinstall the wrkrn.sys file it will start throwing up BSOD messages again.

42 replies

Userlevel 7
Badge +35
We have a new agent release:
 
Following reports of difficulties installing the latest Webroot SecureAnywhere Business (WSAB) update v9.0.15.43, a new agent release titled v9.0.15.50 has been deployed automatically to all of our WSAB customers on Thurs 2nd Feb 2017. This version provides relief to those customers experiencing installation problems.
 
Webroot apologizes for any inconvenience caused by this updated release. Our 24/7 Support team is briefed and available to customers who may have any questions or concerns about this update.
 
Webroot’s Business Technical Support - call 1-866-254-8400 or open a Support Ticket: http://mysupport.webrootanywhere.com/supportwelcome.aspx?SOURCE=ENTERPRISEWSA
Userlevel 2
If this is a fix, can we have a report of what was fixed?  That release notes is more of a notification of a new patch with no new details.
 
There will be other areas where I will raise this concern, but this is all the more reason why Webroot should give its partners the ability to choose whether or not to push a release out selectively or fully.  Auto-Update or No Update is not a great method when you are dealing with enterprise and large numbers of agents.
 
Is there a way to be notified when a new agent version is released so that we may test it?
Userlevel 7
Badge +35
The Root Cause Analysis has been completed and the Engineering team is analyzing the results and assessing the impact on future code builds. We will provide details to customers via support tickets and make available a version for posting later this week. 
 
While we put every release through rigorous testing, in this case a serious issue was discovered after release that was not seen during testing.  Going forward, we are expanding our QA coverage to address an even broader range of customer environments.  We will also improve communication to ensure customers are consistently notified of releases in advance, so they are able to control how they roll out updates in their environments.  Specific details will be announced in the coming weeks.
Userlevel 3
Badge +9
Impressive, thanks! Looking forward to the release and thank you for paying attention.to a community forum :)
 
-Nic
Userlevel 7
Our Team is currently investigating this issue.

Please reach out to Support at your earliest convenience for further instruction:
Business Technical Support: Call 1-866-254-8400
Open a Support Ticket: http://mysupport.webrootanywhere.com/supportwelcome.aspx?SOURCE=ENTERPRISEWSA
The driver that is failing to load is HKEY_LOCAL_MACHINESYSTEMCurrentControlSetEnumSWDWPDBUSENUM{dd7ac8b4-8c50-11e3-80b6-005056945bb1}#0000000000100000
Driver:
WudfRd
Service:
WUDFWpdFs
 
This is the driver that loads the pagefile and is the Windows Driver Foundation - User Mode Driver Framework.
 
Related Processes:
WUDFHpst.exe
wpdfs.dll
WudfRd.sys
WudFPf.sys
 
Since this service is failing to load, its not able to load the page file.
 
Userlevel 7
Badge +35
Webroot released a routine update on Tuesday January 31st, containing general fixes and minor feature enhancements. For most of our millions of customers, the service has run as normal. However, some customers have experienced a problem with the update, so Webroot’s 24-hour support team has been working with them directly to remedy this quickly.
 
If you are one of those customers, we sincerely apologize and ask that you contact Webroot’s Business Technical Support team by calling 1-866-254-8400 or by opening a Support Ticket: http://mysupport.webrootanywhere.com/supportwelcome.aspx?SOURCE=ENTERPRISEWSA
I have verified that the default download link, http://anywhere.webrootcloudav.com/zerol/wsasme.msi, is pulling 9.0.13.75 again. I have not witnessed anything auto-downgrading yet though. I'm hopeful.
Userlevel 7
Badge +35
Good question - here is some clarification from our product team:
 
The 9.0.15.50 release made available yesterday replaces the driver component with a known good driver used previously in v9.0.13.75 which had been operating successfully since November 2016. This means that 9.0.15.50 fixes the issue at hand by removing the code changes made in the v9.0.15.43 build, and will allow affected customers to recover with assistance from our support team.
 
The engineering teams continue to work on determining the root cause of the fault and whilst it was proving difficult to reproduce consistently a new built was determined to be the quickest method of restoring service to impacted customers.
Userlevel 7
@, our Escalations Team has reproduced the issue on their SmallBusinessServer.
A validated fix from development will be patched in the upcoming release.
Userlevel 7
Not a problem, we're always listening and willing to help in any way we can :)
 

Userlevel 7
@ wrote:
Any update on the the status of this release?
-n
This was resolved in build 9.0.17.28
 
SecureAnywhere Business Release Notes
We also experienced this issue accross MANY servers.  Issue seems to revolve around the page file being set to another drive.  We could resolve by removing the drive and booting server back up causing it to recreate it.  Temp resolution is to move page file to C drive.
Badge +3
My problematic servers were indeed not configured for the c: drive to have the pagefile. 
Relevent Eventlog ID:
219
Userlevel 5
defconoi,
 
Do you or anyone else in this thread have a kernel memory dump from the BSOD you can provide to me? They are normally located at C:WindowsMEMORY.DMP. They are written when the BSOD occus, if it isn't present you may have forcefully restarted your computer before allowing the dump to complete or it was purposfully disabled in the system configuration (sysdm.cpm > Advanced > Startup and recovery > Settings...).
 
If anyone is able to reproduce this make sure that "Complete memory dump" is set and the dump file is set to "%SystemRoot%MEMORY.DMP". Also make sure the DisablePagingExecitve registry key is set to 1.
 
Thank you,
Johnny Shaw
Software Engineer | Webroot Inc.
FYI, we have also had this problem with Windows 7 machines. Same fix, boot to recovery environment and rename the c:windowssystem32driverswrkrn.sys file.
This fixed it for a few VM desktops, however not servers with pagefiles oddly, this was the first fix I tried.
We did both, I added the Disable Page exec key in recovery mode to the system hive and checked for a memory.dmp, it was crashing and not dumping memory.  Also verified all servers have memory dump of kernel log enabled.
After crash
File System Filter 'WRkrn' (6.1, ?2016?-?11?-?18T12:49:11.000000000Z) has successfully loaded and registered with Filter Manager. 
Before crash
File System Filter 'WRkrn' (6.1, ?2016?-?09?-?14T17:14:42.000000000Z) has successfully loaded and registered with Filter Manager.
WE have 50+ servers this happened to and can submit logs or assist in the matter.  We need this resolved becaue this effected dozens of our customers.
This could be a compatibility issue between webroot & VMware tools, this seems to be only effecting VMware VM's
It occurred to HyperV VM's as well
We are also seeing this with several Windows 7 desktops and Server 2008 R2 servers (non-VM).   But in one of the server cases, we thought it might have been a failing hardware piece and attempted to boot it on a DR appliance as a VM, and that too suffered the same PAGE_FAULT error 50 result.  So then we knew it was not the hardware, but something software based going on.  And led us here.   In all cases, renaming wrkrn.sys to .old has allowed the machine to boot.  For now on the impacted machines we've uninstalled Webroot to prevent it from happening again until this is figured out.
 
Move the Page file to the C: drive and you can keep Webroot installed and reboot at will.  Uninstall Webroot before reconfiguring the page file.  

Reply