Solved

Technical questions about journaling and other

  • 27 December 2013
  • 24 replies
  • 3152 views

Userlevel 7
  1. How do I figure out which journaling db in WRData corresponds to which program being journaled?
  2. How do I "Allow" journaled applications that are not currently running?
  3. How do I see what applications I have manually Allowed which are not currently running? Is this data stored in the registry like the ID Shield information appear to be?
  4. When do the WRDataJoural registry keys get updated? Is this a reliable source of information yet? What happens if I delete these keys?
  5. In what situations would WSA wait for user input to clean up an infection when all the program settings are set to do everything automatically and not tell the user anything?
  6. Does removing entries from Application Protection have any enduring affect or are they immediately repopulated without prejudice?
  7. What are the pruning settings for WRLog.log, if any?
  8. What are the exact abilities and functions of the filtering extension, especially as it compares to the local HTTP inspection engines of your competitors? Please be extremely specific.
  9. What abilities if any does WSA or its infrastructure have in reevaluating a Good detirmination of a file?
icon

Best answer by Shawn 11 February 2014, 22:36

Hello Explanoit!
Let me take a stab at these very excellent questions...

1. How do I figure out which journaling db in WRData corresponds to which program being journaled?
HKLMSOFTWAREWow6432NodeWRDataJournal shows entries for journaled processes and lists the MD5. These entries line up with dbxxxxx.db files in WRDATA. once a file is determined or overridden, journaling stops.
 
2. How do I "Allow" journaled applications that are not currently running?
We will only journal a file that is unknown. If you do not want a file journaled or monitored, please create a override based on MD5 first.
 
3. How do I see what applications I have manually Allowed which are not currently running? Is this data stored in the registry like the ID Shield information appear to be?
Monitoring takes place in the background so there is no visibility into this but whitelisted or overridden files are not journaled anyways.
HKLMSOFTWAREWow6432NodeWRDataJournal shows entries for journaled processes and lists the MD5. These entries line up with dbxxxxx.db files in WRDATA and are purged every so often as long as the file has been determined.
 
4. When do the WRDatajournal registry keys get updated? Is this a reliable source of information yet? What happens if I delete these keys?
These are updated any time an unknown enters the system and monitoring begins. If you delete a key that is currently linked to a db file, you will end the monitoring and lose all previous roll back info. The agent will start journaling it again but will start from where it got lost.
 
5. In what situations would WSA wait for user input to clean up an infection when all the program settings are set to do everything automatically and not tell the user anything?
I cannot think of any...Folks???
 
6. Does removing entries from Application Protection have any enduring affect or are they immediately repopulated without prejudice?
I am not sure there is an easy way to remove entries from Application Protection.Uninstall/reinstall...???
 
7. What are the pruning settings for WRLog.log, if any?
Development does have measures in place to try to keep the size of WRData to a minimum. Excessive monitoring of files is a common cause of this, and having support determine these or creating overrides if you are comfortable doing so is best practice.
 
8. What are the exact abilities and functions of the filtering extension, especially as it compares to the local HTTP inspection engines of your competitors? Please be extremely specific.
Currently the business product is utilizing a legacy PhishCheck feature which is due for EOL very soon. The replacement will be a BrightCloud implementation which is proving to be very effective on the consumer end of things.
More specifics to come on this as development gets closer to release, STAY TUNED!
 
9. What abilities if any does WSA or its infrastructure have in reevaluating a Good determination of a file?
I have seen good determined files get blocked by ID Shield, FW, heuristics, etc...
It all has to do with how high your settings are for heuristics and what the file is attempting to do. The WSA agent compares every PE file on the system against a LARGE set of rules before and during execution, if any part of that behavior matches a rule, it will be handled appropriately.
 
Whew... Nice work Explanoit! I know you will have some followup questions which we appreciate. You just let me know if anything in the above needs further clarification. More to come on webfiltering as it is changing almost weekly...

 
Thanks again!

View original

24 replies

Userlevel 7
Bump?
Great questions explaniot. I personally am most interested in the answer for number 9 (re-evaluating a good determination).
Userlevel 7
Hi explanoit
 
As Shran said these are excellent questions and whilst I might be able to hasard a guess at some of the answers I would not be confident that they are correct...so I will not do so.
 
Suggest that you put this directly to Joe or one of the other techs that frequent the Community, such as Techtoc...they would surely be able to help us out with some answers? :p  Wouldn't they? :8
 
Regards
 
 
Baldrick
 
Userlevel 7
We can reverese files from Good to Bad as easily as we do from Unknown to Good/Bad. The easiest way to purge the WRLOG is to use the Clear protection statistics from the client. Any program that runs/executes that is U in the log will be monitored. There is no easy way to look at whats being montored by looking at the .db files in the WRdata folder.  
 
Very little information is stored in the windows registry bar the relevant entries for security centre etc.  You can manually add applications and then hit allow if you wanted to pre-emt them to use a better phrase however it shouldnt be needed.
Userlevel 7
Thank you @ 
 However, I'm awaiting a complete point by point answer to my original post.
Userlevel 5
Hello Explanoit!
Let me take a stab at these very excellent questions...

1. How do I figure out which journaling db in WRData corresponds to which program being journaled?
HKLMSOFTWAREWow6432NodeWRDataJournal shows entries for journaled processes and lists the MD5. These entries line up with dbxxxxx.db files in WRDATA. once a file is determined or overridden, journaling stops.
 
2. How do I "Allow" journaled applications that are not currently running?
We will only journal a file that is unknown. If you do not want a file journaled or monitored, please create a override based on MD5 first.
 
3. How do I see what applications I have manually Allowed which are not currently running? Is this data stored in the registry like the ID Shield information appear to be?
Monitoring takes place in the background so there is no visibility into this but whitelisted or overridden files are not journaled anyways.
HKLMSOFTWAREWow6432NodeWRDataJournal shows entries for journaled processes and lists the MD5. These entries line up with dbxxxxx.db files in WRDATA and are purged every so often as long as the file has been determined.
 
4. When do the WRDatajournal registry keys get updated? Is this a reliable source of information yet? What happens if I delete these keys?
These are updated any time an unknown enters the system and monitoring begins. If you delete a key that is currently linked to a db file, you will end the monitoring and lose all previous roll back info. The agent will start journaling it again but will start from where it got lost.
 
5. In what situations would WSA wait for user input to clean up an infection when all the program settings are set to do everything automatically and not tell the user anything?
I cannot think of any...Folks???
 
6. Does removing entries from Application Protection have any enduring affect or are they immediately repopulated without prejudice?
I am not sure there is an easy way to remove entries from Application Protection.Uninstall/reinstall...???
 
7. What are the pruning settings for WRLog.log, if any?
Development does have measures in place to try to keep the size of WRData to a minimum. Excessive monitoring of files is a common cause of this, and having support determine these or creating overrides if you are comfortable doing so is best practice.
 
8. What are the exact abilities and functions of the filtering extension, especially as it compares to the local HTTP inspection engines of your competitors? Please be extremely specific.
Currently the business product is utilizing a legacy PhishCheck feature which is due for EOL very soon. The replacement will be a BrightCloud implementation which is proving to be very effective on the consumer end of things.
More specifics to come on this as development gets closer to release, STAY TUNED!
 
9. What abilities if any does WSA or its infrastructure have in reevaluating a Good determination of a file?
I have seen good determined files get blocked by ID Shield, FW, heuristics, etc...
It all has to do with how high your settings are for heuristics and what the file is attempting to do. The WSA agent compares every PE file on the system against a LARGE set of rules before and during execution, if any part of that behavior matches a rule, it will be handled appropriately.
 
Whew... Nice work Explanoit! I know you will have some followup questions which we appreciate. You just let me know if anything in the above needs further clarification. More to come on webfiltering as it is changing almost weekly...

 
Thanks again!
Userlevel 7
Badge +55
Thanks Shawn excellent job! ;)
 
Daniel
Userlevel 7
@ wrote:
Hello Explanoit!
Let me take a stab at these very excellent questions...

.....
 
2. How do I "Allow" journaled applications that are not currently running?
We will only journal a file that is unknown. If you do not want a file journaled or monitored, please create a override based on MD5 first.
 
3. How do I see what applications I have manually Allowed which are not currently running? Is this data stored in the registry like the ID Shield information appear to be?
Monitoring takes place in the background so there is no visibility into this but whitelisted or overridden files are not journaled anyways.
HKLMSOFTWAREWow6432NodeWRDataJournal shows entries for journaled processes and lists the MD5. These entries line up with dbxxxxx.db files in WRDATA and are purged every so often as long as the file has been determined.
 
.....
 
Hi Shawn
 
Thanks for this.  Points 1. - 3. answer some questions that I have had for quite a while...and as you might guess I have dived into the Registry to take a look...which leads me me on to a couple (or three) follow on questions, if I may:
 
1. How does one "...create a override based on MD5 first".
 
2. I have a large number of HKLMSOFTWAREWow6432NodeWRDataJournal entries for apps that are no longer being journalled and some that have not been for a very long time...so just how often do these 'redundant entries get purged from the Registry, because it does not look like this is happening on my system?
 
3. I presume that I could go into the Registry and remove the entries manually (not that I would without knowing what I am doing) but if I can surely malware could specifically target this area of the Registry to find and delete any entry relating to itself so as to 'disable' the safeguard that journaling provides (hope I am not giving anyone with malevolant intentions any ideas...:().  I assume that WSA has this covered but would be interested to know.
 
Many thanks in advance.
 
Regards, Baldrick
Userlevel 7
Badge +55
Yes I have seen that also Baldrick and wondered? But I think it's needed when support asks for wsalogs and maybe not?
 
Daniel
 
 
Userlevel 7
Left over registry entries wouldnt worry me, there are thousands of entries in your registry like this. As for malware if its known bad it wouldnt be able to execute.
 
Even if it did manage to delete these registry entries (which would be very hard as the malware would have to be tailor written and/or extremely complex -due to the file locationsames not being the same on each PC) it would still be flagged as an unknown which we could look at in support. It would be akin to doing things the old fashioned way (ie using a clean system, infecting it, comparing the afters and seeing what it did).   
Userlevel 7
Hi Roy
 
Thanks for the clarification.  That is very helpful.  So possible...but then again what is not within reason...but so hard to execute as to be not worth the effort...that is reassuring.
 
Regards
 
 
 
Baldrick
Userlevel 5
Howdy all,
Good questions Baldrick, and I knew the time purged on keg keys was definitely coming back, if only I got the answer in that amount of time. :D
 
1. Overrides can be created before a file even hits an environment by logging into your Endpoint Protection Console > Overrides tab > Create. Enter the MD5, set the desired determination, then decide if you want that to be globally applied or a specific policy.
 
2. I have requested the specifics on when these keys are updated and/or purged.
Stay Tuned!

3. Deleting these keys is obviously not recommended but I was surprised that I was able to delete these keys without being blocked or prompted by WSA.
I am going to pass this information along as all of our required keys are protected by the WSA agent so there must be a reason these do not have to be.
 
 
Thanks again all, more to come!

 
Userlevel 7
Hi Shawn
 
Thanks for taking the trouble to come back on these further points.  Much appreciated.
 
Just to clarify...I presume that when you say "....logging into your Endpoint Protection Console" the ability to create overrides is only avaialble in the Business and not the Home version of WSA?
 
Regards
 
 
Baldrick
Userlevel 7
Badge +56
Yep that is correct - Business only.
Userlevel 7
Thanks...Nic, thought so but thought it best to check.  Shame as that would make a nice 'Advanced' feature for users who would like to be more proactive (if that is possible) re. their protection, i.e., they like to fiddle & tweak, etc. ;)
 
Regards
 
 
Baldrick
Userlevel 5
Howdy folks!
 
I am following up on my previously unknown and unanswered questions.
 
I was incorrect in that the journaled reg keys are purged as they are NOT. Please let us know if this becomes a problem and we'll investigate. 
 
Nothing happens if you add/remove/modify any of the entries, these are simply informational and no harm can be caused by tampering with these keys, as I know all of you do! ;)
 
I think that sums it up for now. PLEASE let me know if anyone has any further questions or concerns.
Userlevel 7
Hi Shawn
 
Many thanks for coming back to clarify further. :D
 
Whilst I appreciate that the "redundant" entries are harmless I suspect that it would most probably be useful to consider offering a tool to clear them if a user wants to...otherwise some may well tweak the Registry manually without really understanding what they are doing.
 
Just my twopennies worth.
 
Regards
 
 
 
Baldrick
Sorry that I dig out this old thread, but it answered some of my questions regarding the journaling feature.
 
Just one scenario where I don't know how WSA reacts:
*An unknown executable is run
*WSA starts journaling
*The executable does some changes to the system and then deletes itself
*(as an Business user) you see the unkown executable and create a "bad" override in the console.
 
-> How does WSA now know that it has to do a rollback?

Background: The executable does no longer exist, so WSA won't scan it and won't flag it as bad.
Userlevel 7
Infections deleting themselves is quite common its used a defensive technique to stop people from reverese engineering the malware. Most of the newer crypto samples will do this as well as many of the Zeus and Gameover malware. 
 
In terms of the console local over-rides and console over-rides take priority over the cloud determination so you have to be careful when making them. If there is something that you need whitelisting its best to contact us so we can do it for you and vice versa.
 
WSA will do as much as it can even if parts have been removed (either by another AV/user deleted/or the infection). 
 
I have posted some behaviour of one particular Crypto sample (I have removed parts for safety). You can see on the fourth line the dropper deletes itself but the file it downloaded actually does the encyption. You can see the process starting at the end where it moves the file to the recycler folder to do the encyption before it moves it back.
 
Hopes this helps.
 
Create File%appdata%  M8TOVUIFZUOSORLZ.EXE   DeviceHarddiskVolume1UsersharishattavarAppDataLocalTovuifzuosorlz.exe  
Execute%appdata%  M8XUYBPFQDJNCDJJ.EXE  DeviceHarddiskVolume3UsersExca35b4r_1AppDataLocalXuybpfqdjncdjj.exe  
Write RegistryHKCUSoftwareMicrosoftWindowsCurrentVersionRun  CryptoLocker C:UsersUserAppDataLocalTovuifzuosorlz.exe      
Delete File%temp%phv88eABC.EXE  DeviceHarddiskVolume1UsersharishattavarAppDataLocalTempPHV88Eabc.exe  
Write RegistryHKCUSoftwareMicrosoftWindowsCurrentVersionRunOnce  CryptoLocker  C:UsersUserAppDataLocalTovuifzuosorlz.exe      
Write RegistryHKLMSOFTWAREWow6432NodeMicrosoftTracingTovuifzuosorlz 
Write RegistryHKLMSOFTWAREWow6432NodeMicrosoftTracingTovuifzuosorlz_%windir% racing      
Network Event(Russian Federation) **HTTP Entries removed for safety********
HTTP Post**HTTP Entries removed for safety********
Create File%cache%HOME  DeviceHarddiskVolume1UsersharishattavarAppDataLocalMicrosoftWindowsTemporary Internet FilesN2Content.IE5BXVXDEXhome
Write RegistryHKCUSoftwareCryptoLocker_0388  VersionInfo 
Write RegistryHKCUSoftwareCryptoLocker_0388  PublicKey  
Write RegistryHKCUSoftwareCryptoLocker_0388  VersionInfo  
Create File%recycler%$I01RB4V.PDF.TMP.TMP  DeviceHarddiskVolume1$Recycle.BinS-1-5-21-3025999876-3369108721-882803121-1656  N0$I01RB4V.pdf.tmp.tmp  
Write File%recycler%  M2$I01RB4V.PDF DeviceHarddiskVolume1$Recycle.BinS-1-5-21-3025999876-3369108721-882803121-1656  M2$I01RB4V.pdf 
Thanks for your answer.
I do understand the background, but I do have some difficulties to understand how WSA reacts in this situation.
Perhaps you could explain the workflow or the process which happens on the client after an override is created.
Userlevel 7
There is not much to explain, the client see`s a file if its unknown it checks the cloud for a determination but in the enterprise enviroment it will verify the users console first. In terms of what the client does there isnt a big change really it still gets a flag to tell it if a file is good,unknown or bad.
 
You will see in the logs ->
 
Infection detected: c:usersj rolstonappdata
oamingmicrosoftstationeryetstat.exe
File blocked in realtime: c:usersj rolstonappdata
oamingmicrosoftstationeryetstat.exe
Determination flags modified: c:usersadminappdata
oamingmicrosoftstationeryetstat.exe 
 
It`s the Determination flags that is the change from Unknown [u] to bad [b] or from [u] to good (depending on the file)
I'm sorry but I still don't get it :(
My problem is the deleted file: In our example the Cryptolocker executable deleted itself and after that the determination to bad was set; so when does WSA react?
The file no longer exists so it can't be executed or scanned again.
Does WSA check every unkown/monitored process from time to time?
Userlevel 7
Without seeing the logs from said PC I cant be sure what happened. The client checks in (depending on your poll interval) to the cloud to see if a unknown file has been determined. Even if it deleted itself the client will have a record of what its done. 
Perfect, that's what I wanted to find out.
I was afraid that in that case WSA wouldn't do anything as the file no longer exists; on the other hand that would be really silly 😉

Reply