Change WSA's hashing from MD5 to more secure SHA-256

  • 27 August 2013
  • 13 replies

The MD5 standard is already 22 years old and proven vulnerable over the years, not only vulnerable used in encrypted traffic, but also to verify file authenticity through a hash. It was already proven years ago that you can create 2 different files that have the same hash. Here a developer from Surfright comments that the usual objections against newer more advanced hashes(hashing speed and size) are practically non-issues. Reading the file from disk is the bottleneck, it's way slower than the hashing itself, and if you have like Webroot a large cloud database containing lots of information on each file, then the larger size of the hash makes only a very minimal difference on total database size:

Like with the other idea I just posted, I think it's good for a company selling security to use the lastest, more secure, standards, regardless of the likelyhood the older standards being exploited.

13 replies

Userlevel 7
Interestingly, we've never run into this problem. While it's true that the MD5 hashing algorithm is inferior to modern incarnations of SHA in some ways, it's not so easy to create functioning malware that collides with a good hash. On a strictly academic level, it's true that two files could be engineered to result in the same MD5 value. However, by no stretch is this "easy." In the cited example, the author has the benefit of having created both the "good" and "evil" files. In reality, malware authors do not have this benefit. While it may still be possible to create a collision, the colliding malware file actually being functional and deserving of the name "malware" would be a statistical anomaly - one so rare we've never seen it happen. If this was an issue we'd ever seen before, we would certainly be concerned about it. As it stands, this kind of collision creation is either academic in nature (not real-world) or involves a file that doesn't work to do anything malicious anyway. It might still be worth updating the hash algorithm at some point, but probably not for the reasons specified. Also of great note: MD5 is not the only hashing algorithm we use, though it's the only one you see in the logs. The back end is capable of much more. We've got ourselves covered on that one. 🙂
Userlevel 7
Thank you for posting this https:///t5/user/viewprofilepage/user-id/4366.
You're right in that if Webroot was starting from scratch today, SHA-256 would be the best choice. MD5 is no longer appropriate for use in new products, but as Jim mentioned for this use case it remains a valid algo. Previous comments from Webroot employees indicate they have other checks on file metadata in place that make even a MD5 collision not a huge threat.
I think Jim is referring to PX5 hashing, which was inherited from the PrevX code that WSA is built on. You can see them in the Threat Logs but not the Scan Logs.
Starting Routine> Removing c:usersmalbenchdesktopwebroot customer service_filesmypc backup.exe...#(PX5: B4FE1EBF28B5D0CE84791D558CEF4900E235C636 - MD5: 585C176601480391B616372CFCC6AE1E)...
Deleting File> c:usersmalbenchdesktopwebroot customer service_filesmypc backup.exe
It's basically impossible to create a collision with two different algorithms, so Webroot might be on firm ground here. The security industry is moving to public identification of files with SHA-256 instead of MD5 so it's something Webroot should review as time goes on. A standard hashing algorithm is critical for companies and researchers interacting and sharing information about badware. Hopefully SHA-256 will serve their needs for a long time.
Userlevel 7
Badge +55
@  Correct from the threat log:
Starting Routine> Removing 2EAA88F8004F36DAE0A60052C6E29C00E08E067C - MD5: 7C30DB9CFC5F663497D1709E78F702DA)...
Deleting File>
Userlevel 7
Excellent post and replies.. Thank you everyone!  When I came here, I said I learn something new every day..I still do and I love it!
Userlevel 7
Ah, you like playing with those freaking shippinglabel viruses too? 🙂
Userlevel 7
Badge +55
Aw yea I get many kinds as my ISP uses Yahoo mail and Yahoo mail is protected by N%^t*n so I save the attachment and check with VirusTotal to see how many detect it like 2/46 and a couple of times it's 0/46 and WSA picks it up then I use the MD5 to check later on VirusTotal to see the progress and I know that VT doesn't use the full product on there only the Commandline.
Thanks for the replies 🙂
Userlevel 7
You're so welcome Boerenkool,
I'm glad we have so many viewpoints available from expert users and Webroot's top staff. It's very cool. And thanks again for bringing this concern up. Many things in the product have been changed based on discussions in the forums and SHA-256 was something I've been meaning to ask about.
Userlevel 7
Even if a clash was to occur (I have never seen it myself) we have a number of other options to remove the infection from a customers PC. We can manually remove it using the client if need be and this would sidestep the issue alltogether. Looking at the above example
We could identify the real file based on path,file name/size and behaviour etc
@ Indeed, those discussions are good for creating better products 🙂
Update: "Collision attack against widely used MD5 algorithm took 10 hours, cost just 65 cents."
Userlevel 7
Admittedy, it does cause me some concern, though the article also points out that it is pretty much inable to be done without use of some serious supercomputer time as well.  Still, all in all, maybe this needs to be re-considered.
Userlevel 7
I think the main thing to understand about our product is that it is not a simple MD5 scanner.  Our system "ENZO" is capable of not only using MD5s but a huge array of techniques in monitoring your systems.  We have algorythems that are capable of detecting behaviors, MD5s, and many types of other file signatures.