Scott Johnson is a security consultant with Optiv's application security team. His focus is on penetration testing, source code review, and mobile application assessments.
Black Hat Tools Arsenal: Burp-Hash Plugin, Part 2 - How it Works
This is a follow-up post about our Burp-Hash plugin for the Burp Suite that we presented at the Black Hat USA Tools Arsenal. You can read the backstory that inspired us to create the tool in Part 1 of this post. You also can watch a quick two-minute video overview of the plugin on YouTube.
To install Burp-Hash, you can pull down the latest burp-hash.jar file from GitHub, launch Burp and import it on the Extender tab. Alternatively, you can install Burp-Hash from the BApp Store by navigating to Burp > Extender tab > BApp Store and locating Burp-Hash from there.
When we started to develop the Burp-Hash plugin, we set out to accomplish two things:
- Hash all the things! Locate as many parameters as possible and pre-compute their cryptographic hashes, sort of like miniature, targeted Rainbow Tables.
- Match the hashes! Locate any hashes passed from the server and compare them to the hashes we have pre-computed, so we can find a match.
Along the way, we decided to use SQLite to contain our observed parameters and their hashes. SQLite provides a relational database with SQL syntax for single user applications, such as mobile apps. Additionally, the SQLite database files are portable, which means a Burp-Hash user can re-import the database file and carry the table of observed parameters and corresponding pre-computed hashes across burp installations. SQLite turned out to be a great fit for our project’s goals.
For performance reasons, Burp-Hash runs within Burp’s Passive Scanner and only for URLs that are in the Burp Target Scope. Burp-Hash will not observe or hash parameters passed to or from web applications that are outside the scope. To get Burp-Hash to start scanning a particular application, just add that application’s URL to the Burp Target Scope. Burp-Hash will then scan any proxied traffic that matches the Target Scope.
Burp Suite provides a very extensible API, as demonstrated by the variety of extensions available for download in the Burp Suite BApp Store. The Burp Extension API provides programmatic ways of grabbing many key/value pair parameters such as the query string of HTTP GET requests and URL-encoded values from HTTP POST requests. However, there are limitations we knew we needed to overcome for Burp-Hash to become more powerful. Fortunately, the Burp Extension API also provides access to the raw HTTP requests and responses.
Many web services leverage JSON and XML, but data encapsulated in JSON or XML cannot be extracted by the Burp Extension API’s getParameters() method. This is for good reason: there’s no easy way to identify parameter values in object-oriented data since it can be nested in complex ways, unlike simple name/value pairs found in HTTP GET query strings. As a result, Burp-Hash does some regular expression based searching to identify the parameter data to hash. Besides parsing JSON, Burp-Hash uses regular expressions to help locate data such as email addresses, even if those values live in places like the HTTP headers. Burp-Hash also grabs complete HTTP headers since some web services include those as part of an HMAC calculation. If developers forget to key or salt their HMACs, then Burp-Hash may be able to identify the source HTTP header values.
The following is an example screenshot of the SQLite parameters table with some test data:
Once a parameter is located, Burp-Hash will check the SQLite database to see if it’s a value that was previously found. If it’s a new parameter, Burp-Hash then pre-computes hashes using the cryptographic hash algorithms that are enabled in its configuration. Out of the box, Burp-Hash supports:
However, a Burp-Hash user can choose to disable some or all of the hashes for performance reasons, or even disable the hashing of parameters, leaving Burp-Hash in “Report Only” mode, which simply notifies the user when potential hashes are observed passing to and from the server.
Burp-Hash will even pre-compute the URL-encoded form of the parameter’s value, just in case a developer happens to use that version as the source of the cryptographic hash. Over time, this becomes sort of a Rainbow Table of all observed values:
Identifying potential hashes in an HTTP request or response turns out to be somewhat tricky. The first hurdle is dealing with encodings. Most commonly on the web, hashes are Base64 or ASCII Hexadecimal encoded. Creating regular expressions to locate hexadecimal strings is not that hard, but base64 is another beast entirely. For example, the following two strings both satisfy the character sets for base64:
However, it is much more obvious to the human eye that the second string is not properly base64 encoded. Burp-Hash mostly gets past this hurdle by enforcing total string length rules that match the common cryptographic hash algorithms’ output, though it is possible for a false positive to trick Burp-Hash. It also is possible that those values could be URL encoded by the server in addition to their primary encoding, so Burp-Hash also takes that into consideration and can detect a variety of parameters and hashes with or without URL encoding.
The second hurdle is ensuring that a value previously identified as a longer hash output (e.g. SHA-512) does not also result in a false positive match for a shorter hash output (e.g. SHA-1). Since the character sets are the same, a subset of a larger hash string does appear to be a hash by itself. Burp-Hash attempts to solve this by locating longer hash strings first.
The third hurdle is identifying false positives. It is possible that Burp-Hash is only identifying a string of bytes that looks like a hash, but those bytes could have been generated by a variety of other computations that are not hashes at all. It is for this reason that if Burp-Hash only observes what appears to be a hash, the confidence is marked as “tentative” in Burp’s issues list:
If, however, the value can be matched to a specific pre-computed hash value in Burp-Hash’s SQLite database, then Burp-Hash adds a “Hash Match” to Burp’s issues list, with the plaintext source value clearly identified:
Burp-Hash also will highlight the location of the hash in the request or response where it was observed:
While it is unlikely that a penetration tester who installs the Burp-Hash to immediately find a hash match, at a minimum this extension will start to highlight the presence of hashes as they are passed back and forth among a variety of web applications. If your application happens to have the same type of vulnerability that Scott found in Part 1, then Burp-Hash will likely find it for you without the manual effort of extracting all the suspected parameters and checking them against the hash one by one.
For the Burp-Hash project overview visit http://burp-hash.github.io/.
Get the source code on GitHub and send us a pull request here: https://github.com/burp-hash/burp-hash.
Let us test your web applications for hash collisions and many other types of vulnerabilities.