Checking Pwned Passwords for a Password Offline

Checking Pwned Passwords for a Password Offline

Perusing ArsTechnica a couple of weeks back, my eye was caught by this article about a recent dump of emails and password circulating on hacking forums, referred to as Collection #1. Collection #1 is a collection of 773 million unique email addresses and 21 million unique passwords thought to be an amalgamation of several historical hacks but also containing around 140 million email addresses not previously seen in dumps like this. 

Security researcher Troy Hunt runs the site haveibeenpwned (HIBP) which offers three useful services: 

The Password check is of interest to many people, yet rightly makes some people (myself included) nervous: is it really smart to put a password into an online service to see if it's been used before? Hunt details how HIBP handles transmission of passwords via the k-anonymity algorithm here

Briefly, the password API calculates the SHA1 hash for the password, which can't easily be reversed to find the password. It then sends only the first 5 digits of the hash to the API, which returns a list of all the hashes which start with those 5 digits, and the number of times the password for each of those passwords appears in the various breaches. That list is then grepped to find the full hash calculated. 

Using Pwned Passwords API Manually

We can try this process manually. Let's check to see if the password "password" has ever been compromised! Using sha1-online to calculate the hash for "password", we get "5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8".

Putting the first 5 digits into the pwndpasswords API we get a response with 512 lines, each containing the last 35 digits of a hash that starts with those 5 digits, and the number of times the password that generated that hash appeared in previous breaches. Searching that list for the last 5 digits of our password hash, we see:

1D72CD07550416C216D8AD296BF5C0AE8E0:10
1E2AAA439972480CEC7F16C795BBB429372:1
1E3687A61BFCE35F69B7408158101C8E414:1
1E4C9B93F3F0682250B6CF8331B7EE68FD8:3645804
1F2B668E8AABEF1C59E9EC6F82E3F3CD786:1
20597F5AC10A2F67701B4AD1D3A09F72250:3

So, the password "password" appears 3,645,804 times in breaches indexed by HIBP. DO NOT use that password moving forward. 

If you have access to a Linux command line, user Mon over at medium.com has posted a Bash one-liner to check your passwords via the API. That post breaks down the steps, and you can play with the one-liner to see that indeed it's only sending the first 5 digits of your hashed password. 

Using Pwned Passwords API Offline

Still too paranoid to use either of the methods above? Well, if you've got a few dozen gigs of drive space and some time to download a large torrent, you can do all this offline. 

Head over to the Pwned Password page and scroll down to find links to torrents of the full collection of hashed passwords. Keep in mind it's relatively difficult to reverse hashes so it's not a significant security risk for these to be shared. Download the SHA-1 version, ordered by hash. As I'm writing this, the latest edition is around 10GB, and unzips to about 23GB. 

Going to a Linux command line you can now use a modified version of Mon's one-liner. Instead of sending the first 5 digits to the pwned password API, we instead use the Linux command look to do a binary search of the hash collection, and then process the returned result the same way as in Mon's method. 

Here's the full thing:

echo "pass> "; read -s pass_str; echo $pass_str; sha1=$(echo -n $pass_str | tr -d '\n' | sha1sum | tr [a-z] [A-Z]); result=$(look ${sha1:0:40} pwned-passwords-sha1-ordered-by-hash-v4.txt | tr '[:lower:]' '[:upper:]'); printf "Password \"${pass_str}\" appeared %d times in the database.\\n" "${result#*:}" 2>/dev/null

For a breakdown of most of what's going on here, see Mon's post linked above. The difference is we use tr to change the hash to upper case to match the file from HIBIP, then perform a look for the whole 40 digits of the hash on the file. Having the version of the hash list where it's ordered by hash allows you to do a binary search of the whole file in under a second. If you download the version where it's ordered by prevalence, you have to use grep instead and that's probably upwards of 60 seconds per password check. 

For future versions of these collections from HIBP, update the filename pwned-passwords-sha1-ordered-by-hash-v4.txt to match whatever version you downloaded and unzipped. 

Also note that I tweaked Mon's one-liner to show the password you entered on-screen so you can confirm you're really checking the right password. Obviously don't use this with someone looking over your shoulder. 

Repeating the example above for "password" returns the same result: 3,645,804 appearances. What if we get smart and do some substitutions? Let's try "P@$$w0rd" (that's a zero between w and r). Alas, even that fiendishly clever password appears 4,746 times, showing how little security is added by simple character substitution. This is why it's good to use a password manager instead which can generate long random character passwords and save you the hassle of having to remember them. 

 

 

Add new comment

Guest

  • No HTML tags allowed.
  • Web page addresses and email addresses turn into links automatically.