hashpipe

documentation

hashpipe logo

hashpipe

Multi-threaded hash verification tool. Reads lines containing hash:password pairs (optionally with TYPE hints and salts), verifies them by computing the hash from the password, and outputs verified results in mdxfind stdout format. Unresolved lines go to stderr.

Uses yarn.c for threading and OpenSSL for hash computation.

Usage

hashpipe [-t N] [-i N] [-q N] [-m S] [-L secs] [-J N] [-o|-O outfile] [-e|-E errfile] [-s statfile]
         [-b spec] [-B] [-T] [-G] [-N] [-D] [-Y] [-V] [-h] [file ...]
hashpipe -X 'expr' | -F progfile [-p pass] [-s salt] [-S salt2] [-P pepper] [-u user]

Options

-t N — Thread count (default: number of CPUs)

-i N — Max iteration count for hard pass (default: 128)

-q N — Iteration step size (reserved, default: 128)

-m S — Only try types in S; eN for internal index, bare number for hashcat mode (e.g., -m e1,1000); add auto to fall back to auto-detect. Selection is strict by design: -m is a filter, not a hint — a type that is not listed is never tried, and there is no fallback unless auto is present. If the correct type is not in the list, the line is reported as unresolved on stderr exactly like a failed verify, with nothing to distinguish the right type was not selected from the password does not match. This is the same shape of silent negative that -L produces: append auto whenever the type is uncertain, and whenever measuring coverage over a bundled type list, where an omitted type under-reports silently rather than erroring.

-c — Verify each line against its own leading TYPE label only. No detection and no fallback: a pair that does not verify as its label is written to -E. A label naming an unknown type is fatal (INVALID LABEL on stderr, exit 2). The xNN suffix sets the iteration depth, so -i does not bound it. See Label-Only Verification.

-o F — Append verified results to file (default: stdout)

-O F — Write verified results to file, truncating if it exists (default: stdout)

-e F — Append unresolved lines to file (default: stderr)

-E F — Write unresolved lines to file, truncating if it exists (default: stderr)

-s F — Append statistics to file. Writes three tables at exit: hot list hits (per algorithm/salt-length pair), per-algorithm try counts, and per-algorithm solution counts. Stats are collected unconditionally; this option controls whether they are written out.

-L secs — Maximum estimated time, in seconds, for a single iterated verify (default: 1000). When an input hash specifies an iteration count that would exceed this limit — estimated from the benchmark rates — the verify is skipped. This prevents a misidentified hash with an extreme iteration count from stalling the run. Note that a skipped verify is reported as unresolved on stderr with no diagnostic distinguishing it from a genuine non-match, so a run that finds nothing may simply have declined to try; raise the limit when working with deliberately expensive types.

-b S — Benchmark selected types (e.g., -b e1-e10,e15)

-B — Benchmark all registered types

-T — Run self-tests on all registered types

-G — As -T, but also emit the generated example vectors

-N — Print the type table as TSV: index, name, hashcat modes, flags, self-test vector. One row per type, meant to be consumed rather than read — gen-hash-types.sh turns it into HASH_TYPES.md. The vector is emitted exactly as -T verifies it, so a passing self-test proves every row is correct for that build.

-D — Disassemble the compiled hx program and exit

-Y — Load userdef.txt, print the load report, and exit. Without -Y the same file loads silently. Requires MDXFIND_CACHE to be set

-V — Print version and exit

-h — Print help and list all supported hash types

-X 'expr' — Evaluate an hx (hash expression language) expression. Reads passwords from stdin and outputs the computed hash for each. See hx Language below.

-F file — Read the hx program from a file instead of the command line. This is not a hash-file option

-p pass — Single password (used with -X instead of reading stdin)

-s salt — Salt value (used with -X)

-S salt2 — Secondary salt (used with -X)

-P pepper — Pepper value (used with -X)

-u user — User/userid value (used with -X)

Input Format

Each input line has the form:

[TYPE[xNN] ]hash[:salt]:password

Output

Examples

Verify a potfile from stdin:

cat potfile.txt | hashpipe

Verify with type hints, saving results and rejects to separate files:

hashpipe -o verified.txt -e unresolved.txt potfile.txt

Only try MD5 and SHA1 (no auto-detect):

hashpipe -m e1,e8 potfile.txt

Prefer MD5SALT and SHA1PASSSALT, then fall back to auto-detect:

hashpipe -m e31,e405,auto potfile.txt

Restrict to a range of types:

hashpipe -m e1-e12 potfile.txt

Use 8 threads:

hashpipe -t 8 potfile.txt

Pipe verified results directly into mdsplit:

hashpipe potfile.txt | mdsplit potfile.txt

Verify multiple files:

hashpipe file1.txt file2.txt file3.txt

Benchmark all types:

hashpipe -B

Benchmark specific types:

hashpipe -b e1,e8-e12

Limit expensive verify operations to 10 seconds estimated time:

hashpipe -L 10 potfile.txt

Write statistics to a file:

hashpipe -o verified.txt -E unresolved.txt -s stats.txt potfile.txt
cat stats.txt

--- Hot List ---
Type   Algorithm                       SaltL      HotHits
e31    MD5SALT                             3      8873141
e31    MD5SALT                             4        11305
e31    MD5SALT                             2         9577
e31    MD5SALT                             5        17027
e31    MD5SALT                             6       349991
e31    MD5SALT                            30       659261
e31    MD5SALT                             1           65
e31    MD5SALT                            12           12
e31    MD5SALT                             8         2240
e31    MD5SALT                            17          594
e31    MD5SALT                            10            8
e31    MD5SALT                            11            2
e31    MD5SALT                            18            1
e31    MD5SALT                            19            2
e31    MD5SALT                            13            2
e31    MD5SALT                            21            2
e31    MD5SALT                             9            2
e31    MD5SALT                             7            6
       TOTAL                                      9923255

--- Algorithm Tries ---
Type   Algorithm                             Tries
e1     MD5                                      15
e31    MD5SALT                             9923397
       TOTAL                               9928112

--- Solutions ---
Type   Algorithm                            Solved      HotHits
e1     MD5                                       2            0
e31    MD5SALT                             9923327      9923255
       TOTAL                               9923329      9923255

The Hot List section shows which (algorithm, salt_length) pairs were resolved via the hot list fast path. Algorithm Tries shows total compute/verify calls per type. Solutions shows how many hashes each type solved, and how many of those were hot list hits.

Label-Only Verification (-c)

hashpipe normally identifies the algorithm itself, trying every registered type whose digest length matches. That search is the expensive part: verifying one pair is a single hash, while identifying it may be hundreds of them, and for an ambiguous digest width the candidates include costly KDFs.

When the type is already known the search is pure waste — and after an mdxfind run it always is, because mdxfind writes the type on the line:

TYPE[xNN] hash[:salt]:password

-c takes that label as the contract. Detection is not run, there is no fallback, and a pair that does not verify as the type its label names is written to -E as unresolved.

Why it exists

Measured on the full generated regression corpus — 7,234,771 lines across 1,020 types, one machine, both resolving every line:

mode wall clock
auto-detect more than 1 hour
-c 98.7 seconds

Per type the gap is wider still: an iterated salted type measured 2.41s identifying against 0.01s when the type was known, roughly 240:1. The saving scales with how many candidates a digest shape admits and how expensive the wrong ones are.

This matters for validation-only work — checking a large batch of solutions that are already paired and already typed, where the question is "are these right?" rather than "what are these?".

Strictness is the point

Where -m is a filter applied to the whole run, -c takes the type per line, so a mixed file of many types costs one verify per line rather than a search. And where an omitted type under -m produces a silent negative, -c is deliberately loud:

Where the recall matters, run -c first and re-run only its rejects without it. The second pass sees a few hundred lines instead of millions, so it costs almost nothing and recovers mislabelled-but-valid solutions:

hashpipe -c -O verified.txt -E maybe.txt < solutions.txt
hashpipe    -O verified2.txt -E rejected.txt < maybe.txt

Pot(files) considered harmful

For many years, I have taken the position that potfiles, the place where solved hashes go to die, are not only bad, but also dangerous. They can confuse what hash types are involved, or what the algorithm is, and can introduce bad solutions into an otherwise perfect hash processing stream. Hashpipe is intended to be an automated method to resolve, or verify, questionable hashes. By trying many different hashing methods, in a fully automated fashion, hashpipe can figure out what types of hashes are involved, properly format the output, and can pipe directly into mdsplit for long term hash management. Much more to the point, it also separates out the unsolved hashes, corrupted entries, or other bad data.

Architecture

Threading Model

hashpipe uses a producer-consumer architecture with yarn.c (Mark Adler's thread pool abstraction over pthreads):

Adaptive Batch Sizing

Fixed batch sizes cause slow hash types (bcrypt at ~5 hashes/sec) to bottleneck on a single thread. hashpipe uses adaptive batch sizing to distribute work evenly:

This achieves near-linear scaling for slow types: 1000 bcrypt cost-12 hashes run in ~13 seconds on 16 threads vs ~170 seconds single-threaded.

Hot Type Optimization

hashpipe tracks the most recently matched hash type as a "hot type" and a hot list of recent (type, salt_length) matches. When processing a batch, workers try the hot list first — using the known salt length to extract the salt directly without colon parsing — before falling back to the full candidate scan. For homogeneous input (common in potfiles), this avoids testing hundreds of types per line and eliminates ambiguity when salts or passwords contain colons.

Backward-Scanning Colon Resolution

When both the salt and password may contain colons (e.g., hash:salt:with:colons:password:with:colons), hashpipe scans backward through colon positions in the rest-of-line to try every possible salt/password boundary, right-to-left. This ensures correct identification even when the password itself contains colons.

$HEX[] Literal Password Retry

When a password field contains a valid $HEX[...] encoding (e.g., $HEX[41]), hashpipe first decodes it and tries all hash types with the decoded value (A). If nothing matches, hashpipe retries with the literal string $HEX[41] as the password — because some potfiles contain passwords that are literally the text $HEX[...] rather than hex-encoded binary.

The literal password is output using double-encoding: $HEX[244845585b34315d] (hex encoding of the literal bytes $HEX[41]), which avoids ambiguity with actual hex-encoded passwords.

This retry only applies to passwords starting with $HEX[ that contain valid hex content. Invalid hex (e.g., $HEX[ZZ]) is always treated as a literal string on the first pass.

Cost Limiting

Iterated verify functions (bcrypt, PBKDF2, scrypt, sha256crypt, etc.) estimate their execution time before running, using the formula (parsed_cost / bench_cost) / bench_rate. If the estimate exceeds the -L limit, the verify is skipped. This prevents misidentified hashes with extreme iteration counts (e.g., a hex string that happens to match bcrypt format with cost 31) from causing multi-minute stalls during auto-detection.

Verification Strategies

Hash types use one of three verification strategies:

Per-hashlen Candidate Caches

Rather than scanning all 1000 types for each input line, hashpipe maintains per-hashlen lookup tables: separate caches for unsalted, salted, and composed types indexed by binary hash length (0-64 bytes). A 32-byte hex hash (16 binary bytes) only checks MD5, MD4, GOST, RIPEMD-128, and their composed variants.

hx Hash Expression Language

hashpipe includes hx, a domain-specific language for describing, computing, and verifying cryptographic hash compositions. The -X flag activates hx mode.

Quick Examples

# Simple hash
echo password | hashpipe -X 'md5(pass)'
5f4dcc3b5aa765d61d8327deb882cf99

# Salted compound hash (mdxfind type e587, SHA1MD5SALT)
echo password | hashpipe -X 'sha1(md5(pass) . salt)' -s 12345

# NTLM
echo password | hashpipe -X 'md4(utf16le(pass))'
8846f7eaee8fb117ad06bdd830b7586c

# bcrypt with explicit salt and cost
echo password | hashpipe -X 'bcrypt(pass, fromhex("6162636465666768696a6b6c6d6e6f70"), 12)'

# Binary intermediate → hex encoding
echo password | hashpipe -X 'hex(bswap32(sha1_bin(pass)))'

# Iterated hash
echo password | hashpipe -X 'md5^1000(pass)'

Output Format Suffixes

A suffix on a function name selects a non-default representation of its output:

Suffix Meaning
(none) Canonical form (hex for digests, MCF for crypt)
_bin Raw binary bytes
_hex Lowercase hexadecimal
_uc Uppercase hexadecimal
_b64 RFC 4648 base64
_mcf Modular Crypt Format

Not every suffix applies to every function, and an unsupported one is a compile-time error:

Family Accepts Default
Digests, HMAC, KDFs _bin _hex _uc _b64 _hex
Crypt-family (bcrypt, yescrypt, ...) _bin _hex _b64 _mcf _mcf
String transforms and encoders none —

So md5_uc(pass) and bcrypt_mcf(pass, salt, 12) are valid, while md5_mcf(...), bcrypt_uc(...) and base64_bin(...) are rejected.

_uc is not the same as wrapping a call in upper(). The suffix sets the output role, which is what feeds the next round under iteration, so md5_uc^N chains uppercase hex while upper(md5^N) uppercases only the final result. They agree at N=1 and diverge from N=2 on.

Built-in Functions

All hashpipe hash types are automatically available as hx functions (lowercase names). In addition, hx provides:

hx Language Specification

The full hx language specification is available at www.mdxfind.com/hx.pdf, covering the complete grammar, type system, evaluation model, and a reference table mapping all mdxfind types to their hx expressions.

Supported Hash Types

hashpipe supports many hash types. See HASH_TYPES.md for the complete list with hashcat mode mappings and example hashes, or run hashpipe -h for a quick reference.

Notation

Algorithms below are written as hx expressions. The bound variables are pass, salt, salt2, user, and pepper — supplied by -p, -s, -S, -u, and -P respectively.

hashcat and John write the same thing with a $ sigil, as md5($pass.$salt). hx has no sigil, and the $ form is worth avoiding rather than merely wrong: the lexer prints hx:1: unexpected character '$' on standard error, discards the character, and carries on. So md5($pass.$salt) still exits 0 and still prints the correct hash, and with standard error redirected the run looks clean — while md5($p) prints a hash too, of the empty string. In this document $ appears only where it is a literal part of a hash, such as the $1$ of md5crypt or the $HEX[...] wrapper.

Anything in backticks in the tables below is a real hx expression that runs as written under hashpipe -X. Each was checked by computing a hash with it and feeding the result back through hashpipe -m for that type. A type whose definition is a format rather than a composition carries a plain-text description instead. Where an expression takes a cost or a round count, a literal is shown; the type itself reads that value out of the hash.

Common types

Index Type Algorithm
e1 MD5 md5(pass)
e2 MD5UC md5_uc(pass)
e3 MD4 md4(pass)
e4 MD2 md2(pass)
e8 SHA1 sha1(pass)
e9 SHA224 sha224(pass)
e10 SHA256 sha256(pass)
e11 SHA384 sha384(pass)
e12 SHA512 sha512(pass)
e369 NTLM md4(utf16le(pass))

Salted types

Index Type Algorithm
e31 MD5SALT md5(md5(pass) . salt)
e373 MD5PASSSALT md5(pass . salt)
e394 MD5SALTPASS md5(salt . pass)
e385 SHA1SALTPASS sha1(salt . pass)
e405 SHA1PASSSALT sha1(pass . salt)
e587 SHA1MD5SALT sha1(md5(pass) . salt)
e412 SHA256SALTPASS sha256(salt . pass)
e413 SHA256PASSSALT sha256(pass . salt)
e386 SHA512PASSSALT sha512(pass . salt)
e388 SHA512SALTPASS sha512(salt . pass)
e439 MSCACHE md4(md4_bin(utf16le(pass)) . utf16le(lower(user)))
e857 SKYPE md5(user . fromhex("0a") . "skyper" . fromhex("0a") . pass)

Composed types (selected)

Index Type Algorithm
e160 SHA1MD5 sha1(md5(pass))
e178 MD5SHA1 md5(sha1(pass))
e123 MD5MD5PASS md5(md5(pass) . pass)
e188 MD5SHA1MD5 md5(sha1(md5(pass)))
e497 MD4UTF16MD5 md4(utf16le(md5(pass)))
e368 MD5NTLM md5(md4(utf16le(pass)))
e251 SHA256SHA1 sha256(sha1(pass))
e786 NTLMH md4(utf16le(pass)) (dual-mode, see note below)

A nested digest yields its canonical hex form, so sha1(md5(pass)) is SHA-1 over the 32-character hex string, not over the 16 raw bytes. Append _bin where the raw bytes are fed forward instead — MSCACHE above is md4(md4_bin(...)) for exactly that reason.

Crypt types

Index Type Algorithm
e500 DESCRYPT descrypt(pass, salt) — 13-character DES crypt, no prefix
e511 MD5CRYPT md5crypt(pass, salt) — $1$
e512 SHA256CRYPT sha256crypt(pass, salt) — $5$
e513 SHA512CRYPT sha512crypt(pass, salt) — $6$
e577 BCRYPT256 bcrypt(sha256(pass), salt, 12) — $2a$ / $2b$
e529 CISCO8 PBKDF2-HMAC-SHA256, 20000 rounds — $8$
e917 CISCO9 scrypt N=16384 r=1 p=1 — $9$ (hashcat 9300)
e884 SCRYPT scrypt, parameters carried in the hash — SCRYPT:N:r:p:salt:hash (hashcat 8900)

PBKDF2 / KDF types

Index Type Algorithm
e529 CISCO8 $8$ PBKDF2-SHA256 20000 rounds
e530 PBKDF2-SHA256 PBKDF2-HMAC-SHA256 (incl. Python passlib $pbkdf2-sha256$)
e531 PBKDF2-MD5 PBKDF2-HMAC-MD5
e532 PBKDF2-SHA1 PBKDF2-HMAC-SHA1 (incl. Python passlib $pbkdf2$)
e533 PBKDF2-SHA512 PBKDF2-HMAC-SHA512 (incl. Python passlib $pbkdf2-sha512$)
e534 PKCS5S2 {PKCS5S2} PBKDF2-SHA1 10000 rounds
e895 NETSCALER-PBKDF2 Citrix NetScaler PBKDF2-HMAC-SHA256 2500 rounds
e899 LASTPASS PBKDF2-SHA256 + AES-256-ECB (hashcat 6800)
e905 REDHAT389DS {PBKDF2_SHA256} PBKDF2-SHA256 256-byte output (hashcat 10901)
e918 DCC2 PBKDF2-HMAC-SHA1 over NTLM (hashcat 2100)

LDAP SSHA types

Index Type Algorithm
e833 SSHA1BASE64 "{SSHA}" . base64(sha1_bin(pass . salt) . salt)
e835 SSHA256BASE64 "{SSHA256}" . base64(sha256_bin(pass . salt) . salt)
e836 SSHA512BASE64 "{SSHA512}" . base64(sha512_bin(pass . salt) . salt)

Non-hex / verify types

Index Type Algorithm
e450 BCRYPT bcrypt(pass, salt, 12)
e451 BCRYPTMD5 bcrypt(md5(pass), salt, 12)
e452 BCRYPTSHA1 bcrypt(sha1(pass), salt, 12)
e455 PHPBB3 phpass(pass, salt, 11)
e457 APACHE-SHA "{SHA}" . base64(sha1_bin(pass))
e461 APR1 apr1(pass, salt)
e500 DESCRYPT DES crypt (including BSDi Extended DES)
e521 SHA1SALTCX iterated SHA-1 with -- delimiters (hashcat 14400)
e819 MD5-MD5MD5PASSSALT-PEP md5(md5(md5(pass) . salt) . pepper) (hashcat 31700)
e821 MD5-MD5MD5PASSSALT-PEP2 md5(md5(md5(pass . salt)) . pepper) (hashcat 21900)
e822 MD5-SALT-SHA1PEPPASS md5(salt . sha1(pepper . pass)) (hashcat 21310)
e824 SHA1-SALTSHA1U16 sha1(fromhex(salt) . sha1_bin(utf16le(user) . ":" . utf16le(pass))) (hashcat 29000)
e861 CISCOPIX cisco_pix_encode(md5_bin(pad(pass, 16))) (hashcat 2400)
e862 CISCOASA cisco_pix_encode(md5_bin(pad(pass . salt, 16))) (hashcat 2410)
e876 DRUPAL7 $S$ SHA512 iterated
e884 SCRYPT SCRYPT:N:r:p:salt:hash (hashcat 8900)
e888 ISCSI-CHAP iSCSI CHAP authentication (hashcat 4800)
e899 LASTPASS PBKDF2-SHA256 + AES-256-ECB (hashcat 6800)
e901 DOMINO8 Lotus Notes/Domino 8+ PBKDF2-SHA1 (hashcat 9100)
e905 REDHAT389DS {PBKDF2_SHA256} Red Hat Directory Server (hashcat 10901)
e908 SHIRO1 Apache Shiro SHA-512 iterated (hashcat 12150)
e910 ORACLE12 Oracle 12C PBKDF2-SHA512 (hashcat 12300)
e918 DCC2 $DCC2$ Domain Cached Credentials 2 (hashcat 2100)
e919 PWSAFE3 Password Safe v3 SHA-256 iterated (hashcat 5200)
e929 RACF-KDFAES RACF KDF/AES (hashcat 8500)
e930 TACACS+ TACACS+ authentication (hashcat 16100)
e931 APPLE-SECURE-NOTES Apple Secure Notes (hashcat 16200)
e932 CRAMMD5-DOVECOT CRAM-MD5 Dovecot (hashcat 16400)
e933 JWT JSON Web Token HMAC-SHA (hashcat 16500)
e934 QNX-MD5 QNX /etc/shadow MD5 (hashcat 19000)
e935 QNX-SHA256 QNX /etc/shadow SHA256 (hashcat 19100)
e936 QNX-SHA512 QNX /etc/shadow SHA512 (hashcat 19200)
e937 QNX7-SHA512 QNX 7 /etc/shadow SHA512 (hashcat 19210)
e938 SHA1-S1PS2 sha1(salt . pass . salt2) (hashcat 19300)
e939 RAILS-RESTFUL Ruby on Rails Restful-Auth (hashcat 19500)
e940 KRB5PA-17 Kerberos 5 etype 17 Pre-Auth (hashcat 19800)
e941 KRB5PA-18 Kerberos 5 etype 18 Pre-Auth (hashcat 19900)
e942 WPA-PMKID WPA-PMKID PBKDF2 (hashcat 16800/22000)
e943 WPA-EAPOL WPA EAPOL MIC (hashcat 22000)
e944 ANSIBLE-VAULT Ansible Vault PBKDF2-SHA256 (hashcat 16900)
e945 APFS Apple File System (hashcat 18300)
e946 OTM-SHA256 Oracle Transportation Mgmt SHA256 (hashcat 20600)
e947 TELEGRAM-SHA256 Telegram Mobile Passcode SHA256 (hashcat 22301)
e948 WEB2PY-SHA512 Web2py PBKDF2-SHA512 (hashcat 21600)
e949 SOLARWINDS SolarWinds Orion (hashcat 21500)
e950 SOLARWINDS2 SolarWinds Orion v2 (hashcat 21501)
e951 SIMPLACMS Simpla CMS (hashcat 22800)
e952 APPLE-KEYCHAIN Apple Keychain PBKDF2+3DES (hashcat 23100)
e953 APPLE-IWORK Apple iWork PBKDF2+AES (hashcat 23300)
e954 BITWARDEN Bitwarden double PBKDF2 (hashcat 23400)
e955 MONGODB-SHA1 MongoDB SCRAM-SHA-1 (hashcat 24100)
e956 MONGODB-SHA256 MongoDB SCRAM-SHA-256 (hashcat 24200)
e957 FORTIGATE256 FortiGate SHA256 (hashcat 26300)
e958 UMBRACO Umbraco HMAC-SHA1 (hashcat 24800)
e959 DAHUA-AUTH Dahua Authentication MD5 (hashcat 24900)
e960 BESDER-AUTH Besder Authentication MD5 (hashcat 24901)
e961 SQLCIPHER SQLCipher PBKDF2+AES (hashcat 24600)
e962 RORAILS-SHA1 Ruby on Rails SHA1 (hashcat 27200)
e963 AES128-NOKDF AES-128-ECB no KDF (hashcat 26401)
e964 AES192-NOKDF AES-192-ECB no KDF (hashcat 26402)
e965 AES256-NOKDF AES-256-ECB no KDF (hashcat 26403)
e966 VMWARE-VMX VMware VMX (hashcat 27400)
e967 BCRYPTSHA512 bcrypt(sha512(pass), salt, 12) (hashcat 28400)
e968 POSTGRESSCRAM256 PostgreSQL SCRAM-SHA-256 (hashcat 28600)
e969 AWSSIGV4 Amazon AWS Signature v4 (hashcat 28700)
e970 KRB5DB17 Kerberos 5 etype 17 DB (hashcat 28800)
e971 KRB5DB18 Kerberos 5 etype 18 DB (hashcat 28900)
e991 MD5SALT1SALT2 md5(salt . pass . salt2) (hashcat 33000)
e992 SYMFONY256 Symfony Legacy SHA256 (hashcat 35800)
e993 WPBCRYPT WordPress bcrypt(hmac-sha384) (hashcat 35500)
e994 GOST12512CRYPT $gost12512hash$ gost12512crypt (hashcat 35600)

Note on NTLMH (e786)

NTLM hashing requires converting the password to UTF-16LE before computing MD4. For pure ASCII input, every tool agrees: each byte is zero-extended to a 16-bit value. For non-ASCII input, however, hashcat's zero-extension mode does not perform proper UTF-8 → UTF-16LE conversion — it simply widens each raw byte to 16 bits. The resulting hash is not a valid Microsoft NTLM hash and could never be used for Windows authentication, but it is what hashcat computes and stores in potfiles.

NTLMH (e786) accepts both interpretations:

  1. Proper UTF-8 → UTF-16LE (via iconv with //IGNORE): invalid UTF-8 sequences are silently discarded, and only valid characters are converted.
  2. Blind zero-extension (hashcat-compatible): every input byte is widened to 16 bits regardless of UTF-8 validity.

NTLM (e369) uses only the proper iconv path, since mdxfind always emits valid UTF-8 in its output for e369.

Example: the password $HEX[c0ffeebabe] (5 raw bytes, not valid UTF-8) produces two NTLMH hashes:

Additional algorithm families

hashpipe also supports GOST, GOST-CRYPTO, Streebog, gost12512crypt, RIPEMD-128/160/320, TIGER, HAVAL (all variants), BLAKE-224/256/384/512, BMW, CubeHash, ECHO, Fugue, Groestl, Hamsi, JH, Keccak, SHA-3, Luffa, Panama, RadioGatun, Shabal, SHAvite, SIMD, Skein, Whirlpool, MD6, MDC2, EDON, Snefru, HAS-160, BLAKE2B/2S, MurmurHash, RADMIN2, LM, SipHash, SAP BCODE/PASSCODE, AS/400 DES, PS-TOKEN, WINPHONE, QNX shadow, WPA-PMKID/EAPOL, Apple Keychain/iWork/APFS, Ansible Vault, Bitwarden, MongoDB SCRAM, SolarWinds, VMware VMX, SQLCipher, JWT, TACACS+, Kerberos 5 Pre-Auth/DB (etype 17/18), AWS Signature v4, PostgreSQL SCRAM-SHA-256, WordPress bcrypt, Symfony Legacy, Argon2, and hundreds of composed/chained variants.

Benchmarking

hashpipe -B benchmarks all registered types and reports hashes/second for each:

$ hashpipe -B | head -10
e1      MD5     7613341 16      0x00
e2      MD5UC   4153321 16      0x04
e3      MD4     4765041 16      0x00
e8      SHA1    2297819 20      0x00
e10     SHA256  1951415 32      0x00
e12     SHA512  772145  64      0x00
e369    NTLM    3005558 16      0x08
e450    BCRYPT  5       0       0x40
e455    PHPBB3  1747    0       0x40
e461    APR1    3147    0       0x40

Output format: index name rate hashlen flags

Types that cannot be benchmarked (missing dependencies) show n/a for rate.

Use -b to benchmark specific types: -b e1,e8-e12,e450.

Benchmark rates are used by the -L cost limit to estimate verify time for iterated types. The built-in rates were measured on an Apple M1 (8-core).

Building

make deps      # pull and build all dependencies from source
make hashpipe

Three compression libraries -- zlib, liblzma and libbzip2 -- are taken from the system rather than built by make deps, so install their development packages first. They are needed by the 7ZIP (e1000) verifier:

sudo apt install zlib1g-dev liblzma-dev libbz2-dev     # Debian / Ubuntu
sudo dnf install zlib-devel xz-devel bzip2-devel       # Fedora / RHEL
sudo port install zlib xz bzip2                        # macOS / MacPorts

Without the headers the build stops at fatal error: lzma.h: No such file or directory. The build prefers each library's static archive where one is present and falls back to linking it dynamically, so no manual setup is needed either way. To force one or the other:

make hashpipe COMPRESS='-lz -llzma -lbz2'                  # dynamic
make hashpipe COMPRESS='/path/libz.a /path/liblzma.a /path/libbz2.a'

make deps clones each dependency from its authoritative GitHub repository, pins it to a verified commit hash, and builds a static library. This requires git, a C compiler, make, and autotools (for libmhash and libJudy). Built artifacts are placed in the hashpipe source tree.

If you already have the required static libraries (from a previous make deps or a manual build), make hashpipe is sufficient.

To remove downloaded dependency sources:

make distclean

Docker

Docker can be used to build and run hashpipe without installing dependencies locally:

docker build . -t csp/hashpipe

docker run -v ${PWD}:/data -it --rm csp/hashpipe -m auto potfile.txt

The /data directory inside the container is used as the working directory.

Known Build Issues

sphlib BMW strict aliasing bug (GCC 12+): sphlib's bmw.c contains strict aliasing violations that cause GCC to generate incorrect code for BMW-224 and BMW-256 at -O2 and above. BMW-384/512 (64-bit core) and all other sphlib algorithms are unaffected. Apple clang is unaffected. The make deps target already applies the workaround (-fno-strict-aliasing). If you build sphlib separately, add -fno-strict-aliasing to its compile flags. See sphlib#3.

Dependencies

hashpipe requires the following static libraries:

and these three from the system, which make deps does not build:

Supported Platforms

The Makefile detects the build platform automatically. Tested on:

Type Indices and Hashcat Modes

hashpipe uses the same type index numbering as mdxfind. The -m option accepts both internal indices (with e prefix) and hashcat mode numbers (bare numbers):

# Internal indices
hashpipe -m e1,e8,e450 potfile.txt

# Hashcat mode numbers
hashpipe -m 0 potfile.txt          # MD5 (hashcat mode 0)
hashpipe -m 1000 potfile.txt       # NTLM (hashcat mode 1000)
hashpipe -m 3200 potfile.txt       # bcrypt (hashcat mode 3200)

# Mixed: hashcat modes and internal indices together
hashpipe -m 1000,e1,3200 potfile.txt

# Ranges (internal indices only) with hashcat modes
hashpipe -m e1-e12,1000,3200 potfile.txt

# With auto-detect fallback
hashpipe -m 0,1000,auto potfile.txt

Run hashpipe -h to see the full type list with hashcat mode mappings.

Hashcat Mode Coverage

hashpipe maps hashcat mode numbers to internal type indices via the -m option. Of hashcat's ~590 distinct modes, hashpipe currently resolves ~335. The remaining ~255 modes are not supported for the following reasons:

Reason Modes Description
Full-disk / volume encryption ~80 TrueCrypt, VeraCrypt, LUKS, BitLocker, DiskCryptor, BestCrypt, VirtualBox — require XTS block cipher decryption of 512+ byte sectors to verify a candidate password. hashpipe verifies hashes, not disk images.
Document / archive encryption ~25 MS Office (2003–2016), PDF, 7-Zip, RAR3/5, WinZip, PKZIP, ODF, Stuffit, AxCrypt, AES Crypt — require decrypting a document/archive payload and checking structural integrity (CRC, HMAC, XML parse). The "hash" is really an encrypted blob.
Wallet / blockchain encryption ~25 Bitcoin wallet.dat, Ethereum, Electrum, MultiBit, Exodus, MetaMask, Blockchain.com, MEGA, Terra Station, Bisq, Dogechain, Stargazer, 1Password — PBKDF2/scrypt key derivation followed by AES/3DES decryption of wallet data with structural verification.
Key file / credential store ~20 OpenSSH private keys, GnuPG/PGP, PEM, JKS Java Key Store, DPAPI master keys, KeePass, Mozilla NSS (key3.db/key4.db), SecureCRT, iTunes backup, Radmin3, Windows Hello — require decrypting a key structure and verifying internal consistency (ASN.1, PKCS padding, key check values).
Elliptic curve / cipher operations ~17 Bitcoin WIF/raw private keys (secp256k1 point multiplication), RC4 DropN (stream cipher key recovery), ChaCha20, Skip32, Kremlin NewDES — these are not hash functions; they require elliptic curve arithmetic or cipher-specific operations that have no place in a hash verification tool.
Network protocol / session tokens ~5 SIP digest auth, SNMPv3, KNX IP Secure, Flask/Mojolicious session cookies — require protocol-specific challenge-response state or application-specific secret keys beyond what a password hash tool provides. Note: many network protocol types are supported, including NTLMv1/v2 (5500/5600), WPA-PMKID (16800/22000), WPA-EAPOL, WPA-PMK (22001), TACACS+ (16100), Kerberos Pre-Auth and TGS etype 23 (7500/13100), Kerberos etype 17/18 (19800/19900/28800/28900), PostgreSQL SCRAM (11100/28600), and MongoDB SCRAM (24100/24200).
Non-cryptographic hashes 6 CRC32, CRC32C, CRC64Jones, Java Object hashCode(), MurmurHash (64-bit), STDOUT — trivially reversible checksums or debugging modes with no cryptographic purpose. Infinite collisions make verification meaningless.

The full list of unresolved modes with per-mode explanations is maintained in regress/unresolved-hashcat.txt and can be regenerated with regress/build_unresolved_table.sh.

Acknowledgments

hashpipe depends on the following libraries:

Platform detection in the Makefile was inspired by PR #1 from @0xVavaldi.

License

MIT

John the Ripper output (-J)

hashpipe reads hash:password pairs and reports the type it can recompute. By default it labels results in mdxfind's format. -J selects John's canonical form instead, for people whose tooling downstream speaks John:

-J 0    every line in mdxfind format                        (default)
-J 1    John format for lines that arrived in John format, mdxfind for the rest
-J 2    John format wherever an equivalent exists, mdxfind where none does

$ echo '5f4dcc3b5aa765d61d8327deb882cf99:password' | hashpipe -J 0
MD5x01 5f4dcc3b5aa765d61d8327deb882cf99:password

$ echo '5f4dcc3b5aa765d61d8327deb882cf99:password' | hashpipe -J 2
$dynamic_0$5f4dcc3b5aa765d61d8327deb882cf99:password

HASHPIPE_OUTPUT sets the same thing for callers that cannot reach the command line. An explicit -J always wins, and an environment-selected mode is announced on stderr so a changed output format is never a silent surprise. An unrecognised value — to either — is a fatal error, never a quiet fall back to the default.

Reading John's input

-J 1 and -J 2 also accept John's canonical $dynamic_N$ form, which hashpipe previously rejected outright:

$ echo '$dynamic_6$8d8cac84f234f42e32daeb94e7cd49e8$*.*:solomon1' | hashpipe -J 1
$dynamic_6$8d8cac84f234f42e32daeb94e7cd49e8$*.*:solomon1

The hash/salt boundary is taken from the data — the leading hex run — not from the type the wrapper names. John's salts may contain $ (its own documentation uses $dynamic_6$<hash>$*.*), so splitting on the separator would silently truncate them, and a truncated salt produces a confident "unresolved". $HEX$-encoded salts are decoded.

What it will not do

The wrapper's claim is never passed through. hashpipe reports what it recomputed. Feed it a line tagged $dynamic_0$ whose digest is actually SHA-1 and it comes back corrected, not laundered:

$ echo '$dynamic_0$5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8:password' | hashpipe -J 1
$dynamic_26$5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8:password

dynamic_1000 and above are refused. John reserves 0–999 for built-ins; 1000+ are defined in john.conf and mean whatever the producing machine said. Mapping them would be guessing with a confident face.

-J 2 output is deliberately mixed. Types with no John equivalent fall back to mdxfind format, so the stream is not uniformly parseable by a John-only consumer. That is the specified behaviour, not a defect.

The mapping table

john_map.h is generated by tools/john_map_gen.sh, never written by hand. Every row was confirmed by recomputation: John's own test vector for a format, wrapper stripped, digest reproduced by the named hashpipe type. A row also has to survive a confirm pass in which hashpipe is restricted to that single type and must verify every sampled vector — without it, a narrow-focus type such as MD5CAP (cap(md5(pass)), a no-op whenever the digest starts with a digit) masquerades as the real mapping on any one vector.

Regenerate against a freshly built John; a stale tree silently omits formats. The generated header records the John version and commit it was built from.

tools/john_mode_test.sh is the acceptance test. It checks the refusals above and feeds every line hashpipe emits back through john --show=formats to confirm John still accepts it.