The Mastermind Notes
Hashing
Hashing is a one-way function that creates a string
of characters. You cannot reverse the hash to re-create the original
file. Passwords are often stored as hashes instead of storing the
actual password. Additionally, applications often salt passwords with
extra characters before hashing them.
A cryptographic hash function is an algorithm that takes data of
arbitrary size as its input and returns a fixed size value,
called message digest or checksum, as its output. For
example, sha256sum calculates the SHA256 (Secure Hash Algorithm
256) message digest. SHA256, as the name indicates, returns a
checksum of size 256 bits (32 bytes). This checksum is usually
written using hexadecimal digits. Knowing that a hexadecimal digit
represents 4 bits, the 256 bits checksum can be represented as 64
hexadecimal digits.
Common hashing functions
md5sum
sha256sum
hmac256
MD5
Message Digest 5 (MD5) is a common hashing algorithm that produces
a 128-bit hash. Security experts now consider it cracked and
discourage its
use as a cryptographic hash.
Generating md5 checksum for a file
md5sum file
SHA
Definition
Secure Hash Algorithms (SHA) are a group of hashing algorithms with
variations in grouped four families. SHA-256 creates 256-bit hashes
and SHA-512 creates 512-bit hashes. SHA is also used to verify file
integrity.
SHA1
SHA-1 generates a 160-bit hash value and was widely used in the
past. However, it is no longer recommended for use due to its
vulnerability to collision attacks. In a collision attack, two different
inputs produce the same hash value, which can lead to data tampering
and forgery.
Generating SHA1 checksum for a file
$ sha1sum file
$ shasum file
$ shasum file
SHA2
SHA-2, on the other hand, generates hash values of 224, 256, 384,
and 512 bits, making it more secure than SHA-1. It is commonly used
for digital signatures, data integrity checks, and password storage.
SHA-2 uses the Merkle-Damgard construction, which means that it
divides the input message into blocks, and each block is processed
separately. This makes SHA-2 more resistant to collision attacks.
SHA3
SHA-3 generates hash values of 224, 256, 384, and 512 bits and is
commonly used for digital signatures, data integrity checks, and
password storage. SHA-3 uses the sponge construction, which means
that it absorbs the input message and then squeezes out the hash
value. This construction provides a high level of resistance to collision
attacks and is more efficient than the Merkle-Damgard construction
used in SHA-2.
SHA256 and 512
Generating SHA256 and 512 checksum for a file
shasum -a256 file
Likewise for 512 checksum
shasum -a512 file
HMAC
Hash-based Message Authentication Code (HMAC) verifies both the
integrity and authenticity of a message with the use of a shared
secret.
HMAC can be combined with any standard hash function, such as MD5,
SHA-2, or SHA-3, by using a shared secret key. Therefore, only
communicating parties who know the secret key can generate or verify
the partial digital signature. If the recipient decrypts the message
digest but cannot successfully compare it to a message digest
generated from the original plaintext message, that means the
message was altered in transit.
Blake2
BLAKE2 is a high-performance cryptographic hash function that
outpaces MD5, SHA-1, and SHA-2 in speed while maintaining a strong
security profile. It is considered a successor to the original BLAKE
algorithm and comes in two main variants:
BLAKE2b – Optimized for 64-bit platforms
BLAKE2s – Designed for 8- to 32-bit platforms
✅ High Performance: Notable for its exceptional hashing speed,
especially in software implementations.
✅ Strong Security: Provides a secure alternative to MD5 and SHA-
1/2. As of the latest findings, no practical vulnerabilities have been
identified.
✅ Versatile Functionality: Supports keying (enabling MAC
functionality), personalization, and salting, with customizable digest
sizes (up to 64 bytes for BLAKE2b and 32 bytes for BLAKE2s).
✅ Lightweight: BLAKE2s is particularly suitable for resource-
constrained environments.
Common Applications
🔹 Cryptographic Systems – Used in various cryptographic protocols
due to its speed and security.
🔹 Data Integrity Verification – Applied as a checksum for ensuring
data integrity.
🔹 Password Hashing – While not its primary use, BLAKE2 can be
utilized for password hashing when high-speed processing is needed.
Design Principles
Foundation: Built upon the cryptographic primitive ChaCha, a
stream cipher developed by Daniel J. Bernstein.
Security Architecture: Uses HAIFA construction instead of the
traditional Merkle–Damgård (as seen in SHA-2), preventing length
extension attacks without requiring HMAC.
Notable Implementations
🔹 libsodium – A widely used cryptographic library incorporating
BLAKE2 for encryption, decryption, digital signatures, and password
hashing.
🔹 Argon2 – The winner of the Password Hashing Competition (PHC),
which leverages BLAKE2 internally for secure password hashing.
BLAKE2b
BLAKE2b is a cryptographic hash function known for its speed and
security. It surpasses MD5, SHA-1, and SHA-2 in performance while
maintaining security levels comparable to the SHA-3 standard. As an
enhanced version of the original BLAKE hash function, BLAKE2b is
optimized for modern processors, ensuring efficient hashing.
Superior Performance: BLAKE2b processes data faster than many
traditional hash functions, especially for short messages.
Robust Security: Designed with a strong security margin, making
it resilient against known cryptanalytic attacks.
Versatile Implementation: Available in two versions—BLAKE2b
(optimized for 64-bit platforms) and BLAKE2s (optimized for 8-
to 32-bit platforms).
Simplified Design: Easier to understand and analyze compared to
SHA-3, promoting better security validation.
Patent-Free: BLAKE2 is not restricted by patents, allowing
unrestricted use in any application.
Common Applications
Cryptographic Authentication: Generates message digests to
verify integrity and authenticity.
Password Hashing: Enhances password security for
authentication systems.
File Integrity Verification: Detects file tampering during storage
or transfer.
Technical Insights
Built on the HAIFA construction, similar to its predecessor,
BLAKE.
Utilizes components from the ChaCha stream cipher, ensuring
efficiency and strong diffusion properties.
Comparison with Other Hash Functions
SHA-3: Although not part of the SHA-3 competition, BLAKE2b
delivers comparable security while outperforming SHA-3 in speed.
SHA-256: While widely adopted, SHA-256 is typically slower than
BLAKE2b for similar security levels.
Hash Collision
A hash collision occurs when the hashing algorithm creates the same
hash from different inputs. MD5 is highly susceptible to collision
attacks, which is why it is no longer recommended as a cryptographic
hash.
Collision Resistance in Cryptographic Hash Functions
Overview
Collision resistance is a fundamental property of cryptographic hash
functions, ensuring it is computationally infeasible to find two distinct
inputs that generate the same hash output.
Key Concepts
Cryptographic Hash Functions: These are mathematical algorithms
that take an input (message) and generate a fixed-length hash
value, typically represented as a numeric or alphanumeric string.
Even a minor alteration in the input results in a significantly
different hash.
Weak Collision Resistance: A hash function H is weakly collision-
resistant if, given an input x, it is difficult to find another input
y (where x ≠ y) that results in the same hash, i.e., H(x) =
H(y).
Strong Collision Resistance: A hash function is strongly collision-
resistant if it is challenging to find any two different inputs, x
and y, such that H(x) = H(y).
Examples
Hash Collisions: When two different inputs produce the same
hash, a collision occurs, potentially compromising data integrity
in security-dependent applications.
SHA-1 Vulnerability: Once deemed secure, SHA-1 was later proven
vulnerable to collision attacks, leading to its deprecation in favor
of stronger algorithms like SHA-256.
Mitigation Strategies
Upgrading to Stronger Algorithms: As weaknesses emerge,
transitioning to more secure hash functions (e.g., from SHA-1 to
SHA-256) is crucial.
Ongoing Algorithm Evaluation: Continuous assessment and
research ensure cryptographic hash functions remain robust
against emerging threats.
Hash Length Extension Attacks
Overview
A hash length extension attack is a cryptographic exploit that targets
certain message authentication implementations. It takes advantage of
how some hash algorithms process data in blocks, enabling an
attacker to forge a valid hash for modified data — without knowing the
original secret key.
This vulnerability primarily affects hash functions based on the
Merkle–Damgård construction, such as:
MD5
SHA-1
SHA-256 (if used incorrectly)
When a system builds a signature by hashing secret || message (where
|| means concatenation) and then validating the hash, it can be
tricked if the implementation is naive.
Conditions for a Successful Attack
An attacker can mount a hash length extension attack if the following
are known or guessable:
1. Length of the secret key
The key length doesn’t need to be exact at the start — it
can often be brute-forced within a reasonable range.
2. Original message (cleartext data)
The portion of the data before the attacker’s
modifications must be known.
3. The hashing algorithm used
Only certain algorithms are vulnerable (e.g., MD5, SHA-1,
SHA-256). Algorithms like HMAC are resistant by design.
4. Padding scheme
Most vulnerable algorithms use predictable, standard
padding rules. Knowing or deducing them is essential.
Because the padding length depends on (secret length + data length) ,
knowing the secret length is crucial for reconstruction.
How the Attack Works
In vulnerable hash constructions:
1. The hash function processes data in fixed-size blocks.
2. It maintains an internal state that can be reused if the attacker
can set it to the state after hashing the original message.
3. The attacker appends new data to the original message —
without ever seeing the secret — and still produces a valid
signature.
Example Walkthrough
Original setup:
secret = "secret" data = "data" MD5(secret || data) =
6036708eba0d11f6ef52ad44e8b74d5b
Attacker’s goal: Append "append" so that the system accepts:
secret || data || padding || append
As valid with the new hash.
Steps:
1. Compute the MD5 padding for secret || data .
2. Set the internal state of MD5 to
6036708eba0d11f6ef52ad44e8b74d5b .
3. Append "append" and continue hashing.
4. Output is a valid MD5 for the extended message , even
though the attacker never knew "secret" .
Why It Works
The Merkle–Damgård design allows hashing to resume from any internal
state. Since the attacker can recreate the state after secret || data
and knows the padding rules, they can continue hashing as if they
were the server.
Real-World Risks
This flaw appears in:
Naive API authentication using hash(secret || message)
URL signing mechanisms
Download verification systems
Legacy systems still using MD5 or SHA-1 for authentication**
If exploited, an attacker can:
Modify parameters in API calls
Inject malicious payloads into signed requests
Bypass authentication for protected resources
Mitigation Strategies
1. Use HMAC instead of raw hash-based signatures
HMAC is designed to be immune to length extension
attacks.
Example: HMAC(secret, message)
2. Avoid vulnerable hash functions
Migrate from MD5/SHA-1 to SHA-256, SHA-3, or BLAKE2
with secure constructions.
3. Include the secret after the message
hash(message || secret) resists length extension in most
practical cases.
4. Implement strict input validation
Ensure no external party can control appended data in
signed messages.
Modern Password Hashing Algorithms
When storing passwords securely, the choice of hashing algorithm
matters. Unlike older, fast hash functions such as MD5 or SHA-1,
modern password hashing algorithms are intentionally slow and
configurable. This slowness is a security feature , it increases the
cost of brute-force and GPU-based cracking attempts.
The performance of these algorithms can be fine-tuned using a work
factor or cost parameters, allowing you to adjust how computationally
expensive the hashing process is as hardware capabilities evolve.
Security Note:
There’s no need to hide which password hashing algorithm your
application uses. If configured correctly, modern algorithms remain
secure even when their names are publicly known.
Recommended Algorithms for Secure Password Storage
1. Argon2id
Argon2 won the 2015 Password Hashing Competition and is now widely
considered the most secure and flexible option. Among its three
variants (Argon2i, Argon2d, and Argon2id), Argon2id is recommended
because it provides strong resistance to both side-channel attacks and
GPU-based cracking.
Unlike simpler algorithms that only adjust a single work factor,
Argon2id lets you configure:
m – Minimum memory usage (in KiB)
t – Number of iterations
p – Degree of parallelism (threads)
Suggested configurations (choose based on available system
resources):
m=47104 (46 MiB), t=1, p=1
m=19456 (19 MiB), t=2, p=1
m=12288 (12 MiB), t=3, p=1
m=9216 (9 MiB), t=4, p=1
m=7168 (7 MiB), t=5, p=1
These provide equivalent security; the trade-off lies in CPU vs RAM
usage.
2. scrypt
scrypt is a memory-hard password-based key derivation function
(PBKDF) designed by Colin Percival. While Argon2id is generally
preferred, scrypt is a strong alternative when Argon2id is unavailable.
Configurable parameters:
N – CPU/memory cost factor (must be a power of two)
r – Block size
p – Parallelization factor
Suggested configurations:
N=2^17 (128 MiB), r=8, p=1
N=2^16 (64 MiB), r=8, p=2
N=2^15 (32 MiB), r=8, p=3
N=2^14 (16 MiB), r=8, p=5
N=2^13 (8 MiB), r=8, p=10
Again, the security level is equivalent; differences come down to
CPU/RAM trade-offs.
3. bcrypt
bcrypt is an older password hashing function still in use, mainly for
legacy systems. It is slower than MD5/SHA but lacks the modern
memory-hard design of Argon2 or scrypt.
Minimum recommended work factor: 10 (increase if performance
allows).
Maximum password length: Most implementations cap input at 72
bytes , enforce this limit in your application.
bcrypt Pre-Hashing Considerations
Developers sometimes pre-hash passwords before bcrypt using SHA-2,
HMAC, or BLAKE3 to bypass length limits. However, this approach can
introduce risks:
Null byte truncation , bcrypt treats null-terminated strings as the
end of the password.
Password shucking , If the inner hash (e.g., SHA-512) is used
elsewhere, cracking bcrypt could reduce to cracking the inner
hash.
Safe pre-hash approach (if unavoidable):
bcrypt( base64( HMAC-SHA384( password, pepper ) ), salt, cost )
Store the pepper separately from the database.
4. PBKDF2
PBKDF2 is NIST-recommended and FIPS 140-validated, making it ideal
when compliance is mandatory.
Requires selecting an internal hash (e.g., HMAC-SHA256).
Security is controlled via iteration count.
Suggested iteration counts (as of Dec 2022, RTX 4000 benchmarks):
PBKDF2-HMAC-SHA1: 1,300,000 iterations
PBKDF2-HMAC-SHA256: 600,000 iterations
PBKDF2-HMAC-SHA512: 210,000 iterations
Parallel PBKDF2 settings:
SHA512: cost=2
SHA256: cost=5
SHA1: cost=10
PBKDF2 Pre-Hashing
If a password exceeds the hash function’s block size (e.g., 64 bytes
for SHA-256), it will be pre-hashed automatically. While this is
generally safe, some poor implementations re-hash on every iteration
, leading to performance degradation and potential DoS vectors with
very long passwords (e.g., Django’s 2013 vulnerability).
Mitigation: Manually pre-hash with a salt before PBKDF2.
Summary Table
Algorithm Best Use Memory- Adjustable Compliance Notes
Case Hard Work Ready
Factor
Argon2id Default ✅ m, t, p ❌ Best
modern parameters protection vs
choice GPU & side-
channel
scrypt Alternative ✅ N, r, p ❌ Memory-
when Argon2 parameters hard, strong
unavailable legacy
support
Algorithm Best Use Memory- Adjustable Compliance Notes
Case Hard Work Ready
Factor
bcrypt Legacy ❌ Cost factor ❌ 72-byte
systems limit, pre-
only hash with
caution
PBKDF2 Regulatory ❌ Iteration ✅ Hardware-
compliance count accelerated
(FIPS/NIST) on many
platforms
Done !
Check out other cheat sheets and study notes using the below link
https://shop.motasem-notes.net
https://buymeacoffee.com/notescatalog
