back to blog

Two different bodies, one valid webhook signature

7 min read

#security #webhooks #typescript #lessons

Body A and body B below are different on the wire. Not almost the same. Byte-for-byte different. Sign one on a webhook signer I wrote myself, present the other, and the signature still checks out.

I found this in webhook-hmac-kit, a small webhook signing library I maintain. The whole job of a signer is to prove the bytes that arrive are the bytes that got signed. Mine didn't, for a specific and boring reason.

Body A is the JSON {"note":"�"}, where that character is U+FFFD, the Unicode replacement character, correctly encoded as three bytes: ef bf bd. Body B is the same JSON with those three bytes swapped for one invalid byte, ff, which is not valid UTF-8 on its own. A lossy decode turns an invalid byte into the same replacement character. So raw.toString('utf-8') on body A and on body B produces the identical JS string. A signer that decodes the body before hashing hashes that identical string, and body A's signature verifies body B.

It isn't only a wire-decoding quirk. It's the encoding step itself. An unpaired surrogate on its own, \uD800, encodes to the same three UTF-8 bytes as the replacement character. So sign({ payload: '\uD800' }) and sign({ payload: '�' }) return the same signature, with no adapter involved at all. Any code path that turns a JS string into bytes before or during signing carries this.

The bug I was actually chasing when I found it

I found the surrogate bug while fixing a different one. The library's canonical string, the thing that actually gets hashed, used to be built like this: v1:{timestamp}:{nonce}:{payload}, joined on a colon. Nothing stopped the nonce or the payload from containing a colon.

Take a message signed with nonce abc and payload a:b:c. The canonical string is v1:1700000000:abc:a:b:c. The same bytes also read as nonce abc:a, payload b:c. Or nonce abc:a:b, payload c. All three splits produce the identical string, so all three verify under the same signature.

The nonce is the value a replay cache keys on. An attacker who has seen one valid delivery can present it again with the split point moved over. Each variant hands the cache a nonce it has never seen. A cache doing exactly what it was built to do waves the forged ones straight through anyway. I reproduced this: three separate accepted deliveries of one signed message, against a cache that correctly rejected the honest replay. Nothing was wrong with the cache. The nonce it was keying on wasn't stable.

The fix moved the delimiter to a dot and constrained the nonce to a fixed, dot-free grammar. That makes the split unambiguous instead of merely unlikely to collide. While rebuilding the canonical value I also switched it from a template literal to bytes. I widened the payload type too, so a caller could hand over raw wire bytes directly.

That last change is the one that turned up the surrogate bug.

The property test that was green for the wrong reason

Before shipping the delimiter fix I wrote a property test. Generate random (timestamp, nonce, payload) triples, and assert that no two distinct ones produce the same canonical string. It passed and I moved on.

It was green for the wrong reason. The function it called returns a string, and that function genuinely is injective, a distinct string for a distinct triple. But signing doesn't hash that string. It hashes the UTF-8 encoding of that string, and encoding a JS string as UTF-8 is not injective, for the surrogate reason above. The property test asserted something true about a layer that was already correct, and never touched the layer that wasn't.

Random byte arrays almost never collide by chance either, which is the other reason the first version of the test missed it. The fix that made the property mean something: assert on the signed bytes, not the canonical string. And generate a payload arbitrary that can actually produce an ill-formed one. The relevant part of the test file:

const payloadBytesArb = fc.oneof(
  fc.uint8Array({ maxLength: 24 }),
  fc.uint8Array({ min: 0xf8, max: 0xff, maxLength: 4 }),
);

The second arm draws lead bytes in 0xf8–0xff, invalid as the start of any UTF-8 sequence on their own. Every byte string built from them decodes to the same run of replacement characters. Without that arm, the ill-formed case practically never gets generated, however many runs you add.

What a property test has to target

A property test is only as good as the function it calls. Say the property is "no two distinct inputs produce the same signature". Then the property has to call the signing function. Not a helper three layers upstream that happens to look like the interesting part.

The wire-level version of the bug came from the same mistake, one layer over. The code that pulled the raw body off the request decoded it to a string before handing it to the signer. Two different byte sequences on the wire could collapse to the same string before signing ever ran. The fix is the same shape: carry the payload as bytes end to end. Only decode it if the caller explicitly hands over a string in the first place.

Try it on your own signer

Three lines, against whatever signer you already have:

const a = Buffer.from([0x7b,0x22,0x6e,0x6f,0x74,0x65,0x22,0x3a,0x22,0xef,0xbf,0xbd,0x22,0x7d]); // {"note":"�"}
const b = Buffer.from([0x7b,0x22,0x6e,0x6f,0x74,0x65,0x22,0x3a,0x22,0xff,0x22,0x7d]);             // same, one byte swapped
console.log(verify(b, sign(a))); // true means it decoded before hashing

Swap sign and verify for your own calls. If it comes back true, the body it checked was never the body that got signed.

Check your signer

Five questions:

  • Does it accept raw bytes, or force everything through a string first?
  • Does it decode the body to a string before hashing?
  • Does it hand the parsed string downstream, or the raw bytes?
  • Does anything downstream separately hash the raw body, assuming it's the same thing?
  • Does the fix exist in that path? Byte concatenation instead of a template literal. A TextDecoder with fatal: true, so a bad decode throws instead of substituting. Or an isWellFormed() check before a string gets encoded.

The compare

While I was in there the signer moved off node:crypto and onto Web Crypto, so it runs on Workers and Deno without a compat flag. That put the compare on crypto.subtle, and subtle.verify is the exact thing that had a timing bug. Node's WebCrypto HMAC verify wasn't constant-time until March 2026. CVE-2026-21713 was a timing side channel in memcmp() inside crypto_hmac.cc. The person who fixed it then amended the WebCrypto spec itself, w3c/webcrypto PR #553, requiring HMAC verification to run in constant time.

That line only exists in the editor's draft. I couldn't find a Web Platform Test for it. So it's a requirement with no test behind it, which is roughly how the Node bug went unnoticed for years. I moved verification to a blinded double-HMAC compare instead. Sign both digests under a fresh random key per call, and compare those. That way I'm not trusting a runtime's own verify to already hold a guarantee the spec only just started asking for.

One contributor in the WebCrypto issue tracker calls this pattern a non-solution, an abuse of the API. He argues the platform should ship a real constant-time primitive instead. I still think the workaround is the safer default until it exists, but the objection is fair.

The one-line lesson

An unescaped delimiter between two free-form fields is not a formatting choice. It's a parsing ambiguity. A property test aimed at the wrong function will tell you it's fine.

I checked how common this is elsewhere; that is the next post.

Full changes are in the 2.0.0 changelog.