Skip to content
All projects

Hookstorm

An open-source tool that plays the webhook provider: it delivers your events twice, out of order, late or not at all, then checks that your business rules still hold.

Role
Creator
Year
2026
Status
v0.2.0 on npm
Stack
TypeScript, Node

The problem

Webhook providers promise delivery at least once, in no particular order. Stripe retries a failed delivery for days; Paystack retries every few minutes, then hourly. So a handler will see the same event twice, see events out of order, and get retries for requests it already processed.

Most handlers are still written and tested as if every event arrives exactly once, in order. The gap shows up in production as double charges, refunds overwritten by a late "paid" event, and events lost because the handler answered 200 before doing the work.

What I built

Hookstorm replays recorded events against your endpoint and injects the faults real providers produce:

  • duplicates, one after another and at the same time
  • reordering and delays
  • dropped requests, which the provider then retries
  • dropped responses, where your app did the work but the provider never saw the 200
  • timeouts

It follows each provider's retry rules (Stripe, Paystack, GitHub, or a custom one), compressed from hours to milliseconds, and re-signs every attempt so your real signature check still runs. You write the rules that must always hold, such as "each order gets at most one receipt", as TypeScript. Every run comes from a seed, so any failure comes with a command that replays exactly the same delivery plan.

Putting it to the test

I wrote a small ticket shop the way I'd normally write one: NestJS, Postgres, Stripe in test mode. Its payment handler checks whether the order is already paid, uses a transaction, and verifies the signature with Stripe's SDK. I would have approved it in code review.

I recorded 14 real test-mode events from three checkouts and a refund, and pointed Hookstorm at the handler. With the realistic profile, 24 of 50 seeds broke five of the six rules. Two real bugs were behind it.

Two deliveries at once, two payments

26msdeliver evt_2#0 payment_intent.succeeded → 200
27msdeliver evt_2#1 payment_intent.succeeded → 200
✗ order a09b…0e3b has 2 payment entries

Both copies read the order while it was still pending, both passed the "already paid?" check, and both ran the transaction: two ledger entries, stock taken twice, two receipts. The check sat outside the transaction, so it only protected against duplicates that didn't overlap.

A retry after a refund pays the order again

196mspayment_intent.succeeded attempt 1 → response dropped
199mscharge.refunded attempt 1 → 200
257mspayment_intent.succeeded attempt 2 → 200
✗ order ended as "paid" after its refund was acknowledged

(This trace is from an earlier run with sample events. With the 14 recorded events, this rule broke in 2 of 50 realistic runs and 15 of 30 hostile runs.)

The payment was processed but its 200 was lost, so the provider retried. Meanwhile the refund landed. When the retry arrived, the order wasn't "paid" any more (it was "refunded"), so the guard let it through and the refund was undone.

The fix

Four standard patterns, none of them specific to Stripe:

  1. Apply each event once, in the same transaction as its effects. Claim the event id with INSERT … ON CONFLICT DO NOTHING; a concurrent duplicate waits on the key, then skips.
  2. One payment and one refund per order, enforced by the database, with a unique index and a row lock.
  3. Status only moves forward. A late payment still records the money but can't turn a refunded order back into a paid one.
  4. Receipts go through an outbox, so the webhook no longer waits on the email API.

Results

Same 14 real events, same seeds, same six rules:

RunBeforeAfter
realistic, 50 seeds24 broke 5 of 6 rulesall rules held
hostile, 30 seeds29 broke 5 of 6 rulesall rules held
hostile, 200 more seeds—all held across 9,215 delivery attempts

The fix makes the race impossible rather than unlikely, which is why it holds on every repeat.

What I learned

The most useful lesson was about the tool itself. If a test resets the database and replays events for orders that no longer exist, the handler logs "unknown order", returns 200 and does nothing, and rules like "at most one payment per order" pass because nothing was paid at all. A green run that tests nothing is worse than no test.

So Hookstorm 0.2 now checks that a run actually changed something and reports it as inconclusive if it didn't, and adds a rule that verifies each acknowledged event really took effect.

The full write-up, with the handler code and the fix: My Stripe webhook handler looked safe. Replaying 14 real events broke it in 24 of 50 runs.

Have a role or a project in mind? Let’s talk.