How to Know Your Webhook Handler Won't Break in Production

You shipped your payment integration last sprint. You wrote tests for the happy path. You confirmed that a successful charge returns the right status code. Your CI is green. You closed the ticket.

Three weeks later, a customer emails support because they were charged but never got a confirmation email. Your order is stuck in PENDING. The webhook handler that was supposed to move it to FULFILLED never fired — or fired twice, or fired for an event type it didn't know how to handle.

The charge went through. The webhook didn't. And you had no test that would have caught it.

Why Webhook Handlers Break Differently

Most payment integration bugs live in the webhook handler, not in the checkout. The reason is timing: the checkout is synchronous — you make a request, you get a response, something happens immediately. Webhook delivery is asynchronous — the gateway fires an event to your server minutes, sometimes hours later, and your handler processes it in the background with no user watching.

This creates a category of failures that are almost invisible in development:

• The gateway retries a webhook you already processed
• An event arrives for a type your handler doesn't recognise
• The payment gateway sends a slightly malformed payload
• Your server is under load and responds slowly, triggering more retries
• The webhook never arrives at all, but the money was captured

None of these show up in "does the checkout work?" tests. They all show up in production — usually at the worst possible time, and always with a real customer attached.

What the Webhook Health Check Actually Tests

MockCard's webhook health check fires 5 signed probes directly at your webhook handler and measures exactly how it responds to each one. Here's what each probe is looking for.

1. Successful payment webhook

The baseline. Your handler receives a payment_intent.succeeded event and should return HTTP 200.

If this fails, nothing else matters. Your handler can't process successful payments at all — every paid order will stay in whatever pending state it was in at checkout.

What it looks like when it's broken: Orders show "payment received" but never fulfil. Support queue fills up with "where's my order?" tickets within hours.

2. Payment failed webhook

Your handler receives a payment_intent.payment_failed event. It should also return 200.

Developers often miss this one. The instinct is "if the payment failed, we don't need to do anything." But you do — you need to update the order status, release any reserved inventory, and allow the customer to retry. If your handler crashes on a failed event, the gateway keeps retrying, your error logs fill with noise, and your webhook reliability metrics degrade.

What it looks like when it's broken: Failed payments create ghost orders that stay pending. Customers can't retry because the order is "in progress." Webhook dashboards show 40% failure rates.

3. 3DS challenge event

Your handler receives a payment_intent.requires_action event. This is what arrives when a card needs 3D Secure authentication before the charge can complete.

Many handlers only check for succeeded and payment_failed. They have no case for requires_action, which means the handler throws an unhandled exception, returns 500, and the gateway retries until it gives up. The 3DS flow never completes. The customer is stuck on a loading screen.

What it looks like when it's broken: Customers attempting 3DS-required cards (common in India, Europe) see endless loading or "payment pending" after completing their OTP.

4. Unknown event type

Your handler receives an event with a type it has never seen before — something like charge.dispute.created.

This might sound unlikely, but payment gateways add new event types regularly. If your handler is a simple switch statement without a default case, it throws an unhandled exception on any unknown type. The gateway retries. Eventually the handler gets marked as unreliable.

What it looks like when it's broken: A gateway changelog announces a new event type. Your webhook failure rate spikes the next day. You spend an afternoon debugging something that had nothing to do with your recent deploy.

5. Malformed payload

Your handler receives an event with a missing field — specifically, the top-level id field is absent.

If your first line of handler code is something like event_id = payload["id"] without a null check, this crashes immediately. Malformed events can arrive from gateway bugs, replay tools, or deliberate testing. Your handler should gracefully return 200 or 400 — not 500.

The Two Scenarios Nobody Builds For (Pro)

The 5 standard probes catch the obvious failures. The following two scenarios are different — they don't appear in any staging environment, they don't appear during normal testing, and they cause the kind of production incident that gets a Slack postmortem.

Race condition: duplicate webhook delivery

Every production payment gateway retries webhook delivery when your handler is slow or returns a non-2xx. If your handler takes 3 seconds and the gateway's retry timeout is 2 seconds, the same event gets delivered twice. Both deliveries carry the same event.id.

Without an idempotency check, your handler processes both. The order fulfils twice. You ship two items, charge the customer's card once, and have an inventory discrepancy that reconciliation can't explain for weeks.

The fix is a processed_events table — store the event ID on first processing, skip on second. Simple in principle. But almost no developer writes a test that actually fires the same event twice in quick succession and asserts the order was fulfilled exactly once.

MockCard Pro's simulate_race flag fires identical webhooks ~1 second apart. You can write that assertion. You can know with certainty that your handler is idempotent — not just believe it is.

Limbo state: payment captured, webhook never arrives

This one is quieter and worse. The charge succeeds. The payment is captured. MockCard (or your real gateway) never delivers the webhook. Your order sits in PENDING_WEBHOOK indefinitely. The customer was charged. They have no confirmation. Your support queue gets a ticket 20 minutes later.

This happens in production when:

• A gateway experiences a partial outage affecting webhook delivery
• Your server has a deployment restart between charge and webhook
• A load balancer misconfiguration drops the delivery

The right response is a reconciliation job — a background task that checks orders stuck in PENDING_WEBHOOK for more than an hour and looks them up directly against the gateway's API. Most applications don't have this, because the scenario never appears in development.

MockCard Pro's limbo scenario creates exactly this state on demand: the charge returns success, the webhook never arrives. You can write and test the reconciliation path before a real outage forces you to.

What to Do With the Results

The health check shows you which probes passed and which failed. Here's how to think about the output.

If standard probes fail: Your handler has a correctness bug. An event type is unhandled, or your handler crashes on malformed input. These are worth fixing today — they're affecting a percentage of your live transactions right now.

If everything passes: Good baseline. Your handler is structurally sound. The question is whether it stays that way — every deploy is a chance for a regression.

This is the part that matters most: the health check is a snapshot, not a guarantee. A passing result today means your handler works today. It doesn't mean it still works after the next refactor, the next dependency update, or the next developer who adds a feature to the checkout flow.

The way to make the guarantee is to add MockCard to your CI pipeline and run these same probes on every pull request. That's what the API is for.

// webhook.test.ts — runs in CI on every PR
const SCENARIOS = [
  "success",
  "insufficient_funds",
  "3ds_challenge",
] as const;

for (const scenario of SCENARIOS) {
  it(`webhook handler returns 200 for ${scenario}`, async () => {
    const res = await fetch("https://www.mockcard.io/api/v1/generate", {
      method:  "POST",
      headers: { "Authorization": `Bearer ${process.env.MOCKCARD_API_KEY}` },
      body: JSON.stringify({
        scenario,
        amount:      9900,
        currency:    "inr",
        webhook_url: process.env.WEBHOOK_URL, // your staging handler URL
      }),
    });

    // MockCard delivers the webhook to your handler synchronously before responding
    // Your handler's response is reflected in the delivery log — poll via log_id
    expect(res.status).toBe(201);
  });
}

# test_webhook.py — runs in CI on every PR
import httpx, os, pytest

SCENARIOS = ["success", "insufficient_funds", "3ds_challenge"]

@pytest.mark.parametrize("scenario", SCENARIOS)
def test_webhook_handler_returns_200(scenario):
    res = httpx.post(
        "https://www.mockcard.io/api/v1/generate",
        headers={"Authorization": f"Bearer {os.environ['MOCKCARD_API_KEY']}"},
        json={
            "scenario":    scenario,
            "amount":      9900,
            "currency":    "inr",
            "webhook_url": os.environ["WEBHOOK_URL"],
        },
    )
    assert res.status_code == 201

The Pattern That Prevents 2am Pages

The developers who don't get paged at 2am about payment issues share one habit: they treat their webhook handler like a public API. They define its contract (these event types, these shapes), they test every branch of that contract, and they run those tests automatically.

MockCard is the tool that makes the contract testable. The health check on the homepage shows you where the contract is currently broken. The API and the Pro chaos scenarios let you test the entire contract continuously — including the parts that only break under production conditions.

The race condition, the limbo state, the unknown event type — these are predictable failures. Every payment integration will encounter them eventually. The only question is whether you encounter them in a test or in production.