Webhook Debugging: A Developer's Complete Guide

Why Webhook Debugging Is Hard

Webhooks are the glue of modern software. Stripe sends payment events, GitHub sends push events, Shopify sends order events. But when a webhook doesn't work, debugging it is uniquely painful:

• You can't see the request. It's sent from a third party to your server. Unlike API calls you initiate, you have no control over timing or content.
• Retries mask the problem. Most webhook providers retry failed deliveries. By the time you check, the original error might be buried under successful retries.
• Local development is blind. Your laptop doesn't have a public URL. You can't receive webhooks without a tunnel.
• Payloads vary. The same event type can have different shapes depending on your account configuration, API version, or edge cases the docs don't mention.

The Debugging Workflow

Step 1: Capture the Raw Request

Before you fix anything, you need to see what's actually being sent. Create a temporary endpoint that logs everything:

# Using WebhookVault - instant webhook endpoint, no signup
curl -X POST https://webhookvault.anethoth.com/api/v1/demo/endpoint

# Returns:
# {
#   "endpoint_url": "https://webhookvault.anethoth.com/hook/demo_abc123",
#   "inspect_url": "https://webhookvault.anethoth.com/demo/inspect/abc123",
#   "expires_in": "1 hour"
# }

Point your webhook provider at this URL. Now you can inspect every header, every payload byte, and every response your endpoint would return.

Step 2: Verify the Payload Structure

The most common webhook bug: your code expects a different payload shape than what arrives. Check for:

• Nested objects where you expect flat fields
• Arrays where you expect single objects
• String amounts where you expect numbers (Stripe sends amounts as integers in cents)
• Missing fields in certain event types (not all Stripe events include customer)
• Different field names between test mode and live mode

Step 3: Check Signature Verification

Most webhook providers sign their payloads so you can verify authenticity. This is the second most common failure point after payload parsing.

Common signature verification mistakes:

• Reading the body twice. Many frameworks consume the request body stream on first read. If your middleware reads it before your signature check, the check sees an empty body.
• Wrong encoding. Signature verification requires the raw body bytes, not a parsed JSON object.
• Stale secrets. You rotated the webhook secret in the provider's dashboard but forgot to update your .env file.
• Clock skew. Some providers reject signatures if your server's clock is too far off. Use NTP.

Step 4: Replay Failed Deliveries

Once you've fixed the bug, you need to reprocess the events that failed. Options:

1. Provider replay: Stripe, GitHub, and Shopify all let you re-send events from their dashboards.
2. Endpoint replay: If you captured the original request (via WebhookVault or your own logging), replay it to your fixed endpoint.
3. Idempotency keys: If your handler uses idempotency keys (it should!), replaying is safe — duplicate events are ignored.

Provider-Specific Tips

Stripe Webhooks

• Always verify the stripe-signature header. Use the official SDK: stripe.Webhook.construct_event(payload, sig, secret)
• Handle both checkout.session.completed AND invoice.paid for subscription flows
• Amounts are in cents (integer). 2000 = $20.00
• Test with Stripe CLI: stripe listen --forward-to localhost:8080/webhook

GitHub Webhooks

• Signature is HMAC-SHA256 in the X-Hub-Signature-256 header
• Payload size can be large for push events with many commits — handle gracefully
• The action field differentiates sub-events (e.g., PR opened vs. merged vs. closed)

Shopify Webhooks

• HMAC verification uses X-Shopify-Hmac-Sha256 header with Base64 encoding
• Webhooks are tied to app installations — they stop when the merchant uninstalls
• Mandatory webhooks (GDPR) must be implemented or your app gets rejected

Best Practices for Production Webhooks

1. Return 200 immediately, then process asynchronously. Webhook providers have short timeouts (5-30 seconds). If your processing takes longer, the provider retries, causing duplicate events.
2. Implement idempotency. Store the event ID and skip duplicates. Webhooks are delivered "at least once," not "exactly once."
3. Log everything. Log the full request (minus sensitive data) before processing. When something breaks, you need the original payload.
4. Set up dead letter queues. Events that fail processing after N retries should go somewhere reviewable, not disappear.
5. Monitor delivery health. Track your webhook success rate. A sudden drop from 99% to 90% means something broke.