--- title: "Webhook Troubleshooting Guide" description: "Solutions to common webhook problems and debugging strategies." sidebar_position: 5 last_updated: 2026-08-20 --- # Webhook Troubleshooting Guide Solutions to common webhook problems and debugging strategies. ## Debugging Tools ### Dashboard Delivery Attempts The bex.co dashboard provides powerful debugging tools: 1. Navigate to **Accept Payments** → **Webhooks** 2. Click on an endpoint to view its delivery attempts 3. Expand any attempt to see: - Full request body sent by bex.co - Full response body from your server - HTTP status code - Error messages - Response time - Retry schedule ### Replay Failed Events Don't worry if a webhook delivery fails - you can replay it: 1. Find the failed attempt in the dashboard 2. Click **Replay Event** button 3. The event will be resent immediately 4. Check if it succeeds this time ### Test Your Endpoint Use the **Test** button to send a sample event: 1. Click **Test** on your endpoint 2. A `webhook.test.event` is sent with sample data 3. Verify your server receives and processes it correctly 4. Check server logs and dashboard for results ## Common Issues ### Issue: "Invalid Signature" (401 Unauthorized) **Symptoms:** - Your endpoint returns HTTP 401 - Logs show "Invalid signature" or "Unauthorized" - All webhook deliveries fail with signature errors **Cause 1: Using Parsed JSON Instead of Raw Body** This is the **most common mistake**. You must verify the signature against the raw request body, not the parsed JSON. :::danger Wrong ```javascript // WRONG - Body is already parsed and modified app.use(express.json()); app.post('/webhooks', (req, res) => { const body = JSON.stringify(req.body); // This won't match! const signature = crypto .createHmac('sha256', secret) .update(body) .digest('hex'); // Verification will fail }); ``` ::: :::tip Correct ```javascript // CORRECT - Use raw body app.post('/webhooks', express.raw({ type: 'application/json' }), (req, res) => { const body = req.body; // Raw Buffer const signature = crypto .createHmac('sha256', secret) .update(body) .digest('hex'); // Verification will succeed } ); ``` ::: **Cause 2: Wrong Secret** You may be using an old secret or the wrong endpoint's secret. **Solution:** 1. Go to dashboard → Click **Rotate Secret** on your endpoint 2. **Copy the new secret immediately** (it's only shown once) 3. Update your environment variable: `WEBHOOK_SECRET=your_new_secret` 4. Restart your server 5. Click **Test** to verify it works **Cause 3: Whitespace or Encoding Issues** Ensure you're using the exact raw bytes without modification. ```javascript // Correct approach const signature = crypto .createHmac('sha256', secret) .update(req.body) // Must be Buffer, not modified string .digest('hex'); ``` ### Issue: Webhooks Timing Out **Symptoms:** - Dashboard shows "Timeout" errors - No response received after 30 seconds - Events are automatically retried **Cause 1: Endpoint Takes Too Long to Respond** Your handler is doing too much work synchronously. :::danger Wrong ```javascript // WRONG - Slow synchronous processing app.post('/webhooks', async (req, res) => { const event = parseAndVerify(req); // This takes 60 seconds - too slow! await processOrder(event); await sendEmails(event); await updateAnalytics(event); res.status(200).send('OK'); // Too late - already timed out }); ``` ::: :::tip Correct ```javascript // CORRECT - Respond quickly, process async app.post('/webhooks', async (req, res) => { const event = parseAndVerify(req); // Respond immediately (within 1 second) res.status(200).json({ received: true }); // Process in background processWebhookAsync(event).catch(err => { console.error('Background processing error:', err); // Log to error tracking service }); }); async function processWebhookAsync(event) { // Heavy processing here await processOrder(event); await sendEmails(event); await updateAnalytics(event); } ``` ::: **Cause 2: Endpoint Not Accessible** Your endpoint might be blocked by a firewall, behind a VPN, or using localhost. **Solutions:** - Ensure endpoint is **publicly accessible** over the internet - Use **HTTPS** (HTTP is not supported) - Check cloud provider **security groups** allow inbound HTTPS - For local testing, use [ngrok](https://ngrok.com/): ```bash # Terminal 1: Start your server npm start # Terminal 2: Create secure tunnel ngrok http 3000 # Output: Forwarding https://abc123.ngrok.io -> http://localhost:3000 # Use the ngrok HTTPS URL in bex.co dashboard ``` ### Issue: Missing Webhooks **Symptoms:** - Some events aren't received - Expected webhook never arrived - Inconsistent delivery **Cause 1: Not Subscribed to Event Type** You're not subscribed to that specific event type. **Solution:** 1. Go to dashboard → **Edit** your endpoint 2. Check the boxes for event types you want to receive 3. Click **Save** **Cause 2: Endpoint Disabled** The endpoint was automatically disabled due to repeated failures. **Solution:** 1. Check if endpoint shows **"Disabled"** badge in dashboard 2. Review failure reason in delivery attempts table 3. Fix the underlying issue (signature, timeout, etc.) 4. Click **Edit** → Toggle **"Endpoint active"** to re-enable 5. Click **Test** to verify it works **Cause 3: Event Already Sent** bex.co doesn't send duplicate events - this is correct behavior. **Solution:** Each event is sent once per endpoint. Check your server logs or database to confirm if you actually processed it. ### Issue: Duplicate Event Processing **Symptoms:** - Same order fulfilled twice - Customer charged multiple times - Duplicate database records **Cause: Not Implementing Idempotency** Your handler doesn't check if an event was already processed. :::danger Wrong ```javascript // WRONG - No duplicate check async function handleWebhook(event) { // Process immediately without checking await fulfillOrder(event.data.paymentId); } ``` ::: :::tip Correct ```javascript // CORRECT - Check before processing async function handleWebhook(event) { // Check if already processed using event ID const existing = await db.webhookEvents.findOne({ eventId: event.id }); if (existing) { console.log('Event already processed:', event.id); return; // Skip duplicate } // Process the event await fulfillOrder(event.data.paymentId); // Mark as processed await db.webhookEvents.create({ eventId: event.id, type: event.type, processedAt: new Date() }); } ``` ::: ### Issue: "Connection Refused" Errors **Symptoms:** - Dashboard shows "Connection refused" - Error message: "Cannot connect to endpoint" - Immediate failure (not timeout) **Cause 1: Server Not Running** Your webhook server is down or crashed. **Solution:** - Check your server is running: `systemctl status your-service` - Verify it's listening on the correct port - Check server logs for startup errors - Use process manager like PM2 or systemd for auto-restart **Cause 2: Wrong URL or Port** The webhook URL doesn't match your actual endpoint. **Solution:** - Verify URL in dashboard matches your server - Check the path: `/webhooks` vs `/webhook` vs `/api/webhooks` - Ensure using **HTTPS** (HTTP not supported) - Test with curl: ```bash curl -X POST https://your-domain.com/api/webhooks \ -H "Content-Type: application/json" \ -d '{"test":"data"}' ``` **Cause 3: Firewall Blocking Requests** Corporate firewall or cloud security group is blocking inbound traffic. **Solution:** - Allow incoming HTTPS traffic (port 443) - Check cloud provider security groups (AWS, GCP, Azure) - Verify network ACLs - Contact support for bex.co IP ranges to whitelist ### Issue: 500 Internal Server Error **Symptoms:** - Your endpoint returns HTTP 500 - bex.co automatically retries the event - Errors in your server logs **Cause 1: Unhandled Exception** Your code throws an error that crashes the request handler. :::danger Wrong ```javascript // WRONG - Unhandled errors crash the handler app.post('/webhooks', async (req, res) => { const event = parseAndVerify(req); await processEvent(event); // If this throws, 500 is returned res.status(200).send('OK'); }); ``` ::: :::tip Correct ```javascript // CORRECT - Proper error handling app.post('/webhooks', async (req, res) => { try { const event = parseAndVerify(req); // Always return 200 first (even if processing fails) res.status(200).json({ received: true }); // Process with error handling await processEvent(event); } catch (error) { console.error('Webhook processing error:', error); // Log to error tracking service (Sentry, Datadog, etc.) // Don't throw - webhook was already acknowledged } }); ``` ::: **Cause 2: Database Connection Error** Can't connect to your database. **Solution:** - Check database is running and accessible - Verify connection string - Check network connectivity - Handle database errors gracefully: ```javascript try { await db.orders.create(orderData); } catch (dbError) { console.error('Database error:', dbError); // Still return 200 - retry won't help DB issues } ``` ### Issue: Events Out of Order **Symptoms:** - Receive `payment.confirmed` before `payment.pending` - Events arrive in unexpected sequence - State transitions seem wrong **Cause: Network Delays or Retries** Events may arrive out of order due to network conditions or retries. **Solution: Use Status Field, Not Event Order** Don't rely on event arrival order. Always check the current status: ```javascript async function handlePaymentEvent(event) { const payment = event.data; // Get current status from your database const existingPayment = await db.payments.findOne({ id: payment.paymentId }); // Only update if new status is "higher" than current const statusPriority = { 'pending': 1, 'confirmed': 2, 'failed': 3, 'refunded': 4 }; if (statusPriority[payment.status] > statusPriority[existingPayment.status]) { await db.payments.update(payment.paymentId, { status: payment.status, updatedAt: new Date() }); } else { console.log('Ignoring older event:', event.type); } } ``` ## Best Practices ### 1. Always Verify Signatures Never skip signature verification: ```javascript const signature = req.headers['x-eden-signature']; if (!verifySignature(req.body, signature, WEBHOOK_SECRET)) { return res.status(401).send('Unauthorized'); } ``` ### 2. Return 200 Within 1 Second Acknowledge receipt immediately: ```javascript // Return 200 in < 1 second res.status(200).json({ received: true }); // Process in background (can take minutes) processAsync(event); ``` ### 3. Handle Idempotency Prevent duplicate processing: ```javascript if (await alreadyProcessed(event.id)) { console.log('Duplicate event, skipping'); return; } ``` ### 4. Log Everything Comprehensive logging helps debugging: ```javascript console.log('Webhook received:', { eventId: event.id, type: event.type, paymentId: event.data.paymentId, status: event.data.status, timestamp: new Date().toISOString() }); ``` ### 5. Monitor Failures Set up alerts for webhook failures: - Check dashboard regularly - Set up alerts for high failure rates (>10%) - Replay failed events after fixing issues - Monitor success rate badge on endpoints ### 6. Handle Unknown Events New event types may be added in the future: ```javascript switch (event.type) { case 'payment.confirmed': await handlePaymentConfirmed(event.data); break; case 'payment.failed': await handlePaymentFailed(event.data); break; default: // Log but don't fail for unknown event types console.log('Unknown event type:', event.type); } ``` ## Pre-Launch Testing Checklist Before going live, verify your webhook endpoint: - [ ] Can receive test webhook from dashboard - [ ] Signature verification works correctly - [ ] Returns 200 OK within 1 second - [ ] Logs event details for debugging - [ ] Handles unknown event types gracefully - [ ] Doesn't process same event twice (idempotency) - [ ] Works with production HTTPS URL - [ ] Error handling doesn't crash server - [ ] Background processing completes successfully - [ ] Database updates work correctly ## Local Development Testing ### Using ngrok ```bash # Terminal 1: Start your development server npm run dev # Terminal 2: Create secure tunnel ngrok http 3000 # Output: # Forwarding https://abc123.ngrok.io -> http://localhost:3000 # Use the ngrok HTTPS URL in bex.co dashboard ``` ### Using localtunnel ```bash # Start your server npm start # Create tunnel (no account needed) npx localtunnel --port 3000 # Output: your url is: https://random-name.loca.lt # Use this URL in bex.co dashboard ``` ## Getting Help ### Dashboard Debugging Use the dashboard's built-in debugging tools: 1. **View Delivery Attempts** - See full request/response details 2. **Check Failure Reasons** - Understand why deliveries failed 3. **Replay Failed Events** - Resend events after fixing issues 4. **Test Endpoint** - Send sample events manually ### Check Server Logs Always check your server logs for errors: ```bash # For PM2 pm2 logs your-app-name # For systemd journalctl -u your-service-name -f # For Docker docker logs container-name -f ``` ### Contact Support If you're still stuck, contact support with: **Email**: hello@bex.co **Include**: - Endpoint ID from dashboard - Event ID (from failed delivery) - Error message from your server logs - Delivery attempt details from dashboard (screenshot) - Your webhook handler code (without secrets) **Response Time**: Usually within 24 hours ### Community Support - **Forum**: [https://bex.co/forum](https://bex.co/forum) - **Discord**: [https://discord.gg/eWZvE4RSBw](https://discord.gg/eWZvE4RSBw) - **GitHub Discussions**: Share integration examples ## Next Steps - 📚 **Learn More**: Read the [Quick Start Guide](./quickstart.md) - 🔒 **Security**: Review [Signature Verification](./signatures.md) - 📖 **Events**: Check [Event Reference](./events.md) for all event types - 🏠 **Overview**: Back to [Webhooks Overview](./index.md)