Webhook Service

Referenced Files in This Document - webhook.py - webhook.py - scanner.py - sweeper.py - config.py - init.py - payments.py - payment.py - payment.py - README.md

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction

This document describes the Webhook Service that notifies external systems about payment status changes. It covers the webhook delivery mechanism, payload formatting, HMAC signature verification, integration with ARQ workers for asynchronous processing, retry behavior, configuration of webhook URLs and secrets, and payload validation. It also documents supported events, error handling strategies, and operational guidance for testing and security.

Project Structure

The webhook capability spans several modules: - Service layer for outbound webhook delivery - ARQ task for asynchronous processing - Blockchain scanner that triggers webhooks upon payment confirmation - Configuration for webhook URL and secret - Worker broker initialization for ARQ

graph TB subgraph "Configuration" CFG["Settings
webhook_url, webhook_secret"] end subgraph "Workers" BROKER["ARQ Redis backend
Redis"] ACTOR["Actor send_webhook_task"] end subgraph "Services" SVC["WebhookService
send_webhook"] SCN["ScannerService
confirm_payments"] end subgraph "External" ENDPT["Webhook Endpoint"] end CFG --> ACTOR BROKER <- --> ACTOR SCN --> ACTOR ACTOR --> SVC SVC --> ENDPT

Diagram sources - config.py - init.py - webhook.py - webhook.py - scanner.py

Section sources - config.py - init.py - webhook.py - webhook.py - scanner.py

Core Components

  • WebhookService: Asynchronous HTTP client that sends JSON payloads and optional HMAC signatures.
  • ARQ Task: Background task that invokes WebhookService and leverages built-in retries.
  • ScannerService: Detects and confirms payments; emits webhooks when a payment reaches the confirmed state.
  • Configuration: Provides webhook URL and secret for signing.

Key responsibilities: - Delivery: Async HTTP POST with JSON body and optional X-Webhook-Signature header. - Signing: HMAC-SHA256 over the JSON payload using the configured secret. - Retry: ARQ task configured with a fixed number of retries. - Trigger: Webhook emission occurs when a payment transitions to confirmed.

Section sources - webhook.py - webhook.py - scanner.py - config.py

Architecture Overview

The webhook pipeline integrates blockchain scanning, internal state transitions, and asynchronous delivery via ARQ.

sequenceDiagram participant Scanner as "ScannerService" participant DB as "Database" participant Actor as "ARQ Task" participant Service as "WebhookService" participant Endpoint as "External Webhook Endpoint" Scanner->>DB : "Update payment status to confirmed" Scanner->>Actor : "send_webhook_task(url, payload, secret)" Actor->>Service : "send_webhook(url, payload, secret)" Service->>Endpoint : "POST JSON body
Header : X-Webhook-Signature (optional)" Endpoint-->>Service : "HTTP 2xx" Service-->>Actor : "Success" Actor-->>Scanner : "Task processed"

Diagram sources - scanner.py - webhook.py - webhook.py

Detailed Component Analysis

WebhookService

Responsibilities: - Serialize payload to JSON. - Optionally compute HMAC-SHA256 signature and attach X-Webhook-Signature header. - Perform async HTTP POST with a timeout. - Log successes and failures.

Behavior: - On HTTP status errors, logs the status code. - On exceptions, logs the error and returns failure.

flowchart TD Start(["send_webhook(url, payload, secret)"]) --> Serialize["Serialize payload to JSON"] Serialize --> Headers["Set Content-Type: application/json"] Headers --> HasSecret{"Secret provided?"} HasSecret --> |Yes| Sign["Compute HMAC-SHA256(signature)"] Sign --> AddHeader["Add X-Webhook-Signature header"] HasSecret --> |No| SkipSign["Skip signing"] AddHeader --> Request["POST to url with JSON body"] SkipSign --> Request Request --> Resp{"HTTP 2xx?"} Resp --> |Yes| LogOk["Log success and return True"] Resp --> |No| LogErr["Log HTTP error and return False"] Request --> Except{"Exception?"} Except --> |Yes| LogEx["Log exception and return False"] LogOk --> End(["Done"]) LogErr --> End LogEx --> End

Diagram sources - webhook.py

Section sources - webhook.py

ARQ Task: send_webhook_notification

Responsibilities: - Send webhook notification for a specific payment event (used by admin API). - Wraps WebhookService.send_webhook() for on-demand delivery.

Note: For automatic webhook delivery during payment detection and confirmation, ScannerService._dispatch_webhook() calls WebhookService.send_webhook() directly — no separate task is enqueued for these events. - Invoke WebhookService and raise on failure to trigger ARQ retries. - Log actor-level errors and re-raise to allow broker retry policy.

Notes: - Actor is configured with a fixed number of retries. - The actor sets up an event loop for async operations.

sequenceDiagram participant Broker as "ARQ Redis backend" participant Actor as "send_webhook_task" participant Loop as "Event Loop" participant Service as "WebhookService" Broker->>Actor : "Invoke with (url, payload, secret)" Actor->>Loop : "run_until_complete(run())" Actor->>Service : "send_webhook(url, payload, secret)" alt Success Service-->>Actor : "True" Actor-->>Broker : "Acknowledge" else Failure Service-->>Actor : "False" Actor->>Actor : "Raise Exception" Actor-->>Broker : "Retry per policy" end

Diagram sources - webhook.py - webhook.py

Section sources - webhook.py

Webhook Triggering: ScannerService.confirm_payments

Responsibilities: - Transition payments to confirmed after sufficient confirmations. - Build a standardized payload for the webhook. - Enqueue the ARQ task with configured URL and secret.

Payload fields emitted: - payment_id - status - address - amount - chain - token_id (optional)

flowchart TD ScanStart["Scan detected payments"] --> Confirm["Compute confirmations
Compare with threshold"] Confirm --> Confirmed{"Reached required confirmations?"} Confirmed --> |Yes| SetStatus["Set status to confirmed"] SetStatus --> BuildPayload["Build payload:
payment_id, status, address,
amount, chain, token_id"] BuildPayload --> CheckURL{"webhook_url configured?"} CheckURL --> |Yes| Enqueue["Enqueue send_webhook_task(url, payload, secret)"] CheckURL --> |No| Skip["Skip webhook"] Confirmed --> |No| Wait["Wait for next scan"] Enqueue --> Done["Commit and return"] Skip --> Done Wait --> Done

Diagram sources - scanner.py - config.py

Section sources - scanner.py - config.py

Payload Validation and Supported Events

Supported event type: - payment.confirmed

Payload fields: - payment_id: UUID string - status: "confirmed" - address: destination address - amount: numeric amount as string - chain: blockchain identifier - token_id: UUID string or null

Validation behavior: - The service does not validate payload fields; it forwards whatever is provided to the actor. - The actor and service do not enforce strict schema checks; downstream consumers should validate.

Section sources - scanner.py - payment.py

Security Verification Using HMAC Signatures

Signature computation: - HMAC-SHA256 over the JSON payload. - Header: X-Webhook-Signature.

Verification steps (consumer-side): - Recompute HMAC-SHA256 over the received JSON body using the shared secret. - Compare the computed value with the header value. - Reject if mismatch or header missing.

Notes: - Signature is optional; if no secret is configured, no header is attached. - Consumers must independently manage and protect the shared secret.

Section sources - webhook.py - config.py

Webhook URL Configuration and Secret Management

Configuration keys: - webhook_url: Global URL for payment notifications. - webhook_secret: Secret used to sign payloads.

Environment and defaults: - Values are loaded from settings; defaults are None. - Production environments should override defaults via environment variables.

Operational guidance: - Set webhook_url to the target endpoint. - Set webhook_secret to a strong secret value. - Keep secrets out of source control; use environment variables.

Section sources - config.py

Integration with ARQ Workers

Worker startup: - Redis broker is initialized from settings. - Workers are started with arq CLI pointing to the webhook module.

Actor configuration: - send_webhook_task is decorated with max_retries to enable automatic retries.

Section sources - init.py - webhook.py - README.md

Error Handling Strategies

  • HTTP errors: Logged with status code; returns failure.
  • Exceptions: Logged; returns failure.
  • Actor-level failures: Raise to trigger ARQ retries.

Recommended improvements (not currently implemented): - Exponential backoff with jitter for retries. - Dead letter queue handling for persistent failures. - Idempotency keys to prevent duplicate processing.

Section sources - webhook.py - webhook.py

Payload Formatting and Event Types

Event type: - payment.confirmed

Payload fields: - payment_id: string (UUID) - status: "confirmed" - address: string - amount: string (numeric) - chain: string - token_id: string or null

Note: - The current implementation emits only the confirmed event. - Additional events (e.g., created, failed) are not emitted by the scanner.

Section sources - scanner.py - payment.py

Webhook Endpoint Testing Procedures

Recommended testing steps: - Use a local webhook receiver (e.g., ngrok) to capture requests. - Verify JSON body and presence of X-Webhook-Signature when secret is set. - Simulate network failures and timeouts to exercise retry behavior. - Test signature verification using the same shared secret.

Operational tips: - Start workers before generating payments to ensure timely delivery. - Monitor logs for HTTP errors and actor exceptions.

Section sources - README.md - webhook.py

Dependency Analysis

High-level dependencies: - ScannerService depends on Settings for webhook configuration and enqueues the actor. - Actor depends on WebhookService for delivery. - WebhookService depends on httpx for HTTP operations. - ARQ Redis backend is initialized from settings and used by the actor.

graph LR Settings["Settings
webhook_url, webhook_secret"] --> Actor["send_webhook_task"] Settings --> Scanner["ScannerService"] Scanner --> Actor Actor --> Service["WebhookService"] Service --> HTTP["httpx.AsyncClient"] Settings --> Broker["ARQ Redis Broker"] Broker <- --> Actor

Diagram sources - config.py - scanner.py - webhook.py - webhook.py - init.py

Section sources - config.py - scanner.py - webhook.py - webhook.py - init.py

Performance Considerations

  • Async HTTP client reduces blocking overhead.
  • Fixed retry count in actor limits resource consumption on repeated failures.
  • Consider adding exponential backoff and jitter to reduce thundering herd effects.
  • Tune broker and worker concurrency based on traffic volume.

[No sources needed since this section provides general guidance]

Troubleshooting Guide

Common issues and resolutions: - Webhook delivery fails with HTTP error: - Check endpoint availability and response codes. - Review service logs for detailed error messages. - Signature verification fails: - Ensure the consumer uses the same shared secret. - Verify the header X-Webhook-Signature is present when secret is configured. - Actor keeps retrying: - Confirm the actor’s max_retries setting. - Investigate persistent failures and adjust policies. - No webhooks emitted: - Verify webhook_url is configured. - Ensure payments reach the confirmed state.

Section sources - webhook.py - webhook.py - config.py

Conclusion

The Webhook Service provides a focused, asynchronous notification mechanism for payment confirmations. It supports HMAC-signed payloads, integrates with ARQ for reliable delivery, and can be extended to support additional events and improved retry strategies. Proper configuration of webhook URL and secret, combined with consumer-side signature verification, ensures secure and reliable integrations.

[No sources needed since this section summarizes without analyzing specific files]

Appendices

Appendix A: Supported Events and Payloads

  • Event: payment.confirmed
  • Fields: payment_id, status, address, amount, chain, token_id (optional)

Section sources - scanner.py - payment.py

Appendix B: Configuration Reference

  • webhook_url: Target endpoint URL for webhooks.
  • webhook_secret: Shared secret for HMAC signing.

Section sources - config.py

Appendix C: Worker Startup and Commands

  • Start workers using the arq command with the webhook module.

Section sources - README.md