Data Ingestion Recipe Parsing Workflows

Webhooks vs Polling for POS Sync

This page helps a food-tech developer choose how to pull sales out of a POS: subscribe to event webhooks, or poll the API on a schedule. It is a decision companion to POS API polling strategies; read that for the rate-limiting and pagination mechanics, then use this to decide the ingestion trigger — a choice that determines data freshness, load on the vendor, and how you recover from a missed event.

Webhooks push each event as it happens: fresh, low-load, but at the mercy of delivery reliability. Polling pulls on your schedule: predictable and replayable, but higher-latency and rate-limit-bound. In practice, resilient systems combine them.

Prerequisites and Data Contract

  • Python 3.11+, a web framework for the webhook receiver, and the POS API polling client for the poll path.
  • A dedup key on every event (transaction_id + location_id) so both paths are idempotent — the same sale must never be counted twice regardless of how it arrived.
  • Output is a deduplicated transaction feed the menu schema normalization layer consumes.
Webhooks versus polling for POS sync A comparison of two ingestion triggers. Webhooks push each sale event as it happens, giving low latency and low load, but require signature verification, an idempotent receiver, and a strategy for missed or out-of-order deliveries. Polling pulls sales on a fixed schedule, giving predictable and replayable ingestion, but adds latency between the sale and its capture and consumes the vendor rate limit. A hybrid uses webhooks for freshness and a periodic poll as a reconciliation sweep. Webhooks Polling • low latency, event-driven • low steady load on vendor • needs signature verification • missed events need replay • best for freshness • predictable, on your schedule • trivially replayable by window • higher latency • consumes rate limit • best as a safety net

Step-by-Step Implementation

Step 1 — Receive and verify a webhook idempotently

Verify the signature, then dedup on the transaction key before accepting the event.

from __future__ import annotations

import hashlib
import hmac


def verify_and_accept(body: bytes, signature: str, secret: str,
                      seen: set[str]) -> str | None:
    expected = hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, signature):
        raise PermissionError("bad webhook signature")
    event = parse_event(body)
    key = f"{event['transaction_id']}:{event['location_id']}"
    if key in seen:
        return None                      # duplicate delivery, ignore
    seen.add(key)
    return persist_transaction(event)

Step 2 — Poll a window as a replayable sweep

Polling fetches a time window; because it is keyed by window it can be re-run to backfill anything webhooks missed.

from datetime import datetime, timedelta


def poll_window(client, location_id: str, since: datetime,
                until: datetime, seen: set[str]) -> int:
    ingested = 0
    for event in client.list_transactions(location_id, since, until):
        key = f"{event['transaction_id']}:{location_id}"
        if key not in seen:
            seen.add(key)
            persist_transaction(event)
            ingested += 1
    return ingested

Step 3 — Combine them into a resilient hybrid

Use webhooks for freshness and a periodic poll as a reconciliation sweep that closes gaps. Run webhooks as the primary path for near-real-time cost visibility, then poll the trailing few hours on a schedule to catch any event that failed to deliver. Because both paths dedup on the same key, the poll re-ingests only what the webhook missed. This mirrors the resilience posture the POS API polling strategies guide takes toward rate limits: assume the feed will drop something, and build the recovery in.

Verification and Validation

  • Dedup holds across paths. Deliver a transaction by webhook, then poll a window covering it; the transaction must persist exactly once. This is the core guarantee of the hybrid.
  • Signature rejection. Send a webhook with a tampered body; verification must reject it with no persistence.
  • Replay closes gaps. Drop a webhook deliberately (simulate a delivery failure), run the poll sweep, and confirm the missing transaction is backfilled.
  • Rate-limit budget. Confirm the reconciliation poll’s window and cadence stay within the vendor’s limit alongside any other polling.

Gotchas and Edge Cases

  • Out-of-order webhooks. Events can arrive out of sequence or an updated transaction (a voided sale) can follow the original. Key on transaction id and apply the latest version by timestamp, not arrival order.
  • Webhook retries from the vendor. Most vendors retry a webhook until you return 2xx, so a slow handler causes duplicate deliveries. Return quickly, dedup on the key, and process asynchronously.
  • Silent webhook outage. If webhooks stop, cost data quietly goes stale with no error. The reconciliation poll is the safety net — alert if a poll finds an implausible backlog, signaling the webhook path is down.
  • Clock skew on windows. Poll windows use the vendor’s server time; a client-clock window can miss events near the boundary. Overlap windows slightly and rely on dedup to absorb the overlap.

For a reference on HMAC verification, see the official Python hmac documentation.