Scheduled Report Distribution

Rendering a report is not delivering it. Within the reporting, CI/CD and delivery pipelines domain, this guide isolates the last step: taking a validated artifact and getting it to the right person, in the right channel, at the right time, exactly once — across a multi-location estate where a single failed send must never block the other two hundred. The operational failure this eliminates is the report that was generated perfectly and then never arrived, or arrived twice, or arrived in the wrong franchisee’s inbox carrying another operator’s margins.

Concept Definition and Data Contract

Distribution consumes two inputs and emits delivery records. The first input is a rendered artifact with its metadata (report_id, run_version, period, location_scope, content_hash) produced by batch cost reporting automation. The second is a routing table: for each location_id and consumer role, which channel (email address, chat webhook), which timezone the cadence respects, and which redaction profile applies. The output is a sent-ledger record per delivery: report_id, recipient, channel, status, attempts, and a timestamp.

The routing table is data, not code. Adding a location or changing a recipient is a row edit, reviewed like any other config, keyed on the same multi-location cost center architecture the cost engine uses so that “the estate” means the same thing everywhere.

Architecture Decision Rationale

One artifact, N deliveries. The report is rendered once and delivered many times. Re-rendering per recipient would let two people receive subtly different numbers for the same period. The distributor never touches content; it routes an immutable artifact identified by its content hash.

Idempotency by sent-ledger, not by hope. A scheduler that fires twice, a worker that crashes mid-batch and restarts, a manual re-run after a partial failure — all are normal. The distributor checks a sent-ledger keyed on (report_id, recipient, channel) before every send, so redelivery is a no-op rather than a duplicate in someone’s inbox.

Isolated fan-out. Each recipient’s delivery is an independent job. One bad webhook or bounced address must dead-letter on its own without failing the batch, because a distribution job that aborts on the first error delivers nothing to anyone. The channel-specific mechanics are worked through in distributing cost reports via email and Slack.

Phase 1 — Resolve Routing and Build Send Jobs

The first phase joins the artifact to the routing table, applies RBAC redaction, and produces one send job per recipient — none of which touches a network yet.

from __future__ import annotations

from dataclasses import dataclass

import pandas as pd


@dataclass(frozen=True)
class SendJob:
    report_id: str
    recipient: str
    channel: str
    location_scope: str
    redaction: str


def build_jobs(artifact_meta: dict, routing: pd.DataFrame) -> list[SendJob]:
    """Expand one artifact into per-recipient send jobs via the routing table."""
    scoped = routing[routing["location_id"].isin(artifact_meta["location_scope"])]
    return [
        SendJob(
            report_id=artifact_meta["report_id"],
            recipient=row.recipient,
            channel=row.channel,
            location_scope=row.location_id,
            redaction=row.redaction_profile,
        )
        for row in scoped.itertuples()
    ]

Filtering routing rows to the artifact’s location_scope is the RBAC boundary: a regional manager’s job set contains only their locations, so the fan-out physically cannot address a report to someone outside its scope.

Phase 2 — Idempotent Send with Bounded Retries

Each job checks the sent-ledger, sends through the channel adapter, and handles failure by class. Transient errors retry with backoff; permanent errors dead-letter.

from __future__ import annotations

import time

from costing.channels import ChannelError, TransientError, get_adapter
from costing.ledger import already_sent, record_send


def deliver(job, artifact_path: str, max_attempts: int = 4) -> str:
    if already_sent(job.report_id, job.recipient, job.channel):
        return "skipped_duplicate"

    adapter = get_adapter(job.channel)
    for attempt in range(1, max_attempts + 1):
        try:
            adapter.send(job.recipient, artifact_path, redaction=job.redaction)
            record_send(job.report_id, job.recipient, job.channel, status="sent",
                        attempts=attempt)
            return "sent"
        except TransientError:
            if attempt == max_attempts:
                break
            time.sleep(2 ** attempt)          # exponential backoff
        except ChannelError as exc:
            record_send(job.report_id, job.recipient, job.channel,
                        status="dead_letter", attempts=attempt, error=str(exc))
            return "dead_letter"

    record_send(job.report_id, job.recipient, job.channel,
                status="dead_letter", attempts=max_attempts, error="retries_exhausted")
    return "dead_letter"

The distinction between TransientError (a 503, a timeout — worth retrying) and ChannelError (a revoked webhook, a malformed address — never worth retrying) is what keeps the system from hammering a permanently broken target while still surviving a flaky one.

Phase 3 — Batch Fan-Out and Reconciliation

The batch runs every job independently and reconciles the outcome so a partial failure is visible, not swallowed.

from concurrent.futures import ThreadPoolExecutor

import pandas as pd


def run_batch(jobs: list, artifact_path: str) -> pd.DataFrame:
    with ThreadPoolExecutor(max_workers=8) as pool:
        outcomes = list(pool.map(lambda j: (j.recipient, deliver(j, artifact_path)), jobs))
    summary = pd.DataFrame(outcomes, columns=["recipient", "status"])
    dead = summary[summary["status"] == "dead_letter"]
    if not dead.empty:
        # Surface, do not raise — the other deliveries already succeeded.
        print(f"{len(dead)} deliveries dead-lettered:\n{dead.to_markdown(index=False)}")
    return summary

Running deliveries concurrently but independently means one location’s failure is a row in the summary, not an exception that aborts the estate-wide run. The dead-letter set is surfaced for a human to act on — a revoked Slack webhook is a config fix, not a data problem.

Production Hardening

• Idempotency ledger. Key sends on (report_id, recipient, channel) and check before every dispatch, so retries and double-triggers never duplicate a delivery.
• Dead-letter, don’t loop. Permanent failures land in a review queue with their error; only transient failures retry, and only with bounded backoff.
• RBAC at routing time. Scope the routing join to the artifact’s locations and apply a redaction profile per role, so a recipient can never receive margins they are not entitled to see.
• Timezone-correct cadence. Respect each location’s timezone when deciding when “weekly” fires, so a franchise in a different region is not paged at 3 a.m.
• Structured send logs. Emit report_id, recipient, channel, and attempts on every outcome so a “my report never arrived” ticket is answered from the ledger, not from guesswork.

Failure Modes and Troubleshooting

FAQ

Why render once and deliver many times instead of per recipient?

Because two renders of the same period can diverge — a template tweak, a late data change — and then two recipients hold different “official” numbers. Rendering once produces an immutable artifact identified by a content hash; distribution routes that exact artifact to everyone, so every recipient sees identical figures and the delivery can be reconciled against the render.

How does the system guarantee exactly-once delivery?

Through a sent-ledger keyed on (report_id, recipient, channel). Before any send, the distributor checks whether that combination was already delivered; if so, the send is skipped. Because generation and rendering are idempotent, a retried or double-fired batch resolves to the same jobs and the ledger short-circuits the duplicates.

What happens when one recipient's channel is broken?

That job fails in isolation. A transient error (timeout, 503) retries with bounded backoff; a permanent one (revoked webhook, bad address) is written to a dead-letter queue with its error and surfaced for a human to fix. The rest of the batch delivers normally — one broken target never blocks the estate.

How is a franchisee prevented from seeing another operator's numbers?

RBAC is applied at routing time. The routing join is scoped to the artifact’s location_scope, so a recipient’s job set only ever contains their own locations, and a per-role redaction profile strips fields they are not entitled to. The fan-out is structurally incapable of addressing a report outside a recipient’s scope.

Related

• Up one level: Reporting, CI/CD & Delivery Pipelines
• Distributing Cost Reports via Email and Slack
• Batch Cost Reporting Automation
• Multi-Location Cost Center Architecture
• Async Batch Processing Workflows

For channel specifics, see the official Python smtplib documentation.