Reporting Cicd Delivery Pipelines

Failing a Build on Margin-Regression Thresholds

This page shows a food-tech developer how to turn a set of food-cost deltas into a single build verdict with tunable thresholds — deciding which regressions merely warn and which hard-fail the pipeline. It is the decision-logic companion to food-cost diff checks in CI/CD and slots directly behind running food-cost diffs as GitHub Actions gates, which supplies the deltas this logic classifies.

The problem is calibration. Fail on every fractional-cent movement and the gate becomes noise the team disables; fail on nothing and it protects nothing. The answer is a small, explicit policy — per-item and portfolio thresholds stored as data — that maps each delta to pass, warn, or fail and reduces the set to one exit code.

Prerequisites and Data Contract

  • Python 3.11+, a diff frame with columns menu_item_id, food_cost_pct_base, food_cost_pct_new, delta (all Decimal).
  • A thresholds.toml policy file checked into the repo so changes to sensitivity are reviewed like code.
  • The exit-code convention: 0 pass, 1 fail; warnings annotate but do not fail.

Deltas are in food-cost-percentage points (a delta of 0.02 is two points). Thresholds are signed-aware: only regressions (cost up) fail; improvements are reported but never block.

Margin-regression threshold classifier Each per-item delta enters a classifier that checks it against a per-item threshold band and contributes to a portfolio aggregate checked against a portfolio threshold. Each delta is labelled pass, warn, or fail. The worst label across all items determines a single build exit code: fail produces exit one, otherwise exit zero. Per-item deltas signed, in points Classify item + portfolio pass / warn / fail worst label wins Exit code 0 or 1

Step-by-Step Implementation

Step 1 — Declare thresholds as data

# thresholds.toml
[per_item]
warn_pts = 0.01   # 1 point regression -> warn
fail_pts = 0.03   # 3 points regression -> fail

[portfolio]
fail_pts = 0.015  # weighted average regression across basket -> fail

[overrides]
# item-specific tighter bands for high-volume dishes
SKU_SIGNATURE_BURGER = { fail_pts = 0.015 }

Step 2 — Classify each delta

Only positive deltas (cost increases) are regressions. Item overrides win over the global band.

from __future__ import annotations

from decimal import Decimal

import pandas as pd


def classify(df: pd.DataFrame, policy: dict) -> pd.DataFrame:
    df = df.copy()
    warn = Decimal(str(policy["per_item"]["warn_pts"]))
    fail = Decimal(str(policy["per_item"]["fail_pts"]))
    overrides = policy.get("overrides", {})

    def label(row: pd.Series) -> str:
        d = row["delta"]                      # positive = regression
        item_fail = Decimal(str(overrides.get(row["menu_item_id"], {}).get("fail_pts", fail)))
        if d > item_fail:
            return "fail"
        if d > warn:
            return "warn"
        return "pass"

    df["label"] = df.apply(label, axis=1)
    return df

Step 3 — Add a portfolio guard

Many small regressions can each pass the per-item band yet sink the menu’s blended margin. Guard the volume-weighted average too.

from decimal import Decimal

import pandas as pd


def portfolio_breach(df: pd.DataFrame, weights: pd.Series, policy: dict) -> bool:
    fail = Decimal(str(policy["portfolio"]["fail_pts"]))
    w = weights.reindex(df["menu_item_id"]).fillna(Decimal("0"))
    total_w = w.sum()
    if total_w == 0:
        return False
    weighted = (df["delta"].values * w.values).sum() / total_w
    return weighted > fail

Step 4 — Reduce to one exit code

import sys

import pandas as pd


def verdict(labelled: pd.DataFrame, portfolio_fail: bool) -> int:
    if portfolio_fail or (labelled["label"] == "fail").any():
        fails = labelled[labelled["label"].isin(["fail", "warn"])]
        print(fails.to_markdown(index=False))
        return 1
    if (labelled["label"] == "warn").any():
        print("warnings only — not blocking")
        print(labelled[labelled["label"] == "warn"].to_markdown(index=False))
    return 0

Verification and Validation

  • Boundary tests. A delta exactly at fail_pts should not fail (strict >); one cent above should. Add unit tests at the band edges so a policy change cannot silently invert the boundary.
  • Override precedence. Confirm SKU_SIGNATURE_BURGER fails at 1.5 points while a normal item passes until 3, proving overrides win.
  • Portfolio-only failure. Construct a basket where every item warns but none fails individually, yet the weighted average breaches — the verdict must be 1. This proves the portfolio guard is wired.
  • Improvements never fail. A large negative delta (cost dropped) must classify pass; only regressions block.

Gotchas and Edge Cases

  • Sign confusion. A regression is cost increasing, i.e. food_cost_pct_new > base, a positive delta. Comparing abs(delta) would fail on genuine improvements and erode trust in the gate. Keep the comparison signed.
  • Death by a thousand cuts. Per-item bands alone miss a change that nudges fifty dishes up half a point each. The portfolio guard exists precisely to catch coordinated small regressions.
  • Threshold drift in code. Hard-coding thresholds in the script means every sensitivity tweak is a code review of logic rather than policy. Keep them in thresholds.toml so the diff shows a number changing, reviewable on its own.
  • Weighting by stale volumes. Portfolio weights should come from recent sales, refreshed periodically; weighting by a year-old mix can under-weight a now-popular dish and let its regression slip through.

For language specifics, see the official Python decimal documentation.