Yield Factor Calculation Frameworks

In multi-unit foodservice operations, the gap between as-purchased (AP) weight and edible-portion (EP) weight is the single largest source of untracked margin erosion: a case of whole tenderloin invoiced at one price feeds the plate at a very different one once trim, thermal shrink, and moisture loss are accounted for. This guide, part of the Core Architecture & Cost Mapping Systems framework, isolates one sub-problem: how to model yield factors as a deterministic, versioned pipeline so that every ingredient resolves from an invoice line to an auditable EP unit cost that downstream costing can trust. It defines the data contract, explains why the calculation runs as a stateless batch rather than an ad-hoc spreadsheet, and walks the three-phase build that turns raw scale exports into a clean cost ledger. The category-specific mechanics of trim and shrink are covered in the companion walkthrough on calculating trim and yield factors for produce; here we define the framework those numbers flow through.

Concept Definition and Data Contract

The foundational identity is trivial to state — yield_factor = ep_weight / ap_weight — and deceptively hard to operationalize. A yield factor is not a static constant per ingredient; it is a time series per SKU per location, drifting with season, vendor grade, butcher skill, and equipment. The framework’s job is to accept noisy daily observations and emit one dependable factor per SKU that the rest of the cost architecture can multiply against.

The input contract is deliberately narrow. Each observation is one weighing event:

• sku_id — the canonical purchase SKU, identical to the leaf nodes in your recipe BOM graph so the two systems reconcile.
• location_id — the unit that produced the measurement, because yields legitimately differ across kitchens.
• observed_at — a UTC timestamp; the smoothing step requires an ordered series.
• ap_weight_kg, ep_weight_kg — the raw and edible weights in a canonical base unit (kilograms), converted at ingestion, never at calculation time.
• ap_cost — the as-purchased cost for that lot, carried as an exact Decimal, never a float.
• location_variance — an optional multiplier for regional prep standards, defaulting to 1.

The output contract is equally strict: for each (sku_id, location_id), the pipeline returns a smoothed_yield in (0, 1], an adjusted_ep_cost as a quantized Decimal, and an is_anomalous flag. Anything that cannot satisfy those bounds is not silently corrected — it is routed to a quarantine table and excluded from the ledger, so a bad measurement never propagates a fabricated cost.

Architecture Decision Rationale

Three design decisions shape this framework, and each was chosen over a tempting alternative.

Batch smoothing over per-event costing. The naive approach costs each plate against the most recent single yield observation. That couples plate cost to sensor noise: one mis-tare on a digital scale swings the EP cost of every dish using that ingredient. Instead the framework treats yield as a smoothed signal, applying an exponentially weighted moving average (EWMA) per SKU so that a single outlier is dampened, not amplified. Costing reads the smoothed factor, not the raw event.

Vectorized pandas over row iteration. Reconciling tens of thousands of SKUs across dozens of locations nightly is throughput-bound. Row-by-row iterrows loops are both slow and error-prone; the entire calculation is expressed as vectorized groupby/transform operations that the underlying array engine can execute in bulk.

Decimal at the monetary boundary over float throughout. Yield ratios are dimensionless and tolerate IEEE-754 arithmetic during smoothing. Money does not. The moment a smoothed yield is divided into an AP cost to produce an EP cost, the arithmetic switches to Python’s decimal module and is quantized once at the boundary, matching a NUMERIC(12,4) column in the ledger. This prevents fractional-cent drift from accumulating across a portfolio of millions of plate deductions.

Phase 1 — Ingestion and Normalization

The first phase parses raw scale exports and invoice lines into a validated, canonically-unitted record set. A Pydantic v2 model enforces the contract at the boundary so that malformed rows fail loudly here rather than corrupting a calculation three steps downstream.

from __future__ import annotations

from datetime import datetime
from decimal import Decimal

from pydantic import BaseModel, ConfigDict, Field, field_validator


class YieldObservation(BaseModel):
    """One weighing event: raw and edible weight for a single lot."""

    model_config = ConfigDict(frozen=True, extra="forbid")

    sku_id: str
    location_id: str
    observed_at: datetime
    ap_weight_kg: float = Field(gt=0)          # zero AP => divide-by-zero, reject
    ep_weight_kg: float = Field(ge=0)
    ap_cost: Decimal = Field(gt=0)
    location_variance: Decimal = Field(default=Decimal("1"), gt=0)

    @field_validator("ap_cost", "location_variance", mode="before")
    @classmethod
    def _as_decimal(cls, v: object) -> Decimal:
        # Route money through str() so float literals never taint the Decimal.
        return Decimal(str(v))

Note the gt=0 constraint on ap_weight_kg: the classic yield bug is a divide-by-zero when a scale reports 0 after a failed tare, and pushing that guard into the type means the zero row is rejected at parse time rather than producing a NaN that quietly spreads. Casting money through Decimal(str(v)) — not Decimal(v) — is what keeps a stray float literal from injecting binary-rounding error before the value is even stored.

Ingestion also canonicalizes units here. If an export arrives in pounds or ounces it is converted to kilograms before it becomes a YieldObservation; the same discipline that governs portion size standardization applies — convert at the edge, so every value entering the calculation is already in the base unit and the math never touches unit conversion.

Phase 2 — Validation and Anomaly Routing

Passing the type contract proves a row is well-formed, not that it is plausible. Phase 2 loads the validated observations into a frame and applies boundary logic: a yield above 1.0 is physically impossible for anything but moisture absorption (and usually signals a tare error), while a yield below a category floor signals over-trimming or a mislabeled SKU. Both are flagged and split off, not clamped.

import numpy as np
import pandas as pd


def flag_yield_anomalies(
    df: pd.DataFrame,
    min_yield_floor: float = 0.45,
    max_yield_ceiling: float = 1.05,
) -> pd.DataFrame:
    """Compute raw yield and flag rows outside plausible bounds.

    Expects validated columns:
        ['sku_id', 'location_id', 'observed_at',
         'ap_weight_kg', 'ep_weight_kg', 'ap_cost', 'location_variance']
    """
    df = df.copy()

    # Vectorized yield; ap_weight_kg is guaranteed > 0 by Phase 1, so the
    # np.where guard here is belt-and-suspenders against upstream bypass.
    df["raw_yield"] = np.where(
        df["ap_weight_kg"] > 0,
        df["ep_weight_kg"] / df["ap_weight_kg"],
        np.nan,
    )

    df["is_anomalous"] = (
        df["raw_yield"].isna()
        | (df["raw_yield"] < min_yield_floor)
        | (df["raw_yield"] > max_yield_ceiling)
    )
    return df


def split_quarantine(df: pd.DataFrame) -> tuple[pd.DataFrame, pd.DataFrame]:
    """Partition into clean rows and a quarantine set for human review."""
    clean = df.loc[~df["is_anomalous"]].copy()
    quarantined = df.loc[df["is_anomalous"]].copy()
    quarantined["reason"] = np.select(
        [
            quarantined["raw_yield"].isna(),
            quarantined["raw_yield"] > 1.05,
            quarantined["raw_yield"] < 0.45,
        ],
        ["zero_or_missing_weight", "yield_above_ceiling", "yield_below_floor"],
        default="unknown",
    )
    return clean, quarantined

Quarantined rows carry a machine-readable reason so a culinary manager can triage them — a run of yield_above_ceiling on one location’s scale is a calibration ticket, not a data-science problem. Critically, the job does not abort on anomalies: the clean partition proceeds to smoothing while the quarantine set is written to its own table with a structured log line per row. A missing measurement is never allowed to default to a zero yield, because a zero yield divides into an infinite EP cost.

Phase 3 — Smoothing and Decimal EP-Cost Handoff

The clean set now feeds the smoothing and cost-derivation step. Yields are smoothed per SKU with an EWMA over the ordered series; the resulting factor is then divided into the AP cost using Decimal arithmetic and quantized once to four places for the ledger.

from decimal import ROUND_HALF_UP, Decimal

CENTS = Decimal("0.0001")


def derive_ep_costs(clean: pd.DataFrame, alpha: float = 0.3) -> pd.DataFrame:
    """Smooth yields per SKU and derive Decimal-exact adjusted EP costs."""
    clean = clean.sort_values(["sku_id", "location_id", "observed_at"]).copy()

    # Vectorized EWMA per (sku, location); no row iteration.
    clean["smoothed_yield"] = (
        clean.groupby(["sku_id", "location_id"])["raw_yield"]
        .transform(lambda s: s.ewm(alpha=alpha, adjust=False).mean())
    )

    def _ep_cost(row: pd.Series) -> Decimal:
        y = Decimal(str(row["smoothed_yield"]))
        if y <= 0:
            raise ValueError(f"non-positive smoothed yield for {row['sku_id']}")
        gross = (row["ap_cost"] / y) * row["location_variance"]
        return gross.quantize(CENTS, rounding=ROUND_HALF_UP)

    clean["adjusted_ep_cost"] = clean.apply(_ep_cost, axis=1)
    return clean

The adjusted_ep_cost reflects the true consumable value of the ingredient: an AP cost of one dollar per kilogram on a protein that yields 0.62 after trim and cook loss resolves to roughly 1.6129 per edible kilogram. That number is the join key the rest of the architecture consumes. When it lands in the materialized cost ledger it is read by the recipe BOM roll-up to compute theoretical plate cost, and by the sales-side deduction path defined in POS taxonomy mapping, where a sold line item must draw down inventory in EP equivalents rather than AP bulk weight. The handoff itself is an idempotent upsert:

INSERT INTO ep_cost_ledger (sku_id, location_id, smoothed_yield,
                            adjusted_ep_cost, run_version, computed_at)
VALUES (:sku, :loc, :yield, :ep_cost, :ver, NOW())
ON CONFLICT (sku_id, location_id) DO UPDATE
    SET smoothed_yield   = EXCLUDED.smoothed_yield,
        adjusted_ep_cost = EXCLUDED.adjusted_ep_cost,
        run_version      = EXCLUDED.run_version,
        computed_at      = EXCLUDED.computed_at;

The ON CONFLICT ... DO UPDATE keyed on (sku_id, location_id) is what makes the nightly refresh re-runnable: a retried job produces the identical ledger state, never duplicate rows. Storing adjusted_ep_cost as NUMERIC(12,4) mirrors the Decimal quantization so the value round-trips without float coercion.

Production Hardening

Moving from a working calculation to a dependable nightly job comes down to a handful of disciplines:

• Idempotency keys. Stamp each run with a monotonic run_version (a snapshot id of the input set). Writing the version alongside each cost lets a reader detect stale numbers and lets a retried job overwrite exactly the rows it owns.
• Memory bounds. The observation set grows with history. Bound the EWMA window — keep only the trailing N observations per SKU in memory — rather than loading the full time series; the smoothed factor converges long before the tail matters.
• Category-specific parameters. Proteins, leafy produce, and dry goods have fundamentally different yield behavior, so a single global floor is wrong for all of them. Key min_yield_floor, max_yield_ceiling, and alpha by ingredient category and pass them per group; the produce-specific values are derived in calculating trim and yield factors for produce.
• Location isolation. Never let one unit’s scale drift contaminate another’s factor. The groupby(["sku_id", "location_id"]) key keeps every kitchen’s series independent — the same isolation the multi-location cost center architecture relies on to stop regional prep variance from fracturing a master ingredient’s cost.
• Variance gating. Before committing, diff each SKU’s new EP cost against the previous version and hold any that moved more than a configured threshold (for example > 5%) for culinary review, so a bad price or scale feed cannot silently propagate a margin shock.
• RBAC boundaries. Procurement holds write access to ap_cost inputs; culinary managers get read-only access to the ledger; the pipeline runs as a service account with EXECUTE on the refresh function alone. Yield coefficients and vendor pricing stay isolated from POS and scheduling systems.

Failure Modes and Troubleshooting

The through-line of every failure above is the same: a yield must be validated, smoothed, and costed in exact arithmetic before it is written — never guessed, never defaulted to zero, never computed in floating point where money is involved.

FAQ

Why smooth yields instead of using the latest measurement?

Because a single weighing event carries real sensor and technique noise. Costing every plate against the most recent raw yield couples plate cost to that noise — one mis-tare swings the EP cost of every dish using the ingredient. An exponentially weighted average dampens outliers while still tracking genuine seasonal drift, so the number costing reads is stable without being stale.

Should yield factors be global or per location?

Per location. Butcher skill, equipment, and prep standards legitimately differ across kitchens, and averaging them into one global factor hides the outlier unit that is actually losing margin. The pipeline keys smoothing on (sku_id, location_id) so each kitchen’s series stays independent and its own drift is visible.

What happens to a measurement with zero AP weight?

It is rejected at ingestion. The Pydantic model constrains ap_weight_kg to gt=0, so a failed-tare 0 never reaches the calculation. If a zero slips past validation, the np.where guard produces NaN, which Phase 2 flags as zero_or_missing_weight and quarantines. A zero yield is never allowed to divide into an EP cost.

Can I keep yields as floats and only worry about Decimal for the final price?

Yes, and that is exactly the boundary this framework draws. Yield ratios are dimensionless and tolerate float arithmetic through the smoothing step. The switch to Decimal happens the instant a yield is divided into a monetary AP cost; the result is quantized once and stored as NUMERIC, which is what keeps fractional-cent error from accumulating across millions of deductions.

Related

• Up one level: Core Architecture & Cost Mapping Systems
• Calculating Trim and Yield Factors for Produce
• Designing Recipe BOM Databases
• Mapping POS Taxonomies to Ingredients
• Multi-Location Cost Center Architecture
• Variance Mapping Methodologies

For library specifics, see the official pandas documentation and Pydantic documentation.