Core Architecture Cost Mapping Systems

Resolving Regional Unit Aliases

This page shows a food-tech developer how to turn the chaos of real-world unit strings — fl oz, floz, fluid ounce, FL. OZ., oz., #, ct — into clean canonical tokens before they reach the conversion matrix. It is the upstream prerequisite for unit conversion and canonicalization, and it lives here in the multi-location cost center architecture because unit vocabulary varies by region and supplier across an estate. Read the canonicalization guide for what happens after the token is clean; this page makes it clean.

The failure this prevents is a conversion matrix riddled with special cases, or worse, one that silently maps oz to weight when a supplier meant fluid ounces. Alias resolution is a deterministic normalization step with an explicit table and a quarantine path — never fuzzy matching.

Prerequisites and Data Contract

  • Python 3.11+, pandas 2.x.
  • An unit_aliases table mapping a normalized alias string to a canonical unit token and, where needed, a dimension hint.
  • Input carries a raw unit_text and an ingredient_id (used to disambiguate ambiguous tokens); output is a canonical raw_unit token plus an alias_status.
Regional unit alias normalization Raw unit text is normalized by lowercasing and stripping punctuation and whitespace, then looked up in the alias table. Ambiguous tokens such as ounce are disambiguated using the ingredient's expected dimension. A resolved token is emitted as a canonical unit; an unknown alias is routed to quarantine for a human to map rather than guessed. unknown Raw unit text FL. OZ. Normalize text case · punct Lookup + disambig by dimension Canonical token fl_oz Quarantine

Step-by-Step Implementation

Step 1 — Build the alias table

Aliases are data. Each row maps a normalized string to a canonical token; ambiguous tokens carry no dimension and are resolved in Step 3.

import pandas as pd

unit_aliases = pd.DataFrame(
    [
        ("floz", "fl_oz", "volume"),
        ("fluidounce", "fl_oz", "volume"),
        ("fluidounces", "fl_oz", "volume"),
        ("lb", "lb", "mass"),
        ("lbs", "lb", "mass"),
        ("pound", "lb", "mass"),
        ("ct", "each", "count"),
        ("count", "each", "count"),
        ("oz", "oz", None),          # ambiguous: weight or fluid — resolve by ingredient
    ],
    columns=["alias", "canonical", "dimension_hint"],
)

Step 2 — Normalize the raw text deterministically

Lowercase, strip punctuation and whitespace, and collapse plurals to a stable key. No fuzzy matching — the transform is reproducible.

import re


def normalize_unit_text(text: str) -> str:
    key = text.strip().lower()
    key = re.sub(r"[.\s\-_]", "", key)      # "FL. OZ." -> "floz"
    key = re.sub(r"s$", "", key) if not key.endswith("ss") else key
    return key

Step 3 — Look up and disambiguate by ingredient dimension

For an ambiguous token like oz, use the ingredient’s expected dimension to decide weight versus fluid. If it cannot be resolved, quarantine.

import pandas as pd


def resolve_alias(df: pd.DataFrame, aliases: pd.DataFrame,
                  expected_dim: dict[str, str]) -> pd.DataFrame:
    df = df.copy()
    df["alias_key"] = df["unit_text"].map(normalize_unit_text)
    merged = df.merge(aliases, left_on="alias_key", right_on="alias", how="left")

    def finalize(row: pd.Series) -> pd.Series:
        if pd.isna(row["canonical"]):
            return pd.Series({"raw_unit": None, "alias_status": "QUARANTINE"})
        if row["canonical"] == "oz":                       # ambiguous
            dim = expected_dim.get(row["ingredient_id"])
            if dim == "volume":
                return pd.Series({"raw_unit": "fl_oz", "alias_status": "RESOLVED"})
            if dim == "mass":
                return pd.Series({"raw_unit": "oz", "alias_status": "RESOLVED"})
            return pd.Series({"raw_unit": None, "alias_status": "QUARANTINE"})
        return pd.Series({"raw_unit": row["canonical"], "alias_status": "RESOLVED"})

    return df.join(merged.apply(finalize, axis=1))

Verification and Validation

  • Normalization stability. Assert normalize_unit_text("FL. OZ.") == "floz" and that it maps to fl_oz. Run the normalizer twice on the same input; identical output confirms determinism.
  • Ambiguity resolved by context. An oz on a liquid ingredient must resolve to fl_oz; on a solid, to oz; with no dimension hint, to QUARANTINE.
  • Unknown alias quarantined. A novel string (dram, noggin) must land in quarantine with a status, not silently drop.
  • Handoff. Confirm every RESOLVED token is one the conversion matrix recognizes, so alias output and converter input agree.

Gotchas and Edge Cases

  • The oz trap. Weight ounces and fluid ounces share a token. Never map oz blindly to weight; disambiguate by the ingredient’s expected dimension or quarantine. This single ambiguity is the most common source of order-of-magnitude cost errors.
  • Over-aggressive plural stripping. Naively removing a trailing s breaks gross or pieces. Guard the rule and prefer explicit alias rows over clever regex.
  • Locale decimals leaking in. Some feeds embed the quantity and unit together (1,5 l). Split quantity from unit before normalization, and handle comma decimals in the quantity parser, not here.
  • Silent map to a wrong dimension. An alias table row with the wrong dimension_hint corrupts every row using it. Treat the alias table as reviewed data with tests, not an ad-hoc dictionary.

For library specifics, see the official Python re documentation.