Two independent broadening phases compose combinatorially and explode a payload to an external API

A "widen the search if it's too narrow" heuristic applied in two independent phases — first generalize a specific input *upward* to a coarser parent, then expand every coarse code *downward* into all its children — composes explosively the moment the downward phase can't tell an input that started coarse from one that was *derived* coarse by the upward phase a moment earlier: a single specific input ends up generating dozens to hundreds of values in a request to an external API that has no documented ceiling and simply errors out past some undocumented limit.

The signal that betrays it

- A search/expansion pipeline that has both an "escalate to a broader category" step and a "given a broad category, enumerate all its sub-categories" step, run one after another in the same function, on the same mutable set. - The downward-expansion step decides whether to expand a value using only a property of the value itself (e.g. "is this a 2-level code with no further qualifier?"), never asking *how that value got into the set* — whether the caller supplied it directly or the upward step just produced it. - Log lines that show a broadened list jumping from single digits to double- or triple-digit counts for what looks like a routine, specific query — e.g., for a made-up hierarchical code scheme, broadened ['A.4.2.1'] -> [... on the order of dozens to a hundred elements ...] where the input was one deep, specific code and the output is effectively "every child of its top-level ancestor" (the exact code and count here are illustrative of the shape, not a real case). - The external API call that follows returns a generic server error (not a documented "payload too large" — an undocumented one) exactly when, and only when, the broadened list crosses some threshold nobody wrote down. Below the threshold everything works; above it, every request in the batch fails the same way. - Downstream, every parallel expansion strategy built on top of this same broadening returns zero new results (not partial, not slow — flat zero), because the request never gets a valid response back to extract anything from. A pool-building process that expects to accumulate results across several strategies instead sits stuck at zero growth across all of them simultaneously, which is the tell that the shared broadening step underneath all strategies is the single point of failure, not any one strategy's logic. - Bonus tell: it doesn't show up in synthetic tests built with "clean" broad inputs, only with real inputs that happen to be maximally specific (the deepest level the classification scheme allows) — because only the deepest, most specific inputs generate an upward hop *and then* a downward hop across the same coarse value; broader synthetic test inputs skip the first hop and never trigger the compounding.

How you exploit it (diagnose and fix)

Reproduce the shape, not the exact case. Take any two-phase broadening — up-generalize then down-enumerate — and feed it the single most specific input the classification allows (deepest level, most qualifiers). Trace the intermediate set after each phase, not just the final one: {specific_input} → add ancestors → {specific_input, parent, grandparent} → then run down-enumeration over *that entire set* → the grandparent (now sitting in the set only because the first phase put it there) gets fanned out into every one of its children. The concrete branching factor and resulting count are entirely a property of *your* classification scheme, not a universal constant — for illustration, a scheme with two levels below a coarse root and ~10–100 children per level would let a single specific input balloon into a few dozen to a few hundred values before it ever reaches the network call; your own scheme's numbers will differ.

Instrument the broadened-set size, not just its content. A log line that prints the broadened list is useless if nobody reads it for length; grep for it and alert on cardinality, not presence: pick your own suspicious-cardinality threshold from your scheme's normal fan-out (for illustration only: if a scheme normally produces a handful of elements per input, seeing one input cross into the low tens is already worth a look) well before whatever undocumented ceiling the external API enforces.

Fix at the provenance boundary, not the enumeration logic. The down-enumeration step itself ("given a coarse code, add all its children") is legitimate and wanted — for an input the *user* actually gave as coarse. The bug is that it can't distinguish that from a coarse value the *upward* phase manufactured a line earlier. The fix is to snapshot the original input set before any broadening starts, and gate the downward-enumeration step on membership in that original snapshot, not on membership in the live, mutating "broadened so far" set: - take original = set(input_codes) before touching anything; - run the upward phase freely (cheap: it only adds ancestors, at most one per input level); - run the downward phase only for coarse values that are in original — i.e. only for what the caller actually asked to broaden downward, never for a coarse value that arrived via the upward hop.

Add a hard cap as a second, independent line of defense, because provenance-gating fixes *this* explosion but won't catch every future one someone introduces by adding a third broadening phase later. Pick a ceiling comfortably under whatever the external API's real (even if undocumented) limit turns out to be, and when the set exceeds it, strip the *least specific* elements first — coarse values are usually redundant once you know the API does prefix/parent matching internally, so removing them loses little precision while cutting the count fastest.

Add a final deterministic truncation as a third net. If stripping the coarse tier still leaves the set over the cap, sort deterministically (alphabetically, or by original insertion order — anything reproducible) and take the first N. Non-deterministic truncation (e.g. relying on set iteration order) turns an already-confusing bug into one that also isn't reproducible between runs, which is worse.

Verify with the actual undocumented boundary, not just under it. Once the three fixes are in, deliberately construct an input that *would* have produced an over-cap set pre-fix, and confirm the broadened-list log line now stays in the single digits and the downstream call succeeds with new results > 0. Confirming "it doesn't crash" is not enough — confirm the pipeline actually produces output again, since a silent-zero-results failure and a silent-cap-truncation-to-wrong-scope failure can look identical from the outside (both "no error, but nothing useful happens").

Related patterns from the same family of "cost/limit control interacting badly with correctness": a sibling failure is when a cap combined with *global* state tracking (instead of *provenance* tracking) silently starves a backlog instead of exploding a payload — same root shape (a legitimate-in-isolation limit, broken by what it's combined with), opposite symptom (nothing grows vs. one call explodes). Check what you already own before paying again for the same external query covers the neighboring failure mode on the same class of paid, quota-limited external search API: paying repeatedly for what you already hold, discovered the same way — by testing against real data, where a field-format mismatch invisible in synthetic fixtures only surfaces against production-shaped input.

What does NOT work

- Fixing the downward-enumeration step to be "smarter" about which coarse codes look legitimate. There's no property of a coarse value like "A" alone that tells you whether a user typed it or the upward phase derived it — the information that distinguishes them (provenance) doesn't exist in the value, only in *how it arrived*. Any fix that inspects the value instead of tracking its origin is treating a symptom. - Relying on the external API's own error message to catch this in production. In one observed case, the failure mode was a generic, undocumented server error with no size hint in the response body — not a clean, documented "payload too large" you could branch on. Treating "catch the 4xx/5xx and retry smaller" as the fix means every legitimate specific query silently fails first, and retry logic has to guess *how much* smaller, blind. Prevent the oversized payload from being built at all; don't try to recover gracefully from having built it. - Raising or removing the enumeration limit "because that many sub-codes isn't that many." The absolute count is not the problem — the *combination* of an implicit broadening (upward) feeding an explicit broadening (downward) over the same mutable set is. A wider cap just moves the undocumented failure threshold further out; it doesn't stop a future third broadening phase from recreating the same compounding one level up. - Testing only with synthetic "obviously broad" inputs. The explosion is specifically triggered by the *most specific* inputs a real user or a real upstream generator would produce (maximally-qualified codes), because those are the ones that take the upward hop before the downward hop ever sees them. A test suite built from tidy, already-coarse example inputs will pass green while the production path — fed real, specific, AI- or user-generated queries — explodes.