ResolveProxyIP exists to wake the lazy proxy peer, and it retried only
curl exit 6 — DNS. The wake-up attempt that arrives before WireGuard has
brought the tunnel up fails with exit 7 instead, and that returned
immediately:
no HTTP response from vast-azalea.netbird.local: exit status 7
(curl: (7) Failed to connect ... after 0 ms)
So the one function whose job is to tolerate a not-yet-ready endpoint
failed on the readiness state it was written for, one second after the
client container reported ready. Retry both exit codes within the same
window; anything else would still be failing when the window closed and
still fails immediately.
Raise the access-log ingest window to 60s for the same reason. The proxy
streams each entry with a 10s send timeout of its own, so 30s left
barely three attempts of headroom before a test that had already got its
200 was failed for a row still in flight.
A provider update added the proxy mapping and then rebuilt the middleware
chain. Between the two, the route was live with no chain behind it, and a
request that landed there was served straight through — a successful
inference that was neither routed by policy nor metered.
Rebuild first. The worst a request in the remaining window meets is the new
chain in front of the previous target, which is still counted.
The loop bounded when a new attempt could start, not how long one could
run. The chat container is capped at 90 seconds of its own and the row
lookup at another 20, so an attempt begun just inside the 180-second window
could report a repricing failure nearly two minutes after that window
closed. Run every call in the loop under a context that expires with the
deadline.
Three open review findings.
The discovery filter treated every slash in a listed id as a gateway prefix
and matched the tail against the policy. A self-hosted id carries slashes of
its own, and an upstream may scope ids per tenant, so "tenant-b/claude-sonnet-5"
matched a permitted "claude-sonnet-5" and reached the picker — a model the
policy never named, and one the guardrail denies on sight, since enforcement
compares the id as written. Strip only the namespaces a gateway is known to
prepend, taken from the first slash rather than the last.
The e2e retry loops slept between attempts without watching the context, so a
cancelled run kept retrying calls that fail instantly and spent its remaining
window sleeping between them. They now stop when the context is done.
The streamed provider's setup key outlived its test: deleting the group does
not delete the key that auto-joins it.
The failure message read repriced.InputCostUsd, and repriced is the zero
value on every path that reaches that line — so a run that gave up always
reported "last input_cost_usd=$0.000000", which reads as a row priced at
zero rather than as a row still at the old rate, or as no row at all.
Keep the last cost read and report that, saying so plainly when no row was
ever read.
The poll interval was not bounded by the window: a page that came back
without the row just before the deadline still slept a full two seconds
before the loop noticed it was out of budget, so the caller waited longer
than the window it asked for to be told nothing arrived.
Cap the wait at whatever is left of the window.
Two review findings on the e2e suite.
lookupAccessLogBySession polled under the caller's context, so one stalled
request could hold the loop open well past the 30s ingest window it exists to
enforce — and the caller would read that delay as a missing row rather than a
slow one. Each poll now expires with the window; the parent's cancellation
still applies, since the request context derives from it.
The streaming test asserted only that the total cost was positive. Input and
output are positive on their own, so a cache-read bucket that was parsed and
then never billed would have passed. Assert the sum of the three buckets: the
gap a dropped cache read leaves is 7e-6, well outside the delta.
Six behaviours had unit coverage only, either because they arrived from code
review after the end-to-end tests were written or because no request in the
suite had the shape that reaches them.
Streaming is the important one. Input tokens exist only in a stream's opening
message_start event, and reading a stream with the wrong vendor's parser
misses it — the metering bug this endpoint's protocol work fixed. Nothing in
the suite sent stream: true, so the branch never ran. The mock now serves an
SSE surface on a second listener, reporting counts that differ from its
buffered ones so a passing assertion can only mean the stream accumulator ran,
and one case drives it through a record typed for the wrong surface.
The rest need no new harness capability: the per-model lookup against the
allowlist, the read-method gate on the non-inference paths, dated ids reaching
an undated registration while a pinned build refuses a different one, the
Bedrock inference-profile lookup reaching its upstream rather than a policy
denial, and a custom dated id keeping its own price.
Sub-agent ids stay uncovered: the parser lifts them onto request metadata but
nothing persists them, so there is no queryable surface to assert against
until that half lands. Covered here only to the extent that sending the
headers leaves the request served and metered.
TestPriceChangeUpdatesRecordedCost drives requests in a loop until one is
priced at the new rate, because the price push and the proxy's chain rebuild
are async. The loop could not actually retry: it looked the row up through
findAccessLogBySession, which fails the test outright when no row lands
within 30s, so the first post-update request that produced no row ended the
run instead of yielding to the next attempt.
That is the observed failure — the nightly run has been red on this test
roughly half the time, always with "session id ...-reprice-b-... must be
recorded in an access-log row" after ~41s: container setup, one request, one
30s wait, dead.
A missing row there is expected rather than exceptional. The provider update
rebuilds the middleware chain, and a request served mid-rebuild can complete
without a resolved provider: 200 to the caller, nothing to attribute, so no
row is ever written for it. Split the polling helper into a non-fatal lookup
and keep the fail-fast wrapper for callers whose row must exist, then treat a
miss in the loop like any other not-yet-repriced iteration. Only the outer
deadline is fatal.
Shorten the per-attempt wait to 20s and raise the overall deadline to 180s so
several attempts fit where before the budget allowed barely one.
The routing and parser-selection fixes touch every provider surface, and
the unit tests only prove each side of a seam in isolation. Two suites
close that:
The provider matrix drives one request per wire shape over a single tunnel,
with a record per catalog surface behind it, and asserts the surface each
request was metered under together with the token counts that surface's own
usage block carries. A response read by the wrong provider's parser meters
zero, so a regression fails instead of passing on a coincidental non-zero.
It also covers the Bedrock and Vertex token-counting paths, the warm-up
probe, and the vendor error envelope on a refusal.
The discovery suite covers the configuration that broke: an account with a
model allowlist, where the listing carries no model and the gate failed
closed. It asserts the listing is served, that it is bounded to the
authorised model, and that inference outside the allowlist is still
refused, so the exemption cannot be read as a way around the gate.
The mock upstream grows the Anthropic, Bedrock and token-counting shapes so
one container stands in for every surface, and the client gains GET and
arbitrary-POST helpers for the endpoints that carry no chat body.
Store the per-account gateway endpoint as {domain, proxy_address} with a
global unique index on the full hostname; dedicated = (domain ==
proxy_address). Bootstrap becomes an explicit POST carrying exactly one
of proxy_address (server allocates an adjective-noun label beneath it)
or endpoint (claimed verbatim, address-first); provider create loses its
bootstrap side effect. PUT is a full replace with every field required —
the immutable identity fields must be echoed unchanged and a mismatch is
rejected with 422. A guarded DELETE releases the endpoint: refused with
412 while providers exist or a proxy is actively serving the endpoint
hostname (matched case-insensitively); re-creating bootstraps fresh. A
self-addressed pin excludes its address from the account's cluster allow
list, and the live mapping update path now addresses the serving proxy
from the synthesized service. Existing rows are migrated on all three
store engines.
