[proxy] Publish a rebuilt chain before the route that reaches it

A provider update added the proxy mapping and then rebuilt the middleware
chain. Between the two, the route was live with no chain behind it, and a
request that landed there was served straight through — a successful
inference that was neither routed by policy nor metered.

Rebuild first. The worst a request in the remaining window meets is the new
chain in front of the previous target, which is still counted.
This commit is contained in:
mlsmaycon
2026-08-23 07:20:17 +00:00
parent 427b4c8d41
commit 17525a58bf
2 changed files with 12 additions and 6 deletions

View File

@@ -446,10 +446,10 @@ func TestPriceChangeUpdatesRecordedCost(t *testing.T) {
}
row, ok := lookupAccessLogBySession(repriceCtx, lastSession, repriceIngestWindow)
if !ok {
// No row for this request. The provider update rebuilds the proxy's
// middleware chain, and a request served mid-rebuild can complete
// without a resolved provider — 200 to the caller, nothing to
// attribute, so no row is ever written for it. Fire another one.
// No row for this request. The proxy now publishes a rebuilt chain
// before the route that reaches it, so a request can no longer be
// served unattributed mid-update; this retry covers the ingest
// window alone. Fire another one under a fresh session.
t.Logf("no access-log row for session %q within %s; retrying under a fresh session", lastSession, repriceIngestWindow)
continue
}

View File

@@ -2074,9 +2074,15 @@ func (s *Server) updateMapping(ctx context.Context, mapping *proto.ProxyMapping)
return fmt.Errorf("auth setup for domain %s: %w", mapping.GetDomain(), err)
}
m := s.protoToMapping(ctx, mapping)
s.proxy.AddMapping(m)
s.meter.AddMapping(m)
// The chain is published before the route that leads to it. A request
// arriving at a target whose chain has not been rebuilt yet is served
// straight through, so a provider update that added the route first left a
// window in which an inference could complete unrouted and unmetered.
// Rebuilding first inverts that: the worst a request in the window meets is
// the new chain in front of the previous target, which is still counted.
s.rebuildMiddlewareChains(svcID, m)
s.meter.AddMapping(m)
s.proxy.AddMapping(m)
return nil
}