[misc] Bound each access-log poll, and assert the streamed total

Two review findings on the e2e suite.

lookupAccessLogBySession polled under the caller's context, so one stalled
request could hold the loop open well past the 30s ingest window it exists to
enforce — and the caller would read that delay as a missing row rather than a
slow one. Each poll now expires with the window; the parent's cancellation
still applies, since the request context derives from it.

The streaming test asserted only that the total cost was positive. Input and
output are positive on their own, so a cache-read bucket that was parsed and
then never billed would have passed. Assert the sum of the three buckets: the
gap a dropped cache read leaves is 7e-6, well outside the delta.
This commit is contained in:
mlsmaycon
2026-08-22 22:00:39 +00:00
parent 5c65dc9349
commit a6bcb177fe
2 changed files with 25 additions and 5 deletions
+6 -1
View File
@@ -66,9 +66,14 @@ func TestStreamingResponseMetersInputTokens(t *testing.T) {
// prices the full input count rather than a remainder.
wantInput := float64(harness.VLLMStreamInputTokens) / 1000 * streamInRate
wantOutput := float64(harness.VLLMStreamOutputTokens) / 1000 * streamOutRate
wantCacheRead := float64(harness.VLLMStreamCacheReadTokens) / 1000 * streamCacheReadRate
assert.InDelta(t, wantInput, row.InputCostUsd, 1e-6, "input cost must price the streamed input tokens")
assert.InDelta(t, wantOutput, row.OutputCostUsd, 1e-6, "output cost must price the streamed output tokens")
assert.Greater(t, row.CostUsd, 0.0, "a streamed request must never record as free")
// The total, not merely a positive number: input and output alone are
// positive, so a cache bucket parsed and then never billed would pass any
// weaker assertion. The gap is 7e-6, well outside the delta.
assert.InDelta(t, wantInput+wantOutput+wantCacheRead, row.CostUsd, 1e-6,
"the recorded cost must be every bucket the surface bills, cache reads included")
}
// TestStreamingOnGatewayTypedProvider drives the same streamed Anthropic call