[e2e] validate stored totals and cache costs against published rates

The provider matrix now sends a long-form prompt, recomputes each
access-log row's expected cost_usd and cache_cost_usd from hardcoded
published per-1k rates and the row's persisted token buckets, and
finishes with a raw-SQL audit of every usage row straight in the
management sqlite store. Bedrock defaults to Sonnet 4.6 and the
workflow gains a bedrock_model dispatch input plus a push trigger on
this branch while it is under review.
This commit is contained in:
Maycon Santos
2026-07-25 19:31:55 +00:00
parent 8bc2851f70
commit 4e624a25c2
4 changed files with 217 additions and 12 deletions

View File

@@ -5,6 +5,17 @@ on:
schedule:
- cron: "0 3 * * *"
workflow_dispatch:
inputs:
bedrock_model:
description: >-
Bedrock inference-profile id to drive the matrix with, exactly as
AWS issues it. Leave empty for the Sonnet 4.6 default.
required: false
default: ""
# Auto-run on pushes to the cache-accounting branch until its PR opens.
push:
branches:
- "anet-cache-cost-accounting"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -62,6 +73,8 @@ jobs:
CLOUDFLARE_TOKEN: ${{ secrets.E2E_CLOUDFLARE_TOKEN }}
AWS_BEARER_TOKEN_BEDROCK: ${{ secrets.E2E_AWS_BEARER_TOKEN_BEDROCK }}
AWS_REGION: ${{ secrets.E2E_AWS_REGION }}
# Bedrock model override: dispatch input wins, then the repo variable, else the test default.
AWS_BEDROCK_MODEL: ${{ inputs.bedrock_model || vars.E2E_AWS_BEDROCK_MODEL }}
# Vertex (Anthropic-on-Vertex): SA + project required; region defaults
# to "global", model to a pinned claude snapshot.
GOOGLE_VERTEX_SA_BASE64: ${{ secrets.E2E_GOOGLE_VERTEX_SA_BASE64 }}