mirror of
https://github.com/netbirdio/netbird.git
synced 2026-09-16 19:59:07 +02:00
[e2e] Spike the vendors' own model-discovery endpoints
Management could populate the provider-config model picker from live vendor data instead of the hand-curated catalog, which today carries comments tracking which models a vendor retired on which date and which cannot know what a particular account may actually invoke. Whether that is a small feature or a large one turns on one unknown: ListInferenceProfiles is a control-plane operation on bedrock.<region>.amazonaws.com, while a provider record's upstream must be bedrock-runtime.<region> for InvokeModel. Nothing documents whether a Bedrock API key authorises the control plane, and if it does not, live discovery for Bedrock needs SigV4 signing in management. Probe each vendor directly — no proxy, no tunnel, no containers — under the credential an operator would actually supply, and print a verdict table. Bedrock gets three probes so the control-plane answer is read against the runtime host we already know 404s, rather than as a lone data point; Vertex gets three because which path serves the publisher listing is itself part of the question. Vertex reuses the same JWT mint and cloud-platform scope llm_router uses in production, so a result here transfers. No status code is asserted: the status is the finding. A probe fails only when its credential is present and the request could not be made at all, which is a broken probe rather than an answer. Add a test_pattern dispatch input so this can run on its own in about a minute instead of behind the sixteen-minute container suite. It reaches the shell through an env var, since a dispatch input interpolated into a run script is a script-injection seam.
This commit is contained in:
@@ -12,6 +12,13 @@ on:
|
||||
AWS issues it. Leave empty for the Sonnet 4.6 default.
|
||||
required: false
|
||||
default: ""
|
||||
test_pattern:
|
||||
description: >-
|
||||
Package pattern to run. Defaults to the whole suite; narrow it to one
|
||||
package (e.g. ./e2e/providerdiscovery/...) when a run only needs that
|
||||
package's answer and not the sixteen minutes the container suite costs.
|
||||
required: false
|
||||
default: "./e2e/..."
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
@@ -77,4 +84,8 @@ jobs:
|
||||
GOOGLE_VERTEX_PROJECT: ${{ secrets.E2E_GOOGLE_VERTEX_PROJECT }}
|
||||
GOOGLE_VERTEX_REGION: ${{ secrets.E2E_GOOGLE_VERTEX_REGION }}
|
||||
GOOGLE_VERTEX_MODEL: ${{ secrets.E2E_GOOGLE_VERTEX_MODEL }}
|
||||
run: go test -tags e2e -timeout 40m -v ./e2e/...
|
||||
# Read through an env var rather than interpolated into the run
|
||||
# script: a dispatch input reaching a shell command directly is a
|
||||
# script-injection seam, however trusted the dispatcher.
|
||||
TEST_PATTERN: ${{ inputs.test_pattern || './e2e/...' }}
|
||||
run: go test -tags e2e -timeout 40m -v "$TEST_PATTERN"
|
||||
|
||||
Reference in New Issue
Block a user