mirror of
https://github.com/netbirdio/netbird.git
synced 2026-08-25 00:51:28 +02:00
47cb96c010
Management could populate the provider-config model picker from live vendor data instead of the hand-curated catalog, which today carries comments tracking which models a vendor retired on which date and which cannot know what a particular account may actually invoke. Whether that is a small feature or a large one turns on one unknown: ListInferenceProfiles is a control-plane operation on bedrock.<region>.amazonaws.com, while a provider record's upstream must be bedrock-runtime.<region> for InvokeModel. Nothing documents whether a Bedrock API key authorises the control plane, and if it does not, live discovery for Bedrock needs SigV4 signing in management. Probe each vendor directly — no proxy, no tunnel, no containers — under the credential an operator would actually supply, and print a verdict table. Bedrock gets three probes so the control-plane answer is read against the runtime host we already know 404s, rather than as a lone data point; Vertex gets three because which path serves the publisher listing is itself part of the question. Vertex reuses the same JWT mint and cloud-platform scope llm_router uses in production, so a result here transfers. No status code is asserted: the status is the finding. A probe fails only when its credential is present and the request could not be made at all, which is a broken probe rather than an answer. Add a test_pattern dispatch input so this can run on its own in about a minute instead of behind the sixteen-minute container suite. It reaches the shell through an env var, since a dispatch input interpolated into a run script is a script-injection seam.
92 lines
4.2 KiB
YAML
92 lines
4.2 KiB
YAML
name: Agent Network E2E
|
|
|
|
on:
|
|
# Nightly at 03:00 UTC, plus on demand from the Actions tab.
|
|
schedule:
|
|
- cron: "0 3 * * *"
|
|
workflow_dispatch:
|
|
inputs:
|
|
bedrock_model:
|
|
description: >-
|
|
Bedrock inference-profile id to drive the matrix with, exactly as
|
|
AWS issues it. Leave empty for the Sonnet 4.6 default.
|
|
required: false
|
|
default: ""
|
|
test_pattern:
|
|
description: >-
|
|
Package pattern to run. Defaults to the whole suite; narrow it to one
|
|
package (e.g. ./e2e/providerdiscovery/...) when a run only needs that
|
|
package's answer and not the sixteen minutes the container suite costs.
|
|
required: false
|
|
default: "./e2e/..."
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
e2e:
|
|
name: Agent Network E2E
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 45
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- name: Install Go
|
|
uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
|
|
with:
|
|
go-version-file: "go.mod"
|
|
|
|
# Container-driver builder so the harness can build the combined/proxy/
|
|
# client images from source with a local layer cache.
|
|
- name: Set up Buildx
|
|
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
|
|
|
|
# Persist the Docker layer cache across runs. This caches the base, apt,
|
|
# and go-mod-download layers; the Go compile still re-runs, as BuildKit
|
|
# mount caches cannot be exported to the GitHub cache.
|
|
- name: Cache Docker layers
|
|
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
|
|
with:
|
|
path: /tmp/.buildx-cache
|
|
key: ${{ runner.os }}-anet-e2e-buildx-${{ hashFiles('go.sum', 'combined/Dockerfile.multistage', 'proxy/Dockerfile.multistage', 'e2e/harness/Dockerfile.client') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-anet-e2e-buildx-
|
|
|
|
- name: Run agent-network e2e
|
|
env:
|
|
# Build the images from source (this branch's code) with the shared
|
|
# local layer cache.
|
|
NB_E2E_BUILDX_CACHE: /tmp/.buildx-cache
|
|
# Provider credentials. Each provider scenario skips if its
|
|
# token (and URL, for gateways) is unset, so partial coverage is fine.
|
|
OPENAI_TOKEN: ${{ secrets.E2E_OPENAI_TOKEN }}
|
|
ANTHROPIC_TOKEN: ${{ secrets.E2E_ANTHROPIC_TOKEN }}
|
|
# Moonshot AI platform key (platform.kimi.ai); drives both Kimi wire
|
|
# shapes (OpenAI /v1 and Anthropic /anthropic) through kimi_api.
|
|
KIMI_TOKEN: ${{ secrets.E2E_KIMI_TOKEN }}
|
|
VERCEL_URL: ${{ secrets.E2E_VERCEL_URL }}
|
|
VERCEL_TOKEN: ${{ secrets.E2E_VERCEL_TOKEN }}
|
|
OPENROUTER_URL: ${{ secrets.E2E_OPENROUTER_URL }}
|
|
OPENROUTER_TOKEN: ${{ secrets.E2E_OPENROUTER_TOKEN }}
|
|
CLOUDFLARE_URL: ${{ secrets.E2E_CLOUDFLARE_URL }}
|
|
CLOUDFLARE_TOKEN: ${{ secrets.E2E_CLOUDFLARE_TOKEN }}
|
|
AWS_BEARER_TOKEN_BEDROCK: ${{ secrets.E2E_AWS_BEARER_TOKEN_BEDROCK }}
|
|
AWS_REGION: ${{ secrets.E2E_AWS_REGION }}
|
|
# Bedrock model override: dispatch input wins, then the repo variable, else the test default.
|
|
AWS_BEDROCK_MODEL: ${{ inputs.bedrock_model || vars.E2E_AWS_BEDROCK_MODEL }}
|
|
# Vertex (Anthropic-on-Vertex): SA + project required; region defaults
|
|
# to "global", model to a pinned claude snapshot.
|
|
GOOGLE_VERTEX_SA_BASE64: ${{ secrets.E2E_GOOGLE_VERTEX_SA_BASE64 }}
|
|
GOOGLE_VERTEX_PROJECT: ${{ secrets.E2E_GOOGLE_VERTEX_PROJECT }}
|
|
GOOGLE_VERTEX_REGION: ${{ secrets.E2E_GOOGLE_VERTEX_REGION }}
|
|
GOOGLE_VERTEX_MODEL: ${{ secrets.E2E_GOOGLE_VERTEX_MODEL }}
|
|
# Read through an env var rather than interpolated into the run
|
|
# script: a dispatch input reaching a shell command directly is a
|
|
# script-injection seam, however trusted the dispatcher.
|
|
TEST_PATTERN: ${{ inputs.test_pattern || './e2e/...' }}
|
|
run: go test -tags e2e -timeout 40m -v "$TEST_PATTERN"
|