[management] Withdraw a cluster address claim that is lost after the write

A cluster address is claimed two ways: an account-scoped proxy row, and an
agent network gateway pin on the address. Each side checked the other
before writing — IsClusterAddressAvailable before SaveProxy,
HasForeignAccountProxyAtHost before the settings insert — but check and
write are separate autocommit statements, so two concurrent claimants could
each pass their check and both commit, leaving a pin no proxy will ever
serve next to the proxy row that displaces it.

Both sides now re-read after they write. Manager.Connect re-asks
availability once the proxy row is committed and, if the address is no
longer free or the answer is inconclusive, deletes its own row and returns
ErrClusterAddressUnavailable, which the connect path reports as
AlreadyExists exactly as the pre-write check would have. bootstrapLabeled
re-asks ownership once the settings row is committed and withdraws the pin
on the same terms. Because both write before they re-read, of two
concurrent claimants at least one re-reads after the other has committed
and backs off — on sqlite, postgres and mysql alike, since each statement
sees every commit before it. Both may back off, which costs a retry;
neither keeps a claim the other holds.

No lock spans the proxies and settings tables portably, and a claims table
would be more machinery than the property needs, so the re-read is the
whole mechanism. DeleteProxy is session-guarded like DisconnectProxy, so a
stale session withdrawing itself cannot take out a newer session's row.

Reported by CodeRabbit on #7402 (CWE-362).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sa3DsBDP3VciAi4PPG17L6
This commit is contained in:
mlsmaycon
2026-09-12 12:53:15 +00:00
co-authored by Claude Fable 5.1
parent 39f8ea3f70
commit 68da6bf3aa
11 changed files with 401 additions and 6 deletions
@@ -571,6 +571,12 @@ func (s *ProxyServiceServer) registerProxyConnection(ctx context.Context, params
proxyRecord, err := s.proxyManager.Connect(ctx, params.proxyID, sessionID, params.address, peerInfo, accountID, caps)
if err != nil {
cancel()
if errors.Is(err, proxy.ErrClusterAddressUnavailable) {
// The claim was lost to a concurrent one after validateProxyConnect
// saw the address free; the row has been withdrawn. Same answer
// as the pre-write check gives, so the proxy treats both alike.
return nil, nil, status.Errorf(codes.AlreadyExists, "cluster address %s is already in use", params.address)
}
if accountID != nil {
return nil, nil, status.Errorf(codes.Internal, "failed to register BYOP proxy: %v", err)
}
@@ -3,11 +3,12 @@ package grpc
import (
"context"
"errors"
"fmt"
"testing"
"go.uber.org/mock/gomock"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/mock/gomock"
"google.golang.org/grpc/codes"
grpcstatus "google.golang.org/grpc/status"
@@ -166,3 +167,27 @@ func TestValidateProxyConnect_AuthorizerRunsLast(t *testing.T) {
assert.Equal(t, codes.AlreadyExists, st.Code(), "an address conflict must keep its own status")
assert.Zero(t, auth.called, "a conflicting address must be rejected before policy runs")
}
// TestRegisterProxyConnection_LostClaimIsAlreadyExists pins the status a proxy
// sees when its claim is lost after validateProxyConnect passed: the manager
// withdraws the row and reports ErrClusterAddressUnavailable, and the connect
// path must answer AlreadyExists — the same code the pre-write check gives —
// rather than the Internal it uses for a store failure, so the proxy handles
// the two paths to "address taken" identically.
func TestRegisterProxyConnection_LostClaimIsAlreadyExists(t *testing.T) {
ctrl := gomock.NewController(t)
mgr := proxy.NewMockManager(ctrl)
mgr.EXPECT().
Connect(gomock.Any(), "proxy-1", gomock.Any(), "cluster.example.com", gomock.Any(), gomock.Any(), gomock.Any()).
Return(nil, fmt.Errorf("cluster address cluster.example.com: %w", proxy.ErrClusterAddressUnavailable))
s := &ProxyServiceServer{proxyManager: mgr}
_, _, err := s.registerProxyConnection(scopedCtx("acc-1"), proxyConnectParams{proxyID: "proxy-1", address: "cluster.example.com"}, &proxyConnection{})
require.Error(t, err)
st, ok := grpcstatus.FromError(err)
require.True(t, ok)
assert.Equal(t, codes.AlreadyExists, st.Code(), "a claim lost after the check must read as the address being taken")
assert.Contains(t, st.Message(), "already in use")
_, tracked := s.connectedProxies.Load("proxy-1")
assert.False(t, tracked, "a refused registration must not be tracked as connected")
}