mirror of
https://github.com/netbirdio/netbird.git
synced 2026-10-09 15:09:08 +02:00
* [client] Force interactive login when extending the auth session A session extend must be answered from the account the peer is registered under. With a silent PKCE flow (DisablePromptLogin or max_age=0) the IdP answers from whatever session it already holds, which need not be the peer's account when several are signed in; the token then fails the user match in ExtendAuthSession with no way to pick another account. Mark the PKCE flow request as a session extend so the management server can force prompt=login for it, overriding the configured silent flow. * [client] Reduce cognitive complexity of Server.Login Login sat at cognitive complexity 27, over the 25 the linter allows. Extract the interactive SSO branch into startSSOLogin, and split the nested in-flight-flow reuse check out of it into reuseOAuthFlow, which flattens the original if/else into early returns: it returns the cached auth info when the previous flow targets the same client and still has more than 90s left, otherwise cancels the stale wait and returns nil so the caller requests a fresh flow. The helpers take the contextState through a small statusSetter interface, since internal.contextState is unexported and re-deriving it with CtxGetState inside the helper would resolve against callerCtx rather than rootCtx. No behavior change: same ordering of state transitions, same mutex scope around the oauthAuthFlow write, same error paths. Login is now at 21. * [client] Respect DisablePromptLogin when extending the auth session Forcing prompt=login on a session extend overrode DisablePromptLogin, which is set for IdPs that break on it: Authentik triggers a double authentication and social logins fail outright. Overriding it there trades a recoverable extend for a login that cannot complete at all. Keep the LoginFlag override, which only replaces max_age=0 or none with prompt=login so the IdP honours login_hint, and leave DisablePromptLogin as configured. Those deployments keep the silent flow, and with several accounts signed in an extend answered from the wrong one still fails the user match. * [client] Guard the shared OAuth flow state with the server mutex reuseOAuthFlow read flow, expiresAt, waitCancel and info without holding s.mutex, while startSSOLogin and WaitSSOLogin write them under it. Reading the fields one at a time could also answer with auth info from a flow that was already replaced, or cancel a wait that no longer belongs to the flow just judged stale. Take one snapshot under the lock and decide from it. WaitSSOLogin read oauthAuthFlow.flow twice outside the lock; both now use a value snapshotted in the critical section that already installs actCancel. Its stale waitCancel was read and called in a separate section from the one installing the new one, so two racing calls could read the same predecessor and leave one wait uncancelled. Swap the two in a single critical section. Both cancels run after unlocking: the displaced wait takes s.mutex as it unwinds. * [client] Verify the SSO login came back for the hinted account login_hint is a suggestion the IdP may ignore: with a silent flow configured (DisablePromptLogin or max_age=0) and a live IdP session for another account, the login completes with that account's token. On a registered peer the management server rejects it as a user mismatch, but on a fresh profile the peer silently registers under the wrong account and the profile is then bound to it — every later login follows the stored hint straight back. After the token exchange, compare the ID token's email against the hint the flow was sent with. On a mismatch, do not log in to management with the token; run one more round asking the IdP to re-decide the account (prompt=login, via ForceAccountPrompt — DisablePromptLogin still wins there). If the prompted round also comes back different, proceed with a warning: the address may legitimately have changed, and refusing forever would lock the user out of the profile while the management server still rejects a token that does not own the peer. A token or profile with no email to compare is not judged. The retry differs per platform because of who opens the browser: - CLI (netbird login foreground) and Android run the whole flow in one process, so the mismatch retries automatically: the browser reopens with the account prompt within the same login attempt. - On desktop the login is split between the daemon and the GUI: Login hands the authorize URL to the GUI, WaitSSOLogin blocks for the token, and only the GUI can open a browser. A new URL cannot be handed out from inside WaitSSOLogin (its response has no field for one, kept that way to avoid a proto change), so the daemon arms forceAccountPrompt, fails the round with "connect again to choose the account", and builds the next Login's flow with the prompt — the user's next connect is the retry. The flag and the flow annotations live in daemon memory only; SwitchProfile drops them so the previous profile's hint cannot judge the next profile's token. The device code flow has no prompt parameter (RFC 8628), so a prompted round there runs as-is and a repeated mismatch is let through with the warning rather than looping. * [client] Address review comments on PKCE session extend flow Fail the PKCE authorization flow test on request error instead of continuing into a nil dereference, and make the godoc comments on the touched exported symbols identifier-leading full sentences. * [client] Match accounts only on the email claim of the ID token The name-claim fallback in the ID token parsing is kept for the login hint and display, but account matching now only considers a value that came from the email claim, so a token without one no longer produces a false account mismatch. * [client] Drop the pending session extend on a profile switch The profile-switch cleanup dropped the pending login flow and the account-prompt flag, but left extendAuthSessionFlow untouched. Its device code was issued by the previous profile's IdP client, so a WaitExtendAuthSession still parked on the browser leg would submit the resulting token against the new profile's engine. * [client] Judge the SSO account against the flow that produced the token WaitSSOLogin snapshotted the flow on entry but re-read the info, hint and accountPrompted from the live s.oauthAuthFlow afterwards, in separate critical sections. WaitToken blocks for the whole browser leg, so a concurrent Login or RequestJWTAuth could replace the flow meanwhile and the mismatch check would compare this wait's token against another flow's account: either arming the prompt spuriously or letting a wrong-account token through against an unrelated profile's hint. Take all of it in the entry snapshot. * [client] Keep the forced account prompt from being lost to flow reuse startSSOLogin consumed forceAccountPrompt and applied the prompt to the freshly built flow, but reuseOAuthFlow could then answer from a cached flow for the same client — one built without prompt=login, e.g. by RequestJWTAuth. The user got the same silent authorization URL that produced the mismatch, with the flag already spent, so no later round asked either. Rule reuse out when the prompt is forced, while still cancelling the predecessor's wait. RequestJWTAuth also wrote the flow fields one by one, leaving the previous login's hint and accountPrompted behind for WaitSSOLogin to judge a later token against. Both sites now replace the whole record. * [client] Consume the forced account prompt after the retry forceAccountPrompt was never cleared, so a flow that outlived the retry it was armed for kept sending prompt=login on every later authorization request and re-authenticated the user each time. RequestAuthInfo now takes the flag as it builds the request. * [client] Cancel the caller context in the SSO login tests WaitSSOLogin parks a goroutine on the caller's context for the whole browser leg. The tests passed context.Background(), which never cancels, so each left one goroutine behind for the lifetime of the test binary. * [client] Cancel the wait displaced by an OAuth flow replacement Replacing the shared record with a whole struct value dropped the previous flow's waitCancel, so an SSO browser wait still parked on it lost its cancel: nothing could preempt it, and it could go on to run attemptLogin or mutate the record behind the new flow. Both replacement sites now take the displaced cancel over in the same critical section, via a shared replaceOAuthFlow, and invoke it after the unlock. * [client] Guard OAuth flow mutations by the flow that owns the wait * [client] Arm the account prompt only from the wait that owns the flow * [client] Adopt the three-value parseEmailFromIDToken in the device flow The main merge brought in the device flow's email extraction from #7193, which still used the two-value signature this branch replaced when account matching was narrowed to the email claim. Git merged the files without a textual conflict, so the branch stopped compiling. Take the fromEmailClaim result and fill EmailClaim from it, the same way the PKCE path does, so device-flow clients get the same account matching. * [client] Populate the pending extend flow in the test server helper SwitchProfile cancels and clears the pending session extend flow unconditionally, the same way it clears the SSH JWT cache. New always populates the field, but the hand-assembled test server did not, so TestSwitchProfile_ClearsJWTCache panicked on a nil PendingFlow.
535 lines
17 KiB
Go
535 lines
17 KiB
Go
package auth
|
|
|
|
import (
|
|
"context"
|
|
"net/url"
|
|
"strings"
|
|
"sync"
|
|
"time"
|
|
|
|
"github.com/cenkalti/backoff/v4"
|
|
"github.com/google/uuid"
|
|
log "github.com/sirupsen/logrus"
|
|
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
|
|
"google.golang.org/grpc/codes"
|
|
"google.golang.org/grpc/status"
|
|
|
|
"github.com/netbirdio/netbird/client/internal/profilemanager"
|
|
"github.com/netbirdio/netbird/client/ssh"
|
|
"github.com/netbirdio/netbird/client/system"
|
|
mgm "github.com/netbirdio/netbird/shared/management/client"
|
|
"github.com/netbirdio/netbird/shared/management/client/common"
|
|
mgmProto "github.com/netbirdio/netbird/shared/management/proto"
|
|
)
|
|
|
|
// peerLoginExpiredMsg is the exact phrase the management server returns
|
|
// when a previously SSO-enrolled peer's login has expired. Sourced from
|
|
// shared/management/status/error.go (NewPeerLoginExpiredError). Matched
|
|
// by substring so a future server-side rewording that keeps the phrase
|
|
// still triggers the friendly fallback in Login().
|
|
const peerLoginExpiredMsg = "peer login has expired"
|
|
|
|
// errSetupKeyOnSSOExpiredPeer replaces the raw management error when the
|
|
// user runs `netbird login -k <setup-key>` against a peer that was
|
|
// originally enrolled via SSO. Wrapped in a PermissionDenied gRPC status
|
|
// so callers' existing isPermissionDenied / isAuthError checks still
|
|
// classify it correctly (early-exit from retry backoff, StatusNeedsLogin
|
|
// in the server state machine).
|
|
var errSetupKeyOnSSOExpiredPeer = status.Error(
|
|
codes.PermissionDenied,
|
|
"this peer was originally enrolled via SSO and its session has expired. "+
|
|
"Setup keys can only enrol new peers — run `netbird up` (interactive SSO) to re-login.",
|
|
)
|
|
|
|
// Auth manages authentication operations with the management server
|
|
// It maintains a long-lived connection and automatically handles reconnection with backoff
|
|
type Auth struct {
|
|
mutex sync.RWMutex
|
|
client *mgm.GrpcClient
|
|
config *profilemanager.Config
|
|
privateKey wgtypes.Key
|
|
mgmURL *url.URL
|
|
mgmTLSEnabled bool
|
|
}
|
|
|
|
// NewAuth creates a new Auth instance that manages authentication flows
|
|
// It establishes a connection to the management server that will be reused for all operations
|
|
// The connection is automatically recreated with backoff if it becomes disconnected
|
|
func NewAuth(ctx context.Context, privateKey string, mgmURL *url.URL, config *profilemanager.Config) (*Auth, error) {
|
|
// Validate WireGuard private key
|
|
myPrivateKey, err := wgtypes.ParseKey(privateKey)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
// Determine TLS setting based on URL scheme
|
|
mgmTLSEnabled := mgmURL.Scheme == "https"
|
|
|
|
log.Debugf("connecting to Management Service %s", mgmURL.String())
|
|
mgmClient, err := mgm.NewClient(ctx, mgmURL.Host, myPrivateKey, mgmTLSEnabled)
|
|
if err != nil {
|
|
log.Errorf("failed connecting to Management Service %s: %v", mgmURL.String(), err)
|
|
return nil, err
|
|
}
|
|
|
|
log.Debugf("connected to the Management service %s", mgmURL.String())
|
|
|
|
return &Auth{
|
|
client: mgmClient,
|
|
config: config,
|
|
privateKey: myPrivateKey,
|
|
mgmURL: mgmURL,
|
|
mgmTLSEnabled: mgmTLSEnabled,
|
|
}, nil
|
|
}
|
|
|
|
// Close closes the management client connection
|
|
func (a *Auth) Close() error {
|
|
a.mutex.Lock()
|
|
defer a.mutex.Unlock()
|
|
|
|
if a.client == nil {
|
|
return nil
|
|
}
|
|
return a.client.Close()
|
|
}
|
|
|
|
// IsSSOSupported checks if the management server supports SSO by attempting to retrieve auth flow configurations.
|
|
// Returns true if either PKCE or Device authorization flow is supported, false otherwise.
|
|
// This function encapsulates the SSO detection logic to avoid exposing gRPC error codes to upper layers.
|
|
// Automatically retries with backoff and reconnection on connection errors.
|
|
func (a *Auth) IsSSOSupported(ctx context.Context) (bool, error) {
|
|
var supportsSSO bool
|
|
|
|
err := a.withRetry(ctx, func(client *mgm.GrpcClient) error {
|
|
// Try PKCE flow first
|
|
_, err := a.getPKCEFlow(client, false)
|
|
if err == nil {
|
|
supportsSSO = true
|
|
return nil
|
|
}
|
|
|
|
// Check if PKCE is not supported
|
|
if s, ok := status.FromError(err); ok && (s.Code() == codes.NotFound || s.Code() == codes.Unimplemented) {
|
|
// PKCE not supported, try Device flow
|
|
_, err = a.getDeviceFlow(client)
|
|
if err == nil {
|
|
supportsSSO = true
|
|
return nil
|
|
}
|
|
|
|
// Check if Device flow is also not supported
|
|
if s, ok := status.FromError(err); ok && (s.Code() == codes.NotFound || s.Code() == codes.Unimplemented) {
|
|
// Neither PKCE nor Device flow is supported
|
|
supportsSSO = false
|
|
return nil
|
|
}
|
|
|
|
// Device flow check returned an error other than NotFound/Unimplemented
|
|
return err
|
|
}
|
|
|
|
// PKCE flow check returned an error other than NotFound/Unimplemented
|
|
return err
|
|
})
|
|
|
|
return supportsSSO, err
|
|
}
|
|
|
|
// GetOAuthFlow returns an OAuth flow (PKCE or Device) using the existing management connection.
|
|
// This avoids creating a new connection to the management server.
|
|
//
|
|
// sessionExtend marks the flow as renewing an existing peer's session rather than
|
|
// logging one in; the server needs it to rule out a silent authorization that the
|
|
// IdP could answer from another account. See PKCEAuthorizationFlowRequest.
|
|
func (a *Auth) GetOAuthFlow(ctx context.Context, forceDeviceAuth bool, sessionExtend bool, hint string) (OAuthFlow, error) {
|
|
var flow OAuthFlow
|
|
|
|
err := a.withRetry(ctx, func(client *mgm.GrpcClient) error {
|
|
if forceDeviceAuth {
|
|
deviceFlow, err := a.getDeviceFlow(client)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
deviceFlow.SetLoginHint(hint)
|
|
flow = deviceFlow
|
|
return nil
|
|
}
|
|
|
|
// Try PKCE flow first
|
|
pkceFlow, err := a.getPKCEFlow(client, sessionExtend)
|
|
if err != nil {
|
|
// If PKCE not supported, try Device flow
|
|
if s, ok := status.FromError(err); ok && (s.Code() == codes.NotFound || s.Code() == codes.Unimplemented) {
|
|
deviceFlow, err := a.getDeviceFlow(client)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
deviceFlow.SetLoginHint(hint)
|
|
flow = deviceFlow
|
|
return nil
|
|
}
|
|
return err
|
|
}
|
|
pkceFlow.SetLoginHint(hint)
|
|
flow = pkceFlow
|
|
return nil
|
|
})
|
|
|
|
return flow, err
|
|
}
|
|
|
|
// IsLoginRequired checks if login is required by attempting to authenticate with the server
|
|
// Automatically retries with backoff and reconnection on connection errors.
|
|
func (a *Auth) IsLoginRequired(ctx context.Context) (bool, error) {
|
|
pubSSHKey, err := ssh.GeneratePublicKey([]byte(a.config.SSHKey))
|
|
if err != nil {
|
|
return false, err
|
|
}
|
|
|
|
var needsLogin bool
|
|
|
|
err = a.withRetry(ctx, func(client *mgm.GrpcClient) error {
|
|
err := a.doMgmLogin(client, ctx, pubSSHKey)
|
|
if isLoginNeeded(err) {
|
|
needsLogin = true
|
|
return nil
|
|
}
|
|
needsLogin = false
|
|
return err
|
|
})
|
|
|
|
return needsLogin, err
|
|
}
|
|
|
|
// Login attempts to log in or register the client with the management server
|
|
// Returns error and a boolean indicating if it's an authentication error (permission denied) that should stop retries.
|
|
// Automatically retries with backoff and reconnection on connection errors.
|
|
func (a *Auth) Login(ctx context.Context, setupKey string, jwtToken string) (error, bool) {
|
|
pubSSHKey, err := ssh.GeneratePublicKey([]byte(a.config.SSHKey))
|
|
if err != nil {
|
|
return err, false
|
|
}
|
|
|
|
var isAuthError bool
|
|
|
|
err = a.withRetry(ctx, func(client *mgm.GrpcClient) error {
|
|
err := a.doMgmLogin(client, ctx, pubSSHKey)
|
|
if isRegistrationNeeded(err) {
|
|
log.Debugf("peer registration required")
|
|
_, err = a.registerPeer(client, ctx, setupKey, jwtToken, pubSSHKey)
|
|
if err != nil {
|
|
// The peer pub-key is already on file with the management
|
|
// server (originally enrolled via SSO) and the session has
|
|
// expired. The setup-key path can only enrol new peers, so
|
|
// retrying with -k will keep failing. Replace the raw mgm
|
|
// message with an actionable hint that tells the user to
|
|
// re-authenticate via SSO instead.
|
|
if setupKey != "" && jwtToken == "" && isPeerLoginExpired(err) {
|
|
err = errSetupKeyOnSSOExpiredPeer
|
|
}
|
|
isAuthError = isPermissionDenied(err)
|
|
return err
|
|
}
|
|
} else if err != nil {
|
|
isAuthError = isPermissionDenied(err)
|
|
return err
|
|
}
|
|
|
|
isAuthError = false
|
|
return nil
|
|
})
|
|
|
|
return err, isAuthError
|
|
}
|
|
|
|
// getPKCEFlow retrieves PKCE authorization flow configuration and creates a flow instance
|
|
func (a *Auth) getPKCEFlow(client *mgm.GrpcClient, sessionExtend bool) (*PKCEAuthorizationFlow, error) {
|
|
protoFlow, err := client.GetPKCEAuthorizationFlow(sessionExtend)
|
|
if err != nil {
|
|
if s, ok := status.FromError(err); ok && s.Code() == codes.NotFound {
|
|
log.Warnf("server couldn't find pkce flow, contact admin: %v", err)
|
|
return nil, err
|
|
}
|
|
log.Errorf("failed to retrieve pkce flow: %v", err)
|
|
return nil, err
|
|
}
|
|
|
|
protoConfig := protoFlow.GetProviderConfig()
|
|
config := &PKCEAuthProviderConfig{
|
|
Audience: protoConfig.GetAudience(),
|
|
ClientID: protoConfig.GetClientID(),
|
|
ClientSecret: protoConfig.GetClientSecret(), //nolint:staticcheck
|
|
TokenEndpoint: protoConfig.GetTokenEndpoint(),
|
|
AuthorizationEndpoint: protoConfig.GetAuthorizationEndpoint(),
|
|
Scope: protoConfig.GetScope(),
|
|
RedirectURLs: protoConfig.GetRedirectURLs(),
|
|
UseIDToken: protoConfig.GetUseIDToken(),
|
|
ClientCertPair: a.config.ClientCertKeyPair,
|
|
DisablePromptLogin: protoConfig.GetDisablePromptLogin(),
|
|
LoginFlag: common.LoginFlag(protoConfig.GetLoginFlag()),
|
|
}
|
|
|
|
if err := validatePKCEConfig(config); err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
flow, err := NewPKCEAuthorizationFlow(*config)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
return flow, nil
|
|
}
|
|
|
|
// getDeviceFlow retrieves device authorization flow configuration and creates a flow instance
|
|
func (a *Auth) getDeviceFlow(client *mgm.GrpcClient) (*DeviceAuthorizationFlow, error) {
|
|
protoFlow, err := client.GetDeviceAuthorizationFlow()
|
|
if err != nil {
|
|
if s, ok := status.FromError(err); ok && s.Code() == codes.NotFound {
|
|
log.Warnf("server couldn't find device flow, contact admin: %v", err)
|
|
return nil, err
|
|
}
|
|
log.Errorf("failed to retrieve device flow: %v", err)
|
|
return nil, err
|
|
}
|
|
|
|
protoConfig := protoFlow.GetProviderConfig()
|
|
config := &DeviceAuthProviderConfig{
|
|
Audience: protoConfig.GetAudience(),
|
|
ClientID: protoConfig.GetClientID(),
|
|
ClientSecret: protoConfig.GetClientSecret(), //nolint:staticcheck
|
|
Domain: protoConfig.Domain,
|
|
TokenEndpoint: protoConfig.GetTokenEndpoint(),
|
|
DeviceAuthEndpoint: protoConfig.GetDeviceAuthEndpoint(),
|
|
Scope: protoConfig.GetScope(),
|
|
UseIDToken: protoConfig.GetUseIDToken(),
|
|
}
|
|
|
|
// Keep compatibility with older management versions
|
|
if config.Scope == "" {
|
|
config.Scope = "openid"
|
|
}
|
|
|
|
if err := validateDeviceAuthConfig(config); err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
flow, err := NewDeviceAuthorizationFlow(*config)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
return flow, nil
|
|
}
|
|
|
|
// doMgmLogin performs the actual login operation with the management service
|
|
func (a *Auth) doMgmLogin(client *mgm.GrpcClient, ctx context.Context, pubSSHKey []byte) error {
|
|
sysInfo := system.GetInfo(ctx)
|
|
a.setSystemInfoFlags(sysInfo)
|
|
_, err := client.Login(sysInfo, pubSSHKey, a.config.DNSLabels)
|
|
return err
|
|
}
|
|
|
|
// registerPeer checks whether setupKey was provided via cmd line and if not then it prompts user to enter a key.
|
|
// Otherwise tries to register with the provided setupKey via command line.
|
|
func (a *Auth) registerPeer(client *mgm.GrpcClient, ctx context.Context, setupKey string, jwtToken string, pubSSHKey []byte) (*mgmProto.LoginResponse, error) {
|
|
validSetupKey, err := uuid.Parse(setupKey)
|
|
if err != nil && jwtToken == "" {
|
|
return nil, status.Errorf(codes.InvalidArgument, "invalid setup-key or no sso information provided, err: %v", err)
|
|
}
|
|
|
|
log.Debugf("sending peer registration request to Management Service")
|
|
info := system.GetInfo(ctx)
|
|
a.setSystemInfoFlags(info)
|
|
loginResp, err := client.Register(validSetupKey.String(), jwtToken, info, pubSSHKey, a.config.DNSLabels)
|
|
if err != nil {
|
|
log.Errorf("failed registering peer %v", err)
|
|
return nil, err
|
|
}
|
|
|
|
log.Infof("peer has been successfully registered on Management Service")
|
|
|
|
return loginResp, nil
|
|
}
|
|
|
|
// setSystemInfoFlags sets all configuration flags on the provided system info
|
|
func (a *Auth) setSystemInfoFlags(info *system.Info) {
|
|
info.SetFlags(
|
|
a.config.RosenpassEnabled,
|
|
a.config.RosenpassPermissive,
|
|
a.config.ServerSSHAllowed,
|
|
a.config.DisableClientRoutes,
|
|
a.config.DisableServerRoutes,
|
|
a.config.DisableDNS,
|
|
a.config.DisableFirewall,
|
|
a.config.BlockLANAccess,
|
|
a.config.BlockInbound,
|
|
a.config.DisableIPv6,
|
|
a.config.SyncMessageVersion,
|
|
a.config.EnableSSHRoot,
|
|
a.config.EnableSSHSFTP,
|
|
a.config.EnableSSHLocalPortForwarding,
|
|
a.config.EnableSSHRemotePortForwarding,
|
|
a.config.DisableSSHAuth,
|
|
a.config.RemoteJobsAllowed,
|
|
)
|
|
}
|
|
|
|
// reconnect closes the current connection and creates a new one
|
|
// It checks if the brokenClient is still the current client before reconnecting
|
|
// to avoid multiple threads reconnecting unnecessarily
|
|
func (a *Auth) reconnect(ctx context.Context, brokenClient *mgm.GrpcClient) error {
|
|
a.mutex.Lock()
|
|
defer a.mutex.Unlock()
|
|
|
|
// Double-check: if client has already been replaced by another thread, skip reconnection
|
|
if a.client != brokenClient {
|
|
log.Debugf("client already reconnected by another thread, skipping")
|
|
return nil
|
|
}
|
|
|
|
// Create new connection FIRST, before closing the old one
|
|
// This ensures a.client is never nil, preventing panics in other threads
|
|
log.Debugf("reconnecting to Management Service %s", a.mgmURL.String())
|
|
mgmClient, err := mgm.NewClient(ctx, a.mgmURL.Host, a.privateKey, a.mgmTLSEnabled)
|
|
if err != nil {
|
|
log.Errorf("failed reconnecting to Management Service %s: %v", a.mgmURL.String(), err)
|
|
// Keep the old client if reconnection fails
|
|
return err
|
|
}
|
|
|
|
// Close old connection AFTER new one is successfully created
|
|
oldClient := a.client
|
|
a.client = mgmClient
|
|
|
|
if oldClient != nil {
|
|
if err := oldClient.Close(); err != nil {
|
|
log.Debugf("error closing old connection: %v", err)
|
|
}
|
|
}
|
|
|
|
log.Debugf("successfully reconnected to Management service %s", a.mgmURL.String())
|
|
return nil
|
|
}
|
|
|
|
// isConnectionError checks if the error is a connection-related error that should trigger reconnection
|
|
func isConnectionError(err error) bool {
|
|
if err == nil {
|
|
return false
|
|
}
|
|
s, ok := status.FromError(err)
|
|
if !ok {
|
|
return false
|
|
}
|
|
// These error codes indicate connection issues
|
|
return s.Code() == codes.Unavailable ||
|
|
s.Code() == codes.DeadlineExceeded ||
|
|
s.Code() == codes.Canceled ||
|
|
s.Code() == codes.Internal
|
|
}
|
|
|
|
// withRetry wraps an operation with exponential backoff retry logic
|
|
// It automatically reconnects on connection errors
|
|
func (a *Auth) withRetry(ctx context.Context, operation func(client *mgm.GrpcClient) error) error {
|
|
backoffSettings := &backoff.ExponentialBackOff{
|
|
InitialInterval: 500 * time.Millisecond,
|
|
RandomizationFactor: 0.5,
|
|
Multiplier: 1.5,
|
|
MaxInterval: 10 * time.Second,
|
|
MaxElapsedTime: 2 * time.Minute,
|
|
Stop: backoff.Stop,
|
|
Clock: backoff.SystemClock,
|
|
}
|
|
backoffSettings.Reset()
|
|
|
|
return backoff.RetryNotify(
|
|
func() error {
|
|
// Capture the client BEFORE the operation to ensure we track the correct client
|
|
a.mutex.RLock()
|
|
currentClient := a.client
|
|
a.mutex.RUnlock()
|
|
|
|
if currentClient == nil {
|
|
return status.Errorf(codes.Unavailable, "client is not initialized")
|
|
}
|
|
|
|
// Execute operation with the captured client
|
|
err := operation(currentClient)
|
|
if err == nil {
|
|
return nil
|
|
}
|
|
|
|
// If it's a connection error, attempt reconnection using the client that was actually used
|
|
if isConnectionError(err) {
|
|
log.Warnf("connection error detected, attempting reconnection: %v", err)
|
|
|
|
if reconnectErr := a.reconnect(ctx, currentClient); reconnectErr != nil {
|
|
log.Errorf("reconnection failed: %v", reconnectErr)
|
|
return reconnectErr
|
|
}
|
|
// Return the original error to trigger retry with the new connection
|
|
return err
|
|
}
|
|
|
|
// For authentication errors, don't retry
|
|
if isAuthenticationError(err) {
|
|
return backoff.Permanent(err)
|
|
}
|
|
|
|
return err
|
|
},
|
|
backoff.WithContext(backoffSettings, ctx),
|
|
func(err error, duration time.Duration) {
|
|
log.Warnf("operation failed, retrying in %v: %v", duration, err)
|
|
},
|
|
)
|
|
}
|
|
|
|
// isAuthenticationError checks if the error is an authentication-related error that should not be retried.
|
|
// Returns true if the error is InvalidArgument or PermissionDenied, indicating that retrying won't help.
|
|
func isAuthenticationError(err error) bool {
|
|
if err == nil {
|
|
return false
|
|
}
|
|
s, ok := status.FromError(err)
|
|
if !ok {
|
|
return false
|
|
}
|
|
return s.Code() == codes.InvalidArgument || s.Code() == codes.PermissionDenied
|
|
}
|
|
|
|
// isPermissionDenied checks if the error is a PermissionDenied error.
|
|
// This is used to determine if early exit from backoff is needed (e.g., when the server responded but denied access).
|
|
func isPermissionDenied(err error) bool {
|
|
if err == nil {
|
|
return false
|
|
}
|
|
s, ok := status.FromError(err)
|
|
if !ok {
|
|
return false
|
|
}
|
|
return s.Code() == codes.PermissionDenied
|
|
}
|
|
|
|
func isLoginNeeded(err error) bool {
|
|
return isAuthenticationError(err)
|
|
}
|
|
|
|
func isRegistrationNeeded(err error) bool {
|
|
return isPermissionDenied(err)
|
|
}
|
|
|
|
// isPeerLoginExpired reports whether err is the management server's
|
|
// "peer login has expired" PermissionDenied response. Used by Login to
|
|
// detect the case where the caller passed a setup-key but the peer is
|
|
// actually an SSO-enrolled record whose session needs refreshing — the
|
|
// setup-key path cannot help there.
|
|
func isPeerLoginExpired(err error) bool {
|
|
if !isPermissionDenied(err) {
|
|
return false
|
|
}
|
|
s, _ := status.FromError(err)
|
|
return strings.Contains(s.Message(), peerLoginExpiredMsg)
|
|
}
|