OnRekeyFailed now re-runs the KEM bootstrap over Signal (conn.RequestReoffer ->
handshaker.SendOffer) instead of only logging: a fresh signalling offer starts a new
exchange that overwrites the stalled PSK on both sides, resyncing after a persistent
data-path desync. Chosen over a responder-side awaitingAck revert (which fights the
confirm-less ack timing) and a full tunnel teardown (heavier). The tunnel stays up on
the previous PSK meanwhile since Signal is independent of the broken data path.
Strict mode (NB_PQ_MLKEM_STRICT, default off) closes the initial PQ-vulnerable
window (NET-1408): when enabled, conn.presharedKey programs a per-conn random
sentinel PSK until the ML-KEM exchange derives the real one, so no session can form
on a non-PQ key (the real PSK is pushed via SetPresharedKey once it converges).
Default stays opportunistic.
Also surface PQ status: the peer 'Quantum resistance' flag (RosenpassEnabled) is now
true when an ML-KEM PSK has been derived for the peer, not only for Rosenpass.
The idle-gate reads LastActivities, which only tracks per-peer data in userspace;
in kernel mode it is empty, so the gate treated every kernel peer as idle and
disabled data-path rotation entirely. Detect the bind via IsUserspaceBind and, in
kernel mode, report zero activity age (always 'active') so rotation runs on every
rekey. Lazy back-to-idle is already limited in kernel; the eBPF WG-activity
detection will later supply a real signal that excludes handshake/pqkem traffic.
Replace the raw SHA-256 concat combiner with HKDF-SHA256 (crypto/hkdf, Go 1.24):
IKM = ML-KEM_ss || X25519_ss (draft-ietf-tls-ecdhe-mlkem order), salt = the
domain-separation label, info = full transcript (offer || answer) || canonicalised
peer identities. Keeps the transcript + identity binding while using a proper KDF.
- Add a trace slog level (NB_PQ_MLKEM_LOG_LEVEL=trace) and move the verbose
per-exchange lifecycle logs (offer/answer/PSK/ack/rotation) to it, so debug
stays quiet and troubleshooting is opt-in.
- Stop logging the raw preshared key; drop the temporary pqkem-dbg OnRemoteOffer/
OnRemoteAnswer probes.
- Demote the per-handshake conn log to trace.
Source OnDataPathRekeyed from the WGWatcher's per-handshake callback
(onWGCheckSuccess), which fires only on a fresh handshake, and OnDataPathDown
from the handshake-timeout path. A fresh handshake clocks the next chained
KEM exchange pushed over the data-path UDP transport.
Learn the peer's data-path endpoint from the signalling offer/answer: its WG
overlay IP combined with the advertised pq UDP port (SetRemotePort -> AddPeer).
Registering here is safe before the tunnel is up because sends only ever fire
once it is (clocked by OnDataPathRekeyed). RemovePeer is wired at peer teardown
(engine.removePeer), not on transient disconnect.
We clock the next Offer initiation to the OnDataPathRekeyed, so we have 2 minutes
ahead of us to do our attempts and stuff before to give up.
On failure, we will know because we will not receive a new answer.. but more importantly
the wg handshake will fail :D
Define OnDataPathRekeyed event to transition from control plane path to data plane path over the WG tunnel.
Keep confirm ALWAYS on NEW established WG tunnel (posthandshake with rekeying). We keep an active method
irrelevant of the WG handshake (we might decide that the indirect wg handshake is sufficient in the future).
Optimistic commit on responder(when sending answer), while on initiator we set it on getting the answer
- Have just one manager => one lock
- Session state is needed in driver to => we have it available now.
- Isomorphically align to rosenpass components and functionality
File Role rosenpass equivalent
kem.go primitive pure X25519MLKEM768 crypto.go/handshake
message.go Offer/Answer/Confirm + Encode/Decode messages.go
manager.go Manager stateful, single lock server logic
callbacks.go WGCallbackHandler (seam output) Handler
Transport (interfaccia) seam trasporto pluggable Conn
* [client] Track the WireGuard device on the engine as a lock-free handle
Add an atomic handle on the wg device next to wgInterface, stored once the
interface is up and cleared when it is closed. Nothing reads it yet, so this
is a pure addition with no behavior change; it exists so the next commit can
reach the device without taking syncMsgMux.
* [client] Retune the WireGuard buffer pool without the engine lock
SetPerformance took syncMsgMux before reaching the device. That lock is held
by handleSync while it adds and removes peers, and peer removal is exactly
what blocks when a device's buffer pool is exhausted: Peer.Stop waits on a
keepalive timer callback that is itself parked in WaitPool.Get. Raising the
cap is the way out of that state, so the call must not queue behind the lock
the stall is holding.
Read the device through the atomic handle instead. Device.SetPreallocatedBuffersPerPool
takes the pool's own lock and broadcasts, so the waiters wake up.
* [proxy] Extract the buffer-cap apply loop out of the perf handler
Pure move: the loop over the registered clients becomes applyBufferCap, with
the same sequential behavior and the same return values. Split out so the next
commit can change how it iterates without the diff also carrying the move.
* [proxy] Bound the perf endpoint so one wedged client cannot hold it
The apply loop was sequential and unbounded. embed.Client.SetPerformance goes
through the client lock, which Start holds for the whole of a startup, so a
single account that is busy or wedged delayed the new buffer cap for every
other account on the node -- on the endpoint whose whole purpose is to
un-wedge a node.
Apply to all clients concurrently and give the whole call a 5s budget.
Accounts that do not answer in time are reported in "failed" instead of
blocking the response.
* [client] Drop the device handle before closing the interface
close() cleared the atomic handle only after wgInterface.Close() returned, so a
concurrent SetPerformance could still load it, retune a device that is being
torn down, and report the change as applied for an engine that has stopped.
Clear it first, so the window closes before the teardown begins.
Reported by cubic on PR #7452.
* [proxy] Put the per-client retune behind a field
Pure refactor: applyBufferCap calls h.setPerformance instead of the client
method directly, and NewHandler wires it to setClientPerformance. Same call,
same behavior; the seam is what lets the next two commits be tested without a
live embedded client.
* [proxy] Do not report a finished retune as timed out
When the deadline fires, select chooses at random among the ready cases, so a
result already sitting in the buffered channel could be skipped and its account
reported as timed out even though the cap had been applied. Drain what is
buffered before declaring the rest pending.
Reported by cubic on PR #7452.
* [proxy] Keep one retune per account in flight
The 5s budget bounds how long the endpoint waits, not the work: SetPerformance
goes through the embedded client's lock, and on a wedged account Stop holds that
lock forever, so every retry left one more goroutine parked there.
Route each account through a single worker. A request that finds one already
running takes its result if it has landed, and otherwise reports the account
under "in_flight" instead of starting a second attempt. One stuck account now
costs one goroutine, no matter how often the endpoint is called.
Reported by CodeRabbit and cubic on PR #7452.
* [proxy] Make the retune budget a var
Pure refactor: perfApplyTimeout becomes a var so a test can shorten it instead
of waiting five seconds. Same value, same behavior in production.
* [proxy] Extract the buffered-result drain
Pure refactor: the loop that empties the results channel when the deadline
fires becomes collectBuffered. Same behavior; split out so it can be tested
on its own, which the inline version could not be without racing the deadline.
* [proxy] Cover the retune single-flight and the deadline drain
TestApplyBufferCapSingleFlightPerAccount fails without the worker registry:
five calls against a client stuck in its own lock start five blocked workers
instead of one.
TestCollectBufferedCountsResultsReadyAtTheDeadline pins the drain helper's
contract - buffered results counted, errors recorded, only unanswered accounts
left pending. It drives collectBuffered directly: through applyBufferCap the
two select cases race by construction, so an end-to-end version of it would
pass on the unfixed code about half the time.
* [proxy] Keep the worker alongside each pending account
Pure refactor: the pending set becomes a map to the account's worker instead of
an empty struct. Same membership and same behavior; the next commit needs the
worker to resolve an account whose result has not reached the channel yet.
* [proxy] Publish a retune result before releasing its slot
The worker sent its result last, after taking perfMu to remove itself from the
registry. That lock is taken once per account by every caller walking the fleet,
so a worker that finished on time could queue behind an apply over thousands of
accounts and land after the deadline. Send first, deregister after.
Reported by cubic on PR #7452.
* [proxy] Read the worker, not the clock, for a finished retune
Publishing earlier only narrows the window: a client that answers just before
the deadline can still be reported as timed out. At the deadline the workers
themselves are authoritative - a closed done channel means the retune finished
and w.err carries its outcome, ordered by the close. Consult them instead of
declaring every pending account timed out, and keep the timeout label for the
ones actually still running.
Reported by cubic on PR #7452.
* [proxy] Cover the finished-worker resolution at the deadline
Fails on the previous behavior with "applied = 0, want 1": every pending
account was labelled a timeout, including the one whose retune had already
completed.
* [client] Add a release-wired rootless UBI image variant
* [client] Add ARM64 to the rootless UBI image
* [client] Express license output validation as a guard
riccardom