mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-23 20:25:56 +00:00
Keep deployed agents' command channel across restarts (#2542)
## Problem Agents installed through the one-step cluster deploy lose their command channel for good after their first restart. `POST /api/agents/agent/enroll` bound the runtime token to an invented `agent-<hostname>` and returned that as the agent's ID. The agent used it until Pulse acknowledged its first report with the host's real, machine-derived ID, which the agent persists. On the next start the agent registered under that real ID. The token was by then a version-2 binding to `agent-<hostname>`, so admission refused it with "Agent token mismatch: token is not bound to the registering agent", every few minutes, indefinitely. Seen live on a v6.5.0-rc.1 server with a deployed Proxmox node: reports flowed, but the node could run no commands or actions. ## Fix - **Enrollment leaves identity to the agent.** The runtime token is bound to the node's hostname and the current binding version, marked `deploy_identity_binding=agent`, and the response no longer carries `agentId`. The first command registration backfills the agent's own ID through the existing hostname-bound backfill path, and the config gate takes the same decision. - **One-time repair for tokens already issued.** A token enrollment bound to `agent-<hostname>` (it has `deploy_job_id` and no identity marker) may move once to the registering agent's ID: - The hostname must match exactly (case-insensitive), not by short-name equivalence. - The move persists `deploy_identity_binding=repaired` in the same save, and a failed save restores the previous metadata. - It applies whatever the token's binding version. - Any other identity or hostname is refused rather than handed to the legacy hostname migration. - A marked token never moves again, including after a hostname rebind recreates the `agent-<hostname>` shape. ## Tests - **`TestHandleEnroll_RuntimeTokenBindsTheAgentsOwnIdentityOnFirstCommandRegistration`** replays the field sequence: the identity enrollment handed out, then the acknowledged identity after a restart. It fails on the old code at the "registration after restart" step. - **`TestDeployPlaceholderTokenMovesToTheRegisteringAgentOnce`** covers: - the repair on the exact hostname only; - the persisted marker after a reload; - the repair, rebind to host b, repair-again attack; - config-gate parity. - **`TestVersionlessDeployPlaceholderCannotBeMovedTwice`** covers the versionless variant of that attack. - **`TestCurrentDeployTokenNeverMovesOnceBound`** checks that a token issued by current enrollment never moves once bound. - **Policy unit tests:** `TestEvaluateRepairsDeployPlaceholderBindingOnce`, `TestEvaluateVersionlessDeployPlaceholderOnlyMovesThroughTheRepair` and `TestEvaluateCurrentDeployTokenBackfillsTheAgentIdentity`. - **`TestContractDeployEnrollResponseCarriesNoAgentIdentity`** pins the enroll response keys. - **Mutation checks:** removing the marker check, loosening the hostname match, not persisting the marker, or letting placeholders fall through to the legacy branch each fails at least one of these tests. ## Not in this change Both of these exist for every install path and are not made worse here: - The first command registration binds whatever ID the agent presents. If that differs from the identity Pulse later resolves for its reports (a configured agent ID, or a host continuity fork), the next restart still mismatches. - Tokens that never connect the command channel keep an empty `bound_agent_id`, so collector-authority reduction cannot match them. Before this change they had a non-empty but wrong `agent-<hostname>`, which failed the same way.
This commit is contained in:
@@ -8233,6 +8233,24 @@ to the hostname the agent then reports under.
|
||||
`internal/hostagent/enroll_test.go`, including
|
||||
`TestEnrollPresentsTheIdentityNewReports`, pin that order and outcome.
|
||||
|
||||
### Deploy enrollment leaves agent identity to the agent
|
||||
|
||||
A deployed agent reports under its machine-derived identity, Pulse
|
||||
acknowledges that identity, and the agent persists it, so its command channel
|
||||
must bind to that same identity. Enrollment therefore binds the runtime token
|
||||
to the node's hostname and the current binding version only and returns no
|
||||
agent ID; the first command-channel registration backfills the agent's own ID.
|
||||
Enrollment previously bound the token to, and returned, an invented
|
||||
`agent-<hostname>`. The agent used it until its first acknowledged report
|
||||
replaced it, after which every restart registered under the real ID and the
|
||||
command channel refused it permanently. Tokens issued that way move once to
|
||||
the registering agent's ID on the same host, under the binding rule in the
|
||||
security-privacy contract.
|
||||
`TestHandleEnroll_RuntimeTokenBindsTheAgentsOwnIdentityOnFirstCommandRegistration`
|
||||
in `internal/api/deploy_handlers_test.go` replays that sequence: the identity
|
||||
enrollment handed out, or the agent's own, then the acknowledged identity after
|
||||
a restart.
|
||||
|
||||
### Alert push routing does not create an agent lifecycle channel
|
||||
|
||||
The shared API router may classify an external-probe outage for specialized
|
||||
|
||||
@@ -10975,6 +10975,13 @@ when persistence fails. `TestHandleEnroll_Success`,
|
||||
`internal/api/deploy_handlers_test.go` prove the persisted success state and
|
||||
both failed-commit responses.
|
||||
|
||||
The success response carries exactly `runtimeToken`, `runtimeTokenId`, and
|
||||
`reportInterval`. It names no agent identity: the agent keeps the
|
||||
machine-derived ID its reports carry, and the runtime token binds that ID on
|
||||
the agent's first command registration.
|
||||
`TestContractDeployEnrollResponseCarriesNoAgentIdentity` in
|
||||
`internal/api/contract_test.go` pins the shape.
|
||||
|
||||
### Notification destination writes report committed truth
|
||||
|
||||
`PUT /api/notifications/email`, `PUT /api/notifications/apprise`, and webhook
|
||||
|
||||
@@ -1450,6 +1450,31 @@ binding version through the first-use path, an unrelated hostname still fails
|
||||
closed, and the registration-bound hostname is never overwritten by an
|
||||
equivalent spelling the agent reports, because the install grant compares
|
||||
against it.
|
||||
Deploy-enrolled runtime tokens follow the same immutable-identity rule.
|
||||
`POST /api/agents/agent/enroll` binds the runtime token to the enrolled node's
|
||||
hostname and the current binding version only, marks its identity as coming
|
||||
from the agent (`deploy_identity_binding=agent`), and returns no agent
|
||||
identity, so the agent's first command registration backfills its own
|
||||
machine-derived ID. Enrollment used to bind and return an invented
|
||||
`agent-<hostname>`, which the agent abandons once Pulse acknowledges its
|
||||
reports. A token still carrying that placeholder, its `deploy_job_id`, and no
|
||||
identity marker may move exactly once to the registering agent's ID on exactly
|
||||
the same hostname (case-insensitive, never the short-name equivalence),
|
||||
whatever its binding version, and the move persists
|
||||
`deploy_identity_binding=repaired` in the same save. Any other identity or
|
||||
hostname presented to such a token is refused outright rather than handed to
|
||||
the legacy hostname migration, which would rebind without the marker. A marked
|
||||
token never moves again, even after a hostname rebind recreates the
|
||||
`agent-<hostname>` shape, and a non-deploy token of that shape, a different or
|
||||
merely equivalent hostname, or a deploy token bound to a real ID fails closed;
|
||||
the agent config gate consumes the same decision.
|
||||
`TestEvaluateRepairsDeployPlaceholderBindingOnce` and
|
||||
`TestEvaluateCurrentDeployTokenBackfillsTheAgentIdentity` in
|
||||
`internal/api/agentbinding/policy_test.go`, and
|
||||
`TestDeployPlaceholderTokenMovesToTheRegisteringAgentOnce`,
|
||||
`TestVersionlessDeployPlaceholderCannotBeMovedTwice`, and
|
||||
`TestCurrentDeployTokenNeverMovesOnceBound` in
|
||||
`internal/api/security_regression_test.go` pin that boundary.
|
||||
Pulse-minted install tokens now carry a server-authored command-policy intent
|
||||
in addition to first-use binding metadata. That intent grants no independent
|
||||
authority: first-report convergence requires the same shared binding decision
|
||||
|
||||
@@ -6474,6 +6474,18 @@ a cluster out for good. This adds no backup, restore, retention, snapshot, or
|
||||
recovery authority. `internal/api/deploy_handlers_test.go` and
|
||||
`internal/deploy/store_test.go` pin the admission and the query.
|
||||
|
||||
### Deploy token identity repair is one persisted transition
|
||||
|
||||
The one-time move of a deploy runtime token from the placeholder
|
||||
`agent-<hostname>` identity to the registering agent's ID is written through
|
||||
the existing API-token persistence inside the same command-channel admission
|
||||
as other binding repairs, together with the `deploy_identity_binding=repaired`
|
||||
marker that prevents a second move. When that save fails, the previous binding
|
||||
metadata and marker are restored and registration is refused, so a failed
|
||||
write never leaves a half-moved token. This adds no backup, restore, retention, snapshot, or
|
||||
recovery authority. `internal/api/security_regression_test.go` pins the
|
||||
persisted binding.
|
||||
|
||||
### Destination severity policy preserves notification recovery identity
|
||||
|
||||
Email, webhook, Apprise, and Relay minimum-severity floors persist through
|
||||
|
||||
@@ -60,12 +60,13 @@ func agentExecHostnamesMatch(bound, requested string) bool {
|
||||
// already-issued agent exec token accepts a registering agent identity, and
|
||||
// which metadata repair the admission path must persist when it does.
|
||||
type agentExecBindingDecision struct {
|
||||
admit bool
|
||||
firstBind bool
|
||||
legacyMigrate bool
|
||||
rebindHostname bool
|
||||
backfillID bool
|
||||
backfillHost bool
|
||||
admit bool
|
||||
firstBind bool
|
||||
legacyMigrate bool
|
||||
rebindHostname bool
|
||||
backfillID bool
|
||||
backfillHost bool
|
||||
repairDeployIdentity bool
|
||||
}
|
||||
|
||||
// evaluateAgentExecBinding computes the admission decision for a token record
|
||||
@@ -84,6 +85,8 @@ func evaluateAgentExecBinding(record *config.APITokenRecord, requestedID, reques
|
||||
rebindHostname: decision.RebindHostname,
|
||||
backfillID: decision.BackfillID,
|
||||
backfillHost: decision.BackfillHost,
|
||||
|
||||
repairDeployIdentity: decision.RepairDeployIdentity,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -235,11 +238,15 @@ func (r *Router) admitAgentExecToken(token string, agentID string, hostname stri
|
||||
"bound_at",
|
||||
agentExecBindingVersionKey,
|
||||
agenttokens.RuntimeRoleMetadataKey,
|
||||
agentbinding.DeployIdentityKey,
|
||||
)
|
||||
record.Metadata["bound_agent_id"] = requestedID
|
||||
record.Metadata["bound_at"] = time.Now().UTC().Format(time.RFC3339)
|
||||
record.Metadata[agentExecBindingVersionKey] = agentExecBindingVersion
|
||||
record.Metadata[agenttokens.RuntimeRoleMetadataKey] = agenttokens.CredentialKindLegacyFullTrust
|
||||
if decision.repairDeployIdentity {
|
||||
record.Metadata[agentbinding.DeployIdentityKey] = agentbinding.DeployIdentityRepaired
|
||||
}
|
||||
if r.persistence != nil {
|
||||
if err := r.persistence.SaveAPITokens(r.config.APITokens); err != nil {
|
||||
restoreAgentExecMetadata(record.Metadata, previousMetadata)
|
||||
|
||||
@@ -14,6 +14,15 @@ const (
|
||||
Version = "2"
|
||||
IssuedViaConfig = "config_agent_install_command"
|
||||
IssuedViaHosted = "hosted_agent_install_command"
|
||||
|
||||
// DeployIdentityKey records where a deploy runtime token's agent identity
|
||||
// comes from. Enrollment that leaves identity to the agent writes
|
||||
// DeployIdentityAgent; the one-time repair of a token enrollment bound to
|
||||
// an invented agent-<hostname> writes DeployIdentityRepaired. A token with
|
||||
// either value is never repaired again.
|
||||
DeployIdentityKey = "deploy_identity_binding"
|
||||
DeployIdentityAgent = "agent"
|
||||
DeployIdentityRepaired = "repaired"
|
||||
)
|
||||
|
||||
type Decision struct {
|
||||
@@ -23,6 +32,10 @@ type Decision struct {
|
||||
RebindHostname bool
|
||||
BackfillID bool
|
||||
BackfillHost bool
|
||||
// RepairDeployIdentity marks a LegacyMigrate that moves a historical deploy
|
||||
// placeholder binding; the admission path must persist
|
||||
// DeployIdentityRepaired with the new identity so it cannot recur.
|
||||
RepairDeployIdentity bool
|
||||
}
|
||||
|
||||
// EvaluateActionRunner admits only a pre-bound, typed action credential whose
|
||||
@@ -65,6 +78,17 @@ func Evaluate(record *config.APITokenRecord, requestedID, requestedHost string)
|
||||
if canBindAutoRegisteredInstallToken(record, requestedID, requestedHost) {
|
||||
return Decision{Admit: true, FirstBind: true}
|
||||
}
|
||||
// A historical deploy placeholder moves to a new identity only through the
|
||||
// marked one-time repair, whatever its binding version. It must never fall
|
||||
// through to the legacy branch below, which accepts equivalent hostnames
|
||||
// and leaves no marker, so a later hostname rebind could recreate the
|
||||
// placeholder shape and unlock another move.
|
||||
if requestedID != boundID && historicalDeployPlaceholder(record, boundID, boundHost) {
|
||||
if requestedID != "" && strings.EqualFold(boundHost, requestedHost) {
|
||||
return Decision{Admit: true, LegacyMigrate: true, RepairDeployIdentity: true}
|
||||
}
|
||||
return Decision{}
|
||||
}
|
||||
if strings.TrimSpace(record.Metadata[VersionKey]) != Version &&
|
||||
boundHost != "" && hostnamesMatch(boundHost, requestedHost) {
|
||||
return Decision{Admit: true, LegacyMigrate: true}
|
||||
@@ -84,6 +108,19 @@ func Evaluate(record *config.APITokenRecord, requestedID, requestedHost string)
|
||||
}
|
||||
}
|
||||
|
||||
// historicalDeployPlaceholder reports a runtime token that deploy enrollment
|
||||
// bound to the agent-<hostname> identity it used to invent. No agent keeps
|
||||
// that identity past its first acknowledged report, so such a token may move
|
||||
// once to the registering agent's real identity on exactly the same hostname.
|
||||
// Tokens issued by current enrollment carry DeployIdentityAgent and a
|
||||
// repaired token carries DeployIdentityRepaired, so neither qualifies, even if
|
||||
// a later hostname rebind reproduces the agent-<hostname> shape.
|
||||
func historicalDeployPlaceholder(record *config.APITokenRecord, boundID, boundHost string) bool {
|
||||
return strings.TrimSpace(record.Metadata["deploy_job_id"]) != "" &&
|
||||
strings.TrimSpace(record.Metadata[DeployIdentityKey]) == "" &&
|
||||
boundHost != "" && boundID == "agent-"+boundHost
|
||||
}
|
||||
|
||||
func CanBindInstallToken(record *config.APITokenRecord, agentID, hostname string) bool {
|
||||
if record == nil || strings.TrimSpace(agentID) == "" || strings.TrimSpace(hostname) == "" {
|
||||
return false
|
||||
|
||||
@@ -74,3 +74,89 @@ func TestEvaluateActionRunnerRequiresExactTypedPrebinding(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestEvaluateRepairsDeployPlaceholderBindingOnce(t *testing.T) {
|
||||
historical := func() *config.APITokenRecord {
|
||||
return &config.APITokenRecord{Metadata: map[string]string{
|
||||
"bound_agent_id": "agent-delly2",
|
||||
"bound_hostname": "delly2",
|
||||
"deploy_job_id": "dep_1",
|
||||
VersionKey: Version,
|
||||
}}
|
||||
}
|
||||
|
||||
decision := Evaluate(historical(), "machine-id", "delly2")
|
||||
if !decision.Admit || !decision.LegacyMigrate || !decision.RepairDeployIdentity {
|
||||
t.Fatalf("historical deploy placeholder decision = %+v, want a one-time identity repair", decision)
|
||||
}
|
||||
if decision := Evaluate(historical(), "machine-id", "DELLY2"); !decision.RepairDeployIdentity {
|
||||
t.Fatalf("case-only hostname difference blocked the repair: %+v", decision)
|
||||
}
|
||||
for _, host := range []string{"other-node", "delly2.lan", "delly2.site-b"} {
|
||||
if decision := Evaluate(historical(), "machine-id", host); decision.Admit {
|
||||
t.Fatalf("placeholder repaired for hostname %q: %+v", host, decision)
|
||||
}
|
||||
}
|
||||
if decision := Evaluate(historical(), "agent-delly2", "delly2"); !decision.Admit || decision.RepairDeployIdentity {
|
||||
t.Fatalf("placeholder identity itself = %+v, want an ordinary admit", decision)
|
||||
}
|
||||
|
||||
for _, marker := range []string{DeployIdentityAgent, DeployIdentityRepaired} {
|
||||
record := historical()
|
||||
record.Metadata[DeployIdentityKey] = marker
|
||||
if decision := Evaluate(record, "machine-id", "delly2"); decision.Admit {
|
||||
t.Fatalf("token marked %q moved to a new identity: %+v", marker, decision)
|
||||
}
|
||||
}
|
||||
|
||||
notDeployed := historical()
|
||||
delete(notDeployed.Metadata, "deploy_job_id")
|
||||
if decision := Evaluate(notDeployed, "machine-id", "delly2"); decision.Admit {
|
||||
t.Fatalf("a non-deploy token bound to agent-<hostname> moved: %+v", decision)
|
||||
}
|
||||
|
||||
realIdentity := historical()
|
||||
realIdentity.Metadata["bound_agent_id"] = "machine-id"
|
||||
if decision := Evaluate(realIdentity, "another-machine", "delly2"); decision.Admit {
|
||||
t.Fatalf("a deploy token bound to a real identity moved: %+v", decision)
|
||||
}
|
||||
}
|
||||
|
||||
// Deploy tokens issued before binding versions were stamped at enrollment, and
|
||||
// never connected since, carry no version. They must still move only through
|
||||
// the marked repair: the generic legacy branch would accept an equivalent
|
||||
// hostname and leave no marker behind.
|
||||
func TestEvaluateVersionlessDeployPlaceholderOnlyMovesThroughTheRepair(t *testing.T) {
|
||||
versionless := func() *config.APITokenRecord {
|
||||
return &config.APITokenRecord{Metadata: map[string]string{
|
||||
"bound_agent_id": "agent-a",
|
||||
"bound_hostname": "a",
|
||||
"deploy_job_id": "dep_1",
|
||||
}}
|
||||
}
|
||||
if decision := Evaluate(versionless(), "agent-b", "a.example"); decision.Admit {
|
||||
t.Fatalf("versionless placeholder moved on an equivalent hostname: %+v", decision)
|
||||
}
|
||||
if decision := Evaluate(versionless(), "machine-id", "a"); !decision.Admit || !decision.RepairDeployIdentity {
|
||||
t.Fatalf("versionless placeholder decision = %+v, want the marked repair", decision)
|
||||
}
|
||||
if decision := Evaluate(versionless(), "agent-a", "a"); !decision.Admit || decision.RepairDeployIdentity {
|
||||
t.Fatalf("versionless placeholder identity itself = %+v, want admitted without a move", decision)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEvaluateCurrentDeployTokenBackfillsTheAgentIdentity(t *testing.T) {
|
||||
record := &config.APITokenRecord{Metadata: map[string]string{
|
||||
"bound_hostname": "delly2",
|
||||
"deploy_job_id": "dep_1",
|
||||
VersionKey: Version,
|
||||
DeployIdentityKey: DeployIdentityAgent,
|
||||
}}
|
||||
decision := Evaluate(record, "machine-id", "delly2")
|
||||
if !decision.Admit || !decision.BackfillID || decision.LegacyMigrate || decision.RepairDeployIdentity {
|
||||
t.Fatalf("current deploy token decision = %+v, want an ordinary identity backfill", decision)
|
||||
}
|
||||
if decision := Evaluate(record, "machine-id", "other-node"); decision.Admit {
|
||||
t.Fatalf("current deploy token admitted another host: %+v", decision)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -44,6 +44,7 @@ import (
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/api/resourceapi"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/bootstrap"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/deploy"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/license/entitlements"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/mock"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
@@ -121,6 +122,35 @@ func TestAgentConfigFetchAuditCapacityContract(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// The deploy enrollment response hands the agent its runtime credential and
|
||||
// nothing that names its identity. An agent identity invented here once bound
|
||||
// the command channel to an ID the agent abandons after its first report.
|
||||
func TestContractDeployEnrollResponseCarriesNoAgentIdentity(t *testing.T) {
|
||||
h, store := newEnrollTestHandlers(t)
|
||||
jobID, targetID := seedEnrollJobAndTarget(t, store, deploy.TargetEnrolling)
|
||||
rec := mintTestBootstrapToken(t, h.config, jobID, targetID, "pve-node2")
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/agents/agent/enroll", enrollJSON(t, "pve-node2"))
|
||||
attachAPITokenRecord(req, rec)
|
||||
rr := httptest.NewRecorder()
|
||||
h.HandleEnroll(rr, req)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("enroll status = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var resp map[string]any
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
keys := make([]string, 0, len(resp))
|
||||
for key := range resp {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
if want := []string{"reportInterval", "runtimeToken", "runtimeTokenId"}; !reflect.DeepEqual(keys, want) {
|
||||
t.Fatalf("enroll response keys = %v, want %v", keys, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleVersionBuildIdentityContract(t *testing.T) {
|
||||
oldBuildVersion := updates.BuildVersion
|
||||
t.Cleanup(func() { updates.BuildVersion = oldBuildVersion })
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/agentexec"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/api/agentbinding"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/deploy"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/monitoring"
|
||||
@@ -915,11 +916,17 @@ func (h *DeployHandlers) HandleEnroll(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
runtimeRecord.OrgID = bootstrapToken.OrgID
|
||||
canonicalAgentID := fmt.Sprintf("agent-%s", req.Hostname)
|
||||
// Bind the runtime token to the deployed node's hostname only. The
|
||||
// agent's identity is the machine-derived ID its reports carry, which
|
||||
// Pulse acknowledges and the agent persists, so the first command-channel
|
||||
// registration backfills that ID. Enrollment used to invent
|
||||
// agent-<hostname> here and bind to it, which pinned the command channel
|
||||
// to an identity the agent abandons after its first acknowledged report.
|
||||
runtimeRecord.Metadata = map[string]string{
|
||||
"bound_agent_id": canonicalAgentID,
|
||||
"bound_hostname": req.Hostname,
|
||||
"deploy_job_id": jobID,
|
||||
"bound_hostname": req.Hostname,
|
||||
"deploy_job_id": jobID,
|
||||
agentExecBindingVersionKey: agentExecBindingVersion,
|
||||
agentbinding.DeployIdentityKey: agentbinding.DeployIdentityAgent,
|
||||
}
|
||||
setAPITokenOwnerUserID(runtimeRecord, apiTokenOwnerUserID(*bootstrapToken))
|
||||
|
||||
@@ -969,7 +976,6 @@ func (h *DeployHandlers) HandleEnroll(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
// 12. Return runtime token + config to agent.
|
||||
resp := map[string]any{
|
||||
"agentId": canonicalAgentID,
|
||||
"runtimeToken": runtimeRaw,
|
||||
"runtimeTokenId": runtimeRecord.ID,
|
||||
"reportInterval": "30s",
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/agentexec"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/api/agentbinding"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/deploy"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
@@ -550,8 +551,8 @@ func TestHandleEnroll_Success(t *testing.T) {
|
||||
t.Fatal("expected runtimeTokenId in response")
|
||||
}
|
||||
runtimeTokenID, _ := resp["runtimeTokenId"].(string)
|
||||
if resp["agentId"] != "agent-pve-node2" {
|
||||
t.Fatalf("expected agentId=agent-pve-node2, got %v", resp["agentId"])
|
||||
if _, invented := resp["agentId"]; invented {
|
||||
t.Fatalf("enrollment returned agentId %v; the agent must keep the machine-derived identity its reports carry", resp["agentId"])
|
||||
}
|
||||
|
||||
config.Mu.Lock()
|
||||
@@ -569,6 +570,18 @@ func TestHandleEnroll_Success(t *testing.T) {
|
||||
if got := runtimeRecord.Metadata[apiTokenMetadataOwnerUserID]; got != "alice" {
|
||||
t.Fatalf("runtime token owner_user_id = %q, want alice", got)
|
||||
}
|
||||
if got := runtimeRecord.Metadata["bound_agent_id"]; got != "" {
|
||||
t.Fatalf("runtime token bound_agent_id = %q, want unbound until the agent's first command registration", got)
|
||||
}
|
||||
if got := runtimeRecord.Metadata["bound_hostname"]; got != "pve-node2" {
|
||||
t.Fatalf("runtime token bound_hostname = %q, want pve-node2", got)
|
||||
}
|
||||
if got := runtimeRecord.Metadata[agentExecBindingVersionKey]; got != agentExecBindingVersion {
|
||||
t.Fatalf("runtime token %s = %q, want %q", agentExecBindingVersionKey, got, agentExecBindingVersion)
|
||||
}
|
||||
if got := runtimeRecord.Metadata[agentbinding.DeployIdentityKey]; got != agentbinding.DeployIdentityAgent {
|
||||
t.Fatalf("runtime token %s = %q, want %q so it is never treated as a historical placeholder", agentbinding.DeployIdentityKey, got, agentbinding.DeployIdentityAgent)
|
||||
}
|
||||
persistedTokens, err := h.persistence.LoadAPITokens()
|
||||
if err != nil {
|
||||
t.Fatalf("load persisted tokens: %v", err)
|
||||
@@ -597,6 +610,68 @@ func TestHandleEnroll_Success(t *testing.T) {
|
||||
config.Mu.Unlock()
|
||||
}
|
||||
|
||||
// A deployed agent reports under its machine-derived identity, Pulse
|
||||
// acknowledges that identity, and the agent persists it. Its command channel
|
||||
// must therefore bind to that identity on first registration and keep
|
||||
// admitting it across restarts. Enrollment once bound the runtime token to an
|
||||
// invented agent-<hostname>, so every deployed agent lost command execution
|
||||
// after its first restart.
|
||||
func TestHandleEnroll_RuntimeTokenBindsTheAgentsOwnIdentityOnFirstCommandRegistration(t *testing.T) {
|
||||
h, store := newEnrollTestHandlers(t)
|
||||
jobID, targetID := seedEnrollJobAndTarget(t, store, deploy.TargetEnrolling)
|
||||
rec := mintTestBootstrapToken(t, h.config, jobID, targetID, "pve-node2")
|
||||
h.persistence = config.NewConfigPersistence(h.config.DataPath)
|
||||
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"hostname": "pve-node2", "os": "linux", "arch": "amd64", "agentVersion": "6.0.0", "commandsEnabled": true,
|
||||
})
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/agents/agent/enroll", bytes.NewBuffer(body))
|
||||
attachAPITokenRecord(req, rec)
|
||||
rr := httptest.NewRecorder()
|
||||
h.HandleEnroll(rr, req)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("enroll status = %d: %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var resp map[string]any
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
runtimeToken, _ := resp["runtimeToken"].(string)
|
||||
if runtimeToken == "" {
|
||||
t.Fatal("enrollment returned no runtime token")
|
||||
}
|
||||
|
||||
// The agent registers first under whatever identity enrollment handed it,
|
||||
// falling back to its own, then restarts under the identity Pulse
|
||||
// acknowledged for its reports: its machine-derived ID.
|
||||
router := &Router{config: h.config, persistence: h.persistence}
|
||||
const machineID = "49a2dbb3-3cc8-43b7-ad57-eb1711142454"
|
||||
firstID := machineID
|
||||
if handed, _ := resp["agentId"].(string); handed != "" {
|
||||
firstID = handed
|
||||
}
|
||||
if _, ok := router.admitAgentExecToken(runtimeToken, firstID, "pve-node2"); !ok {
|
||||
t.Fatalf("first command registration as %q was refused", firstID)
|
||||
}
|
||||
admission, ok := router.admitAgentExecToken(runtimeToken, machineID, "pve-node2")
|
||||
if !ok || admission.AgentID != machineID {
|
||||
t.Fatalf("registration after restart = %+v ok=%v, want the agent's acknowledged identity", admission, ok)
|
||||
}
|
||||
if _, ok := router.admitAgentExecToken(runtimeToken, machineID, "pve-node2"); !ok {
|
||||
t.Fatal("the bound agent was refused on its next registration")
|
||||
}
|
||||
if _, ok := router.admitAgentExecToken(runtimeToken, "another-machine", "pve-node2"); ok {
|
||||
t.Fatal("a second identity was admitted on a token already bound to the agent")
|
||||
}
|
||||
persisted, err := h.persistence.LoadAPITokens()
|
||||
if err != nil {
|
||||
t.Fatalf("load persisted tokens: %v", err)
|
||||
}
|
||||
if len(persisted) != 1 || persisted[0].Metadata["bound_agent_id"] != machineID {
|
||||
t.Fatalf("persisted runtime token = %+v, want bound to %s", persisted, machineID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleEnroll_InstallingState(t *testing.T) {
|
||||
h, store := newEnrollTestHandlers(t)
|
||||
jobID, targetID := seedEnrollJobAndTarget(t, store, deploy.TargetInstalling)
|
||||
@@ -1018,8 +1093,11 @@ func TestHandleEnroll_CommandsEnabledAddsAgentExecScope(t *testing.T) {
|
||||
if hasManage {
|
||||
t.Errorf("runtime collector token must not have %s scope, got scopes: %v", config.ScopeAgentManage, found.Scopes)
|
||||
}
|
||||
if got := found.Metadata["bound_agent_id"]; got != "agent-cmd-host" {
|
||||
t.Errorf("runtime token bound_agent_id = %q, want %q", got, "agent-cmd-host")
|
||||
if got := found.Metadata["bound_agent_id"]; got != "" {
|
||||
t.Errorf("runtime token bound_agent_id = %q, want unbound until the agent's first command registration", got)
|
||||
}
|
||||
if got := found.Metadata["bound_hostname"]; got != "cmd-host" {
|
||||
t.Errorf("runtime token bound_hostname = %q, want %q", got, "cmd-host")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@ import (
|
||||
"github.com/gorilla/websocket"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/agentexec"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai/approval"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/api/agentbinding"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/api/agenttokens"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
@@ -5500,3 +5501,135 @@ func TestSystemSettingsWebhookAllowlistPropagatesToAllTenantManagers(t *testing.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Agents already deployed hold a runtime token bound to the invented
|
||||
// agent-<hostname>; once they restart under their acknowledged identity the
|
||||
// command channel refused them for good. The token moves once to the real
|
||||
// identity on exactly the same hostname, persists that it was repaired, and is
|
||||
// strict again afterwards, even if a hostname rebind recreates the
|
||||
// agent-<hostname> shape.
|
||||
func TestDeployPlaceholderTokenMovesToTheRegisteringAgentOnce(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
cfg := &config.Config{DataPath: dir}
|
||||
raw, err := auth.GenerateAPIToken()
|
||||
if err != nil {
|
||||
t.Fatalf("generate token: %v", err)
|
||||
}
|
||||
record, err := config.NewAPITokenRecord(raw, "deploy-runtime-a", []string{config.ScopeAgentReport, config.ScopeAgentExec})
|
||||
if err != nil {
|
||||
t.Fatalf("new token record: %v", err)
|
||||
}
|
||||
record.Metadata = map[string]string{
|
||||
"bound_agent_id": "agent-a",
|
||||
"bound_hostname": "a",
|
||||
"deploy_job_id": "dep_1",
|
||||
agentExecBindingVersionKey: agentExecBindingVersion,
|
||||
}
|
||||
cfg.APITokens = []config.APITokenRecord{*record}
|
||||
persistence := config.NewConfigPersistence(dir)
|
||||
if err := persistence.SaveAPITokens(cfg.APITokens); err != nil {
|
||||
t.Fatalf("persist token: %v", err)
|
||||
}
|
||||
router := &Router{config: cfg, persistence: persistence}
|
||||
|
||||
if _, ok := router.admitAgentExecToken(raw, "agent-b", "a.example"); ok {
|
||||
t.Fatal("a placeholder token moved on an equivalent but different hostname")
|
||||
}
|
||||
if !commandConfigAllowedForToken(&cfg.APITokens[0], models.Host{ID: "agent-b", Hostname: "a"}) {
|
||||
t.Fatal("the config gate refused the repair the command channel admits")
|
||||
}
|
||||
if admission, ok := router.admitAgentExecToken(raw, "agent-b", "a"); !ok || admission.AgentID != "agent-b" {
|
||||
t.Fatalf("placeholder repair admission = %+v ok=%v, want the registering agent", admission, ok)
|
||||
}
|
||||
persisted, err := persistence.LoadAPITokens()
|
||||
if err != nil {
|
||||
t.Fatalf("reload tokens: %v", err)
|
||||
}
|
||||
if got := persisted[0].Metadata; got["bound_agent_id"] != "agent-b" || got[agentbinding.DeployIdentityKey] != agentbinding.DeployIdentityRepaired {
|
||||
t.Fatalf("persisted binding after repair = %v, want agent-b marked repaired", got)
|
||||
}
|
||||
|
||||
// The same identity may re-bind a renamed hostname, which recreates the
|
||||
// agent-<hostname> shape on host b. It must not unlock a second move.
|
||||
if _, ok := router.admitAgentExecToken(raw, "agent-b", "b"); !ok {
|
||||
t.Fatal("the bound identity could not re-bind its renamed hostname")
|
||||
}
|
||||
if _, ok := router.admitAgentExecToken(raw, "another-machine", "b"); ok {
|
||||
t.Fatal("the repaired token moved a second time")
|
||||
}
|
||||
if commandConfigAllowedForToken(&cfg.APITokens[0], models.Host{ID: "another-machine", Hostname: "b"}) {
|
||||
t.Fatal("the config gate allowed an identity the command channel refuses")
|
||||
}
|
||||
}
|
||||
|
||||
// The exact attack on a versionless historical token: an equivalent-hostname
|
||||
// registration must not migrate it without the repair marker, or a hostname
|
||||
// rebind could recreate the placeholder shape and unlock a second move.
|
||||
func TestVersionlessDeployPlaceholderCannotBeMovedTwice(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
cfg := &config.Config{DataPath: dir}
|
||||
raw, err := auth.GenerateAPIToken()
|
||||
if err != nil {
|
||||
t.Fatalf("generate token: %v", err)
|
||||
}
|
||||
record, err := config.NewAPITokenRecord(raw, "deploy-runtime-a", []string{config.ScopeAgentReport, config.ScopeAgentExec})
|
||||
if err != nil {
|
||||
t.Fatalf("new token record: %v", err)
|
||||
}
|
||||
record.Metadata = map[string]string{
|
||||
"bound_agent_id": "agent-a",
|
||||
"bound_hostname": "a",
|
||||
"deploy_job_id": "dep_1",
|
||||
}
|
||||
cfg.APITokens = []config.APITokenRecord{*record}
|
||||
router := &Router{config: cfg, persistence: config.NewConfigPersistence(dir)}
|
||||
|
||||
if _, ok := router.admitAgentExecToken(raw, "agent-b", "a.example"); ok {
|
||||
t.Fatal("a versionless placeholder migrated on an equivalent hostname")
|
||||
}
|
||||
// The deployed agent first registers under the identity enrollment handed
|
||||
// it, then under its acknowledged identity after a restart.
|
||||
if _, ok := router.admitAgentExecToken(raw, "agent-a", "a"); !ok {
|
||||
t.Fatal("the deployed agent's first registration was refused")
|
||||
}
|
||||
if admission, ok := router.admitAgentExecToken(raw, "machine-id", "a"); !ok || admission.AgentID != "machine-id" {
|
||||
t.Fatalf("restart registration = %+v ok=%v, want the one-time repair", admission, ok)
|
||||
}
|
||||
if got := cfg.APITokens[0].Metadata[agentbinding.DeployIdentityKey]; got != agentbinding.DeployIdentityRepaired {
|
||||
t.Fatalf("repair marker = %q, want %q", got, agentbinding.DeployIdentityRepaired)
|
||||
}
|
||||
if _, ok := router.admitAgentExecToken(raw, "another-machine", "a"); ok {
|
||||
t.Fatal("the repaired token moved a second time")
|
||||
}
|
||||
}
|
||||
|
||||
// A token issued by current enrollment leaves identity to the agent. Even if
|
||||
// the agent it first binds happens to be named agent-<hostname>, the token is
|
||||
// not a historical placeholder and never moves.
|
||||
func TestCurrentDeployTokenNeverMovesOnceBound(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
cfg := &config.Config{DataPath: dir}
|
||||
raw, err := auth.GenerateAPIToken()
|
||||
if err != nil {
|
||||
t.Fatalf("generate token: %v", err)
|
||||
}
|
||||
record, err := config.NewAPITokenRecord(raw, "deploy-runtime-c", []string{config.ScopeAgentReport, config.ScopeAgentExec})
|
||||
if err != nil {
|
||||
t.Fatalf("new token record: %v", err)
|
||||
}
|
||||
record.Metadata = map[string]string{
|
||||
"bound_hostname": "c",
|
||||
"deploy_job_id": "dep_2",
|
||||
agentExecBindingVersionKey: agentExecBindingVersion,
|
||||
agentbinding.DeployIdentityKey: agentbinding.DeployIdentityAgent,
|
||||
}
|
||||
cfg.APITokens = []config.APITokenRecord{*record}
|
||||
router := &Router{config: cfg, persistence: config.NewConfigPersistence(dir)}
|
||||
|
||||
if _, ok := router.admitAgentExecToken(raw, "agent-c", "c"); !ok {
|
||||
t.Fatal("first registration of a current deploy token was refused")
|
||||
}
|
||||
if _, ok := router.admitAgentExecToken(raw, "machine-id", "c"); ok {
|
||||
t.Fatal("a current deploy token moved after its first binding")
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user