Apply managed Proxmox TLS trust to exact polling endpoints

Persist endpoint-scoped decisions alongside discovery pins, including explicit CA mode, without clearing unrelated member trust or accepting renewed certificates automatically. Use the same policy for saved-credential probes, qualify their scope, and keep failed saves out of live PVE configuration.

Change-source: pulse-maintainer
This commit is contained in:
pulse-triage[bot]
2026-10-27 07:07:44 +00:00
parent 32d70b57a7
commit aa8e809fc8
16 changed files with 693 additions and 31 deletions
@@ -15,6 +15,17 @@
## Purpose
### Managed Proxmox TLS edits do not provision agents
The node editor's endpoint-scoped fingerprint edits and saved-credential tests
manage only existing API connection trust. They do not enrol, restart or probe
an agent, rotate its credentials, broaden access or mark agent health current.
Tests explicitly qualify a successful API endpoint connection as distinct from
background collection and the other members' health. Discovery observations
cannot overwrite a saved exact-endpoint trust override.
`TestPVEEndpointTrustSavedPolling` verifies the producer boundary with synthetic
TLS endpoints; installed/agent recovery remains a separate outcome.
### Configuration-import retries preserve the remote-agent boundary
The import API's process-local replay admission rejects duplicate committed
@@ -21,6 +21,25 @@
## Purpose
### Scoped Proxmox trust edits and connection tests
PVE updates accept `clusterEndpointTrustOverrides` entries containing an
existing HTTPS `endpoint` and required string `fingerprint`. An empty string
selects the existing verification setting for that endpoint; missing/null,
duplicate equivalent endpoints, unknown scopes and contradictory primary
settings are rejected before mutation. A primary `fingerprint` edit also
reconciles that exact endpoint in polling trust. Candidate PVE configuration
is published in memory only after persistence succeeds.
Cluster member responses expose the permitted runtime `tlsEndpoint` and
`usesPrimaryTLS`, with the effective fingerprint. Saved-credential Test accepts
an optional existing PVE `endpoint` and temporary TLS edits. It makes no saved
trust or health changes and states that other endpoints/background collection
were not tested. Omitted TLS edits use polling's saved trust, including an
ambiguity failure. Other provider types cannot select a PVE endpoint.
`TestPVEEndpointTrustAdmission` and `TestPVEEndpointTrustSavedPolling` cover
rejected inputs, failed persistence, bounded probes and unchanged credentials.
### Configuration imports distinguish saved files from live recovery
After an archive commits, every attempted live alert, webhook, email and guest
@@ -17,6 +17,24 @@
## Purpose
### Explicit Proxmox endpoint trust survives save and reload
A managed primary fingerprint edit reconciles only that exact HTTP authority
and base path in the polling pin set. A member decision uses its actual
runtime `tlsEndpoint`, not a node-name alias. Other member pins and conflicting
scopes remain intact. Persisted `tlsFingerprintOverrides` take precedence over
old discovery observations for the same scope; an explicit empty value means
use the existing `VerifySSL` policy, not primary-pin fallback or first-use
capture. Discovery does not replace the operator's decision or change routing.
Conflicting equivalent explicit decisions fail closed. Unknown members retain
the established first-use boundary; a known member's renewal is not accepted
automatically. Init and reconnect use the same builder.
`TestPVEEndpointTrustSavedPolling`, `TestPVEEndpointTrustOverridesScoped` and
`TestClusterExplicitEmptyEndpointTrust` prove saved/reopened policy, real
synthetic TLS collection and exact authority/base-path isolation. They do not
establish installed agent health, reporter-specific causality or recovery.
### Retained local archives do not change kind when a VMID is reused
A PVE VM archive cannot advance the backup date or protect a live LXC using
@@ -18,6 +18,21 @@
## Purpose
### Proxmox trust choices are endpoint-scoped
Persisted PVE `tlsFingerprintOverrides` are exact authority/base-path operator
choices, including deliberate empty pins. Empty does not authorise TOFU or
inherit another endpoint's pin; with verification enabled, normal certificate
chain and hostname checks remain required. A known member's renewed certificate
is not automatically trusted. Unrelated saved pins, credentials, routing and
access restrictions remain unchanged. Primary and per-member edits cannot
silently disagree for the same authority. Trust maps are deeply copied before
candidate mutation and survive encrypted node save/reopen.
`TestPVEEndpointTrustOverridePersistenceAndClone` and
`TestPVEEndpointTrustSavedPolling` cover clone isolation, encrypted persistence,
valid-CA/renewed-pin controls and no authenticated HTTP on rejected TLS.
### Agent delivery is separate from authenticated transport
A saved agent enrollment or an authenticated rejected report does not establish
@@ -21,6 +21,17 @@
## Purpose
### Proxmox TLS recovery preserves storage and guest state
Reconciliation of saved API certificate trust is confined to the selected
endpoint authority/base path. It leaves backup/History data, guest identity,
credentials, discovered member records and unrelated pins intact. A saved
connection test reads the nodes API only and does not perform backup, restore,
freeze/thaw, filesystem, storage or guest-agent operations. Failure to persist
the PVE candidate does not publish its trust in memory.
`TestPVEEndpointTrustAdmission` and `TestPVEEndpointTrustSavedPolling` prove
these scoped save/probe boundaries, not installed backup or filesystem recovery.
### Backup retention never substitutes for verification
The existing restore-point State cell and drawer status prioritise running or
+4 -1
View File
@@ -798,6 +798,7 @@ type NodeConfigRequest struct {
Enabled *bool `json:"enabled,omitempty"` // Lifecycle toggle; nil on update preserves current
// PVE only: per-cluster-member connection address overrides. Only the
// members named here are touched; an empty ipOverride clears the override.
ClusterEndpointTrustOverrides []ClusterEndpointTrustOverrideRequest `json:"clusterEndpointTrustOverrides,omitempty"`
ClusterEndpointOverrides []ClusterEndpointOverrideRequest `json:"clusterEndpointOverrides,omitempty"`
ClusterNodeDisplayNameOverrides []ClusterNodeDisplayNameOverrideRequest `json:"clusterNodeDisplayNameOverrides,omitempty"`
guestURLSet bool `json:"-"`
@@ -956,6 +957,8 @@ type ClusterEndpointResponse struct {
GuestURL string `json:"guestURL,omitempty"`
IP string `json:"ip,omitempty"`
IPOverride string `json:"ipOverride,omitempty"`
TLSEndpoint string `json:"tlsEndpoint,omitempty"`
UsesPrimaryTLS bool `json:"usesPrimaryTLS,omitempty"`
Fingerprint string `json:"fingerprint,omitempty"`
Online bool `json:"online"`
LastSeen time.Time `json:"lastSeen,omitempty"`
@@ -1660,7 +1663,7 @@ func (h *ConfigHandlers) GetAllNodesForAPI(ctx context.Context) []NodeResponse {
Status: h.getNodeStatus(ctx, "pve", pve.Name),
IsCluster: pve.IsCluster,
ClusterName: pve.ClusterName,
ClusterEndpoints: toClusterEndpointResponses(pve.ClusterEndpoints, pve.ClusterNodeIdentities),
ClusterEndpoints: pveClusterEndpointTLSResponses(pve, h.getConfig(ctx).Discovery),
Source: pve.Source,
}.NormalizeCollections()
nodes = append(nodes, node)
@@ -3,21 +3,34 @@ package configapi
import (
"bytes"
"context"
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/json"
"encoding/pem"
"fmt"
"github.com/rcourtman/pulse-go-rewrite/internal/api/apicontext"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/monitoring"
"github.com/rcourtman/pulse-go-rewrite/internal/websocket"
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
"io"
stdlog "log"
"math/big"
"net"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"path/filepath"
"reflect"
"strings"
"sync/atomic"
"testing"
"github.com/rcourtman/pulse-go-rewrite/internal/api/apicontext"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/websocket"
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
"time"
)
func TestHandleTestConnection(t *testing.T) {
@@ -308,3 +321,257 @@ func TestNodeMutationEventsStayWithRequestOrganization(t *testing.T) {
})
}
}
// The standard root pool is process-cached. Keep the synthetic CA experiment in
// a fresh copy of this test process so the normal full suite is order-independent.
func TestPVEEndpointTrustSavedPolling(t *testing.T) {
if os.Getenv("PULSE_TLS_TEST_CHILD") != "1" {
executable, err := os.Executable()
if err != nil {
t.Fatal(err)
}
cmd := exec.Command(executable, "-test.run=^TestPVEEndpointTrustSavedPolling$", "-test.v")
cmd.Env = append(os.Environ(), "PULSE_TLS_TEST_CHILD=1")
output, err := cmd.CombinedOutput()
t.Log(string(output))
if err != nil {
t.Fatal(err)
}
return
}
pub, key, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
ca := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "isolated polling CA"}, NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour), IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign}
der, err := x509.CreateCertificate(rand.Reader, ca, ca, pub, key)
if err != nil {
t.Fatal(err)
}
ca, err = x509.ParseCertificate(der)
if err != nil {
t.Fatal(err)
}
caFile := filepath.Join(t.TempDir(), "ca.pem")
if err := os.WriteFile(caFile, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}), 0600); err != nil {
t.Fatal(err)
}
t.Setenv("SSL_CERT_FILE", caFile)
t.Setenv("SSL_CERT_DIR", t.TempDir())
makeLeaf := func(serial int64) tls.Certificate {
pub, key, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
leaf := &x509.Certificate{SerialNumber: big.NewInt(serial), NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour), IPAddresses: []net.IP{net.ParseIP("127.0.0.1")}, ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}, KeyUsage: x509.KeyUsageDigitalSignature}
der, err := x509.CreateCertificate(rand.Reader, leaf, ca, pub, key)
if err != nil {
t.Fatal(err)
}
return tls.Certificate{Certificate: [][]byte{der, ca.Raw}, PrivateKey: key}
}
pin := func(c tls.Certificate) string { return fmt.Sprintf("%x", sha256.Sum256(c.Certificate[0])) }
old := [2]tls.Certificate{makeLeaf(2), makeLeaf(3)}
renewed := [2]tls.Certificate{makeLeaf(4), makeLeaf(5)}
var active [2]atomic.Pointer[tls.Certificate]
var authenticated [2]atomic.Int32
var servers [2]*httptest.Server
for i := range servers {
active[i].Store(&renewed[i])
servers[i] = httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api2/json/nodes" {
t.Errorf("unexpected request path %s", r.URL.Path)
http.NotFound(w, r)
return
}
if r.Header.Get("Authorization") != "PVEAPIToken=fixture@pve!pulse=synthetic-only" {
t.Error("saved credential changed")
w.WriteHeader(401)
return
}
authenticated[i].Add(1)
fmt.Fprint(w, `{"data":[{"node":"alpha","status":"online"},{"node":"beta","status":"online"}]}`)
}))
servers[i].Config.ErrorLog = stdlog.New(io.Discard, "", 0)
servers[i].TLS = &tls.Config{MinVersion: tls.VersionTLS12, GetConfigForClient: func(*tls.ClientHelloInfo) (*tls.Config, error) {
return &tls.Config{MinVersion: tls.VersionTLS12, Certificates: []tls.Certificate{*active[i].Load()}}, nil
}}
servers[i].StartTLS()
t.Cleanup(servers[i].Close)
}
// Admit the intended, hostname-matching leaf independently of pinning.
roots := x509.NewCertPool()
roots.AddCert(ca)
for i, s := range servers {
conn, err := tls.DialWithDialer(&net.Dialer{Timeout: time.Second}, "tcp", strings.TrimPrefix(s.URL, "https://"), &tls.Config{RootCAs: roots, MinVersion: tls.VersionTLS12})
if err != nil {
t.Fatal(err)
}
got := fmt.Sprintf("%x", sha256.Sum256(conn.ConnectionState().PeerCertificates[0].Raw))
conn.Close()
if got != pin(renewed[i]) {
t.Fatal("wrong fixture leaf presented")
}
}
for _, tc := range []struct {
name, action string
original bool
wantHealth [2]bool
wantTest int
}{
{"unapproved-rotation", "", false, [2]bool{false, false}, 400},
{"primary-edit-is-scoped", "primary-pin", false, [2]bool{true, false}, 200},
{"primary-CA-clear-is-scoped", "primary-ca", false, [2]bool{true, false}, 200},
{"explicit-both-pins", "both-pins", false, [2]bool{true, true}, 200},
{"explicit-both-CA", "both-ca", false, [2]bool{true, true}, 200},
{"member-CA-does-not-inherit-primary-pin", "member-ca", false, [2]bool{true, true}, 200},
{"original-leaves-control", "", true, [2]bool{true, true}, 200},
} {
t.Run(tc.name, func(t *testing.T) {
for i, s := range servers {
if tc.original {
active[i].Store(&old[i])
} else {
active[i].Store(&renewed[i])
}
s.CloseClientConnections()
authenticated[i].Store(0)
}
inst := config.PVEInstance{Name: "fixture-cluster", Host: servers[0].URL, TokenName: "fixture@pve!pulse", TokenValue: "synthetic-only", VerifySSL: true, VerifySSLExplicit: true, IsCluster: true, ClusterName: "fixture-cluster", Fingerprint: pin(old[0]), ClusterEndpoints: []config.ClusterEndpoint{
{NodeIdentity: "alpha-id", NodeName: "alpha", Host: servers[0].URL, Fingerprint: pin(old[0])},
{NodeIdentity: "beta-id", NodeName: "beta", Host: servers[1].URL, Fingerprint: pin(old[1])},
}}
cfg := &config.Config{DataPath: t.TempDir(), PVEInstances: []config.PVEInstance{inst}}
t.Setenv("PULSE_DATA_DIR", cfg.DataPath)
persistence := config.NewConfigPersistence(cfg.DataPath)
if err := persistence.SaveNodesConfig(cfg.PVEInstances, nil, nil); err != nil {
t.Fatal(err)
}
reloads := 0
h := &ConfigHandlers{defaultConfig: cfg, defaultPersistence: persistence, runtime: defaultRuntimeDependencies(), reloadFunc: func() error { reloads++; return nil }}
if tc.action != "" {
primary := pin(renewed[0])
if strings.Contains(tc.action, "ca") && tc.action != "member-ca" {
primary = ""
}
update := map[string]any{"verifySSL": true, "fingerprint": primary}
if strings.HasPrefix(tc.action, "both") || tc.action == "member-ca" {
member := pin(renewed[1])
if strings.Contains(tc.action, "ca") {
member = ""
}
update["clusterEndpointTrustOverrides"] = []map[string]string{{"endpoint": servers[1].URL, "fingerprint": member}}
}
body, _ := json.Marshal(update)
rec := httptest.NewRecorder()
h.HandleUpdateNode(rec, httptest.NewRequest("PUT", "/api/config/nodes/pve-0", strings.NewReader(string(body))))
if rec.Code != 200 || reloads != 1 {
t.Fatalf("save=%d reloads=%d: %s", rec.Code, reloads, rec.Body.String())
}
}
disk, err := persistence.LoadNodesConfig()
if err != nil {
t.Fatal(err)
}
cfg.PVEInstances = disk.PVEInstances
saved := cfg.PVEInstances[0]
if saved.TokenValue != inst.TokenValue || !saved.VerifySSL || !reflect.DeepEqual(saved.ClusterEndpoints, inst.ClusterEndpoints) {
t.Fatal("trust edit changed credentials, verification or discovered member records")
}
before, _ := json.Marshal(saved)
rec := httptest.NewRecorder()
h.HandleTestNode(rec, httptest.NewRequest("POST", "/api/config/nodes/pve-0/test", strings.NewReader(`{}`)))
if rec.Code != tc.wantTest {
t.Fatalf("saved Test=%d want=%d: %s", rec.Code, tc.wantTest, rec.Body.String())
}
if rec.Code == 200 && !strings.Contains(rec.Body.String(), "Other endpoints and background collection were not tested") {
t.Fatal("primary test claims cluster collection")
}
projection := pveClusterEndpointTLSResponses(saved, cfg.Discovery)
if len(projection) != 2 || !projection[0].UsesPrimaryTLS || projection[1].UsesPrimaryTLS || projection[1].TLSEndpoint != servers[1].URL {
t.Fatalf("incorrect effective endpoint projection: %+v", projection)
}
// Init/reconnect share this same exported production builder.
endpoints, pins, conflicts := monitoring.BuildPVEClusterClientEndpoints(saved, cfg.Discovery)
clientCfg := config.CreateProxmoxConfig(&saved)
clientCfg.Timeout = time.Second
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
cc, err := proxmox.NewClusterClientWithContext(ctx, saved.Name, clientCfg, endpoints, pins, conflicts...)
if err != nil {
cancel()
t.Fatal(err)
}
nodes, pollErr := cc.GetNodes(ctx)
cancel()
health := cc.GetHealthStatus()
if health[servers[0].URL] != tc.wantHealth[0] || health[servers[1].URL] != tc.wantHealth[1] {
t.Fatalf("health=%v want=%v", health, tc.wantHealth)
}
if tc.wantHealth[0] || tc.wantHealth[1] {
if pollErr != nil || len(nodes) != 2 {
t.Fatalf("ordinary nodes poll: %v, %d", pollErr, len(nodes))
}
} else if pollErr == nil {
t.Fatal("unapproved renewal accepted")
}
t.Logf("primary=%t member=%t verifySSL=%t", health[servers[0].URL], health[servers[1].URL], saved.VerifySSL)
// A member probe uses the same scoped policy, saved credentials and no
// persistent trust or health update. It never captures a replacement pin.
rec = httptest.NewRecorder()
body, _ := json.Marshal(map[string]string{"endpoint": servers[1].URL})
h.HandleTestNode(rec, httptest.NewRequest("POST", "/api/config/nodes/pve-0/test", strings.NewReader(string(body))))
if (rec.Code == 200) != tc.wantHealth[1] {
t.Fatalf("member probe=%d: %s", rec.Code, rec.Body.String())
}
after, _ := json.Marshal(cfg.PVEInstances[0])
if string(before) != string(after) {
t.Fatal("test mutated saved state")
}
for i, healthy := range tc.wantHealth {
if !healthy && authenticated[i].Load() != 0 {
t.Fatal("rejected TLS sent authenticated HTTP")
}
}
})
}
}
func TestPVEEndpointTrustAdmission(t *testing.T) {
instance := config.PVEInstance{Name: "cluster", Host: "https://primary.test:8006", Fingerprint: "primary-pin", VerifySSL: true, TokenValue: "preserved", ClusterEndpoints: []config.ClusterEndpoint{{NodeName: "member", Host: "https://member.test:8006", Fingerprint: "member-pin"}}}
for _, body := range []string{
`{"clusterEndpointTrustOverrides":[{"endpoint":"https://outside.test:8006","fingerprint":""}]}`,
`{"clusterEndpointTrustOverrides":[{"endpoint":"https://member.test:8006"}]}`,
`{"clusterEndpointTrustOverrides":[{"endpoint":"https://member.test:8006","fingerprint":null}]}`,
`{"clusterEndpointTrustOverrides":[{"endpoint":"https://member.test:8006","fingerprint":""},{"endpoint":"https://MEMBER.test:8006/","fingerprint":""}]}`,
`{"fingerprint":"new","clusterEndpointTrustOverrides":[{"endpoint":"https://primary.test:8006","fingerprint":"different"}]}`,
`{"clusterEndpointTrustOverrides":[{"endpoint":"http://member.test:8006","fingerprint":""}]}`,
`{"clusterEndpointTrustOverrides":[{"endpoint":"https://member.test:8007","fingerprint":""}]}`,
`{"clusterEndpointTrustOverrides":[{"endpoint":"https://member.test:8006/other","fingerprint":""}]}`,
} {
t.Run(body, func(t *testing.T) {
cfg := &config.Config{DataPath: t.TempDir(), PVEInstances: []config.PVEInstance{instance.DeepCopy()}}
h := &ConfigHandlers{defaultConfig: cfg, defaultPersistence: config.NewConfigPersistence(cfg.DataPath), runtime: defaultRuntimeDependencies()}
before, _ := json.Marshal(cfg)
rec := httptest.NewRecorder()
h.HandleUpdateNode(rec, httptest.NewRequest("PUT", "/api/config/nodes/pve-0", strings.NewReader(body)))
after, _ := json.Marshal(cfg)
if rec.Code != 400 || string(before) != string(after) {
t.Fatalf("invalid trust update mutated config: %d %s", rec.Code, rec.Body.String())
}
})
}
// A storage failure must not publish the trust/credential candidate in memory.
cfg := &config.Config{DataPath: t.TempDir(), PVEInstances: []config.PVEInstance{instance.DeepCopy()}}
blocked := filepath.Join(cfg.DataPath, "not-a-directory")
if err := os.WriteFile(blocked, []byte("blocked"), 0600); err != nil {
t.Fatal(err)
}
h := &ConfigHandlers{defaultConfig: cfg, defaultPersistence: config.NewConfigPersistence(blocked), runtime: defaultRuntimeDependencies()}
before, _ := json.Marshal(cfg)
rec := httptest.NewRecorder()
h.HandleUpdateNode(rec, httptest.NewRequest("PUT", "/api/config/nodes/pve-0", strings.NewReader(`{"fingerprint":"new-pin"}`)))
after, _ := json.Marshal(cfg)
if rec.Code != 500 || string(before) != string(after) {
t.Fatalf("failed save published trust: %d %s", rec.Code, rec.Body.String())
}
}
+43 -7
View File
@@ -1432,6 +1432,7 @@ func (h *ConfigHandlers) handleUpdateNode(w http.ResponseWriter, r *http.Request
r.Body = http.MaxBytesReader(w, r.Body, 32*1024)
var pendingPMG []config.PMGInstance
var pendingPVE []config.PVEInstance
var req NodeConfigRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, "Invalid request body", http.StatusBadRequest)
@@ -1535,7 +1536,16 @@ func (h *ConfigHandlers) handleUpdateNode(w http.ResponseWriter, r *http.Request
updated.Disabled = !*req.Enabled
}
*pve = updated
updated, err = applyPVEEndpointTrustUpdates(updated, req, h.getConfig(r.Context()).Discovery)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
pendingPVE = make([]config.PVEInstance, len(h.getConfig(r.Context()).PVEInstances))
for i, instance := range h.getConfig(r.Context()).PVEInstances {
pendingPVE[i] = instance.DeepCopy()
}
pendingPVE[index] = updated
} else if nodeType == "pbs" && index < len(h.getConfig(r.Context()).PBSInstances) {
pbs := &h.getConfig(r.Context()).PBSInstances[index]
current := *pbs
@@ -1667,16 +1677,31 @@ func (h *ConfigHandlers) handleUpdateNode(w http.ResponseWriter, r *http.Request
}
// Save configuration to disk using our persistence instance
h.normalizePVEConfigState(r.Context())
pveToSave := h.getConfig(r.Context()).PVEInstances
if pendingPVE != nil {
if normalized, changed := config.ConsolidatePVEInstances(pendingPVE); changed {
pendingPVE = normalized
}
config.EnsurePVEClusterNodeIdentities(pendingPVE)
pveToSave = pendingPVE
} else {
h.normalizePVEConfigState(r.Context())
pveToSave = h.getConfig(r.Context()).PVEInstances
}
pmgToSave := h.getConfig(r.Context()).PMGInstances
if pendingPMG != nil {
pmgToSave = pendingPMG
}
if err := h.getPersistence(r.Context()).SaveNodesConfig(h.getConfig(r.Context()).PVEInstances, h.getConfig(r.Context()).PBSInstances, pmgToSave); err != nil {
if err := h.getPersistence(r.Context()).SaveNodesConfig(pveToSave, h.getConfig(r.Context()).PBSInstances, pmgToSave); err != nil {
log.Error().Err(err).Msg("Failed to save nodes configuration")
http.Error(w, "Failed to save configuration", http.StatusInternalServerError)
return
}
if pendingPVE != nil {
config.Mu.Lock()
h.getConfig(r.Context()).PVEInstances = pendingPVE
config.Mu.Unlock()
}
if pendingPMG != nil {
config.Mu.Lock()
h.getConfig(r.Context()).PMGInstances = pendingPMG
@@ -2146,6 +2171,7 @@ func (h *ConfigHandlers) handleTestNodeConfig(w http.ResponseWriter, r *http.Req
type nodeConnectionTestTLSOptions struct {
VerifySSL *bool `json:"verifySSL"`
Fingerprint *string `json:"fingerprint"`
Endpoint string `json:"endpoint,omitempty"`
}
func (o nodeConnectionTestTLSOptions) apply(verifySSL bool, fingerprint string) (bool, string) {
@@ -2195,12 +2221,22 @@ func (h *ConfigHandlers) handleTestNode(w http.ResponseWriter, r *http.Request)
}
}
if nodeType != "pve" && tlsOptions.Endpoint != "" {
http.Error(w, "Endpoint selection is only supported for PVE", http.StatusBadRequest)
return
}
// Find the node to test
var testResult map[string]interface{}
if nodeType == "pve" && index < len(h.getConfig(r.Context()).PVEInstances) {
pve := h.getConfig(r.Context()).PVEInstances[index]
verifySSL, fingerprint := tlsOptions.apply(pve.VerifySSL, pve.Fingerprint)
pve := h.getConfig(r.Context()).PVEInstances[index].DeepCopy()
endpoint, fingerprint, trustErr := pveConnectionTestTrust(pve, tlsOptions, h.getConfig(r.Context()).Discovery)
if trustErr != nil {
http.Error(w, trustErr.Error(), http.StatusBadRequest)
return
}
verifySSL, _ := tlsOptions.apply(pve.VerifySSL, pve.Fingerprint)
// Create a temporary client to test connection
authUser := pve.User
@@ -2208,7 +2244,7 @@ func (h *ConfigHandlers) handleTestNode(w http.ResponseWriter, r *http.Request)
authUser = normalizePVEUser(authUser)
}
clientConfig := proxmox.ClientConfig{
Host: pve.Host,
Host: endpoint,
User: authUser,
Password: pve.Password,
TokenName: pve.TokenName,
@@ -2237,7 +2273,7 @@ func (h *ConfigHandlers) handleTestNode(w http.ResponseWriter, r *http.Request)
latency := time.Since(startTime).Milliseconds()
testResult = map[string]interface{}{
"status": "success",
"message": fmt.Sprintf("Connected to PVE cluster with %d nodes", len(nodes)),
"message": fmt.Sprintf("Connected to this PVE endpoint (%d nodes reported). Other endpoints and background collection were not tested.", len(nodes)),
"latency": latency,
}
}
+120
View File
@@ -0,0 +1,120 @@
package configapi
import (
"fmt"
"strings"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/monitoring"
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
)
// ClusterEndpointTrustOverrideRequest reconciles one existing dial authority,
// not a node-name alias or every member of the cluster. A required empty string
// explicitly selects the existing VerifySSL policy without a certificate pin.
// Missing/null fingerprints are not instructions to clear trust.
type ClusterEndpointTrustOverrideRequest struct {
Endpoint string `json:"endpoint"`
Fingerprint *string `json:"fingerprint"`
}
func applyPVEEndpointTrustUpdates(instance config.PVEInstance, req NodeConfigRequest, discovery config.DiscoveryConfig) (config.PVEInstance, error) {
instance = instance.DeepCopy()
endpoints, _, _ := monitoring.BuildPVEClusterClientEndpoints(instance, discovery)
allowed := make(map[string]bool, len(endpoints))
for _, endpoint := range endpoints {
if key, ok := proxmox.ClusterEndpointIdentity(endpoint); ok {
allowed[key] = true
}
}
updates := make(map[string]string)
for _, override := range req.ClusterEndpointTrustOverrides {
key, ok := proxmox.ClusterEndpointIdentity(override.Endpoint)
if !ok || !strings.HasPrefix(key, "https://") || !allowed[key] {
return instance, fmt.Errorf("TLS trust can only be updated for an existing HTTPS endpoint; reload the connection before saving")
}
if override.Fingerprint == nil {
return instance, fmt.Errorf("fingerprint is required for each TLS endpoint update")
}
if _, duplicate := updates[key]; duplicate {
return instance, fmt.Errorf("TLS endpoint was provided more than once")
}
updates[key] = *override.Fingerprint
}
if req.hasFingerprintField() {
if key, ok := proxmox.ClusterEndpointIdentity(instance.Host); ok {
if other, exists := updates[key]; exists && other != req.Fingerprint {
return instance, fmt.Errorf("primary and member TLS settings disagree for the same endpoint")
}
updates[key] = req.Fingerprint
}
}
if len(updates) > 0 && instance.TLSFingerprintOverrides == nil {
instance.TLSFingerprintOverrides = make(map[string]string)
}
for key, fingerprint := range updates {
// Remove equivalent spellings only within this exact trust scope. A
// different host, port, scheme or base path retains its own saved policy.
for stored := range instance.TLSFingerprintOverrides {
if storedKey, ok := proxmox.ClusterEndpointIdentity(stored); ok && storedKey == key {
delete(instance.TLSFingerprintOverrides, stored)
}
}
instance.TLSFingerprintOverrides[key] = fingerprint
}
return instance, nil
}
func pveConnectionTestTrust(instance config.PVEInstance, options nodeConnectionTestTLSOptions, discovery config.DiscoveryConfig) (string, string, error) {
endpoint := instance.Host
if options.Endpoint != "" {
endpoint = options.Endpoint
}
key, valid := proxmox.ClusterEndpointIdentity(endpoint)
if !valid {
return "", "", fmt.Errorf("invalid PVE endpoint")
}
endpoints, pins, conflicts := monitoring.BuildPVEClusterClientEndpoints(instance, discovery)
allowed := false
for _, configured := range endpoints {
configuredKey, ok := proxmox.ClusterEndpointIdentity(configured)
if ok && key == configuredKey {
endpoint = configured
allowed = true
break
}
}
if !allowed {
return "", "", fmt.Errorf("only existing PVE endpoints can be tested; reload the connection")
}
// A submitted pin is the exact prospective save for this endpoint. Omitted
// TLS fields test saved polling policy, including ambiguity and explicit CA.
if options.Fingerprint != nil {
return endpoint, *options.Fingerprint, nil
}
fingerprint, known, err := proxmox.ResolveEndpointFingerprint(endpoint, pins, conflicts...)
if err != nil {
return "", "", err
}
if !known {
fingerprint = instance.Fingerprint
}
return endpoint, fingerprint, nil
}
func pveClusterEndpointTLSResponses(instance config.PVEInstance, discovery config.DiscoveryConfig) []ClusterEndpointResponse {
responses := toClusterEndpointResponses(instance.ClusterEndpoints, instance.ClusterNodeIdentities)
_, pins, conflicts := monitoring.BuildPVEClusterClientEndpoints(instance, discovery)
primary, primaryValid := proxmox.ClusterEndpointIdentity(instance.Host)
for i, endpoint := range instance.ClusterEndpoints {
target := monitoring.PVEClusterEndpointURL(endpoint, instance.VerifySSL, discovery)
responses[i].TLSEndpoint = target
key, valid := proxmox.ClusterEndpointIdentity(target)
responses[i].UsesPrimaryTLS = valid && primaryValid && key == primary
fingerprint, known, err := proxmox.ResolveEndpointFingerprint(target, pins, conflicts...)
if err == nil && known {
responses[i].Fingerprint = fingerprint
}
}
return responses
}
+13 -10
View File
@@ -561,16 +561,19 @@ func looksLikeNumericSeconds(value string) bool {
// PVEInstance represents a Proxmox VE connection
type PVEInstance struct {
Name string
Host string // Primary endpoint (user-provided)
GuestURL string // Optional guest-accessible URL (for navigation)
User string
Password string
TokenName string
TokenValue string
Fingerprint string
VerifySSL bool
VerifySSLExplicit bool `json:"verifySSLExplicit,omitempty"` // operator-chosen VerifySSL; survives automatic consolidation (#2140)
Name string
Host string // Primary endpoint (user-provided)
GuestURL string // Optional guest-accessible URL (for navigation)
User string
Password string
TokenName string
TokenValue string
Fingerprint string
VerifySSL bool
// Exact endpoint-scoped operator decisions, including an explicit empty pin
// (CA verification when VerifySSL is true). Discovery cannot replace them.
TLSFingerprintOverrides map[string]string `json:"tlsFingerprintOverrides,omitempty"`
VerifySSLExplicit bool `json:"verifySSLExplicit,omitempty"` // operator-chosen VerifySSL; survives automatic consolidation (#2140)
MonitorVMs bool
MonitorContainers bool
MonitorStorage bool
+24
View File
@@ -724,3 +724,27 @@ func TestPMGCollectionPersistenceDefaults(t *testing.T) {
})
}
}
func TestPVEEndpointTrustOverridePersistenceAndClone(t *testing.T) {
original := PVEInstance{Host: "https://primary.test:8006", VerifySSL: true, TokenValue: "synthetic-preserved", TLSFingerprintOverrides: map[string]string{"https://primary.test:8006": "", "https://member.test:8006": "member-pin"}}
clone := original.DeepCopy()
clone.TLSFingerprintOverrides["https://member.test:8006"] = "changed"
if original.TLSFingerprintOverrides["https://member.test:8006"] != "member-pin" {
t.Fatal("candidate trust mutation escaped its clone")
}
persistence := NewConfigPersistence(t.TempDir())
if err := persistence.SaveNodesConfig([]PVEInstance{original}, nil, nil); err != nil {
t.Fatal(err)
}
loaded, err := persistence.LoadNodesConfig()
if err != nil {
t.Fatal(err)
}
if len(loaded.PVEInstances) != 1 {
t.Fatal("saved connection disappeared")
}
saved := loaded.PVEInstances[0]
if pin, known := saved.TLSFingerprintOverrides[original.Host]; !known || pin != "" || !saved.VerifySSL || saved.TokenValue != original.TokenValue || saved.TLSFingerprintOverrides["https://member.test:8006"] != "member-pin" {
t.Fatal("reload lost explicit scoped CA policy or unrelated state")
}
}
+17
View File
@@ -1,6 +1,7 @@
package config
import (
"maps"
"net"
"net/url"
"strings"
@@ -39,6 +40,7 @@ func clonePVEInstances(instances []PVEInstance) []PVEInstance {
out := make([]PVEInstance, len(instances))
copy(out, instances)
for i := range out {
out[i].TLSFingerprintOverrides = maps.Clone(instances[i].TLSFingerprintOverrides)
if len(instances[i].ClusterEndpoints) > 0 {
out[i].ClusterEndpoints = make([]ClusterEndpoint, len(instances[i].ClusterEndpoints))
copy(out[i].ClusterEndpoints, instances[i].ClusterEndpoints)
@@ -90,6 +92,11 @@ func ProxmoxFingerprintsConflict(a, b string) bool {
// certificate rotated between discoveries stays as two views rather than two
// distinct clusters being silently folded into one.
func pveClusterInstancesHaveFingerprintConflict(a, b PVEInstance) bool {
for endpoint, fingerprint := range a.TLSFingerprintOverrides {
if other, known := b.TLSFingerprintOverrides[endpoint]; known && fingerprint != other {
return true
}
}
if ProxmoxFingerprintsConflict(a.Fingerprint, b.Fingerprint) {
return true
}
@@ -330,6 +337,16 @@ func mergePVEInstanceData(dst *PVEInstance, src PVEInstance) {
if dst == nil {
return
}
if len(src.TLSFingerprintOverrides) > 0 {
if dst.TLSFingerprintOverrides == nil {
dst.TLSFingerprintOverrides = make(map[string]string)
}
for endpoint, fingerprint := range src.TLSFingerprintOverrides {
if _, exists := dst.TLSFingerprintOverrides[endpoint]; !exists {
dst.TLSFingerprintOverrides[endpoint] = fingerprint
}
}
}
if dst.Host == "" && strings.TrimSpace(src.Host) != "" {
dst.Host = strings.TrimSpace(src.Host)
@@ -4,6 +4,7 @@ import (
"strings"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
"github.com/rs/zerolog/log"
)
@@ -21,11 +22,22 @@ func (m *Monitor) buildClusterEndpointsForReconnect(pve config.PVEInstance) ([]s
// addresses remain useful failover and reachability evidence, but they must
// never displace the URL the operator proved reachable when adding the cluster.
func (m *Monitor) buildClusterClientEndpoints(pve config.PVEInstance) ([]string, map[string]string, []string) {
return BuildPVEClusterClientEndpoints(pve, monitorDiscoveryConfig(m))
}
// PVEClusterEndpointURL exposes the same permitted dial target used by polling.
// It neither probes nor resolves a different address for the editor/test flow.
func PVEClusterEndpointURL(endpoint config.ClusterEndpoint, verifySSL bool, discoveryCfg config.DiscoveryConfig) string {
return clusterEndpointRuntimeURL(endpoint, verifySSL, false, discoveryCfg)
}
// BuildPVEClusterClientEndpoints constructs polling trust, including explicit
// decisions for exact endpoint identities. It performs no network operation.
func BuildPVEClusterClientEndpoints(pve config.PVEInstance, discoveryCfg config.DiscoveryConfig) ([]string, map[string]string, []string) {
endpoints := make([]string, 0, len(pve.ClusterEndpoints)+1)
endpointFingerprints := make(map[string]string)
var conflictingEndpoints []string
conflicts := make(map[string]bool)
discoveryCfg := monitorDiscoveryConfig(m)
hasFingerprint := pve.Fingerprint != ""
seen := make(map[string]struct{}, len(pve.ClusterEndpoints)+1)
@@ -71,6 +83,36 @@ func (m *Monitor) buildClusterClientEndpoints(pve config.PVEInstance) ([]string,
}
}
// An explicit save reconciles only this exact authority/base path, including
// equivalent URL spellings. Keep other member pins and conflicts untouched.
for _, endpoint := range endpoints {
fingerprint, known, err := proxmox.ResolveEndpointFingerprint(endpoint, pve.TLSFingerprintOverrides)
if !known {
continue
}
key, valid := proxmox.ClusterEndpointIdentity(endpoint)
if !valid {
continue
}
if err != nil {
conflictingEndpoints = append(conflictingEndpoints, endpoint)
continue
}
for saved := range endpointFingerprints {
if savedKey, ok := proxmox.ClusterEndpointIdentity(saved); ok && savedKey == key {
delete(endpointFingerprints, saved)
}
}
filtered := conflictingEndpoints[:0]
for _, conflict := range conflictingEndpoints {
if conflictKey, ok := proxmox.ClusterEndpointIdentity(conflict); !ok || conflictKey != key {
filtered = append(filtered, conflict)
}
}
conflictingEndpoints = filtered
endpointFingerprints[endpoint] = fingerprint
}
if configuredAuthority != "" {
log.Info().
Str("instance", pve.Name).
@@ -3,14 +3,13 @@ package monitoring
import (
"context"
"fmt"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
"reflect"
"strings"
"testing"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
)
func TestMapPVENodeNetworkInterfacesKeepsConfiguredBridgesAndAddresses(t *testing.T) {
@@ -547,3 +546,44 @@ func TestClusterMemberBrowserURL(t *testing.T) {
})
}
}
func TestPVEEndpointTrustOverridesScoped(t *testing.T) {
instance := config.PVEInstance{Host: "https://primary.test:8006", Fingerprint: "new-primary", VerifySSL: true, ClusterEndpoints: []config.ClusterEndpoint{
{Host: "https://primary.test:8006", Fingerprint: "old-primary"},
{Host: "https://PRIMARY.test:8006/./", Fingerprint: "contradictory-primary"},
{Host: "https://member.test:8006", Fingerprint: "old-member"},
{Host: "https://MEMBER.test:8006/", Fingerprint: "contradictory-member"},
{Host: "https://member.test:8006/other", Fingerprint: "separate-path"},
}, TLSFingerprintOverrides: map[string]string{"https://primary.test:8006": "new-primary"}}
before := instance.DeepCopy()
m := &Monitor{config: &config.Config{}}
for _, build := range []struct {
name string
fn func(config.PVEInstance) ([]string, map[string]string, []string)
}{{"init", m.buildClusterEndpointsForInit}, {"reconnect", m.buildClusterEndpointsForReconnect}} {
t.Run(build.name, func(t *testing.T) {
_, pins, conflicts := build.fn(instance)
pin, known, err := proxmox.ResolveEndpointFingerprint(instance.Host, pins, conflicts...)
if err != nil || !known || pin != "new-primary" {
t.Fatalf("primary save not applied: %q %t %v", pin, known, err)
}
if _, _, err := proxmox.ResolveEndpointFingerprint("https://member.test:8006", pins, conflicts...); err == nil {
t.Fatal("primary save discarded unrelated member ambiguity")
}
pin, known, err = proxmox.ResolveEndpointFingerprint("https://member.test:8006/other", pins, conflicts...)
if err != nil || !known || pin != "separate-path" {
t.Fatal("primary save affected different base path")
}
copy := instance.DeepCopy()
copy.TLSFingerprintOverrides["https://member.test:8006"] = ""
_, pins, conflicts = build.fn(copy)
pin, known, err = proxmox.ResolveEndpointFingerprint("https://member.test:8006", pins, conflicts...)
if err != nil || !known || pin != "" {
t.Fatalf("explicit member CA mode lost: %q %t %v", pin, known, err)
}
})
}
if !reflect.DeepEqual(before, instance) {
t.Fatal("building polling trust mutated source config")
}
}
+17 -3
View File
@@ -257,9 +257,6 @@ func (cc *ClusterClient) knownEndpointFingerprintLocked(endpoint string) (string
var fingerprint string
known := false
for storedEndpoint, fp := range cc.endpointFingerprints {
if fp == "" {
continue
}
storedKey, storedValid := clusterEndpointKey(storedEndpoint)
if storedEndpoint != endpoint && (!valid || !storedValid || storedKey != key) {
continue
@@ -282,6 +279,23 @@ func (cc *ClusterClient) knownEndpointFingerprintLocked(endpoint string) (string
return fingerprint, known, nil
}
// ClusterEndpointIdentity returns the existing certificate-trust scope: HTTP
// origin and base path, not DNS aliases, node names or cluster membership.
func ClusterEndpointIdentity(endpoint string) (string, bool) {
return clusterEndpointKey(endpoint)
}
// ResolveEndpointFingerprint resolves only saved endpoint trust, without a
// fallback, network call or trust capture. A present empty value is deliberate
// CA/explicit-opt-out policy, not an unknown first-use member.
func ResolveEndpointFingerprint(endpoint string, fingerprints map[string]string, conflictingEndpoints ...string) (string, bool, error) {
cc := ClusterClient{endpointFingerprints: fingerprints, conflictingEndpointFingerprints: make(map[string]bool)}
for _, conflict := range conflictingEndpoints {
cc.conflictingEndpointFingerprints[conflict] = true
}
return cc.knownEndpointFingerprintLocked(endpoint)
}
// CheckEndpointFingerprintConflicts checks saved trust without constructing a
// client, probing TLS or changing any pin. Discovery must not discard ambiguity
// by collapsing saved rows or replacing their addresses during a refresh.
+22
View File
@@ -1022,3 +1022,25 @@ func TestClusterProducerPinConflictsAreOwnedAndScoped(t *testing.T) {
}
}
}
func TestClusterExplicitEmptyEndpointTrust(t *testing.T) {
primary := "https://primary.test:8006"
member := "https://member.test:8006"
cc := &ClusterClient{config: ClientConfig{Host: primary, Fingerprint: "primary-pin"}, endpoints: []string{primary, member}, endpointFingerprints: map[string]string{member: ""}}
fingerprint, err := cc.getEndpointFingerprint(member)
if err != nil || fingerprint != "" {
t.Fatalf("explicit CA policy inherited primary pin: %q %v", fingerprint, err)
}
if cc.canTrustMemberOnFirstUseLocked(member) {
t.Fatal("explicit CA policy allowed automatic trust capture")
}
cc.endpointFingerprints["https://MEMBER.test:8006/"] = "different-pin"
if _, err := cc.getEndpointFingerprint(member); !errors.Is(err, errConflictingEndpointFingerprints) {
t.Fatalf("explicit CA/pin ambiguity did not fail closed: %v", err)
}
for _, endpoint := range []string{"https://alias.test:8006", "https://member.test:8007", "http://member.test:8006", "https://member.test:8006/other"} {
if _, known, err := ResolveEndpointFingerprint(endpoint, map[string]string{member: ""}); known || err != nil {
t.Fatalf("explicit trust escaped authority/base path: %s", endpoint)
}
}
}