shankar0123
0834bc1ad5
Phase 14 of the deploy-hardening II master bundle. The procurement- team headline doc + per-connector operator guides for the top 5 most-deployed connectors. NEW docs/deployment-vendor-matrix.md (~30 rows): - Per (connector × vendor-version) status: ✓ / CI / mock / pending / n/a - Known issues + workarounds + e2e test name reference - LTS + current-stable scope per frozen decision 0.1 - Quarterly re-pin cadence guidance for sidecar digests - "How to add a new vendor version" recipe Per frozen decision 0.14: a (connector × vendor-version) cell is "verified" only when ALL apply: ≥1 happy-path e2e green; ≥1 specific-quirk test green for that version; operator manual smoke completed at least once. Cells lacking the third criterion show "CI" status (auto-tests green but pending operator validation). Status snapshot at bundle close: - NGINX 1.25 + 1.27: CI - Apache 2.4: CI - HAProxy 2.6 + 2.8 + 3.0: CI - Traefik 2.x + 3.x: CI - Caddy 2.x: CI - Envoy 1.30 + 1.32: CI (file-mode SDS only; gRPC SDS V3-Pro) - Postfix 3.6 + 3.8: CI - Dovecot 2.3: CI - IIS 10 (2019, 2022): pending (Windows-host-only CI) - F5 v15.1 + v17.0 + v17.5: mock (real-F5 vagrant box documented) - SSH OpenSSH 8.x + 9.x: CI - WinCertStore (2019, 2022): pending (Windows-host-only) - JavaKeystore JDK 11 + 17 + 21: pending - K8s 1.28 + 1.30 + 1.31: CI NEW per-connector deep-dive docs: - docs/connector-nginx.md (~150 lines, 10 quirks documented) - docs/connector-k8s.md (~110 lines, 10 quirks) - docs/connector-iis.md (~120 lines, 10 quirks; Windows-host-only CI constraint loud) - docs/connector-apache.md (~80 lines, 10 quirks) - docs/connector-f5.md (~190 lines, 10 quirks; two-tier validation recipe for operator-supplied real-F5 vagrant box) Each doc follows the same structure: - Overview - Vendor versions tested - Per-quirk operator guidance (one section per TestVendorEdge_<vendor>_<edge>_E2E) - Troubleshooting matrix - V3-Pro deferrals - Related docs cross-refs Other connector docs (HAProxy, Traefik, Caddy, Envoy, Postfix, Dovecot, SSH, WinCertStore, JavaKeystore) live in docs/connectors.md + are referenced from the matrix. Phase 15 next: per-vendor CI matrix job in .github/workflows/ci.yml.
3.8 KiB
Kubernetes Secrets Connector — Operator Deep-Dive
Per Phase 14 of the deploy-hardening II master bundle.
Overview
The K8s connector (internal/connector/target/k8ssecret/) deploys
TLS certs into kubernetes.io/tls Secrets. Atomic at the API
server level (Update is transactional); the post-deploy verify
SHA-256-compares the returned Secret data against deployed bytes
(defends against admission webhooks that modify cert data).
Vendor versions tested
- Kubernetes 1.28 LTS
- Kubernetes 1.30
- Kubernetes 1.31 (current stable)
Per-quirk operator guidance
Kubelet sync wait contract
TestVendorEdge_K8s_KubeletSyncWaitContract_DefaultTimeout60s_E2E
After Secret update, kubelet projects new cert bytes into
pod-mounted volumes. Default sync interval ~60s. The connector
waits up to CERTCTL_K8S_DEPLOY_KUBELET_SYNC_TIMEOUT (default
60s).
Operator action: for slow clusters (large pod count, slow node DNS), tune the env var upward. For fast clusters, the default is fine.
Admission webhook mutation
TestVendorEdge_K8s_AdmissionWebhookModifiesSecretData_DeployDetectsViaSHA256Compare_E2E
Some admission webhooks (Vault Agent Injector, OPA Gatekeeper) mutate Secret data on Update. The connector pulls the Secret back after Update and SHA-256-compares against deployed bytes. Mismatch surfaces as deploy failure.
Multi-version API stability
TestVendorEdge_K8s_K8s128LTS_vs_130_vs_131_SecretAPIContractStable_E2E
kubernetes.io/tls Secret schema (data.tls.crt + data.tls.key)
is stable across 1.28-1.31. No per-version branch needed.
Typed vs Opaque Secret
TestVendorEdge_K8s_TypedKubernetesIOTLSVsUntypedOpaque_DeployRespectsType_E2E
Connector preserves operator-supplied Secret type. Typed
kubernetes.io/tls is the canonical form; untyped Opaque is
preserved for operators with legacy automation that expects it.
Cert-manager interop
TestVendorEdge_K8s_CertManagerInterop_RawSecretVsCertificateCRD_E2E
Connector targets raw Secrets, NOT cert-manager Certificate CRs.
Operators using cert-manager should NOT also point certctl at the
same Secret name (cert-manager will overwrite). Documented
coexistence: certctl handles non-cert-manager Secrets;
cert-manager handles its own.
Multi-namespace
TestVendorEdge_K8s_MultiNamespaceDeploy_DeployUpdatesCorrectNamespace_E2E
Connector targets the configured Namespace only. Cross-namespace
deploys require multiple connector entries.
RBAC errors
TestVendorEdge_K8s_RBACInsufficientPermissions_DeployFailsWithActionableError_E2E
Connector surfaces the K8s API's forbidden: secrets is restricted
error verbatim. Operator action: bind a Role with
secrets: get,update,create verbs to the agent's ServiceAccount.
Labels + annotations preservation
TestVendorEdge_K8s_LabelsAnnotationsPreserved_E2E
Connector merges (not replaces) operator-supplied metadata. Custom labels/annotations on the Secret survive cert rotation.
Pod-mounted Secret rollover
TestVendorEdge_K8s_PodMountedSecretRollover_E2E
When a pod mounts the Secret as a volume, kubelet projects new cert bytes into the pod's filesystem after sync. Pods watching the file (via inotify or polling) pick up the new cert without restart.
Immutable Secret flag
TestVendorEdge_K8s_ImmutableSecretFlag_E2E
K8s Secrets can be marked immutable: true for performance.
Update fails with actionable error; operator must drop the flag,
update, then re-apply if desired.
V3-Pro deferrals
- cert-manager
CertificateCR interop as first-class deploy target (V3-Pro: certctl as cert-manager external issuer). - Multi-cluster federation (deploy a single cert across N clusters with single connector entry).