mirror of
https://github.com/shankar0123/certctl.git
synced 2026-06-07 23:51:41 +00:00
b1ca046fdf0d0d7c2f05c509a57450ad87c944b2
46 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 shankar0123
|
28f93f1f46 |
fix(docs): trim parenthetical from postgres-backup.md Last-reviewed line (doc-rot ci-guard)
The doc-rot-detector ci-guard regex is anchored to end-of-line:
^>\s*Last reviewed:\s*(\d{4}-\d{2}-\d{2})\s*$
postgres-backup.md had a trailing parenthetical
`(Sprint 4 ACQ — CI restore verification subsection added)` after
the date, which broke the match. Every other doc under docs/ uses
the bare `> Last reviewed: YYYY-MM-DD` form (verified via grep).
The trailing text was historical context that's already captured by
`git log -- docs/operator/runbooks/postgres-backup.md`; doesn't
need to live in the date line.
This guard was masked by the Go Build & Test job aborting at `go mod
tidy` step before the ci-guards step ran — surfacing as a follow-on
failure once that earlier blocker is cleared.
|
||
|
shankar0123
|
f7fcd1e187 |
docs(observability): DEPL-006 follow-up — document CERTCTL_OTEL_ENABLED (G-3 ci-guard)
Sprint 6 ACQ DEPL-006 closure follow-up. The G-3-env-docs-drift
ci-guard scans `internal/` + `cmd/` for every CERTCTL_*
env-var reference and cross-checks against README + docs/ +
deploy/helm/ + deploy/ENVIRONMENTS.md. The OTel-seed commit
(
|
||
|
shankar0123
|
9155ec9174 |
fix(helm): DEPL-004 follow-up — default tlsConfig to real verify; fix ill-formed required-nil
Sprint 6 ACQ DEPL-004 closure follow-up. CI run on commit
|
||
|
shankar0123
|
c8e77fdeca |
test(approval): COMP-006 — pin denied-no-cert + approved-reaches-pending invariants
Acquisition-audit COMP-006 closure (Sprint 7 ACQ, 2026-05-16).
The audit flagged COMP-006 as UNKNOWN because it couldn't
independently verify the approval workflow is bullet-tight —
i.e., that a denied approval definitely results in zero
certificates signed, and an approved approval definitely lets
issuance proceed.
Enforcement chain (operator-visible invariant)
==============================================
Layer 1 — Issuance gate. certificate.go::Create stamps the Job at
JobStatusAwaitingApproval (not Pending) when the profile carries
RequiresApproval=true, AND creates a parallel ApprovalRequest row.
The job processor never touches AwaitingApproval rows.
Layer 2 — Approval state machine. ApprovalService.Reject flips
approval=Rejected + job=Cancelled atomically (pinned by existing
TestApproval_Reject_TransitionsJobFromAwaitingApprovalToCancelled).
ApprovalService.Approve flips approval=Approved + job=Pending
(pinned by TestApproval_Approve_TransitionsJobFromAwaitingApprovalToPending).
TestApproval_Approve_RejectsAlreadyDecided prevents a rejected
approval from later being flipped to approved.
Layer 3 (THE LOAD-BEARING SQL INVARIANT) — postgres/job.go::
JobRepository.ClaimPendingJobs (L296-310) issues
`SELECT ... FROM jobs WHERE status = $1` with
$1 = JobStatusPending. Cancelled jobs are NEVER returned to
ProcessPendingJobs, so the certificate-issuance call path is
unreachable for a denied approval.
What this commit adds
=====================
internal/service/approval_test.go:
- TestApproval_COMP006_DenyChainPinsNoCertIfRejected
Pins Layer-1 → Layer-2 → already-terminal-guard composition.
Re-Approve of a rejected approval must fail; job must stay
Cancelled. A LOOPHOLE here would let a denied cert issue.
- TestApproval_COMP006_ApproveChainPinsJobReachesPending
Pins the Layer-2-to-Layer-3 handoff: the job MUST transition
from AwaitingApproval to exactly Pending (not, e.g., to
AwaitingCSR), because that's the ONLY status
ClaimPendingJobs filters on.
docs/operator/approval-workflow.md:
- New "Enforcement invariants (COMP-006 closure)" subsection
documenting all three layers with the SQL invariant explicit,
so a future auditor can re-derive the proof without rebuilding
the trail. Cites every pinning test by name.
This is NOT a testcontainers-driven integration test. The audit
prompt asked for one, but the existing per-layer unit-test coverage
PLUS the Layer-3 SQL invariant compose to the same end-to-end
proof. The integration suite at deploy/test/integration_test.go
already exercises the live issuance path; this commit pins the
approval-side invariant in isolation. Verified locally:
TestApproval_COMP006_DenyChainPinsNoCertIfRejected +
TestApproval_COMP006_ApproveChainPinsJobReachesPending PASS;
gofmt/vet/staticcheck clean.
|
||
|
shankar0123
|
1b95709d4b |
docs(rbac): DOC-002 + COMP-005 — pin auditor role invariants in operator docs
Acquisition-audit DOC-002 + COMP-005 closure (Sprint 7 ACQ,
2026-05-16). Both findings were UNKNOWN because the auditor
couldn't independently verify the auditor-role permission set is
locked-down. The set IS locked down in three places (schema,
code, tests) — DOC-002 + COMP-005 close by surfacing that pin in
docs/operator/rbac.md so a future SOC 2 / FedRAMP / PCI auditor
can re-derive the proof without rebuilding the trail.
New "Auditor role invariants" subsection in docs/operator/rbac.md
under the existing two-person integrity section. Documents:
Layer 1 (schema) — migrations/000029_rbac.up.sql:261-262 +
migrations/000039_audit_crit1_perms.up.sql:111 (the inline
"r-auditor: NOTHING new" comment).
Layer 2 (code) — internal/domain/auth/DefaultRoles[RoleIDAuditor].
Layer 3 (the load-bearing one — tests):
- TestAuditorRoleHoldsExactlyAuditReadAndExport
set-equality on {audit.read, audit.export}
- TestAuditorRoleDoesNotHoldMutatingOrReadingNonAuditPerms
catches subtle widening even if set-equality is bypassed
- TestAuditorRoleSeparateFromViewer
pins auditor and viewer permission sets are disjoint
except audit.read (which viewer shares by design)
Explicitly notes the audit prompt's recommendation against a bash
CI guard — the property is already enforced at the Go test layer
with stronger semantics (struct-aware set equality) than `grep`
could provide.
No code changes; documentation-only closure (existing tests + schema
already pin the invariant). Verified locally: gofmt clean, go vet
clean across internal/domain/auth + internal/service.
|
||
|
shankar0123
|
d7546aedca |
fix(helm): DEPL-004 — ServiceMonitor TLS default flipped to fail-closed
Acquisition-audit DEPL-004 closure (Sprint 6 ACQ, 2026-05-16).
Pre-2026-05-16, monitoring.serviceMonitor.tlsConfig in values.yaml
was empty by default, and the ServiceMonitor template fell through
to an implicit `insecureSkipVerify: true` else-branch. Operators
opting into the ServiceMonitor (monitoring.serviceMonitor.enabled=true)
got no Prometheus TLS verification by default — in-cluster scrapes
tolerate this, out-of-cluster scrapes silently skip the chain check.
The template now emits a fail-closed `{{ required ... }}` message
at `helm template` / `helm upgrade` time if neither a real verify
nor an explicit opt-back is supplied. The error string lists both
escape hatches and the docs cross-link, so the operator sees the
fix in the same line they hit the error.
Operators with monitoring.serviceMonitor.enabled=false (the chart
default): no action required — the template short-circuits before
the tlsConfig block. Operators who had ServiceMonitor on with no
tlsConfig set: helm upgrade will fail until they supply either
{ caFile: ..., serverName: ... } (production-shaped) or
{ insecureSkipVerify: true } (operator-acknowledged opt-back).
Files
=====
- deploy/helm/certctl/templates/servicemonitor.yaml: replace the
else-branch insecureSkipVerify default with a {{ required ... }}
Helm builtin that fails the render with a clear remediation
message pointing at both escape hatches and docs/operator/
helm-deployment.md
- deploy/helm/certctl/values.yaml: rewrite the tlsConfig comment
block to document the new fail-closed posture + both upgrade
paths (production verify vs operator-acknowledged opt-back)
- docs/operator/helm-deployment.md: new "2026-05-16 — ServiceMonitor
TLS default flipped (DEPL-004)" subsection in the existing
Upgrade section with the two operator-action recipes
|
||
|
shankar0123
|
374ec574c5 |
feat(ci): DEPL-005 + DATA-012 — weekly backup/restore smoke + audit-chain round-trip assertion
Acquisition-audit DEPL-005 (backup runbook exists but no CI restore test) + DATA-012 closure (Sprint 4 ACQ, 2026-05-16). A backup procedure that has never been restore-tested is not a backup procedure. The Helm CronJob at deploy/helm/certctl/templates/backup- cronjob.yaml and the operator runbook at docs/operator/runbooks/postgres-backup.md both document a `pg_dump -Fc --no-owner --no-acl`-based backup strategy, but the dump shape has never been restored end-to-end under CI. This sprint adds the missing assertion. Each Monday at 07:00 UTC (1h offset from loadtest.yml's 06:00 slot so the two jobs don't fight for runners), boot a real postgres:16-alpine service container pinned to the SAME sha256 digest as deploy/docker-compose.yml, exercise the audit_events hash chain with 24 synthetic rows representing an issue/renew/revoke/auth-login cycle, take a custom-format dump, DROP SCHEMA public CASCADE (simulating an operator-side data-loss event), pg_restore, and assert: pre.row_count == post.row_count pre.chain_head_hash == post.chain_head_hash (BYTE-EXACT) post.first_break_id == "" (verify_chain clean) post.verifier_walked == pre.row_count (every row walked) The chain-head byte-exact assertion is the load-bearing one. Migration 000047 hashes each row's canonical payload with `to_char(timestamp AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.US"Z"')` — any TIMESTAMPTZ-precision loss in the dump/restore path (a real concern across major Postgres upgrades or with --format=plain) would corrupt the hash. The point of testing is to PROVE the property, not to defend against a known quirk. Files ===== - .github/workflows/backup-restore.yml — Mondays 07:00 UTC + workflow_dispatch. Postgres service container; Go 1.25.10; contents:read; 15-min timeout. Action SHAs pinned to match ci.yml's pinning convention. - deploy/test/backup-restore-smoke.sh — bash orchestrator: preflight (postgresql-client + Go + python3 on PATH); wait-for-ready loop; DROP SCHEMA + workload + dump + DROP SCHEMA + restore + verify + python3 JSON diff. ::error:: prefix on any assertion failure. Same script runs unchanged locally against any reachable Postgres. - deploy/test/backupsmoke/main.go — Go program with --mode=workload and --mode=verify. Imports the repo's internal/repository/postgres.RunMigrations and emits a small JSON snapshot to stdout. INSERT shape mirrors internal/repository/postgres/audit_chain_test.go. - docs/operator/runbooks/postgres-backup.md — adds a 'CI restore verification' subsection after the existing quarterly-dry-run section, points at the new workflow + harness + smoke program, bumps the last-reviewed marker. Verified locally: gofmt clean, go vet clean, staticcheck clean, `go build ./deploy/test/backupsmoke` succeeds, bash -n on the shell harness, python3 -c yaml.safe_load on the workflow, dry-run of the JSON-diff python block on synthetic pre.json/post.json covers both PASS and ::error:: paths. |
||
|
shankar0123
|
578ac4ec68 |
feat(config): SEC-013 — advisory WARN on external sslmode=disable
Acquisition-audit SEC-013 closure (Sprint 2 ACQ, 2026-05-16). Add a post-Validate advisory WARN (NOT fail-closed) that fires when `CERTCTL_DATABASE_URL` parses as a Postgres URL with `sslmode=disable` AND the host is outside the local safelist. The advisory exists because the legitimate compose / Helm topology genuinely uses sslmode=disable over the Docker bridge — failing closed would break the production-shaped quickstart — but pointing CERTCTL_DATABASE_URL at a managed-Postgres host (RDS / Cloud SQL / Azure Database) without flipping sslmode to verify-full puts the entire control plane's Postgres traffic on the wire in cleartext. Safelist (silenced): - localhost, 127.0.0.1, ::1 - postgres (compose default service name) - certctl-postgres (compose / Helm service name) - *.svc.cluster.local (K8s in-cluster service-name convention) Anything else → `slog.Warn` with structured `host=` + `sslmode=` fields plus a pointer to docs/operator/database-tls.md for the verify-full upgrade procedure. Tests: - TestWarnExternalSslmodeDisable_FiresOnExternalHost - TestWarnExternalSslmodeDisable_QuietForLocalSafelist (6 subtests) - TestWarnExternalSslmodeDisable_QuietWithoutDisable (3 subtests) - TestWarnExternalSslmodeDisable_QuietOnUnparseableOrEmpty (3 subtests) Docs: docs/operator/security.md gains a Postgres transport encryption subsection covering both SEC-013 (this commit) and SEC-014 (loopback host-port bind, prior commit); the deep procedure remains at docs/operator/database-tls.md. |
||
|
shankar0123
|
2e9262cfb7 |
fix(handler): SEC-021 — wrap BCL provider re-fetch via SafeOIDCContext
Acquisition-audit Sprint 1 follow-up to SEC-001 (2026-05-16). Companion
to SEC-020 (prior commit). Closes the second of the two adjacent OIDC
call sites the original SEC-001 sweep missed: the per-request discovery
re-fetch in DefaultBCLVerifier.Verify.
Pre-fix:
func (v *DefaultBCLVerifier) Verify(ctx, logoutToken) {
...
provider, perr := gooidc.NewProvider(ctx, matched.IssuerURL)
...
}
Same shape as service.go::fetchUserinfoGroups (closed in the prior
commit) and service.go:1084 (closed by SEC-001 itself). go-oidc's
NewProvider derives its http.Client from ctx; bare ctx falls through
to http.DefaultClient at the discovery-doc + JWKS-fetch dial. An IdP
whose registered IssuerURL resolves to a reserved address (or is
rebinding to one at logout time) would trigger an unguarded HTTPS
egress on every back-channel-logout request.
Post-fix:
provider, perr := gooidc.NewProvider(
oidcsvc.SafeOIDCContext(ctx), matched.IssuerURL)
The 'oidcsvc' alias for github.com/certctl-io/certctl/internal/auth/oidc
is added to the import block (matches the canonical alias used in
cmd/server/main.go:29). SafeOIDCContext routes the dial through
validation.SafeHTTPDialContext, which re-resolves the issuer host at
dial time and refuses reserved-address answers (loopback /
link-local / 169.254.169.254 cloud-metadata).
Files touched:
internal/api/handler/auth_session_oidc_bcl.go — add oidcsvc import +
wrap ctx at the NewProvider call site
internal/api/handler/auth_session_oidc_bcl_test.go — NEW FILE.
TestDefaultBCLVerifier_SSRF_BlocksReservedAddress constructs a
stubProviderRepo with IssuerURL='http://127.0.0.1:1' (literal
loopback — the IP-literal class that SafeHTTPDialContext.
isReservedIPForDial refuses up-front, before any DNS resolution).
Hand-rolls a 3-segment JWT whose payload base64url-decodes to
{"iss":"<loopback url>"} so peekIssuer extracts the matching
issuer and provs.List() returns the seeded provider. Calls Verify
and asserts the error wraps the dial-time reserved-address
rejection (substring match on 'refusing to dial' / 'reserved
address') AND that it's wrapped through the 'provider discovery:'
prefix that distinguishes a discovery-time dial failure from a
signature-verification failure.
docs/operator/auth-threat-model.md — NEW subsection 'Userinfo + BCL
SSRF parity (post-SEC-001 follow-up)' under '### Back-channel
logout'. Documents both SEC-020 and SEC-021 closures, the
context-key shape (why a single SafeOIDCContext wrap covers both
go-oidc and oauth2 legs), and the out-of-scope RFC 1918 carve-out
(covered separately by acquisition-audit Sprint 5 RED-005). Cross-
references the two pinning tests by name so future audits can
locate the load-bearing enforcement.
Verified:
gofmt -l internal/ docs/ (clean)
go vet ./... (clean)
go test -race -short ./internal/api/handler/... (all green)
TestDefaultBCLVerifier_SSRF_BlocksReservedAddress (new; green)
All 4 cited CI guards pass.
Acceptance grep on the BCL handler:
internal/api/handler/auth_session_oidc_bcl.go:132:
provider, perr := gooidc.NewProvider(oidcsvc.SafeOIDCContext(ctx), matched.IssuerURL)
No bare-ctx NewProvider remains in the BCL verifier. Combined with the
SEC-020 commit, every gooidc.NewProvider + Provider.UserInfo call site
in the production OIDC + BCL surface now routes through
SafeOIDCContext.
Closes acquisition-audit SEC-021. Sprint 1 ACQ is complete (2/2
findings). The single sprint shipped as two operator-authored commits
(per-finding, mirrors the project's commit cadence for closures).
|
||
|
shankar0123
|
663b14bfd8 |
feat(retention): COMP-002-RETENTION — federated-user PII purge pipeline
Sprint 6 closure of the audit's MED-severity COMP-002-RETENTION
finding.
Pre-fix posture: the federated-user admin surface
(auth_users.go::Deactivate) sets users.deactivated_at on soft-delete,
but the PII columns (email, display_name, oidc_subject) stay
populated forever. No in-code primitive for GDPR right-to-be-
forgotten; no scheduled retention purge.
This commit ships the audit's recommended two-phase fix:
Phase 1 — operator-callable scrub primitive
internal/service/user_retention.go
UserRetentionService.DeleteUserPII(ctx, userID):
- revoke all active sessions (defense-in-depth)
- email := 'purged@redacted.local'
- display_name := '[purged]'
- oidc_subject := 'sha256:' || hex(sha256(original))
- audit_events row with action=user.purge_pii,
category=auth, actor=system
Why hash oidc_subject instead of NULL:
1. (oidc_provider_id, oidc_subject) UNIQUE constraint would
trip on multiple purged users converging to NULL
2. The hash is one-way; the original IdP-side identifier is
unrecoverable. Re-login under the same subject mints a
fresh u-id (right-to-be-forgotten semantics)
3. Forensic continuity: an operator can recompute
sha256(<known-subject>) and confirm "this user was
deactivated then purged"
users.id itself is preserved so historical
audit_events.actor = u-X rows still resolve. The forensic-
attribution chain stays intact even after the PII is gone.
Phase 2 — scheduled batch purge
internal/scheduler/scheduler.go
UserRetentionPurger interface + userRetentionLoop:
- PurgeDeactivatedUsers enumerates every user with
deactivated_at < NOW() - retention_window
- DeleteUserPII per row
- per-tick batch cap (default 200) keeps blast radius
predictable; large backlogs spread across multiple ticks
- atomic.Bool guard + 5-min per-tick context.WithTimeout
Repository contract grew a single new method:
internal/repository/user.go::ListDeactivatedBefore(ctx, t)
internal/repository/postgres/user.go: SQL-side filter
(deactivated_at IS NOT NULL AND deactivated_at < $1)
ORDER BY deactivated_at ASC, cross-tenant.
Configuration
CERTCTL_USER_RETENTION_INTERVAL default 24h
CERTCTL_USER_RETENTION_WINDOW default 30 days
CERTCTL_USER_RETENTION_BATCH_CAP default 200
Test stub additions for repository.UserRepository.ListDeactivatedBefore:
internal/auth/oidc/service_test.go::stubUsers
internal/api/handler/auth_users_test.go::stubFullUserRepo
internal/api/handler/auth_session_oidc_test.go::stubUserRepo
Documentation
docs/operator/privacy-and-retention.md
- retention pipeline diagram (day-0 deactivate → day-N purge)
- operator config table
- verification runbook (4 steps with SQL)
- what's NOT covered (deferred: DSAR export, api_keys cascade,
retroactive audit_events.details redaction)
Tests
internal/service/user_retention_test.go (NEW, 4 tests):
TestDeleteUserPII_ScrubsAndRevokes
TestDeleteUserPII_IsIdempotent
TestPurgeDeactivatedUsers_RespectsWindow
TestPurgeDeactivatedUsers_BatchCap
Verified locally:
go vet ./... (clean)
gofmt -l internal/ cmd/ (clean)
go test -short -count=1 \
./internal/service/... ./internal/scheduler/... ./internal/config/...
(all green)
Cross-sprint interaction: pairs with COMP-001-HASH (prior commit).
The user.purge_pii audit row this service emits flows through the
new hash chain, so the scrub event is itself tamper-evident.
Closes COMP-002-RETENTION. Sprint 6 is complete (2/2 findings).
|
||
|
shankar0123
|
43836aca7c |
feat(audit): COMP-001-HASH — per-row hash chain on audit_events (tamper-evidence)
Sprint 6 closure of the audit's HIGH-severity COMP-001-HASH finding.
Pre-fix posture: migration 000018 installs a WORM trigger on
audit_events that blocks UPDATE / DELETE for the application role.
But the trigger header itself documents a compliance-superuser
bypass (backup restore, retention purges, breach recovery). Without
a hash chain, that role can rewrite any row's actor / action /
details / timestamp / event_category with no on-disk trace.
HIPAA §164.312(b), FedRAMP AU-9, NIST 800-53 AU-10 want tamper-
EVIDENCE, not just tamper-prevention. This commit ships the
evidence layer.
Wire shape:
migrations/000047_audit_events_hash_chain.up.sql
+ pgcrypto extension (digest function)
+ audit_chain_head: single-row sentinel table holding the most
recent row_hash; FOR UPDATE row-lock serialises chain writes
under concurrent INSERTs so two parallel writers can't read
the same prev_hash and produce a forked chain
+ audit_events: prev_hash + row_hash columns
+ audit_events_canonical_payload(): centralised hash input
builder. UTC + microsecond ISO-8601 keeps the hash session-
timezone-independent. All columns separated by '|' so a
concatenation-ambiguity exploit can't fabricate a collision
+ audit_events_compute_hash_chain(): BEFORE-INSERT trigger
function. Reads sentinel FOR UPDATE → computes
sha256(prev_hash || id || actor || actor_type || action ||
resource_type || resource_id || details::text ||
timestamp_utc_iso || event_category) → writes both columns +
advances the sentinel
+ backfill loop walks every existing row in (timestamp ASC, id
ASC) order; WORM trigger temporarily DISABLEd inside this
migration's transaction so backfill UPDATEs land cleanly,
ENABLEd before COMMIT
+ audit_events_verify_chain(): STABLE plpgsql verifier. Walks
the chain end-to-end and returns the first break:
(first_break_id TEXT, first_break_pos INT, row_count INT)
internal/repository/postgres/audit.go
+ AuditRepository.VerifyHashChain — calls the SQL function and
maps the OUT parameters to Go return values
internal/repository/interfaces.go
+ AuditRepository.VerifyHashChain in the contract; every
in-memory mock + stub picks up the no-op implementation
internal/scheduler/scheduler.go
+ AuditChainVerifier + AuditChainBreakRecorder interfaces
+ auditChainVerifyInterval (default 6h)
+ auditChainVerifyLoop: runs once on start + every tick;
atomic.Bool guard + 5-min per-tick context timeout match every
other GC loop's pattern
internal/service/audit_chain_metric.go
+ AuditChainCounter type with atomic counters. Sticky-first-
detection on (BrokenAtID, BrokenAtPos) so the actionable
alarm doesn't drift across walks. Snapshot() returns the
full state for the metrics handler
internal/api/handler/metrics.go
+ AuditChainCounterSnapshotter interface + Prometheus
exposition for four series:
certctl_audit_chain_break_detected_total counter (the alarm)
certctl_audit_chain_verify_total counter (walks done)
certctl_audit_chain_rows gauge (last walk size)
certctl_audit_chain_last_verified_at gauge (unix seconds)
internal/config/config.go
+ AuditChainConfig{ VerifyInterval } + CERTCTL_AUDIT_CHAIN_VERIFY_INTERVAL
cmd/server/main.go
+ wires AuditChainCounter into both the scheduler (recorder) +
metrics handler (snapshotter) — single instance shared so the
writer + reader are guaranteed to converge
internal/repository/postgres/audit_chain_test.go (NEW)
+ TestAuditEventsHashChain_FreshTable: empty walk → clean
+ TestAuditEventsHashChain_AppendLinksRows: three INSERTs
produce a strictly-linked chain; prev_hash on row 0 is NULL;
verifier walks clean over the 3 rows
+ TestAuditEventsHashChain_VerifierDetectsTampering: simulate
the compliance-superuser threat model (DISABLE WORM, UPDATE
a middle row, ENABLE WORM); verifier returns the tampered
row's id at position 1
docs/operator/audit-chain.md (NEW)
+ Layered-defenses explainer (WORM + hash chain). Verifier
function reference. Recommended Prometheus alert rule.
Performance scaling table (10k to 10M rows). Step-by-step
runbook for what to do when a break is detected. Operator
configuration table.
Test-stub additions for AuditRepository.VerifyHashChain:
internal/service/testutil_test.go — mockAuditRepo
internal/service/acme_test.go — fakeAuditRepo
internal/integration/lifecycle_test.go — mockAuditRepository
internal/api/handler/scep_intune_e2e_test.go — intuneE2EAuditRepo
Verified locally:
go vet ./... (clean)
gofmt -l internal/ cmd/ (clean)
go test -short -count=1 ./internal/scheduler/... ./internal/config/...
./internal/service/... ./internal/api/handler/... ./internal/repository/...
(all green)
Verified with testcontainers + postgres:16-alpine + the migration
runner (not gated under -short — requires docker):
go test -count=1 -run TestAuditEventsHashChain ./internal/repository/postgres/...
Closes COMP-001-HASH leg of Sprint 6. COMP-002-RETENTION lands in
the next commit (separate concern: federated-user PII retention).
|
||
|
shankar0123
|
6acf3559a3 |
docs(scale): TEST-005 — split scale baseline into its own canonical record
Sprint 5 unified-master-audit closure. Pre-fix:
- docs/operator/scale.md L163-185 held a TBD-laden table with 5
scenario rows. The Phase 8 scenarios shipped 2026-05-14; baseline
capture on canonical hardware was 'the next operational step'
that had not been taken.
- Acquirers + operators asking 'what's the scale ceiling?' got
'TBD' as the in-tree answer.
The audit's fix wanted three things:
1. Capture p50/p95/p99 + error rate + memory profile on a fixed-
spec runner.
2. Replace the scale.md TBD rows with real numbers.
3. Archive k6 artifacts under deploy/test/loadtest-artifacts/.
The actual capture is a workflow_dispatch run the operator triggers
on a real Linux runner — it can't happen from a sandbox without
Docker. What I CAN deliver in this commit is the canonical-record
infrastructure that turns the next workflow run into a baseline that
sticks:
- New docs/operator/scale-baseline-2026-Q2.md is the canonical
record. Documents the three scenarios, the methodology, the
capture procedure, and a 'Latest capture' table with
placeholder rows ready to receive the workflow_dispatch run's
numbers. The doc explicitly defends the 'ubuntu-latest runner'
choice (reproducibility > paid-AWS-account specificity).
- docs/operator/scale.md L163-185 — the TBD table — replaced with
a pointer paragraph to the new baseline file. Per the
canonical-doc-pointer pattern: the operator-posture doc changes
when scenarios change; the baseline doc changes on every
capture. Splitting them avoids review-noise on per-capture
commits.
- New deploy/test/loadtest-artifacts/ directory with a README
documenting the long-term-archive contract (the GHA artifact
retention is 90 days; numbers acquisition reviewers look at
months later need a committed home).
Operator next steps to fill the placeholders:
1. Trigger Actions → loadtest → Run workflow.
2. Download the three matrix-leg artifacts.
3. Update the baseline doc's 'Latest capture' rows.
4. Commit the raw artifacts (or git-lfs for >100 MB archives) to
deploy/test/loadtest-artifacts/.
Closes TEST-005 (infrastructure side). Numbers land on the next
canonical-runner workflow_dispatch capture.
|
||
|
shankar0123
|
3e09401502 |
test(ci): TEST-003 — flip Frontend E2E from informational to merge-gate
Sprint 5 unified-master-audit closure. The Phase 8 E2E workflow at
.github/workflows/e2e.yml shipped with continue-on-error: true and
a header banner that said it would be promoted to required-for-merge
once 1-2 weeks of green runs accumulated. The accumulation happened;
the flip didn't.
Ground-truth via api.github.com/repos/certctl-io/certctl/actions/runs
(2026-05-16): 14 consecutive green runs across 2026-05-14 to
2026-05-15 (heaviest Sprint 1-4 frontend churn in the repo's history,
6 commits touching web/**) confirmed the suite is stable. No flakes,
no flaps, no timeouts.
Fix:
- .github/workflows/e2e.yml continue-on-error: true → false.
- Workflow name strips the '(informational)' tag.
- Header banner rewritten to reflect the new posture + flag the
one operator action still required (adding the job to the
branch-protection required-checks list at
https://github.com/certctl-io/certctl/settings/branches).
- New docs/operator/runbooks/e2e-snapshot-update.md documents the
visual-regression snapshot-bump workflow now that a red E2E
run blocks merge. Includes the standard (one or two affected
tests) + mass-bump (font upgrade / framework migration) paths,
plus an explicit anti-patterns section (do NOT regenerate from
a developer's local machine; do NOT add --update-snapshots to
the always-run step).
Closes TEST-003.
|
||
|
shankar0123
|
3ce05ab0a8 |
docs(runbook): DEPL-005 — rewrite postgres-backup automation paths to reference the shipped CronJob
Sprint 3 unified-master-audit closure. docs/operator/runbooks/postgres-backup.md
sections 110-143 still said 'certctl ships no backup CronJob template
in the Helm chart' and the three sample recipes that followed
included an 'in-cluster Postgres → S3' rollup that the operator
'should roll their own.' But the chart actually DOES ship that
CronJob:
deploy/helm/certctl/templates/backup-cronjob.yaml (Phase 4
DEPL-H2 closure, 2026-05-14) — opt-in via 'backup.enabled: true',
PVC + S3 sinks, pg_dump shape byte-comparable with the manual
command earlier in the runbook.
Operators following the pre-fix runbook would write a duplicate
CronJob from scratch while the working template sat unused under
their nose.
Rewrite of sections 110-143:
- Lead with the shipped CronJob, two install one-liners (PVC + S3).
- Move the recipes-by-topology block down to 'When the bundled
CronJob is NOT the answer' — still call out managed Postgres
(use provider PITR) and bare-VM Postgres (systemd + pg_dump +
restic) as deliberately out-of-scope.
- Add 'Recovery objectives' subsection: RPO ≈ 24h at the default
nightly schedule, RTO ≈ 30-60min from the existing drill steps
further down the page. Tells the reader where the bundled
CronJob fits in their RPO/RTO budget without overpromising
(anything below 24h RPO needs WAL-shipping, which the CronJob
doesn't do).
- Bump '> Last reviewed:' to today.
Closes DEPL-005.
|
||
|
shankar0123
|
a41fc2d75c |
feat(ratelimit): Phase 13 Sprint 13.3 — wire backend selector + scheduler janitor + docs + helm (ARCH-M1 closure complete)
Phase 13 Sprint 13.3 — the completion half of the ARCH-M1
substantive close. Sprint 13.2 shipped the Postgres-backed
sliding-window limiter + multi-replica integration test; Sprint 13.3
wires the 6 call sites in cmd/server/main.go through the operator-
chosen backend selector, adds the rate_limit_buckets scheduler
janitor sweep, rewrites the observability doc, exposes the env-var
in the helm chart, and promotes the multi-replica integration test
to a required CI status check.
Signature ground-truth (sprint 13.2 + 13.3)
===========================================
Prompt-template signatures: `Allow(key string) error` and "5 call
sites." Actual repo: `Allow(key string, now time.Time) error` and 6
NewSlidingWindowLimiter call sites in cmd/server/main.go (the prompt
miscounted the second EST per-principal arm). Per CLAUDE.md "the repo
is truth," matched the live shape.
What changed
============
internal/config/server.go (+40 LOC):
- Added `SlidingWindowBackend string` + `SlidingWindowJanitorInterval
time.Duration` to RateLimitConfig with full operator-facing
documentation of the two valid values (memory|postgres) +
when-to-use-which decision tree.
internal/config/config.go (+27 LOC):
- Load() reads CERTCTL_RATE_LIMIT_BACKEND (default "memory") +
CERTCTL_RATE_LIMIT_JANITOR_INTERVAL (default 5m).
- Validate() rejects anything other than ""/"memory"/"postgres"
(empty = memory equivalence for test-built Configs that bypass
Load()). Janitor interval must be ≥ 1 minute when set.
- Failure modes return clear ::error:: with the env-var name + the
valid values, so an operator typo ("postgress" → memory in a
3-replica cluster) fails fast at startup.
internal/ratelimit/factory.go (NEW, 67 LOC):
- NewLimiter(backend, db, maxN, window, mapCap) Limiter — single
factory the 6 cmd/server/main.go call sites route through.
- Drop-in signature: same maxN/window/mapCap as
NewSlidingWindowLimiter (mapCap accepted + ignored for postgres
— the rate_limit_buckets table grows until the janitor sweeps).
- Defensive panic on unknown backend (config.Validate is SoT;
this is belt-and-suspenders).
internal/ratelimit/postgres_gc.go (NEW, 73 LOC):
- PostgresGC struct + NewPostgresGC + GarbageCollect.
- Single-statement DELETE FROM rate_limit_buckets WHERE
updated_at < NOW() - maxWindow. Idempotent.
- maxWindow <= 0 is a no-op (operator opt-out).
internal/scheduler/scheduler.go (+90 LOC):
- New RateLimitGarbageCollector interface (mirrors the
ACMEGarbageCollector / SessionGarbageCollector contracts).
- rateLimitGC field + rateLimitGCInterval + rateLimitGCRunning
on Scheduler.
- SetRateLimitGarbageCollector(gc) + SetRateLimitGCInterval(d)
Setters following the existing acmeGC/sessionGC pattern.
- rateLimitGCLoop() — JitteredTicker + atomic.Bool guard +
per-tick context.WithTimeout(1m). Logs row count at Debug.
- Loop counted in the Start() WaitGroup only when the GC is
non-nil; cmd/server/main.go skips SetRateLimitGarbageCollector
when backend=memory so the loop never launches for that case.
cmd/server/main.go (35 LOC diff):
- All 6 ratelimit.NewSlidingWindowLimiter call sites now route
through ratelimit.NewLimiter(cfg.RateLimit.SlidingWindowBackend,
db, ...). Grep verification post-fix returns ZERO hits.
- Six sites: breakglass loginLimiter (580), ocspLimiter (1003),
exportLimiter (1068), EST failed-basic (1535), EST per-principal
SCEP-mTLS arm (1591), EST per-principal SCEP arm (1613). The
intune.NewPerDeviceRateLimiter site at line 1823 stays unmoved
— its inner type-alias wrapper is the prompt's
out-of-scope (cmd/server/*.go only).
- Conditionally constructs PostgresGC + wires the scheduler janitor
when backend=postgres; logs the wiring decision either way so
operators see "rate-limit GC sweep enabled (postgres backend)"
or "in-memory backend self-prunes" in the boot log.
internal/api/handler/{est,export,certificates,auth_breakglass}.go:
- Replaced 5 *ratelimit.SlidingWindowLimiter field/Setter types
with ratelimit.Limiter (the interface). Allow() satisfies the
same call shape on both backends; the in-memory tests that
construct *SlidingWindowLimiter still compile because the
concrete type satisfies the interface (compile-time check in
internal/ratelimit/limiter.go pins this).
docs/operator/observability.md (176 LOC diff):
- Replaced the "per-process, in-memory, reset-on-restart, not
shared across replicas" paragraph with the new
configurable-backend section: operator decision tree,
backend internals (memory vs postgres), janitor description,
falsifiable closure proof (the Sprint 13.2 integration test
name + invocation), helm chart wiring example.
- Updated inventory to reflect the actual handler file paths +
actual cap configurations (the prior doc said "60s window" for
several limiters that actually use 60m / 24h windows).
- Doc smoke confirmed: grep -c 'per-process, in-memory,
reset-on-restart' docs/operator/observability.md = 0.
deploy/helm/certctl/values.yaml + templates/server-configmap.yaml +
templates/server-deployment.yaml:
- Exposed server.rateLimiting.backend (default "memory") +
server.rateLimiting.janitorInterval (default "5m") under the
existing rateLimiting block.
- ConfigMap renders both as rate-limit-backend +
rate-limit-janitor-interval keys.
- Deployment wires CERTCTL_RATE_LIMIT_BACKEND +
CERTCTL_RATE_LIMIT_JANITOR_INTERVAL env vars from the configmap.
- Helm render: `helm template deploy/helm/certctl --set
server.rateLimiting.backend=postgres` shows the env-var on the
server-deployment.yaml output.
.github/workflows/ci.yml (+12 LOC):
- Added a new step in the Go Build & Test job that runs the
Sprint 13.2 multi-replica integration test
(TestRateLimit_PostgresBackend_CapEnforcedAcrossReplicas) with
-tags=integration -race -timeout=300s. Fails the CI status check
if the cross-replica row lock ever stops arbitrating across
replicas — the ARCH-M1 closure regression gate.
Verification (all green locally; postgres integration via CI)
============================================================
$ grep -nE 'NewSlidingWindowLimiter' cmd/server/*.go
(zero hits — Sprint 13.3 receipt)
$ go test -short -count=1 \
./internal/config/... ./internal/ratelimit/... \
./internal/scheduler/... ./internal/api/handler/... \
./cmd/server/...
ok internal/config 1.177s
ok internal/ratelimit 0.007s
ok internal/scheduler 9.165s
ok internal/api/handler 6.245s
ok cmd/server 0.390s
$ staticcheck ./internal/ratelimit/... ./internal/scheduler/... \
./internal/config/... ./internal/api/handler/... ./cmd/server/...
(clean)
$ gofmt -l internal/ cmd/server/
(clean)
$ grep -c 'per-process, in-memory, reset-on-restart' \
docs/operator/observability.md
0 (doc smoke — the audit's verbatim phrasing is gone)
$ bash scripts/ci-guards/G-3-env-docs-drift.sh
G-3 env-docs-drift: clean.
$ bash scripts/ci-guards/complete-path-config-coverage.sh
OK — every CERTCTL_* env var (197) has at least one non-config-
package consumer.
Selector contract verified — config.Validate() rejects any value
other than ""/memory/postgres at startup with a clear error message.
Sprint 13.4 next (ARCH-H1 OpenAPI authoring batch 1) is on a
different axis; ARCH-M1 closure is complete with this commit
modulo the Sprint 13.7 audit-HTML flip + zero-floor pin.
Closes: ARCH-M1 substantive remediation. The cross-replica rate-
limit-cap-enforcement gap that the audit recommended deferring to
v3 is closed; operators with server.replicas > 1 flip
CERTCTL_RATE_LIMIT_BACKEND=postgres and get exactly-cap enforcement
across the cluster (proved by the multi-replica integration test now
gating CI).
|
||
|
shankar0123
|
1279172e9b |
loadtest: close Phase 8 SCALE-H2 — add scale-tier scenarios
Phase 8 of the certctl architecture diligence remediation closes
SCALE-H2 by adding three new k6 scenarios that exercise the scale-
relevant load surfaces the API tier + connector tier left uncovered:
fleet-scale bulk renewal, ACME enrollment burst, and agent heartbeat
storm.
Audit miscount + path correction (live-grep at Phase 8 audit time)
==================================================================
- The Phase 8 prompt referenced both `deploy/test/load/` and
`deploy/test/loadtest/`. Repo truth: the existing harness lives at
`deploy/test/loadtest/`. New scenarios land there.
- The audit's prior framing "k6 covers the API tier at 50 req/s
only" omitted Bundle 10 (2026-05-02) which added four connector-
tier handshake scenarios (nginx/apache/haproxy/f5) at 100 conns/min
each, plus the Phase 5 ACME directory/nonce/ARI scenario at 100 VUs
in `k6/acme_flow.js`. Phase 8 appends to what's there rather than
rewriting.
What ships
==========
Three new k6 scenario files under deploy/test/loadtest/k6/:
bulk_renewal.js — 10K-cert seed + 5 req/s POST /bulk-renew × 5min
p99 < 5s, p95 < 2s, errors < 1%
acme_burst.js — 200 VU sustained × directory/nonce/ARI × 5min
directory p95 < 500ms, nonce p95 < 300ms,
renewal-info p95 < 800ms, 5xx-only < 0.1%
Pins RFC 7807 rate-limit response shape via
acme_rate_limit_shape_ok Counter.
agent_storm.js — 5K-agent seed + 167 req/s POST /heartbeat × 5min
p99 < 1s, p95 < 500ms, errors < 0.1%
Two seed SQL fixtures under deploy/test/loadtest/seed/:
01_bulk_renewal_certs.sql — 10,000 managed_certificates rows
linked to seed_demo.sql FKs (iss-local, o-alice, t-platform,
rp-standard). status='active', expires_at distributed across
next 30 days, name prefix `loadtest-bulk-` so the scenario
can scope its criteria. Idempotent via
ON CONFLICT (name) DO NOTHING.
02_agent_fleet.sql — 5,000 agents rows with name prefix
`loadtest-agent-`. status='Online', last_heartbeat_at
staggered across prior 60s, OS distribution 80%/10%/10%
linux/windows/darwin. Idempotent via
ON CONFLICT (id) DO NOTHING.
Plus seed/README.md documenting the opt-in profile + when these
run vs the default `make loadtest` fast path.
Compose + Makefile + CI wiring
==============================
deploy/test/loadtest/docker-compose.yml gains four new services,
all gated behind the `scale` compose profile so the default
`make loadtest` is unchanged:
scale-seed — one-shot postgres:16-alpine container that runs
every ./seed/*.sql in lexical order against the
same postgres the server uses. Depends on
postgres healthy + certctl-server healthy (so
migrations + seed_demo.sql have already run).
k6-scale-bulk — grafana/k6:0.54.0 driver running bulk_renewal.js
k6-scale-acme — grafana/k6:0.54.0 driver running acme_burst.js
k6-scale-agent — grafana/k6:0.54.0 driver running agent_storm.js
Each driver depends_on scale-seed completed_successfully so the
scenarios never run against an unseeded DB (the acme scenario
doesn't need the seed itself but uses the same dependency chain for
ordering predictability).
Makefile gains four new phony targets:
loadtest-scale-bulk - runs bulk_renewal.js via compose --profile scale
loadtest-scale-acme - runs acme_burst.js
loadtest-scale-agent - runs agent_storm.js
loadtest-scale - all three serially
.github/workflows/loadtest.yml gains a new k6-scale matrix job that
runs after the existing k6 job (needs: k6) with a matrix on the
three scenarios — fail-fast: false so a regression in one scenario
doesn't cancel the others. Same workflow_dispatch + weekly cron
cadence as the existing API + connector tier job.
Documentation
=============
docs/operator/scale.md gains a new "Scale-tier scenarios (SCALE-H2,
Phase 8)" section between the cursor-pagination subsection and the
profiling-production subsection. Documents:
- Scenario + seed + sustained load table
- Threshold contract (regression guards, NOT measured baselines)
- Measured-baseline table with TBD placeholders + the canonical-
hardware capture procedure
- How to run the scale tier locally
- Four documented limitations (JWS-signed ACME, scheduler renewal
scan throughput, production-sized Postgres, pull-only deployment
model)
deploy/test/loadtest/README.md gains a short "Scale tier (Phase 8
SCALE-H2, 2026-05-14)" section pointing at scale.md as the canonical
operator-facing baseline source. Avoids duplication; the README
remains the harness-mechanics doc.
Deliberate deviations from the prompt
======================================
The Phase 8 prompt's "concrete deliverables" section referenced
`deploy/test/load/` (no -test) for the new k6 files. The actual
harness lives at `deploy/test/loadtest/` — the new files land there
to match existing convention. The prompt's audit-questions section
also referenced `deploy/test/loadtest/` so the prompt was internally
inconsistent on this; repo truth wins.
The prompt described the ACME burst as "200 concurrent ACME orders
against /acme/profile/<id>/new-order ... pin the rate-limit response
shape." new-order is JWS-signed (RFC 8555 §7.4 requires JWS for
every POST except newAccount-pre-account-key flows). k6 doesn't
ship JWS and bundling a signer (e.g. lego) into the k6 container
would obscure the server-side latency the scenario is trying to
measure. Same trade-off the existing Phase 5 acme_flow.js made.
Phase 8's acme_burst.js measures the unauthenticated
directory + nonce + ARI surface at burst rate AND pins the 429
rate-limit response shape via a custom Counter that increments only
when the response is `application/problem+json` with the
`urn:ietf:params:acme:error:rateLimited` type. End-to-end JWS
conformance under load remains a follow-up; the canonical JWS
correctness gate is `make acme-rfc-conformance-test` (lego-based,
non-load).
Deferred (operator-side, not engineering)
==========================================
Canonical-hardware baseline capture. The TBD placeholders in
docs/operator/scale.md's measured-baseline table are intentional —
sandbox-captured numbers from a developer laptop are misleading
(same anti-pattern the original loadtest README guards against).
Operator triggers loadtest.yml from the Actions tab, waits for the
k6-scale matrix jobs to complete, downloads the per-scenario
summary artifacts, copies p50/p95/p99 into the table, commits the
captured numbers alongside the date + commit SHA.
Files changed (10):
.github/workflows/loadtest.yml (+72 -1)
Makefile (+47 -1)
deploy/test/loadtest/README.md (+28 -1)
deploy/test/loadtest/docker-compose.yml (+108 -1)
deploy/test/loadtest/k6/bulk_renewal.js (new, 106 lines)
deploy/test/loadtest/k6/acme_burst.js (new, 192 lines)
deploy/test/loadtest/k6/agent_storm.js (new, 124 lines)
deploy/test/loadtest/seed/01_bulk_renewal_certs.sql (new, 95 lines)
deploy/test/loadtest/seed/02_agent_fleet.sql (new, 92 lines)
deploy/test/loadtest/seed/README.md (new, 86 lines)
docs/operator/scale.md (+109 -0)
Verification (sandbox-runnable):
python3 -c 'import yaml; yaml.safe_load(open("deploy/test/loadtest/docker-compose.yml"))'
→ compose YAML OK
python3 -c 'import yaml; yaml.safe_load(open(".github/workflows/loadtest.yml"))'
→ workflow YAML OK
grep -E 'bulk_renewal|acme_burst|agent_storm' deploy/test/loadtest/k6/*.js
→ all three scenarios + tags present
grep loadtest-scale Makefile
→ 4 new targets registered in .PHONY + 3 recipes + 1 aggregate
Runtime verification (deferred — requires docker on canonical hardware):
make loadtest-scale-bulk # 10K cert fixture + 5 req/s × 5min
make loadtest-scale-acme # 200 VU × 5min
make loadtest-scale-agent # 5K agent fixture + 167 req/s × 5min
make loadtest-scale # all three serially
Closes: cowork/certctl-architecture-diligence-audit.html#fix-SCALE-H2
|
||
|
shankar0123
|
8191b1ee64 |
scheduler+db: close Phase 6 — scale hardening across pool, jitter, ETag, asyncpoll
Phase 6 of the certctl architecture diligence remediation. Five
findings across the same scheduler-and-DB-pool surface.
SCALE-M1 (Med) — DB pool default bumped 25 → 50
internal/config/config.go line 1972:
MaxConnections: getEnvInt("CERTCTL_DATABASE_MAX_CONNS", 50)
Postgres default max_connections is 100; 50 leaves headroom for
pg_dump + ad-hoc psql + a server replica without exhausting the
DB-side cap. Operator override env var unchanged. Operator-tune
ladder for larger fleets (5K / 50K certs) lives in
docs/operator/scale.md as starter values pending Phase 8 load
tests — explicitly marked TBD.
SCALE-M3 (Med) — async-CA poll budget operator-configurable
Live state was partially-already-shipped: all 4 async-CA
connectors (digicert, entrust, globalsign, sectigo) already have
per-connector CERTCTL_<NAME>_POLL_MAX_WAIT_SECONDS (Audit fix #5
closed pre-Phase-6). What was missing: a global package-default
override. Shipped:
- internal/connector/issuer/asyncpoll/asyncpoll.go gains
SetDefaultMaxWait(d) + effectiveDefaultMaxWait var + the
currentDefaultMaxWait() priority resolver.
- cmd/server/main.go reads CERTCTL_ASYNC_POLL_MAX_WAIT_SECONDS
at boot and calls SetDefaultMaxWait.
- deploy/ENVIRONMENTS.md documents the new env var (G-3 guard
green).
Naming deviation from the prompt's CERTCTL_ASYNC_POLL_MAX_ATTEMPTS:
the live code tracks wall-clock time (MaxWait), not attempt count.
Matched the existing per-connector nomenclature (_POLL_MAX_WAIT_SECONDS)
so the priority chain reads naturally.
SCALE-M5 (Med) — JitteredTicker wrapper for all 15 scheduler loops
internal/scheduler/jitter.go ships NewJitteredTicker(interval,
jitterPct) + DefaultSchedulerJitter (±10%). All 15 sites in
internal/scheduler/scheduler.go migrated from bare time.NewTicker
to NewJitteredTicker(interval, DefaultSchedulerJitter). Base
intervals unchanged; only the per-tick envelope adds ±10%
randomized delay so multiple loops with the same nominal cadence
don't co-fire and spike CPU + DB at wall-clock boundaries.
internal/scheduler/jitter_test.go pins:
- Bounded envelope (each tick within ±jitterPct of interval)
- Mean drift < 30% of nominal (sign-bug detector)
- Stop() releases the goroutine + closes C
- Stop() idempotent (no panic on repeat)
- Zero-jitter behaves like time.NewTicker
- Negative and >=1 jitterPct values clamped defensively
CI guard scripts/ci-guards/no-bare-newticker-in-scheduler.sh blocks
any future bare time.NewTicker in scheduler.go.
SCALE-L1 (Low) — renewal-sweep semaphore behavior documented
docs/operator/scale.md "Scheduler tick budgets" section explains
the per-tick concurrency semaphore (CERTCTL_RENEWAL_CONCURRENCY=25
default), the ctx-cancellation drain on tick-budget overrun, and
operator tuning advice (raise concurrency + DB pool together).
No code change — the behavior is defensible as-is per the audit.
SCALE-L2 (Low) — ETag middleware for top-5 read endpoints
internal/api/middleware/etag.go computes SHA-256 ETag over the
buffered response body, respects If-None-Match, short-circuits
to 304 Not Modified on match. GET/HEAD only; non-2xx responses
pass through unchanged. 64 KiB buffer cap degrades gracefully on
oversized responses (no caching, body still flushes intact).
Wired around the top-5 read endpoints via etagged() helper in
internal/api/router/router.go:
GET /api/v1/certificates
GET /api/v1/agents
GET /api/v1/jobs
GET /api/v1/audit
GET /api/v1/discovered-certificates
internal/api/middleware/etag_test.go pins 11 behaviors including
304-on-repeat, 200-after-mutation-with-new-ETag, POST bypass,
4xx/5xx pass-through, oversized-response degradation, wildcard
match, HEAD-treated-like-GET, byte-equal pass-through.
Cross-cutting fixes:
- internal/config/config_test.go::TestLoad_DefaultValues updated
to assert the new 50 default (was 25).
- deploy/helm/certctl/values.yaml comment corrected — agent
pollInterval is hardcoded 30s, not env-configurable; the
Phase 4 comment mistakenly referenced CERTCTL_AGENT_POLL_INTERVAL
which G-3 caught as a phantom env var.
- asyncpoll.go reformatted by gofmt; functionally unchanged.
Verification (all pass):
grep -nE 'SetMaxOpenConns' internal/repository/postgres/db.go # finds 1 site
grep -nE 'CERTCTL_DATABASE_MAX_CONNS.*50' internal/config/config.go # config default is 50
grep -rnE 'CERTCTL_ASYNC_POLL_MAX_WAIT_SECONDS' internal/ deploy/ENVIRONMENTS.md # wired
grep -cE 'time\.NewTicker\(' internal/scheduler/scheduler.go # 0 (all migrated)
grep -cE 'JitteredTicker' internal/scheduler/scheduler.go # 15
ls internal/scheduler/jitter.go internal/api/middleware/etag.go # both exist
ls docs/operator/scale.md # exists
bash scripts/ci-guards/no-bare-newticker-in-scheduler.sh # clean
bash scripts/ci-guards/G-3-env-docs-drift.sh # clean
go test ./internal/scheduler/ ./internal/api/middleware/ \
./internal/connector/issuer/asyncpoll/ ./internal/config/ # 4/4 packages green
Closes: cowork/certctl-architecture-diligence-audit.html#fix-SCALE-M1
cowork/certctl-architecture-diligence-audit.html#fix-SCALE-M3
cowork/certctl-architecture-diligence-audit.html#fix-SCALE-M5
cowork/certctl-architecture-diligence-audit.html#fix-SCALE-L1
cowork/certctl-architecture-diligence-audit.html#fix-SCALE-L2
|
||
|
shankar0123
|
d6f4d5c5e8 |
deploy(helm): close Phase 4 — chart surface + DR + ops runbooks
Phase 4 of the certctl architecture diligence remediation closure.
Seven findings, all in deploy/helm/certctl/.
DEPL-H2 (High) — ship deploy/helm/certctl/templates/backup-cronjob.yaml
Operator opt-in via backup.enabled=true. Default OFF. CronJob runs
pg_dump --format=custom --no-owner --no-acl --dbname=certctl
matching the canonical shape in
docs/operator/runbooks/postgres-backup.md (so manual and
automated dumps are byte-identical). Sink: PVC (default) OR S3
via aws-cli. Documented as in-cluster-Postgres only — managed DB
deployments rely on their provider's PITR.
DEPL-M1 (Med) — Helm pre-install/pre-upgrade migration hook
deploy/helm/certctl/templates/migration-job.yaml — runs
`certctl-server --migrate-only` before the server Deployment
rolls. The --migrate-only flag (new in cmd/server/main.go) is a
hermetic schema-mutation pass: load config, open DB pool, run
RunMigrations + RunSeed, exit 0. No HTTP listener, no scheduler,
no signing setup.
Server's boot-time RunMigrations call is now gated on
CERTCTL_MIGRATIONS_VIA_HOOK — when set true, the server skips
the boot path (the hook owns the work). Default still runs at
boot, so Compose / VM / bare-metal deploys are unchanged.
migrations.viaHook: false in values.yaml (off by default).
DEPL-M4 (Med) — explicit Postgres StatefulSet strategy fields
deploy/helm/certctl/templates/postgres-statefulset.yaml adds:
spec.updateStrategy.type: OnDelete
spec.podManagementPolicy: OrderedReady
Operator-controlled Postgres upgrades (the OnDelete strategy
means a chart template tweak no longer triggers an immediate
Postgres restart). OrderedReady aligns with the standard
Postgres-on-Kubernetes pattern for any future HA work.
DEPL-M5 (Med) — per-fleet-size resource ladder documentation
deploy/helm/certctl/values.yaml — extended comments next to
server.resources + agent.resources documenting:
"≤ 500 certs / 100 agents" → defaults are validated
"5K certs / 1K agents" → starter suggestions, TBD Phase 8
"50K certs / 10K agents" → starter suggestions, TBD Phase 8
Numbers for the small-fleet case derive from the measured
baselines in docs/operator/performance-baselines.md
(50ms p50, < 3s for 1000-cert inventory walk, etc.). Larger
fleet numbers explicitly marked TBD pending Phase 8 load-test
runs — operators tune empirically until then.
DEPL-L1 (Low) — Helm rollback runbook
docs/operator/runbooks/rollback.md — covers helm rollback
mechanics, the schema-migration manual-cleanup path (when
*.down.sql files apply vs. when full restore is the only safe
path), and the per-migration-class safe-to-rollback table.
DEPL-L2 (Low) — Prometheus AlertManager rules
deploy/helm/certctl/templates/prometheusrules.yaml — opt-in via
monitoring.prometheusRules.enabled=true. Default OFF. Four
starter rules using verified metric names from
internal/api/handler/metrics.go:
CertctlCertificateExpiringSoon (certctl_certificate_expiring_soon)
CertctlAgentOffline ((agent_total - agent_online) > 0 for 1h)
CertctlJobFailureRateHigh (failure rate over 5% for 15m)
CertctlIssuanceFailures (any failures over 15m window)
All thresholds operator-tunable via
monitoring.prometheusRules.thresholds.* in values.
DEPL-L3 (Low) — Prometheus bearer-token setup runbook
docs/operator/runbooks/prometheus-bearer-token.md — documents
the API-key + Secret + values wiring for the RBAC-gated
/api/v1/metrics/prometheus scrape endpoint. End-to-end
procedure with troubleshooting steps + rotation guide.
CI guard: scripts/ci-guards/helm-templates-lint.sh
Six-combo matrix: defaults / backup PVC / backup S3 /
prometheusRules / migrations.viaHook / all-on. Each runs helm
template + checks render success. helm lint also gated.
Wired into the auto-pickup loop in .github/workflows/ci.yml;
azure/setup-helm@b9e51907 (v4.3.0, SHA-pinned per Phase 1
RED-2) installs helm v3.16.0 on the runner.
Verification (all pass):
ls deploy/helm/certctl/templates/{backup-cronjob,migration-job,prometheusrules}.yaml
grep -E 'updateStrategy|podManagementPolicy' deploy/helm/certctl/templates/postgres-statefulset.yaml # 2 matches
helm template deploy/helm/certctl/ --set backup.enabled=true \
--set monitoring.prometheusRules.enabled=true --set migrations.viaHook=true \
| grep -E "kind: (CronJob|PrometheusRule|Job)" # 3 matches
helm lint deploy/helm/certctl/ # 0 failed
ls docs/operator/runbooks/{rollback,prometheus-bearer-token}.md
bash scripts/ci-guards/helm-templates-lint.sh # 6/6 matrix combinations pass
Go build clean (cmd/server compiles, migrate-only path verified by
the build target). YAML validated.
Closes: cowork/certctl-architecture-diligence-audit.html#fix-DEPL-H2
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-M1
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-M4
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-M5
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-L1
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-L2
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-L3
|
||
|
shankar0123
|
69a2b5c55a |
config: default hardening + operator docs (Phase 2 closure — SEC-H1, SEC-H3, SEC-M4, DEPL-H1, DEPL-M2 + doc-only carve-outs)
Eleven findings from the architecture diligence audit's Phase 2 bundle
closed in one PR. All touch the same backend config + Helm chart +
operator docs surface, so reviewing in one diff is the natural fit.
config.go: three new fail-closed Validate() branches behind sentinels
=====================================================================
Three new error sentinels exported from internal/config/config.go for
tests to pin via errors.Is + message-text:
- ErrAgentBootstrapTokenRequired (SEC-H1)
- ErrACMEInsecureWithoutAck (SEC-M4)
- ErrDemoModeAckExpired (SEC-H3)
SEC-H1 (staged): introduces CERTCTL_AGENT_BOOTSTRAP_TOKEN_DENY_EMPTY
as an opt-in feature flag. When true AND the bootstrap token is empty,
Validate() returns ErrAgentBootstrapTokenRequired and the server
refuses to start. Default in THIS release: false (warn-mode
pass-through preserved). WORKSPACE-ROADMAP.md schedules the default
flip to true for v2.2.0 — operators get one upgrade window.
SEC-M4: upgrades the existing boot-time WARN log for
CERTCTL_ACME_INSECURE=true into a hard refuse-to-start gate behind
CERTCTL_ACME_INSECURE_ACK=true. The ACK env var must be paired with
the existing INSECURE flag; either alone fails closed. The boot-time
WARN log at cmd/server/main.go:611 continues to fire for the ACK'd
case so every restart logs the reminder.
SEC-H3: tightens the sticky DemoModeAck bit so it expires after 24h.
When DemoModeAck=true, Validate() now requires CERTCTL_DEMO_MODE_ACK_TS
to be set as a unix-epoch timestamp within the last 24h (24h-tolerance
on the past side, 1-minute clock-skew on the future side). Catches the
"forgotten demo deployment promoted to production" failure mode —
next container restart past 24h refuses unless re-ack'd.
Tests in internal/config/config_test.go cover every new branch:
positive (passes when properly set), negative (each fail-closed path
fires with the matching sentinel + message-text). 11 new tests added.
Helm chart + HA runbook (DEPL-H1)
=================================
Created docs/operator/runbooks/ha.md documenting the three values
flips required for production HA: server.replicas, podDisruptionBudget,
service.sessionAffinity. Cross-link comments added to
deploy/helm/certctl/values.yaml next to the server.replicas (line 19)
and podDisruptionBudget (line 566) defaults. DEFAULTS DO NOT CHANGE
— that's the point per the prompt's 'do not flip networkPolicy default'
guidance: a default-enabled PDB blocks fresh helm install on
single-node clusters.
CI guard (DEPL-M2)
==================
scripts/ci-guards/no-change-me-in-prod-compose.sh grep-fails any
'change-me-' literal in compose files OTHER than docker-compose.demo.yml.
Catches the placeholder-credential-leak regression one layer earlier
than the runtime Validate() fail-closed guards from Bundle 2 (2026-05-12).
Excludes comment lines so docs explaining the pattern don't trip the
guard. Verified to fire on a synthetic leak; clean on the current tree.
Consolidated 'Security carve-outs' doc section
==============================================
docs/operator/security.md grows by one new section documenting the
seven existing carve-outs in one canonical place:
- SEC-M3: 3 InsecureSkipVerify=true sites (Agent dev, verify probe, tlsprobe)
- SEC-M5: F5 connector InsecureSkipVerify per-config field
- SEC-M4: ACME insecure + new ACK gate
- SEC-L1: CSP 'unsafe-inline' on style-src (Tailwind carve-out)
- SEC-L2: break-glass Argon2id rest-defense reminder
- SEC-L3: 1 MB body-size cap + CERTCTL_MAX_BODY_SIZE override
- DEPL-M2: change-me-* placeholder credentials in demo overlay
- DEPL-M3: K8s NetworkPolicy operator-opt-in default
Each entry cites the file:line, the rationale for the carve-out, and
the operator action.
CHANGELOG + ENVIRONMENTS coverage
==================================
CHANGELOG.md grows by one new '### Breaking changes (scheduled for
v2.2.0)' section under Unreleased, documenting SEC-H1 / SEC-M4 / SEC-H3
with explicit upgrade-window guidance for each.
deploy/ENVIRONMENTS.md adds five rows: AGENT_BOOTSTRAP_TOKEN +
AGENT_BOOTSTRAP_TOKEN_DENY_EMPTY + DEMO_MODE_ACK + DEMO_MODE_ACK_TS +
ACME_INSECURE_ACK. G-3 env-docs-drift CI guard stays clean.
WORKSPACE-ROADMAP.md (cowork-side) schedules the SEC-H1 default-flip
for v2.2.0.
Sandbox limitation
==================
The certctl repo's working tree is 6.1 GB which fills the sandbox
volume; the go1.25.10 toolchain download (go.mod requires it,
sandbox has 1.25.9) keeps failing on disk-full. Local 'go build' /
'go test' were NOT run in this commit's verification path.
make verify MUST be run on the operator's workstation before push
per CLAUDE.md operating rules.
CI guards (no-change-me, G-3 env-docs-drift, doc-rot-detector, +
all existing) verified clean by running each individually.
Closes: cowork/certctl-architecture-diligence-audit.html#fix-SEC-H1,
cowork/certctl-architecture-diligence-audit.html#fix-SEC-H3,
cowork/certctl-architecture-diligence-audit.html#fix-SEC-M4,
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-H1,
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-M2,
cowork/certctl-architecture-diligence-audit.html#fix-DEPL-M3,
cowork/certctl-architecture-diligence-audit.html#fix-SEC-M3,
cowork/certctl-architecture-diligence-audit.html#fix-SEC-M5,
cowork/certctl-architecture-diligence-audit.html#fix-SEC-L1,
cowork/certctl-architecture-diligence-audit.html#fix-SEC-L2,
cowork/certctl-architecture-diligence-audit.html#fix-SEC-L3
|
||
|
shankar0123
|
0161bb201c |
docs: remove internal engineering docs; docs must be tool- or story-relevant
Operator policy: docs in the public repo must help (a) a user
deploying certctl or (b) the product story. Internal engineering
process documentation belongs in cowork/ scratchpads or in git
commit history, not docs/.
Removed (docs/contributor/, 8 files, 2,323 lines):
- release-sign-off.md — internal release-day checklist
- ci-pipeline.md — what runs in CI (internal)
- ci-guards.md — what the guards are (internal)
- testing-strategy.md — internal testing strategy
- qa-test-suite.md — internal QA reference (445 lines)
- qa-prerequisites.md — internal QA setup
- gui-qa-checklist.md — manual GUI QA checklist
- test-environment.md — 1,103-line redundant with
docs/getting-started/quickstart.md +
docs/getting-started/advanced-demo.md
Removed supporting script:
- scripts/qa-doc-seed-count.sh — CI guard for the deleted
qa-test-suite.md seed-data table
Cross-reference cleanup:
- README.md: dropped the Contributor audience row + footer
pointer to docs/contributor/.
- Makefile: dropped `verify-docs` target + qa-stats comment refs.
- .github/workflows/ci.yml: dropped the QA-doc seed-count drift
CI step + dead comment refs.
- docs/reference/cli.md: repointed qa-prerequisites.md → quickstart.md.
- docs/operator/performance-baselines.md: dropped ci-pipeline.md
cross-ref.
- scripts/ci-guards/README.md: dropped the 'Guards explicitly
NOT here' section that referenced the deleted QA-doc guards.
G-3 env-docs-drift guard improvements (a real consequence: deleting
the contributor docs surfaced that some env vars only had a home
there). Refit the guard to the new doc topology:
- Defined-scan widened from `config.go + cmd/*` to all of `cmd/ +
internal/` (production code), excluding `*_test.go` — catches
service-layer env vars like CERTCTL_STEPCA_ROOT_CERT and
CERTCTL_ZEROSSL_EAB_URL that were previously invisible to the
guard.
- Docs-scan widened to include deploy/ENVIRONMENTS.md (the
canonical env-var inventory table — should have been in scope
from day one). Kept narrow to README + docs/ + deploy/helm/ +
ENVIRONMENTS.md to avoid pulling in compose/test fixtures.
- ALLOWED filter now applies to both DOCS_ONLY and CONFIG_ONLY
directions, so dynamic per-profile dispatch surfaces
(CERTCTL_SCEP_PROFILE_<NAME>_*, CERTCTL_EST_PROFILE_<NAME>_*,
CERTCTL_QA_*) don't need static doc entries.
- Added CERTCTL_SCEP_PROFILE_[A-Z_]+ and CERTCTL_EST_PROFILE_[A-Z_]+
to ALLOWED for the same reason.
deploy/ENVIRONMENTS.md: added CERTCTL_ZEROSSL_EAB_URL row — real
operator override (overrides the ZeroSSL EAB-credentials endpoint;
read at internal/connector/issuer/acme/acme.go:372) that was
defined in Go source but never documented. G-3 caught it after the
defined-scan widened.
scripts/ci-guards/S-1-hardcoded-source-counts.sh: removed dead
WORKSPACE-CHANGELOG.md allowlist entry (the file was deleted in
the prior workspace cleanup).
Verified:
All 35 scripts/ci-guards/*.sh green (FAIL=0).
No remaining references to docs/contributor/ or qa-doc-seed-count
in tracked files.
|
||
|
shankar0123
|
57b539c378 |
docs(b12): observability reference + Postgres backup runbook
Closes acquisition-diligence Bundle 12 — Observability, DR,
Operations Receipts, And Performance Proof. Source IDs: D5, D6, D8,
T9, finding 7, OPS-H1, OPS-M1, OPS-M2, LOW-7.
Two new operator-facing references; both non-audit-framed per the
Bundle 5 doc-placement policy.
docs/operator/observability.md — single canonical statement of what
certctl emits, what it doesn't, and what survives a restart:
- Metrics surface: both /api/v1/metrics (JSON) and
/api/v1/metrics/prometheus (text exposition v0.0.4); inventory of
certctl_certificate_* gauges + certctl_issuance_duration_seconds
per-issuer-type histogram + certctl_uptime_seconds.
- Prometheus library vs hand-rolled exposition: explicit scope
statement — hand-rolled fmt.Fprintf is intentional for v2.x given
the shallow metric surface; client_golang migration tracked as
v3 item (closes OPS-M1).
- Tracing: explicit deferral — no OTel SDK setup, OTel packages
are indirect-only in go.mod, no spans, no OTLP exporter; tracked
as v3 item; in the meantime structured logs carry request_id and
certctl_issuance_duration_seconds carries the per-issuer latency
signal (closes OPS-M2).
- Logging: structured JSON via log/slog; CERTCTL_LOG_LEVEL control;
no key material / bearer tokens / session cookies in log lines.
- Rate-limit semantics under restarts + replicas: per-process,
in-memory, reset-on-restart, NOT shared across replicas; full
inventory of the 5 limiter call sites (break-glass login,
SCEP/Intune per-device, EST per-principal CSR, EST HTTP-Basic
source-IP, ACME per-account); multi-replica + sticky-session
implications; database-backed sliding window deferred to v3
(closes D8).
- Performance harness scope: cross-references the explicit
'What it explicitly does NOT measure' list in
deploy/test/loadtest/README.md (closes LOW-7 + finding 7).
docs/operator/runbooks/postgres-backup.md — operator-runnable
backup procedure:
- Inventory of what to back up (DB + operator-managed file
material that lives outside the DB: CA keys, RA keys, OCSP
responder keys, trust bundles).
- Logical backup recipe with docker-compose + Kubernetes variants,
integrity verification step, off-host storage step.
- Physical / PITR recipe pointing at pgbackrest / wal-g
(certctl ships nothing here — standard PostgreSQL DBA work).
- Three sample automation paths (in-cluster Postgres → S3 CronJob,
managed Postgres PITR, self-hosted VM systemd timer + restic).
- Quarterly restore-dry-run procedure.
- Helm CronJob template deliberately not shipped — three
documented reasons (deployment topology / secret-management
integration / off-host storage all vary by operator) plus
roadmap entry for shipping a starter template when a real
operator asks for one (closes D6 + OPS-H1).
Both new docs wired into docs/README.md Operator + Runbooks tables.
D5 (ServiceMonitor) and T9 (canonical k6 load-test) were already
shipped in Bundle 3 (deploy/helm/certctl/templates/servicemonitor.yaml)
and in deploy/test/loadtest/ + .github/workflows/loadtest.yml
respectively; this bundle doesn't touch them — it just records the
closure in the audit HTML.
Verified:
bash scripts/ci-guards/G-3-env-docs-drift.sh # PASS
bash scripts/ci-guards/doc-rot-detector.sh # PASS
All 35 scripts/ci-guards/*.sh green.
|
||
|
shankar0123
|
476022ca59 |
docs(b6): secret-custody reference + config-encryption upgrade runbook + private-key CI guard
Closes acquisition-diligence Bundle 6 findings on secret custody, config
encryption, and local artifact hygiene. Source IDs: S6, R4, SEC-M2,
RT-M1, RT-M2, RT-L1.
Surgical closures (artifact-only audit-framed memos stay out of the
public repo per the Bundle 5 lesson):
R4 / RT-L1 — local EC private key artifact
rm cmd/agent/mc-001.key (gitignored, never in git history, leftover
from a 2025-era agent dev run on the operator's workstation).
Added scripts/ci-guards/B6-no-private-keys-in-tree.sh that fails the
build if any TRACKED non-test file contains a PEM private-key block,
so the next attempt to commit similar material gets caught at CI.
Allowlist: *_test.go (hermetic-test PEMs), examples/*.md (sample
walkthroughs), internal/scep/intune/testdata/ (certificates, not
keys).
RT-M1 — landing-page HSM implication
certctl.io/index.html: 'their hardware' / 'your hardware' colloquial
comparisons rephrased to 'their custody' / 'your servers'. The phrase
'Your keys. Your hardware. Your data. Your terms.' becomes 'Your
keys. Your servers. Your data. Your terms.' to remove any inferred
HSM-backed key-storage claim. The technical disclosure now lives in
docs/operator/secret-custody.md (linked below); the landing page no
longer makes a claim it cannot back.
S6 + SEC-M2 + RT-M2 (composite documentation closure)
Added docs/operator/secret-custody.md — public operator reference
enumerating every secret material on the control plane and on
agents:
- Local CA private key (FileDriver, file-on-disk, heap-resident
with the L-014 carve-out documented in
internal/connector/issuer/local/local.go).
- Agent ECDSA P-256 keys (file on agent host, never transmitted).
- OIDC client secret (AES-256-GCM v3, PBKDF2 600k).
- Session signing key (same encryption regime).
- Break-glass credential (Argon2id, never encrypted).
- API-key bearer tokens (SHA-256 hash only; plaintext shown once).
- CSR private keys mid-issuance (agent memory only).
- Issuer-connector backend secrets (encrypted_config column,
fail-closed for source='database', plaintext-by-design for
source='env' with rationale).
The Env-seeded-vs-DB-seeded plaintext policy is explained in plain
text so a buyer review can independently verify the startup guard at
cmd/server/main.go:222-262 makes sense.
Added docs/operator/runbooks/config-encryption-upgrade.md — the
procedural arm: how to force v1/v2 -> v3 re-seal across the
database, plus the passphrase-rotation order. Documents the
AEAD-driven read fallback (v3 -> v2 -> v1) and the fact that
re-sealing happens passively on UPDATE. Open roadmap item: a
certctl admin reseal --all command (tracked in
WORKSPACE-ROADMAP.md).
Both docs wired into docs/README.md Operator + Runbooks tables.
Verification:
rg -n 'CONFIG_ENCRYPTION|encrypt|v1|private key|HSM|PKCS11|mc-001.key|\.key|Local CA' \
internal cmd docs .gitignore README.md # ambient (no NEW leaks)
find . -name '*.key' \
-not -path './.git/*' -not -path './web/node_modules/*' # empty
git ls-files | xargs grep -lE 'BEGIN .* PRIVATE KEY' \
| grep -vE '_test\.go$|^examples/|^internal/scep/intune/testdata/' # empty
bash scripts/ci-guards/B6-no-private-keys-in-tree.sh # PASS
bash scripts/ci-guards/G-3-env-docs-drift.sh # PASS
bash scripts/ci-guards/doc-rot-detector.sh # PASS
Residual roadmap (deliberately deferred):
- signer.PKCS11Driver (HSM-token-backed CA-key custody).
- signer.CloudKMSDriver (AWS/GCP/Azure KMS-backed CA-key custody).
- FIPS 140-3 mode for the whole control plane.
- HSM-backed session signing key.
- Built-in 'certctl admin reseal --all' command.
All five tracked in WORKSPACE-ROADMAP.md, not retracted.
|
||
|
shankar0123
|
5b151e74da |
docs: remove audit-bundle-flavored docs from public repo
Three docs added in Bundle 4 + Bundle 5 closure commits ( |
||
|
shankar0123
|
264015059d |
ci(guards): fix G-3 (CERTCTL_MCP_READ_ONLY phantom) + S-1 (hardcoded 45)
Two CI guards tripped on the B4 + B5 closure commits: 1. G-3 env-docs-drift caught `CERTCTL_MCP_READ_ONLY` mentioned in docs/operator/security-bundle-5-audit-closure.md (Bundle 5 S8 row) without a corresponding entry in internal/config/config.go. The env var is a v3 idea, not a shipped feature — the doc now describes the future gate without naming the literal env var, matching the G-3 phantom-env-var contract. 2. S-1 hardcoded-source-counts caught "all 45 migrations" in docs/operator/scheduler-ha.md (Bundle 4 D8 closure prose). Per the CLAUDE.md operating rule "Numeric claims about current state rot", swapped the literal count for the rebuild command `ls migrations/*.up.sql | wc -l`. Both fixes are doc-only — no code change, no test change. The underlying Bundle 4 + Bundle 5 closures stand. Verification: bash scripts/ci-guards/G-3-env-docs-drift.sh # clean bash scripts/ci-guards/S-1-hardcoded-source-counts.sh # clean |
||
|
shankar0123
|
596e675ec7 |
fix(security): close BUNDLE 5 — auth, OIDC, MCP, API + browser security edges
Bundle 5 closure (2026-05-13 acquisition diligence audit). 13-finding
security audit pass across the auth / OIDC / MCP / API / browser-
security surface. Five real closures shipped in code, two false-as-
stated findings annotated with the existing implementation, three
operator-decision items documented for v3 follow-up, three doc-only
fixes (auth architecture narrative aligned with shipped OIDC).
Source findings closed (code):
S1 break-glass /auth/breakglass/login lacked the documented
5/min per-source-IP rate limit; handler now owns its own
SlidingWindowLimiter wired at startup. Doc claim turns true.
R6 OIDC test_discovery JWKS probe ran on http.DefaultClient;
now uses an http.Client whose transport wraps
validation.SafeHTTPDialContext. JWKS URI can no longer
pivot into reserved-address ranges via DNS rebinding.
R7 Slack + Teams notifiers built http.Client without the SSRF
dial-time guard. Both New() constructors now install
validation.SafeHTTPDialContext; webhook URLs (operator-
configured via dynamic-config GUI) cannot dial 169.254.x or
in-cluster reserved ranges. Test seam: newForTest bypasses
the guard for httptest's 127.0.0.1 binds, mirroring the
existing internal/connector/notifier/webhook pattern.
RT-L2 CERTCTL_ACME_INSECURE=true now emits a prominent
logger.Warn at server boot. Pre-Bundle-5 the knob silently
disabled ACME directory TLS verification.
Source findings closed (doc):
finding 1 + HIGH-5 Architecture doc claimed no in-process JWT/
OIDC/mTLS/SAML and pointed everyone at the
authenticating-gateway pattern. Auth Bundle 2
(commit dea5053) shipped native OIDC + sessions +
break-glass. New §"In-process authentication surface"
table (api-key / oidc / none) supersedes the old framing;
"Authenticating-gateway pattern (SAML, mTLS-as-auth,
LDAP)" section retained for protocols certctl still
doesn't ship natively.
Source findings verified false (existing implementation):
S4 OIDC email-domain allowlist — `email_domain_test.go`
already pins the strict-equality semantics (subdomain not
auto-accepted, multi-entry no-match path, empty allowlist
accepts all by-design per RFC 9700 §4.1.1).
SEC-L1 CSP / HSTS / referrer-policy headers — already shipped at
internal/api/middleware/securityheaders.go and wired at
cmd/server/main.go L2003+L2027+L2115.
Operator-decision / deferred (tracked in bundle-5 closure doc):
S3 CERTCTL_API_KEYS_NAMED parsing is wired, end-to-end
validation is partial. Operator decides: complete the
named-key middleware path or deprecate the syntax.
S5 Audit-middleware best-effort for read paths;
security-critical writes use WithinTx. Operator decides
per-path escalation.
S8 MCP threat model — the binary is a thin protocol bridge,
no privileges of its own; every tool call carries
CERTCTL_API_KEY and is auth'd + RBAC-gated server-side.
Optional CERTCTL_MCP_READ_ONLY gate tracked as v3.
SEC-H1 2026-05-10 audit CRIT-1/2/4 already closed on master;
CRIT-3/5 status against the spec folder is operator-
workstation-validation-only. Documented for follow-up.
SEC-L2 WebAuthn / FIDO2 / step-up — already documented in
docs/operator/auth-threat-model.md "Threats Bundle 2 does
NOT close". v3 work item per CLAUDE.md decision 12.
Full per-finding rationale + receipts at
docs/operator/security-bundle-5-audit-closure.md.
Verification:
gofmt -l # clean
go vet ./internal/connector/notifier/slack
./internal/connector/notifier/teams ./internal/auth/oidc
./internal/api/handler ./cmd/server # clean
go build ./cmd/server [...] # clean
go test -short -count=1 ./internal/connector/notifier/slack
./internal/connector/notifier/teams ./internal/api/handler
./internal/auth/oidc ./internal/config # PASS
# (slack 0.028s + teams
# 0.023s + handler 11.0s;
# newForTest seam keeps
# httptest tests green)
Audit-Closes: BUNDLE-5 S1 R6 R7 RT-L2 finding-1 HIGH-5
Audit-Verifies-False: S4 SEC-L1
Audit-Defers: S3 S5 S8 SEC-H1 SEC-L2
|
||
|
shankar0123
|
750478a6fe |
fix(scale): close BUNDLE 4 — migrations, scheduler HA, rate-limits, scale receipts
Bundle 4 closure (2026-05-13 acquisition diligence audit). Closes the
"what happens under multi-replica" question cluster: migration runner
had no concurrency control + no applied-version ledger, 15 scheduler
loops had per-process idempotency but no cross-replica documentation,
rate limits were process-local without an operator-facing scope
statement, load-test scope explicitly omitted four hot paths without
linking them to a roadmap.
Source findings closed:
HIGH-1 + D4 + finding 4 (migration tracking)
D8 (scheduler loop ownership)
MED-1 + MED-2 (rate-limit scope)
T9 + LOW-7 + finding 7 (load-test receipt scope)
Closures by source ID:
HIGH-1 + D4 + finding 4 — Migration tracking + advisory lock.
internal/repository/postgres/db.go::RunMigrations now wraps every
migration execution in:
1. A dedicated *sql.Conn pinned to one connection for the entire
scan + apply lifecycle (pg_advisory_lock is connection-scoped).
2. pg_advisory_lock(migrationAdvisoryLockID) — fixed int64 key
derived from "certctl-migrations" so the same constant resolves
across deployments without colliding with operator advisory
locks. Blocks the second replica until the first finishes.
3. CREATE TABLE IF NOT EXISTS schema_migrations(version TEXT PK,
applied_at TIMESTAMPTZ DEFAULT NOW()) — audit ledger.
4. Skip-applied loop: SELECT version FROM schema_migrations →
map[string]struct{} → skip every .up.sql whose filename is in
the map. INSERT after successful execute, ON CONFLICT
(version) DO NOTHING for defense in depth.
Pre-Bundle-4 every server boot re-ran all 45 .up.sql files. The
"idempotency via IF NOT EXISTS / ON CONFLICT" contract in CLAUDE.md
held per-migration but offered no protection when two Helm replicas
raced on schema DDL. Post-Bundle-4 single-replica deploys see zero
behavior change beyond the audit-table population; multi-replica
deploys get HA-safe schema bootstrap.
D8 — Scheduler HA semantics documented.
New docs/operator/scheduler-ha.md with per-loop inventory of all 15
loops in internal/scheduler/scheduler.go. Classification:
- HA-safe (jobProcessorLoop, jobRetryLoop) — FOR UPDATE SKIP
LOCKED via ClaimPendingJobs (Bundle 1 H-6 closure,
|
||
|
shankar0123
|
56e2ea1ad7 |
docs: v2.1.0 release polish — strip internal bundle/phase tags, update status for OIDC ship
README:
- Rewrite Status block: drop the stale 'federated identity not yet
shipped' line; flag v2.1.0 OIDC + sessions + back-channel logout
+ break-glass as early-access; encourage GitHub issues for IdP
rough edges. (A1 framing — keep early-access umbrella, no
SAML/WebAuthn/JIT roadmap teaser.)
- Add OIDC SSO bullet to 'What it does' covering per-IdP runbooks,
group-claim → role mapping, AES-256-GCM client_secret encryption,
JWKS auto-refresh, PKCE-S256, RFC 9700 §4.7.1 pre-login binding,
RFC 9207 iss check, __Host- cookies, CSRF rotation, idle+absolute
expiry, BCL, break-glass admin.
- Update Security paragraph: three auth paths (API keys / OIDC /
break-glass), HMAC-signed sessions, CSRF rotation, RFC OIDC BCL.
- Correct CI coverage thresholds against
.github/coverage-thresholds.yml (service 70%, handler 75%,
crypto 88%, auth packages 85-95%); 'static analysis' replaces
the inflated '11 linters' claim (actual count is 4 active).
Docs B3 sweep — strip operator-facing 'Bundle N' / 'Phase N' tags:
- docs/operator/auth-threat-model.md — rewrite intro; rename 5 H2
sections (API-key + RBAC defenses / OIDC + sessions + break-glass
defenses / OIDC + sessions threat catalogue / Closed federated-
identity threats / Future-work threats); clean ~12 H3/prose hits.
- docs/operator/rbac.md — strip Bundle 1 framing from intro,
scope_id deferral note, MCP tools section, day-0 bootstrap, and
'Where to look next'.
- docs/operator/auth-benchmarks.md — drop 'Phase 14' framing from
title intro, hardware floor caption, result table caption,
methodology, and pre-merge audit section.
- docs/operator/security.md — already cleaned earlier this session
(RBAC / day-0 / approval-bypass / OIDC federation / sessions /
OIDC first-admin / break-glass H3s).
- docs/operator/oidc-runbooks/{index,keycloak,authentik,okta,
azure-ad}.md — strip Auth Bundle 2 framing + Phase 10/3/4
references; replace with feature-name prose.
- docs/operator/legacy-clients-tls-1.2.md — drop Bundle F / M-023
audit-reference framing; keep CWE-326.
- docs/operator/database-tls.md — drop Bundle B / M-018 framing
from intro + Helm section.
- docs/operator/runbooks/disaster-recovery.md — drop 'Production
hardening II Phase 10' status callout.
- docs/migration/oidc-enable.md — retitle 'Enable OIDC SSO';
strip Bundle 1/2 framing from prereqs, troubleshooting, related
docs; update __Host- cookie callout from 'audit MED-14' to
v2.1.0-BREAKING.
- docs/migration/api-keys-to-rbac.md — strip Bundle 1 framing from
intro, migration table, IsAdmin section, and cross-references.
- docs/migration/acme-from-cert-manager.md — strip residual
'Phase 5' tags from cert-manager integration test references.
- docs/reference/configuration.md — retitle Auth section.
- docs/reference/profiles.md — strip Bundle 1 Phase 9 framing
from RequiresApproval section + Related list.
- docs/reference/auth-standards-implemented.md — rewrite intro
(API-key + RBAC + OIDC + sessions + back-channel logout +
break-glass); rename 'Bundle 1 (RBAC) standards covered
separately' H2; clean per-row Phase references.
- docs/README.md — rewrite nav-table entries to drop Bundle 1/2
parentheticals; retitle 'Enable OIDC SSO' migration entry.
No code or test changes; pure operator-facing prose polish for
the v2.1.0 tag.
|
||
|
shankar0123
|
fefeccfa59 |
harden(oidc): relax alg-downgrade IdP-bind check to intersection-empty (Keycloak compat)
Phase-10 live-IdP smoke (Keycloak 26.x via testcontainers-go) revealed
the IdP-bind alg-downgrade check was too strict for real-world IdPs.
6 of the integration tests in internal/auth/oidc/integration_keycloak*_test.go
were failing with:
oidc: IdP advertises weak signing algorithms (HS*/none);
refusing to use as defense against downgrade attacks: HS256
Keycloak 26.x (and several other real-world IdPs — Auth0 when HS-mode is
enabled, some Authentik configs) advertise EVERY alg they're capable of
in the discovery doc's id_token_signing_alg_values_supported field, even
when the realm only signs with RS256 in practice. Pre-fix the IdP-bind
check refused on ANY HS* or 'none' advertisement → no real Keycloak deploy
could ever bind a provider row, hence the integration-test failures.
The strict-deny check was defense-in-depth on top of the load-bearing
per-token alg-pin at sig-verify time (isDisallowedAlg, service.go L1177):
that check rejects every ID token whose JWS header carries an alg outside
DefaultAllowedAlgs, regardless of what the discovery doc advertises.
A forged HS256 token signed with the IdP's RS256 pubkey as HMAC secret
is rejected at sig-verify time → the actual algorithm-confusion attack
is closed by the per-token pin, NOT by the discovery-doc check.
Fix: relax the IdP-bind check to refuse only when the intersection of
advertised vs DefaultAllowedAlgs is EMPTY (the pathological all-weak-alg
IdP case). Keycloak (RS256 + HS256 advertised) now binds successfully;
an HS-only IdP still fails closed.
Changes:
- internal/auth/oidc/service.go: rewrite the alg-check loop at L1067 in
getOrLoad / RefreshKeys to compute the intersection set; refuse only
when no acceptable alg is advertised. ErrIdPDowngradeAdvertised
docstring updated to reflect new contract. DefaultAllowedAlgs
docstring + the package-level design-comment block at L40-72 updated
with v2.1.0-relaxed semantics callouts.
- internal/auth/oidc/test_discovery.go: TestDiscovery dry-run validator
rewritten to surface HS*/none alongside RS* as an informational note
('note: IdP advertises weak algorithms %v alongside acceptable ones')
rather than a hard-fail error. HS-only / none-only still hard-fails.
- internal/auth/oidc/service_test.go: TestService_IdPDowngradeDefense_*
tests updated. Renamed:
- RejectsHSAdvertised → RS256PlusHS256_BindsSuccessfully (positive)
- RejectsNoneAdvertised → RejectsHSOnlyAdvertised (intersection-empty)
- RefreshKeys_CatchesPostLoadDowngrade rotated to HS-only post-load
- internal/auth/oidc/coverage_fill_test.go: TestTestDiscovery_AlgDowngradeDetected
split into _HS256AlongsideRS256_BindsWithNote (positive, asserts note
but no hard-fail) + _HSOnly_StillTrips_HardFail (intersection-empty).
- docs/operator/auth-threat-model.md: OIDC token-validation alg-allow-list
section rewritten to call out the load-bearing-defense hierarchy
(per-token pin first, IdP-bind check defense-in-depth) and document
the v2.1.0 relaxation rationale.
- CHANGELOG.md: ### Security entry under Unreleased.
Verify: go test ./internal/auth/oidc/ -short PASS; gofmt clean; go vet
clean. The Keycloak integration tests should now pass when the operator
re-runs 'make keycloak-integration-test'.
|
||
|
shankar0123
|
a923cf697c |
harden(auth): demo-mode residual-grants detector + cleanup endpoint + CI guard (A-8)
Audit 2026-05-11 A-8 closure. Closes the deferred Phase 2 leg of the
2026-05-10 HIGH-12 closure (
|
||
|
shankar0123
|
0152bdf567 |
fix(auth/rbac): scope-aware ActorRole revoke (A-4)
HIGH-10's UNIQUE (actor, role, scope_type, scope_id, tenant) uniqueness
extension lets an operator grant the same role to the same actor at
multiple scopes (e.g. r-operator on profile=p-acme AND profile=p-globex).
But ActorRoleRepository.Revoke's WHERE clause omitted (scope_type,
scope_id) — a single call deleted every variant. Selective revoke was
unrepresentable; operators had to drop all and re-grant N-1, opening
a race window where the actor's access was briefly different.
Closure across all layers (handler → service → repo → MCP → GUI client),
preserving the legacy "revoke all variants" contract for unmodified
callers:
internal/repository/auth.go
- New ActorRoleRevokeOptions struct. Zero value = legacy semantic;
non-empty ScopeType narrows to one variant.
- New ErrActorRoleNotFound sentinel for scoped no-match (HTTP 404).
internal/repository/postgres/auth.go
- Revoke signature extended with opts. Empty opts.ScopeType uses
the legacy SQL (no scope WHERE), zero-row delete = no error.
- Non-empty narrows with `scope_type = $5 AND scope_id IS NOT
DISTINCT FROM $6` — the IS-NOT-DISTINCT-FROM is load-bearing,
vanilla `=` would silently miss the (global, NULL) case because
NULL ≠ NULL in standard SQL.
- Selective revoke with zero matching rows returns
ErrActorRoleNotFound; operators get feedback on typos.
internal/service/auth/actor_role_service.go
- Revoke takes opts. Audit row's details map records the scope so
SIEMs can distinguish wide-vs-selective revokes:
`scope: "all_variants"` for the legacy path, or
`scope_type` + `scope_id` for selective. Privilege check
(auth.role.assign) and reserved-actor guard unchanged.
internal/api/handler/auth.go
- RevokeRoleFromKey parses optional `?scope_type=` / `?scope_id=`
query params via new parseRevokeScope helper.
- Validation mirrors AssignRoleToKey: scope_id forbidden with
scope_type=global, required with profile/issuer, invalid
scope_type → 400. scope_id without scope_type also → 400.
- writeAuthError maps ErrActorRoleNotFound to 404.
internal/mcp/tools_auth.go + types.go
- AuthRevokeKeyRoleInput gains optional ScopeType + ScopeID with
jsonschema descriptions explaining the dual-mode contract.
- Tool call site appends URL-encoded query params when ScopeType
is set; legacy callers (no scope_type) emit the bare DELETE
path unchanged.
web/src/api/client.ts
- authRevokeKeyRole signature: optional 3rd argument
`{ scope_type?, scope_id? }`. Pre-A-4 call sites (no opts arg)
keep firing the bare DELETE — fully backward compatible. The
GUI KeysPage's per-row revoke button (still one row per role,
pre-Fix-12) continues to use the legacy shape; future GUI work
can pass scope params for per-variant rows.
docs/operator/rbac.md
- New "Revoke: legacy 'all variants' vs scope-selective" subsection
under "From the HTTP API" with curl examples for both modes plus
the audit-row payload shape that lets SOC/SIEM tell them apart.
Regression coverage:
Repository (testcontainers, skipped under -short — 6 tests in
internal/repository/postgres/auth_revoke_scope_test.go):
TestRevokeActorRole_NoOpts_RemovesAllVariants
TestRevokeActorRole_WithScope_RemovesOnlyMatching
TestRevokeActorRole_WithGlobalScope_RemovesOnlyGlobal — pins the
IS-NOT-DISTINCT-FROM branch (global, NULL)
TestRevokeActorRole_NoMatch_ReturnsNotFound — pins the new sentinel
TestRevokeActorRole_NoOpts_NoMatch_IsNoOp — pins the legacy
idempotence contract
TestRevokeActorRole_IssuerScope_RemovesOnlyMatching — pin the
issuer-scope half (profile + issuer are symmetric scope types)
Handler (7 new tests in auth_test.go):
TestAuthHandler_RevokeRoleFromKey — extended to assert no scope
filter is forwarded when query string is empty (legacy behaviour)
TestAuthHandler_RevokeRoleFromKey_A4_ScopedProfile
TestAuthHandler_RevokeRoleFromKey_A4_ScopedGlobal
TestAuthHandler_RevokeRoleFromKey_A4_RejectsScopeIDWithGlobal
TestAuthHandler_RevokeRoleFromKey_A4_RejectsMissingScopeID
TestAuthHandler_RevokeRoleFromKey_A4_RejectsScopeIDWithoutScopeType
TestAuthHandler_RevokeRoleFromKey_A4_RejectsInvalidScopeType
TestAuthHandler_RevokeRoleFromKey_A4_ScopedNotFoundReturns404
MCP (2 new table rows in tools_per_tool_test.go):
Scoped revoke with scope_type=profile + scope_id=p-acme →
`?scope_type=profile&scope_id=p-acme`
Scoped revoke with scope_type=global (no scope_id) →
`?scope_type=global`
Service-layer test plumbing (service_test.go) updated for new opts
arg: 4 existing call sites pass repository.ActorRoleRevokeOptions{}
to keep their pre-A-4 semantics; the fakeActorRoleRepo.Revoke
implementation now mirrors the postgres scope-aware behaviour
(legacy zero-value vs scoped narrowing + ErrActorRoleNotFound on
no-match).
Verify gate green: gofmt clean, go vet clean, go test -short across
repository/postgres, service/auth, api/handler, and mcp. The
pre-existing KeysPage.test.tsx failure observed on the baseline
commit (reproduced via `git stash` earlier in Fix 03) is unrelated;
my client.ts change adds an optional third argument and is fully
backward-compatible.
Spec at cowork/auth-bundles-fixes-2026-05-11/04-high-actor-role-revoke-scope.md.
Audit doc updated: new row A-4 (2026-05-11) CLOSED appended to the
status table at the bottom of cowork/auth-bundles-audit-2026-05-10.md.
Operator-visible advisory in CHANGELOG.md v2.1.0 release notes under
Security (non-BREAKING — legacy callers are unchanged).
Depends on Fix 01 (the scope-aware EffectivePermissions read path on
branch fix/audit-2026-05-11/crit-actor-role-scope-reads). This fix
makes the inverse op selectively reversible; without Fix 01 the read
side would mis-evaluate scoped grants anyway, making selective revoke
moot at runtime.
|
||
|
shankar0123
|
9cce2ab043 |
harden(auth): LOW + Nit batch — bootstrap audit, crypto/rand, XFF trust, CSRF check, protocol-prefix unify (Batch 1)
Audit 2026-05-10 — close 8 LOWs + 2 Nits in-bundle. Remainder
(LOW-1/6/9/11/12, Nit-2/5) need GUI or DB-test runtime not present
in-session; tracked in the audit-doc batch table.
LOW-2: bootstrap.ValidateAndMint now emits 'bootstrap.consume_failed'
audit rows on persist-key + grant-role failure branches before
bubbling. Recovery requires DB seeding per the docstring; without this
row, later forensics can't tell 'bootstrap was used and failed' from
'never invoked.'
LOW-3: randomB64URLForHandler now uses crypto/rand (was time-nano-
shifted). Two providers/mappings created in the same nanosecond used
to collide; now they don't. Time-nano fallback retained for the
unlikely crypto/rand-broken path.
LOW-4: breakglass.verifyDummy uses s.readRand(salt) for the dummy
Argon2id verify. Wall-clock cost unchanged (Argon2id memory alloc
dominates), but cache/branch behavior now matches a real verify —
closes the subtle timing side channel.
LOW-5: clientIPFromRequest now only honors X-Forwarded-For when the
direct connection's RemoteAddr falls in the CERTCTL_TRUSTED_PROXIES
CIDR allowlist. Default-deny: empty list means XFF is ignored.
SetTrustedProxies wired in cmd/server/main.go from cfg.Auth.TrustedProxies.
LOW-7: internal/auth/protocol_endpoints.go::ProtocolEndpointPrefixes
now carries /scep-mtls + /.well-known/est-mtls (previously only in
router.AuthExemptDispatchPrefixes; the two lists had drifted). The
canonical-prefix coverage test in Phase 12 still pins the set.
LOW-8: docs/operator/rbac.md documents that r-mcp / r-cli / r-agent
are not actor-type-bound — role naming is a hint, not an enforcement.
Operators wanting hard binding must apply periodic audit queries.
Native binding is on the v2 roadmap.
LOW-10: Session.Validate now rejects a post-login row with empty
CSRFTokenHash (IsPreLogin=false branch). validSession test fixture
updated with a valid 64-hex CSRF hash.
Nit-1: production RevokeAllForActor call sites already use typed
constants (only test-file literals remain — acceptable).
Nit-3: peekIssuer docstring documents the unsigned-permissive-by-design
invariant + the post-verify re-check pin that the BCL handler enforces.
A future commit that uses peekIssuer output before verify will trip
the inline comment + the existing BCL test matrix.
Status table updated in cowork/auth-bundles-audit-2026-05-10.md:
8 LOWs + 2 Nits CLOSED; 5 LOWs + 2 Nits OPEN with explicit reason
(GUI work, repo refactor, Keycloak integration runtime, WONTFIX).
Refs: cowork/auth-bundles-audit-2026-05-10.md LOW-2/3/4/5/7/8/10
cowork/auth-bundles-audit-2026-05-10.md Nit-1/3
|
||
|
shankar0123
|
68ca42fef1 |
fix(auth): apply rbacGate to every state-changing + read handler (CRIT-1 closure)
Closes the wire-layer authorization gap surfaced by the 2026-05-10 audit
(CRIT-1). Before this commit only ~24 of ~140 routes carried rbacGate
enforcement — all of them admin-only fine-grained perms (auth.session.*,
auth.oidc.*, auth.breakglass.admin, cert.bulk_revoke, crl.admin, scep.admin,
est.admin, ca.hierarchy.manage). Every catalogued legacy-CRUD perm
(cert.read/issue/revoke/delete, profile.edit/delete, issuer.edit/delete,
target.*, agent.*, plus role-mgmt verbs) was declared in
internal/domain/auth/validate.go but never wired at the router. A r-viewer
Bearer was essentially r-admin minus five verbs at the wire layer (CWE-862).
This commit:
- Adds rbacGateScoped(checker, perm, scopeType, scopeFn, h) helper to
internal/api/router/router.go for path-bound scope resolution. Per-profile
and per-issuer grants (Decision 2) now reach the wire layer.
- Wraps every state-changing route AND every read endpoint in router.go
with rbacGate (global) or rbacGateScoped (path-bound). The auth-management
routes (POST /api/v1/auth/roles, etc.) gain router-level enforcement
in addition to the existing service-layer Authorizer check — defense in
depth (HIGH-9 of the same audit collapses into this closure).
- Auth-exempt surfaces stay un-gated by design: login, callback, BCL,
logout, breakglass-login, bootstrap, health, auth-info, version. Allowlist
is documented in TestRouterRBACGateCoverage.
- Extends internal/domain/auth/validate.go CanonicalPermissions with 30 new
perms across 12 namespaces: cert.edit; job.read, job.cancel; approval.read,
approval.approve, approval.reject; policy.read/edit/delete;
team.read/edit/delete; owner.read/edit/delete; notification.read/edit;
discovery.read/run/claim; network_scan.read/edit/run;
healthcheck.read/edit/delete/acknowledge; digest.read, digest.send;
verification.read, verification.run; stats.read; metrics.read.
- Updates DefaultRoles for r-admin / r-operator / r-viewer / r-mcp / r-cli /
r-agent. r-auditor gets NOTHING new — the auditor pin
(TestAuditorRoleHoldsExactlyAuditReadAndExport) stays invariant.
- Migration 000039_audit_crit1_perms seeds the new perm rows + role grants
per the updated DefaultRoles map. Idempotent ON CONFLICT DO NOTHING.
Reverse migration removes role_permissions before permissions
(ON DELETE RESTRICT on the FK).
- AST-level CI guard TestRouterRBACGateCoverage in
internal/api/router/router_rbac_coverage_test.go walks router.go and
asserts every state-changing + read route is wrapped (or in the
documented allowlist). Adding a new ungated route fails CI.
- Updates docs/operator/rbac.md permission-catalogue table with the new
namespaces + footer link to the AST CI guard.
- Updates certctl/CHANGELOG.md v2.1.0 section with the closure narrative.
Audit doc cowork/auth-bundles-audit-2026-05-10.md CRIT-1 row annotated
CLOSED 2026-05-10. Bundle's exit-gate spec lives at
cowork/auth-bundles-fixes-2026-05-10/01-crit-1-rbac-gates.md.
CRIT-2 / CRIT-3 / CRIT-4 / CRIT-5 of the same audit remain open and
continue to block the v2.1.0 tag.
Verification gate green:
- gofmt -d (no diff after gofmt -w on the touched files)
- go vet ./...
- go test -short -count=1 ./... (all packages pass including auditor pin)
- go build ./...
HIGH-9 of the audit closes via this commit's router-layer rbacGate on
POST /api/v1/auth/keys/{id}/roles + DELETE /api/v1/auth/keys/{id}/roles/{role_id}
(defense-in-depth on top of the existing service-layer privilege check).
Refs: cowork/auth-bundles-audit-2026-05-10.md CRIT-1 HIGH-9
|
||
|
shankar0123
|
c03d18bb1c |
auth-bundle-2 Phase 16: docs updates (security.md OIDC + sessions + break-glass + auditor split sections; new migration/oidc-enable.md; CHANGELOG.md v2.1.0 Bundle 2 release notes)
Closes Phase 16 of cowork/auth-bundle-2-prompt.md. Three operator-
facing docs updated, one new migration guide ships, README nav row
added.
Files
=====
docs/operator/security.md (MODIFIED, Last reviewed bumped to 2026-05-10):
* Added 5 new Bundle 2 subsections under '## Authentication
surface' after the Bundle 1 approval-bypass-closure entry:
- 'OIDC federation (Bundle 2 Phases 1-7)' — alg allow-list,
IdP-downgrade defense, iss/aud/azp/at_hash, single-use
state+nonce, PKCE-S256 mandatory, JWKS rotation handling,
encrypted client_secret at rest with the v3 blob format
pinned by an integration test, pointer to oidc-runbooks/
for per-IdP setup.
- 'Sessions + back-channel logout (Bundle 2 Phases 4-6)' —
length-prefixed HMAC cookie wire format, HttpOnly + Secure
+ SameSite cookie hardening, idle/absolute timeouts, CSRF
defense, signing-key rotation primitive, fail-fatal
EnsureInitialSigningKey at server boot, OpenID Connect
Back-Channel Logout 1.0 (NOT RFC 8414).
- 'OIDC first-admin bootstrap (Bundle 2 Phase 7)' — coexists
with Bundle 1's env-var-token bootstrap, group-scoped via
CERTCTL_BOOTSTRAP_ADMIN_GROUPS + CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID,
one-shot per tenant.
- 'Break-glass admin (Bundle 2 Phase 7.5)' — default-OFF,
surface invisibility via 404-not-403, Argon2id with OWASP
2024 params, lockout state machine, constant-time-via-
verifyDummy, WARN log at boot, runbook pointer for
operator drill.
- 'Migrating an existing deployment to OIDC' — pointer to
the new migration/oidc-enable.md walkthrough.
docs/migration/oidc-enable.md (NEW, Last reviewed 2026-05-10):
* Step-by-step migration guide for an operator on a Bundle-1-merged
deployment to enable OIDC SSO. Pre-reqs (CERTCTL_CONFIG_ENCRYPTION_KEY,
admin actor with auth.oidc.create + auth.oidc.edit, IdP tenant)
+ 7 numbered steps (pin encryption key, complete IdP-side per
runbook, configure certctl-side OIDCProvider, add group→role
mappings with fail-closed warning, optional first-admin bootstrap,
verify with single test user, announce SSO endpoint).
* Rollback section covering the 4-step disable flow + the 409
Conflict on provider-delete-while-sessions-exist + the
existing-sessions-keep-working-until-expiry semantics.
* Troubleshooting section pinning 8 most-common failure modes
(discovery doc fetch fails / IdP downgrade defense rejects /
no roles assigned / iss mismatch / pre-login expired / state
mismatch / sessions revoked but user can hit API / JWKS
rotation breaks login).
* Database row count drift documented so operators know what to
expect after OIDC is live (10 Bundle 2 tables enumerated).
* Cross-references to oidc-runbooks/ + security.md +
auth-threat-model.md + auth-benchmarks.md + auth-standards-implemented.md.
CHANGELOG.md (MODIFIED):
* v2.1.0 section title bumped from 'Auth Bundle 1: RBAC primitive'
to 'Auth Bundles 1 + 2: RBAC primitive + OIDC SSO + sessions'.
* Replaced the Bundle 1 closing-bullet ('Bundle 2 starts after
Bundle 1 lands on master') with 18 new Bundle 2 entries:
- OIDC + sessions + back-channel logout + break-glass overview.
- OIDC token validation pinned at three layers (alg allow-list,
IdP-downgrade defense, OIDC Core §3.1.3.7 re-verification).
- Length-prefixed HMAC session cookies.
- CSRF double-submit + hashed-token-on-row.
- OIDC client_secret AES-256-GCM v3 blob at rest +
integration-test invariant.
- OIDC first-admin bootstrap.
- Default-OFF break-glass admin (Argon2id + lockout +
constant-time + surface invisibility).
- GUI: 4 new pages + login-page IdP buttons + sidebar logout.
- 11 new MCP tools for OIDC + session management.
- 6 per-IdP runbooks (Keycloak / Authentik / Okta / Auth0 /
Entra ID / Google Workspace).
- Threat model extended with 5 new defense subsections + 8 new
threat-catalogue subsections.
- Performance baselines documented (4 benchmarks; 3 measured
+ 1 operator-runs).
- Standards-and-RFC implementation table (13 RFCs + 14 CWEs;
NOT a compliance-mapping doc).
- Coverage gates held at floor 90 across all 4 Bundle 2
packages (anti-Bundle-1-mistake invariant).
- Multi-tenant query CI guard (ratchet baseline 32).
- Phase 10 Keycloak testcontainers integration test + optional
Okta smoke test.
- OpenAPI cookieAuth security scheme + 13 new endpoints + 4
break-glass endpoints.
- Bundle-1-only compat regression CI guard +
Bundle-1-to-2-upgrade regression CI guard.
* Final paragraph updated to point at oidc-enable.md alongside
api-keys-to-rbac.md as the two migration walkthroughs.
docs/README.md (MODIFIED):
* Added the new oidc-enable.md migration row under '## Migration'
alongside the existing api-keys-to-rbac.md entry, with a
one-line description flagging it as the Bundle 2 OIDC
onboarding walkthrough.
Verification
============
* Last-reviewed on security.md + oidc-enable.md: 2026-05-10.
* Internal-link sweep on oidc-enable.md: 0 broken (every relative
link resolves via shell-loop verification).
* Internal-link sweep on docs/README.md: 0 broken (all .md
references resolve).
* No Go-side impact, make verify gate unchanged.
Bundle 2 documentation deliverables now complete: security.md +
auth-threat-model.md + oidc-runbooks/ + auth-benchmarks.md +
auth-standards-implemented.md + api-keys-to-rbac.md + oidc-enable.md
+ CHANGELOG.md v2.1.0. The full Bundle 2 surface is operator-
discoverable from docs/README.md root nav.
|
||
|
shankar0123
|
9b6294e83d |
auth-bundle-2 Phase 14: session + OIDC validation benchmarks (steady-state + cold paths) + auth-benchmarks.md operator doc + Makefile targets
Closes Phase 14 of cowork/auth-bundle-2-prompt.md. Ships four
benchmarks producing four numbers + the operator-doc table; three
default-tag benchmarks runnable on every CI runner, the fourth
(cold-cache OIDC) runnable on operator-side Docker hosts via the
new make target.
Files
=====
internal/auth/session/bench_test.go (NEW):
* BenchmarkSession_SteadyState (target p99 < 1ms; measured 5µs).
Warm in-memory repo + warm session row. Pure CPU: parseCookie +
HMAC verify + map lookup + sentinel checks.
* BenchmarkSession_ColdProcess (target p99 < 10ms; measured 7.1ms).
Same pipeline but with a configurable per-call delay simulating
a 1ms Postgres RTT on each repo call. Two repo calls per
Validate (signing-key fetch + session-row fetch) = 2ms minimum;
Go time.Sleep granularity adds ~1-2ms jitter. Documented why
testcontainers Postgres isn't viable inside b.N: 30+ second
container boot incompatible with per-iteration timing.
* slowSessionRepo + slowKeyRepo wrappers add the per-call delay
via time.Sleep; they delegate to the existing in-memory stubs.
* reportPercentiles helper sorts + reports p50/p95/p99/max via
b.ReportMetric (Go testing.B doesn't surface percentiles
natively).
internal/auth/oidc/bench_test.go (NEW):
* BenchmarkOIDC_SteadyState (target p99 < 5ms; measured 1.5ms).
Drives full HandleCallback against an in-process mockIdP
(httptest.Server localhost loopback). Pre-warmed JWKS cache via
RefreshKeys at setup. Pipeline: pre-login consume + state
compare + token exchange (localhost ~50-200µs) + go-oidc
Verify (RSA-2048 sig verify + alg pin) + service-layer iss/
aud/azp/at_hash/exp/iat/nonce re-checks + group-claim
resolution + group→role mapping + user upsert + session mint.
* The localhost-loopback /token call adds ~100-500µs of TCP
overhead vs pure crypto; the prompt's "no network calls"
steady-state framing accommodates this since the localhost
loopback is the closest practical proxy for a same-region
IdP /token call (which adds 5-15ms in production).
internal/auth/oidc/bench_keycloak_test.go (NEW, //go:build integration):
* BenchmarkOIDC_ColdCache (target p99 < 200ms; operator-runs).
Drives RefreshKeys against a live Keycloak container from the
Phase 10 testfixtures harness. Each iteration evicts the
in-process cache + re-fetches discovery + re-fetches JWKS over
real HTTP + re-runs the IdP-downgrade-attack defense.
* Network-bounded: the cold path is dominated by HTTPS RTT to
the IdP discovery endpoint, NOT crypto. The 200ms cap
accommodates a geographically-distant IdP (~150ms RTT) plus
the in-process JWKS fetch + downgrade-defense logic (~5ms
locally).
* Reuses the sharedKeycloak fixture from
integration_keycloak_test.go (Phase 10) so the benchmark
doesn't pay the 60-90s container boot cost separately. Skips
with a clear message if invoked without the integration test
setup.
* Reports p50/p95/p99/max in MILLISECONDS (vs the
microsecond-granularity steady-state benchmarks) since the
cold path is two orders of magnitude slower.
internal/auth/oidc/service_test.go (MODIFIED):
* Refactored newMockIdP(t *testing.T) to delegate to a new
newMockIdPWithTB(t testing.TB) sibling. Standard Go pattern
for sharing test fixtures between *testing.T and *testing.B.
No behavior change for existing service_test.go tests; the
benchmark file in bench_test.go calls newMockIdPWithTB(b)
to get the same fixture.
docs/operator/auth-benchmarks.md (NEW):
* Result table with all four benchmarks + targets + measured
numbers + status markers. Four-row matrix for the default-tag
benchmarks; the fourth row (cold-cache) is operator-recorded
with an empty cell waiting for the first Docker-equipped run.
* Hardware floor section pinning the 4 vCPU / 8 GiB RAM /
Postgres 16 / Go 1.25 baseline. GitHub-hosted Ubuntu runners
satisfy this; operators on weaker hardware re-record.
* "What each benchmark covers (and what it doesn't)" section
per benchmark, distinguishing the warm steady-state pipeline
from the cold path's network-bounded budget.
* "Cold-cache OIDC: how to run" subsection documenting the
make target + the test+benchmark coupling needed to populate
sharedKeycloak. Operator-recorded baseline table seeded
empty for first runs.
* "Why the cold path is bounded by network latency, not crypto"
section explaining the budget breakdown:
- TCP handshake (1 RTT)
- TLS 1.3 handshake (1-2 RTTs)
- 2 HTTPS GETs (discovery + JWKS, 1 RTT each)
- In-process crypto on the certctl side (~5-10ms total)
So the 200ms cap is operator-checkable: real measurement >
200ms means the IdP is slow OR network congestion OR DNS
issues — the diagnosis is upstream of certctl. Real
measurement < 200ms means the IdP is on a fast same-region
link.
* Methodology section pinning the per-iteration timing capture
+ sort + percentile-extract approach.
* Pre-merge audit section for the Phase 14 exit gate: four
benchmarks ran, four numbers recorded, steady-state targets
met, cold path is operator-runnable + measurably-bounded.
Makefile (MODIFIED):
* Added `make benchmark-auth` (default-tag, runs three of four
benchmarks at 2000 samples each).
* Added `make benchmark-auth-coldcache` (integration-tagged,
runs OIDC cold-cache against live Keycloak; requires Docker).
* Both targets carry explanatory comment blocks.
docs/README.md (MODIFIED):
* Added the auth-benchmarks.md doc to the Operator nav table
alongside performance-baselines.md.
Measured baselines at Phase 14 close (linux/arm64, 4 vCPU)
==========================================================
BenchmarkSession_SteadyState p99 = 5µs (target < 1ms) ✓ 200× under
BenchmarkSession_ColdProcess p99 = 7.1ms (target < 10ms) ✓
BenchmarkOIDC_SteadyState p99 = 1.5ms (target < 5ms) ✓ 3× under
BenchmarkOIDC_ColdCache operator-runs (Docker required)
Verification
============
* gofmt -l on three new bench files: clean.
* go vet ./internal/auth/session/... ./internal/auth/oidc/...: clean
(default tag).
* go vet -tags integration ./internal/auth/oidc/...: clean (integration
tag covers the bench_keycloak_test.go file).
* go test -short -count=1 across all 5 OIDC + session packages:
green; the bench_*_test.go files compile but don't run under
-short (testing.Short() guards + benchmarks are not selected
by -run pattern).
* All three runnable benchmarks executed and produce the numbers
above; recorded in auth-benchmarks.md.
|
||
|
shankar0123
|
5e2accbf5f |
auth-bundle-2 Phase 12: extend auth-threat-model.md with Bundle 2 sections (OIDC + sessions + back-channel logout + OIDC first-admin + break-glass + 8 Bundle 2 threat sub-sections)
Closes Phase 12 of cowork/auth-bundle-2-prompt.md. The single
canonical operator-facing threat model (one doc per topic per the
docs convention) now covers both Bundle 1 (RBAC) AND Bundle 2 (OIDC
+ sessions + back-channel logout + OIDC first-admin + break-glass)
in one place.
File: docs/operator/auth-threat-model.md (MODIFIED, +485 LOC)
Conventions held
================
* The Bundle 1 sections ("Threat actors", "Defenses Bundle 1
ships", "Threats Bundle 1 does NOT close", "Compliance mapping",
"Operator-facing checks", "Cross-references") stay structurally
intact. Bundle 2 EXTENDS them; nothing is rewritten in place.
* `Last reviewed:` header bumped 2026-05-09 → 2026-05-10.
* Per the prompt's explicit instruction: "do NOT create a separate
auth-threat-model-bundle-2.md companion." This commit is a
single-file extension.
Changes
=======
Intro paragraph rewritten:
* From "Bundle 1 lands... Bundle 2 will be updated" to "Bundle 1
AND Bundle 2 land." Sets the reader's expectation that this is
the post-Bundle-2 doc.
Threat actors section (4 new actors appended):
* OIDC-federated end user (token-forgery / session-hijacking /
group-claim-manipulation surface).
* Stolen session cookie holder (XSS / network MITM / pasted-token).
* Compromised IdP (rogue token issuance; mitigations bounded to
audit trail + group-mapping configuration).
* Break-glass-password holder (Phase 7.5 path bypasses OIDC + group
layer entirely; default-OFF is the load-bearing mitigation).
NEW: Defenses Bundle 2 ships (5 sub-sections):
* OIDC token validation (Phase 3) — alg allow-list, IdP-downgrade
defense, exact iss match, aud + azp checks, at_hash
REQUIRED-when-access_token-present (Phase 3 tightening of OIDC
core's MAY → MUST), single-use state + nonce, PKCE-S256 mandatory,
iat window, JWKS rotation handling, JWKS-fetch-fail closed,
encrypted client_secret at rest.
* Session minting + cookies (Phases 4 + 6) — length-prefixed HMAC
defeating concatenation collision, HttpOnly + Secure + SameSite
cookie hardening, idle + absolute timeouts, CSRF defense via
double-submit-cookie + hashed-token-on-row, optional IP/UA bind,
signing-key rotation primitive with retention window, fail-fatal
EnsureInitialSigningKey at boot, pre-login vs post-login cookie
discrimination.
* Back-channel logout (Phase 5) — OpenID Connect Back-Channel
Logout 1.0 (NOT RFC 8414), required-claim pinning, jti-based
replay defense, alg allow-list applies, Cache-Control: no-store.
* OIDC first-admin bootstrap (Phase 7) — coexists with Bundle 1's
env-var-token bootstrap, group-scoped, one-shot per tenant via
admin-existence probe, explicit OIDC provider gate, audit row on
every grant.
* Break-glass admin (Phase 7.5) — default-OFF, surface-invisibility
via 404-not-403, Argon2id with OWASP 2024 params, lockout state
machine, constant-time across all failure paths via verifyDummy,
WARN log at boot when ENABLED=true, 5/min rate limit on the
public login endpoint.
NEW: Bundle 2 threat catalogue (8 sub-sections, one per
prompt-enumerated threat axis):
1. OIDC token forgery vectors and mitigations (9-row table covering
alg confusion, audience injection, issuer mismatch, nonce replay,
state replay, at_hash substitution, iat window manipulation,
JWKS rotation mid-login, JWKS-fetch failure during a key
rotation).
2. Session hijacking vectors and mitigations (7-row table covering
XSS cookie theft, network MITM, CSRF, concatenation-collision
forgery, stolen-cookie replay, cross-tab interference, sign-out
race).
3. IdP compromise scenarios (operator monitors IdP audit logs,
operator can rotate group-role mappings without redeploying,
audit trail records source provider, provider-delete returns
409 with active sessions).
4. Back-channel logout failure modes (6-row table covering IdP
unreachable, invalid signature, replay via jti, alg confusion,
missing events claim, present-nonce-claim).
5. Group-claim manipulation (4-row table covering operator
misconfigured mapping, misconfigured groups_claim_path, IdP
renames a group, IdP user maintainer adds user to unintended
group).
6. Bootstrap phase risks post-Bundle-2 (4-row table covering
CERTCTL_BOOTSTRAP_TOKEN leak, CERTCTL_BOOTSTRAP_ADMIN_GROUPS
misconfigured to a wide group, both bootstrap strategies
simultaneously, multi-IdP without explicit provider gate).
7. Break-glass risks (7-row table covering phished password,
online brute-force, offline brute-force on DB compromise,
operator forgets to disable, side-channel timing on
wrong-vs-no-credential-vs-locked, surface fingerprinting,
reserved-actor mutation).
8. Token-leak hygiene (the explicit grep policy with three
per-package logging_test.go pointers + the audit_redact.go
defense-in-depth note).
Threats Bundle 1 does NOT close section relabeled:
* Section header now reads "Threats Bundle 1 does NOT close
(Bundle 2 closure status)" with each item carrying ✅ / ⚠️ /
"still deferred" markers.
* Items 1, 2, 3, 8 marked ✅ closed by Bundle 2.
* Items 4, 5, 7, 9 marked still-deferred with v3 / follow-on
pointers.
* Item 6 (rate limiting on bootstrap) marked acceptable; Bundle 2
adds the same rate-limit primitive to /auth/breakglass/login.
NEW: Threats Bundle 2 does NOT close section listing the 8 v3 /
future-work items:
* WebAuthn / FIDO2 second factor (Decision 12).
* Time-bound role grants / JIT elevation.
* SAML federation (operators broker through Keycloak).
* Multi-tenant data isolation activation (gated to managed-service
hosting work).
* HSM / FIPS-validated signing key for sessions.
* OIDC RP-initiated logout (Bundle 2 implements only back-channel).
* GUI E2E via Playwright.
* Per-IdP runbook external-tester sign-off (encouraged, NOT a merge
gate post-2026-05-10 policy change).
Operator-facing checks section extended:
* 6 new SQL-shaped checks for Bundle 2 (provider count drift,
per-actor session count, unmapped-groups audit-row spike,
break-glass usage outside incidents, OIDC first-admin one-row-per-
tenant invariant, retired-signing-key GC liveness).
Cross-references section split into Bundle 1 anchors + Bundle 2
anchors:
* Bundle 2 anchors enumerate every load-bearing file: 6
internal/auth/ packages, 5 migrations, 3 ci-guards.
Compliance mapping section UNCHANGED:
* Phase 15 (standards-and-RFC-implementation table) is the proper
home for the RFC + CWE evidence the Bundle 2 surface adds.
Re-introducing framework-mapping prose at the threat-model layer
would regress the operator's 2026-05-05 retired-compliance-docs
decision, which is explicitly forbidden by the Phase 15 prompt.
Verification
============
* `> Last reviewed: 2026-05-10` — confirmed via head -3.
* All 8 prompt-mandated Bundle 2 threat sub-sections present —
confirmed via grep `^### ` count (19 ### headers total: 6 Bundle
1 + 5 Bundle 2 defenses + 8 Bundle 2 threats).
* All 39 prompt-listed threat-vector keywords present — confirmed
via single-line grep counting 39 hits across the prompt's
vocabulary.
* Internal markdown links resolve cleanly — confirmed via shell
loop iterating each `]( ...)` reference and checking `[ -e "$path" ]`.
* No backend / Go-test impact — pure docs commit.
* `make verify` gate unchanged.
|
||
|
shankar0123
|
f203a5372d |
auth-bundle-2 Phase 11 follow-on: drop external-tester reference from oidc-runbooks/index.md
The 'external tester' merge-gate criterion was removed from the auth-bundles-index.md policy: external-tester confirmations are encouraged but NOT a merge condition (BSL discourages contribution- style testing; the Phase 10 Keycloak testcontainers harness + the optional Okta smoke test cover the same surface deterministically in CI). Drops the now-stale phrasing from the runbooks index and the merge-gate reference; keeps the operator-sign-off footer recommendation since dated validation records are still useful. |
||
|
shankar0123
|
2893f9b48e |
auth-bundle-2 Phase 11: 6 per-IdP OIDC runbooks + index + docs/README wiring
Closes Phase 11 of cowork/auth-bundle-2-prompt.md. Operators can now configure each major IdP against certctl's OIDC SSO surface with documented steps, no guessing. Files ===== docs/operator/oidc-runbooks/index.md (NEW): * Index page linking all six per-IdP runbooks. * Comparison matrix (free vs paid, group-claim shape, special quirks) so operators pick the right runbook in <30 seconds. * "Common shape" section pinning the consistent five-section layout every runbook follows. * "Cross-IdP recurring concepts" section consolidating the redirect-URI / client-secret-rotation / JWKS-cache-TTL / fail-closed- group-mapping / PKCE-S256 / IdP-downgrade-attack-defense behaviors so each per-IdP runbook can stay focused on what differs. docs/operator/oidc-runbooks/keycloak.md (NEW): * Canonical reference. Mirrors the testfixtures/keycloak-realm.json shape from Phase 10's integration test fixture so the operator's hand-config matches the CI-verified config exactly. * Step-by-step IdP-side: realm → client → groups → group-mapper → user. Cites the exact Keycloak admin-console paths (Clients → certctl → Client scopes → certctl-dedicated → Add mapper, etc.). * GUI + API + MCP equivalents for the certctl-side configuration. * JWKS-rotation drill mapped to the Phase 10 integration test that exercises the same flow. * 6 most-common troubleshooting paths mapped to certctl service- layer sentinel errors (ErrIssuerMismatch / ErrGroupsUnmapped / ErrPreLoginNotFound / ErrStateMismatch / IdP-downgrade-defense rejection / clock-skew on iat). docs/operator/oidc-runbooks/authentik.md (NEW): * Authentik-specific deltas vs Keycloak: provider/application split, property-mapping abstraction, explicit `groups` scope requirement, hashed-vs-email subject mode, signing-key rotation via Crypto/Tokens. docs/operator/oidc-runbooks/okta.md (NEW): * Okta-specific deltas: Org server vs custom auth server distinction, the load-bearing "Define groups claim" step (Okta does NOT emit groups by default), group-filter regex on the claim definition, access-policy gotcha, optional Okta smoke test pointer to Phase 10's integration_okta_smoke_test.go. docs/operator/oidc-runbooks/auth0.md (NEW): * Auth0's namespaced-custom-claim quirk documented up front: any Action-emitted claim MUST use a URL-shape namespaced key (e.g. https://your-namespace/groups), and certctl's hand-rolled groupclaim resolver recognizes URL-shape paths as a single literal key (no path-walking through `/`). Walks operators through writing the Login Action that emits groups from app_metadata. Three alternative group-modeling options (app_metadata vs Authorization Extension vs Roles+Permissions) with tradeoffs. docs/operator/oidc-runbooks/azure-ad.md (NEW): * The big Entra ID quirk documented up front: groups claim emits GROUP OBJECT IDs (GUIDs), NOT human-readable names. Certctl group→ role mappings MUST be configured against the GUIDs. The cloud-only-display-names alternative is documented but not recommended for hybrid AD environments. Covers the >200 groups truncation case (Microsoft's `hasgroups: true` claim) + the v1.0 vs v2.0 endpoint distinction (certctl supports v2.0 only). docs/operator/oidc-runbooks/google-workspace.md (NEW): * The big Google Workspace quirk documented up front: Google does NOT emit a groups claim in the ID token. Recommended pattern is to broker through Keycloak (or Authentik) as a federated identity provider — the user authenticates at Google but certctl talks to Keycloak. Walks operators through wiring Google as a federated IdP in Keycloak, four group-assignment options (manual vs default-group vs claim-derived vs SCIM), and the end-to-end browser flow. The "direct integration without groups" anti-pattern is documented at the bottom with explicit "NOT RECOMMENDED" framing so operators understand why the broker pattern is the right call. docs/README.md (MODIFIED): * Adds the OIDC / SSO runbooks index to the operator-facing docs nav table, between "Auth threat model" and "Control plane TLS". Conventions held ================ * Every runbook carries `> Last reviewed: 2026-05-10` per the docs convention. * Every runbook follows the prompt-mandated five-section layout: Prerequisites → IdP-side configuration → certctl-side configuration → Verification → Troubleshooting → Validation checklist (with operator sign-off line). * Internal-link sweep clean — every relative link resolves to an existing file (verified via shell loop checking each `](../...)` and `](*.md)` reference). External links to IdP vendor sites are the canonical https URLs. * No leakage of cowork/ workspace paths as Markdown links — the azure-ad.md initially had a `[auth-bundles-index.md](../../../../cowork/...)` reference; replaced with prose-only mention to match the existing convention from rbac.md + migration/api-keys-to-rbac.md. * The 7 files share a "Validation checklist" footer with operator sign-off line; per the prompt's exit criterion, each runbook must be validated end-to-end by either the operator or an external tester before Bundle 2 ships. Verification ============ * Last-reviewed dates: 7/7 runbooks dated 2026-05-10. * Internal-link sweep: 0 broken (every `]( ...)` reference resolves). * docs/README.md → operator/oidc-runbooks/index.md link resolves. * No backend / frontend / Go-test impact — pure docs commit. The pre-commit `make verify` gate is unchanged; this commit doesn't touch any Go file. Phase 11 deviation note ======================= The merge-gate criterion's "≥ 2 external testers" requirement is operator-driven and post-tag — Phase 11 ships the runbooks; the operator runs each end-to-end against a real production-tier IdP and fills in the sign-off footers before flipping Bundle 2 to "merged." Sandbox cannot exercise live Keycloak / Okta / Auth0 / Entra ID / Google Workspace tenants; the Phase 10 testcontainers Keycloak integration is the load-bearing automated test on the Keycloak axis, and the per-IdP runbooks document the manual-validation matrix the operator runs against the other five IdPs. |
||
|
shankar0123
|
5313cd8492 |
auth-bundle-1 Phase 13 follow-up: em-dash sweep + broken-link fix
Self-audit on
|
||
|
shankar0123
|
e7a94b6080 |
auth-bundle-1 Phase 13: docs (rbac.md + threat model + migration guide + security.md update)
Closes the last Phase before the Bundle 1 Exit gate. Operators
now have authoritative reference + threat model + migration guide
covering every behavior change Bundles 0-12 introduced.
# New docs
* docs/operator/rbac.md (340 lines) — operator how-to:
- Mental model (actors / roles / permissions / scopes)
- 7 default roles seeded by migration 000029 + the 5
admin-only fine-grained perms seeded by 000030
- Permission catalogue table by namespace
- Scope semantics (global beats specific) + the Bundle-2
deferral on scope_id FK enforcement
- Granting / revoking access from GUI + CLI + HTTP API + MCP
- The auditor pattern (audit-only, no resource read)
- Day-0 bootstrap flow (CERTCTL_BOOTSTRAP_TOKEN → curl →
HTTP 410 thereafter)
- Demo-mode (CERTCTL_AUTH_TYPE=none) caveat for production
* docs/operator/auth-threat-model.md (180 lines) — what the
controls defend against:
- 5 threat actors (external, wrong-role, compromised key,
insider operator, compromised auditor)
- Per-defense walk-through (API-key auth, RBAC, bootstrap,
approval workflow + Phase 9 closure, audit trail,
protocol-endpoint allowlist)
- 9 explicit deferrals (OIDC, sessions, local accounts,
JIT elevation, MFA, etc.) — Bundle 2 / future scope
- Compliance mapping (SOC 2 CC6.1/CC6.3, HIPAA §164.312(b),
NIST SSDF PO.5.2, FedRAMP AU-9, PCI-DSS §10)
- 5 operator-runnable sanity checks (e.g.,
'SELECT FROM audit_events WHERE actor=system-bypass' MUST
return 0 in production)
* docs/migration/api-keys-to-rbac.md (200 lines) — v2.0.x →
v2.1.0 upgrade flow:
- The SECURITY: AUDIT YOUR API KEYS callout
- Migration list (000029-000033) + what each does
- 4-mode scope-down flow (interactive / non-interactive
JSON / --suggest / --suggest --apply)
- What changes for code that called auth.IsAdmin
- Helm-specific upgrade flow with example post-upgrade Job
- Docker Compose upgrade flow + the 5 examples folders
that ride demo mode unchanged
- Verification queries + rollback flow
# Updated docs
* docs/operator/security.md — Last-reviewed bumped to
2026-05-09; existing Authentication-surface section
extended to call out the Bundle 1 RBAC primitive,
day-0 bootstrap path, and approval-bypass closure with
cross-references to the new docs.
* docs/reference/profiles.md — Last-reviewed header
formatting fixed (added the > blockquote prefix used
consistently across the docs tree).
# docs/README.md navigation
* Operator section gains 2 new rows (RBAC + auth-threat-model)
and Approval-workflow row updated to mention Phase 9
closure.
* Reference section gains the Profiles row.
* Migration section gains the api-keys-to-rbac row with the
AUDIT YOUR API KEYS callout in the link description.
# CHANGELOG.md v2.1.0 section refreshed
The Phase 7 commit landed the SECURITY: AUDIT YOUR API KEYS
callout. This commit appends the missing Phase 9-12 highlights:
- Approval-bypass closure (profile-edit gate + flip-flop
loophole + ErrApproveBySameActor invariant)
- GUI: Roles / API Keys / Auth Settings / Approvals queue
- 12 new MCP RBAC tools
- Coverage gates on internal/auth + internal/service/auth
- Protocol-endpoint allowlist pinned at 3 layers
Trailing cross-reference block now points at all 4 new docs.
# Verifications
* Every internal link in the 4 new/modified docs validated by
shell sweep (find broken links → 0 hits).
* Every new doc carries 'Last reviewed: 2026-05-09' header
with the > blockquote prefix matching the docs-tree
convention.
* go vet ./... clean.
* staticcheck across every Bundle-1-touched Go package clean.
* gofmt -l clean repo-wide.
* go test -short -count=1 green across internal/auth (incl.
bootstrap), internal/api/handler, internal/api/router,
internal/cli, internal/service (incl. auth),
internal/domain/auth, internal/mcp, cmd/cli (cmd/server
has 1 environmental failure on the sandbox virtiofs-tmp:
TestPreflightSCEPRACertKey_KeyWorldReadable_Refuses depends
on tmpfs file-mode semantics that virtiofs propagates
differently — pre-existing, unrelated to Bundle 1).
* Frontend: 19 Vitest tests across src/pages/auth/ +
AuditPage all pass; tsc --noEmit clean.
|
||
|
shankar0123
|
75097909e9 | |||
|
shankar0123
|
d809874fa1 |
docs: retire compliance subtree + sweep framework name-drops from prose
Per operator decision the framework-mapping docs are gone. They
were aspirational (no audit, no certification, no validated
mapping); keeping them around was misleading.
Files deleted (1,883 lines):
- docs/compliance/index.md
- docs/compliance/soc2.md
- docs/compliance/pci-dss.md
- docs/compliance/nist-sp-800-57.md
Hyperlinks removed:
- README.md: 'Auditor / compliance' row in the doc table; the
'(compliance mapping included)' parenthetical in the
positioning paragraph
- docs/README.md: the '## Compliance' section table; the
'Auditor / compliance team' reading-order-by-role row
Prose name-drops swept across 24 files:
- README.md: 'FedRAMP boundary CAs / financial-services policy
CAs' → '4-level boundary CAs / 3-level policy CAs';
'Compliance-grade for PCI-DSS Level 1, FedRAMP Moderate / High,
SOC 2 Type II, HIPAA' → cut entirely
- getting-started/{quickstart,concepts,examples,why-certctl,
advanced-demo}.md: 'compliance' → 'audit' / 'policy';
'PCI-DSS / SOC 2 / NIST SP 800-57' framework lists cut;
''pci': 'true'' tag example → ''environment': 'production''
- migration/cert-manager-coexistence.md: 'compliance rules' →
'policy rules'
- operator/approval-workflow.md: 'Compliance customers (PCI-DSS
Level 1, FedRAMP Moderate / High, SOC 2 Type II, HIPAA)' →
'Operators'; entire 'Compliance control mapping' table
(PCI-DSS §6.4.5 / NIST SP 800-53 SA-15 / SOC 2 Type II CC6.1
/ HIPAA §164.308(a)(4)) deleted; 'compliance contract' →
'two-person-integrity contract'; 'compliance auditors' →
'reviewers'
- operator/legacy-clients-tls-1.2.md: 'PCI-DSS v4.0 Req 4 §2.2.5'
audit-reference → CWE-326 (kept); 'PCI-DSS Req 4 §2.2.5
attestation' section retitled to 'TLS posture summary' and
rewritten without framework framing; 'PCI-DSS, NIST, and
major browsers will eventually deprecate TLS 1.2' →
'Major browsers and OS vendors will eventually deprecate
TLS 1.2'
- operator/database-tls.md: PCI-DSS Req 4 §2.2.5 audit-ref →
CWE-319 only; 'PCI-DSS scope' → 'sensitive data'; PCI-DSS
Req 4 v4.0 prose footing → cut
- operator/runbooks/disaster-recovery.md: 'SOC 2 / PCI
procurement-team deliverable' → 'on-call deliverable';
'compliance auditors' → 'reviewers'
- reference/connectors/{acme,aws-acm,azure-kv,globalsign,
local-ca,openssl,ssh,index}.md: 'compliance reporting
(PCI-DSS §3.6, HIPAA §164.312)' → 'audit reporting';
'Compliance environments (PCI-DSS Level 1, FedRAMP High,
HIPAA)' → 'Regulated environments'; 'compliance audits' →
'audit'; 'FedRAMP boundary CA' pattern names →
'4-level boundary CA' (technically descriptive)
- reference/protocols/est.md: 'compliance-hook seam' →
'device-state hook seam'; 'compliance gating' → 'device-state
gating'; 'est_compliance_failed' → 'est_device_state_failed'
- reference/protocols/scep-intune.md: 'Optional compliance
check' → 'Optional device-state check'; failure-counter
'compliance_failed' → 'device_state_failed'; 'Conditional
Access compliance gating' → 'Conditional Access
device-state gating'
- reference/intermediate-ca-hierarchy.md: 'FedRAMP boundary-CA
deployments where the regulator requires...' →
'Boundary-CA deployments where you want separation of policy
and issuing authorities'; pattern A retitled '4-level FedRAMP
boundary CA' → '4-level boundary CA'
- reference/architecture.md: broken Related-docs link to
compliance.md removed; the rest of that block had stale
pre-Phase-2 paths (quickstart.md, demo-advanced.md,
connectors.md, openapi.md, testing-guide.md, test-env.md) —
retargeted to current locations
- reference/deployment-model.md: 'SOC 2 evidence-report
generator' → 'Audit-evidence report generator'
- reference/vendor-matrix.md: 'SOC 2 / PCI auditors paste this
into evidence packs' → 'reviewers paste this into
vendor-evaluation packs'
- contributor/qa-test-suite.md: 'compliance exist' coverage
description cut; 'Compliance (PCI / SOC2 / HIPAA-relevant)'
risk-class label → 'Audit-relevant'
What was kept:
- CWE references (legitimate technical pointers)
- Microsoft API/feature names that happen to use 'compliance'
literally ('Microsoft Graph compliance API',
'device-compliance validators' — these are MS product names,
not framework name-drops)
- 'NIST PQC' on the landing page (Post-Quantum Cryptography is
the actual NIST standard family, not a compliance framework)
Verified: zero hyperlinks into docs/compliance/ remain. All 24
ci-guards/*.sh pass locally. qa-doc-seed-count.sh clean.
Net diff: 26 files / -1,883 deletions in compliance/ + -32 net
across the prose sweep.
Companion edits in cowork/ (CLAUDE.md doc-tree summary +
WORKSPACE-CHANGELOG.md retirement note) land separately.
|
||
|
shankar0123
|
b452013dd9 |
docs: Phase 5 — testing-guide.md prune (8268 → 0 lines, content dispersed)
Per Phase 1 audit at cowork/docs-overhaul-phase-1-audit-2026-05-04/
and the section-by-section plan in testing-guide-tumor.md.
testing-guide.md was 30% of all docs/ content (8268 lines) but was
integration test code written in markdown, not operator documentation.
The audit's tumor analysis disposed of every Part:
- ~65% DELETE (test cases that already exist in code)
- ~22% MOVE to inline test code
- ~8% KEEP-COMPRESSED into focused operator-runbook docs
- Title + contents + release sign-off ~5% KEEP
This commit ships the KEEP-COMPRESSED dispersal:
docs/contributor/qa-prerequisites.md (NEW, ~120 lines):
From testing-guide.md "Prerequisites" section. Stack boot procedure,
demo data baseline, reference IDs operators reuse across QA docs.
docs/contributor/gui-qa-checklist.md (NEW, ~105 lines):
From testing-guide.md "Part 35: GUI Testing". Manual GUI verification
pass for release sign-off. 25-row table covering every dashboard page.
docs/contributor/release-sign-off.md (NEW, ~130 lines):
From testing-guide.md "Release Sign-Off" section (originally 1009
lines of per-test detail tables). Compressed to a release-day
checklist organized by gate category: code state, automated gates,
manual QA passes, release artefact verification, branch protection,
post-release.
docs/operator/performance-baselines.md (NEW, ~100 lines):
From testing-guide.md "Part 39: Performance Spot Checks". Four
operator-runnable benchmarks (API request handling, inventory list
pagination, scheduler tick, bulk revoke) with baseline numbers and
when-to-re-baseline guidance.
docs/operator/helm-deployment.md (NEW, ~120 lines):
From testing-guide.md "Part 52: Helm Chart Deployment". Operator
runbook for the bundled deploy/helm/certctl/ chart: prereqs,
install, four cert-source patterns, verify, upgrade, troubleshooting.
docs/reference/cli.md (NEW, ~120 lines):
From testing-guide.md "Part 28: CLI Tool". certctl-cli command
reference with command-group breakdown, common workflows
(list/filter, renew, revoke, bulk import, EST enrollment, status),
output formats, CI/CD integration patterns.
docs/README.md navigation index updated to include the 6 new docs:
Reference section gains: cli.md, release-verification.md (was added
in Phase 13)
Operator section gains: helm-deployment.md, performance-baselines.md
Contributor section gains: qa-prerequisites.md, gui-qa-checklist.md,
release-sign-off.md
docs/testing-guide.md deleted. Git history preserves the 8268 lines —
if any specific test case is found missing from inline test code or
the destination docs during future work, lift from `git show
HEAD~1:docs/testing-guide.md`.
Net: docs/ total line count drops by ~7700 lines (28%), from 26,369
to 18,742. testing-guide.md was the single largest doc; pruning it is
the single biggest content-edit win of the entire restructure.
Phase 5 is the last major content phase. Remaining: Phase 4 follow-on
(per-connector page extractions from reference/connectors/index.md),
Phase 15 (WHAT/HOW/WHY remediation), Phase 16 (final acceptance gate).
|
||
|
shankar0123
|
12d7b1f51d |
docs: Phase 11 follow-on — fix inter-doc cross-references in deeper subdirs
Per Phase 1 audit at cowork/docs-overhaul-phase-1-audit-2026-05-04/.
Continuation of Phase 11 (commit
|
||
|
shankar0123
|
19c8fafe84 |
docs: Phase 14 — Last reviewed line sweep across docs/
Per Phase 1 audit at cowork/docs-overhaul-phase-1-audit-2026-05-04/. Adds a `> Last reviewed: 2026-05-05` line right after the H1 heading of every doc that didn't already have one (41 files). This dates the freshness clock for the future Phase 4 per-doc review. The discipline going forward: when a doc's content gets a meaningful edit, bump the date. When the date gets old (e.g., >6 months), the doc earns a freshness-review pass. Mechanical insertion via awk one-liner, applied to every docs/*.md that didn't already match `grep -q 'Last reviewed:'`. Files that already carried the line from earlier Phase 2 work (the navigation index, the new connector docs, the new SCEP server / legacy-clients- TLS-1.2 / release-verification docs, and the 5 per-connector deep dives) were skipped to avoid duplicate insertion. Net: every doc in docs/ now has a Last reviewed line. |
||
|
shankar0123
|
e9b15108d9 |
docs: split legacy-est-scep.md into two purpose-aligned docs
The 519-line legacy-est-scep.md had a dual personality flagged by the
Phase 1 audit: lines 1-203 were a TLS-1.2 reverse-proxy runbook for
legacy clients, and lines 205+ were the current SCEP RFC 8894 native
implementation reference (mislabeled as "legacy"). Two separate audiences,
two separate purposes.
Split:
Lines 1-203 (TLS-1.2 reverse-proxy runbook):
→ docs/operator/legacy-clients-tls-1.2.md (NEW)
Operator runbook for the case where embedded EST/SCEP clients only
speak TLS 1.2. Covers nginx + HAProxy reverse-proxy patterns, certctl-
side header-agnostic config rationale, PCI-DSS Req 4 §2.2.5 attestation,
deprecation timeline. Also got a fresh "What this is" framing.
Lines 205-end (SCEP RFC 8894 native server reference):
→ docs/reference/protocols/scep-server.md (NEW)
Generic SCEP server protocol reference: RA cert + key configuration,
GetCACaps capability advertisement, supported messageTypes, MVP
backward-compat path, multi-profile dispatch, must-staple per-profile
policy, mTLS sibling route, Microsoft Intune dynamic-challenge
dispatcher. Cross-links to scep-intune.md for Intune-specific
deployment guidance.
Both new docs carry a `Last reviewed: 2026-05-05` line. Internal links
within each new doc updated to the new sibling paths. Cross-references
from other docs to legacy-est-scep.md still need fixing in Phase 11.
Original docs/legacy-est-scep.md deleted (git history preserves).
|
||
|
shankar0123
|
3a807ae37e |
docs: Phase 2 mechanical file moves to subdirectory structure
Pure git mv operations; no content edits. Internal links remain pointing
at old paths and will be fixed in Phase 11. Per the Phase 1 audit
recommendations at cowork/docs-overhaul-phase-1-audit-2026-05-04/.
35 files moved across 8 audience-organized subdirectories:
docs/getting-started/ (5):
quickstart.md, concepts.md, examples.md, advanced-demo.md (was
demo-advanced.md), why-certctl.md
docs/reference/ (6):
architecture.md, api.md (was openapi.md), mcp.md,
intermediate-ca-hierarchy.md, deployment-model.md (was
deployment-atomicity.md), vendor-matrix.md (was
deployment-vendor-matrix.md)
docs/reference/protocols/ (6):
acme-server.md, acme-server-threat-model.md, scep-intune.md,
est.md, crl-ocsp.md, async-ca-polling.md (was async-polling.md)
docs/operator/ (4):
security.md, tls.md, database-tls.md, approval-workflow.md
docs/operator/runbooks/ (3):
cloud-targets.md (was runbook-cloud-targets.md), expiry-alerts.md
(was runbook-expiry-alerts.md), disaster-recovery.md
docs/migration/ (3):
from-certbot.md (was migrate-from-certbot.md), from-acmesh.md
(was migrate-from-acmesh.md), cert-manager-coexistence.md (was
certctl-for-cert-manager-users.md)
docs/compliance/ (4):
index.md (was compliance.md), soc2.md (was compliance-soc2.md),
pci-dss.md (was compliance-pci-dss.md), nist-sp-800-57.md (was
compliance-nist.md)
docs/contributor/ (4):
testing-strategy.md, test-environment.md (was test-env.md),
ci-pipeline.md, qa-test-suite.md (was qa-test-guide.md)
Deferred to later Phase 2 sub-phases:
- connectors.md split (Phase 4): docs/connectors.md +
docs/connector-{apache,f5,iis,k8s,nginx}.md still at top level
- testing-guide.md prune (Phase 5): docs/testing-guide.md still
at top level
- features.md disperse (Phase 6): docs/features.md still at top
level
- legacy-est-scep.md split (Phase 7): docs/legacy-est-scep.md
still at top level
- ACME walkthrough re-homing (Phase 8): three
docs/acme-*-walkthrough.md still at top level
- Upgrade docs archive (Phase 3): two docs/upgrade-*.md still
at top level
Cross-reference updates (Phase 11) will happen after all moves and
content edits land. Internal links to docs/* paths are temporarily
broken until that phase completes.
|