mirror of
https://github.com/shankar0123/certctl.git
synced 2026-06-07 17:02:43 +00:00
shankar0123
35e18bfc56
Closes the #9 acquisition-readiness blocker from the 2026-05-01 issuer coverage audit. Pre-fix, JobService.ProcessPendingJobs ran every claimed job sequentially in a single goroutine: safe but slow, and operators with large fleets had no lever to dial throughput up. Switching to fire-and-forget per-job goroutines would have unbounded the upstream-CA call rate and tripped DigiCert / Entrust / Sectigo rate limits — certctl's response to 429 was to retry on the next tick, re-fanning out the same calls and digging deeper into the limit. Operators need a knob. This commit: - Adds CERTCTL_RENEWAL_CONCURRENCY env var (default 25) loaded via the existing getEnvInt pattern in internal/config/config.go. Documented inline as the cap for the per-tick renewal/issuance/ deployment goroutine fan-out, with operator-tuning guidance: permissive upstream limits + large fleets (>10k certs) → 100; strict limits or async-CA-heavy fleets → 25 or lower. - Wires golang.org/x/sync/semaphore.Weighted around the per-job goroutine launch in JobService.ProcessPendingJobs. Acquire(ctx, 1) is the load-bearing piece — it BLOCKS the loop when at the cap, providing real backpressure rather than fire-and-forget. The fan-out is split into processPendingJobsSequential (legacy, preserved for unit-test wiring that doesn't call SetRenewalConcurrency) and processPendingJobsConcurrent (production, delegates to a generic boundedFanOut helper). - boundedFanOut takes the per-job work as a closure so the cap can be tested directly without standing up the renewal/deployment service graph. processed/failed counters use atomic.Int64 to avoid mutex overhead on every job completion; final log line reads both AFTER wg.Wait so the counts reflect every dispatched job. ctx-aware Acquire ensures a shutdown ctx cancel interrupts the dispatch loop promptly; in-flight goroutines drain via Wait before the function returns so no goroutine outlives the scheduler tick. - shouldSkipJob extracted as a package-private helper so the agent-routed-deployment skip logic is shared between the sequential and concurrent paths byte-for-byte (the audit prompt's "channel-based semaphore without ctx-aware acquire" anti-pattern is explicitly avoided — semaphore.Weighted.Acquire returns on ctx done; channel <- struct{}{} would block forever). - SetRenewalConcurrency setter on JobService normalises ≤0 to 1. semaphore.NewWeighted(0) constructs a semaphore that blocks every Acquire forever; the normalisation prevents a misconfigured env var from wedging the scheduler. - cmd/server/main.go wires SetRenewalConcurrency(cfg.Scheduler. RenewalConcurrency) on the freshly-built jobService, immediately after SetAuditService. Production deployments always take the bounded path; tests that build JobService directly via NewJobService keep their strict-sequential behaviour because renewalConcurrency is the zero value. - Tests in internal/service/job_concurrency_test.go: * TestBoundedFanOut_CapHolds — primary regression guard. 50 jobs × 50ms work × cap=5 → asserts peak in-flight never exceeds 5 AND reaches 5 at least once (catches both upper-bound regressions and gates that incorrectly cap below the configured value). Lock-free max via CompareAndSwap so the measurement instrument doesn't itself constrain concurrency. * TestBoundedFanOut_AllJobsRun — lower-bound: every non-skipped job is dispatched. * TestBoundedFanOut_SkipsAgentRoutedDeployments — pins the shouldSkipJob contract. * TestBoundedFanOut_CtxCancelInterrupts — ctx cancellation interrupts a stuck fan-out within the timeout budget. * TestBoundedFanOut_FailedJobsCounted — per-job errors don't abort the fan-out. * TestSetRenewalConcurrency_NormalizesNonPositive — ≤0 → 1 fail-safe pinned across negative/zero/positive inputs. - docs/features.md: scheduler-loop table augmented with the concurrency-cap env-var pointer alongside the job-processor row. - docs/architecture.md: Concurrency Safety section gains a paragraph explaining the cap, the operator-tuning guidance, the ctx-aware Acquire semantics, and the audit reference. Operator-facing impact: the first big renewal sweep no longer takes down the upstream CA's rate-limit budget. Existing deployments get the bounded path automatically (default 25); operators can override via env var without code changes. Verified locally: - gofmt -l . clean - go vet ./... clean - staticcheck ./... clean - go test -short -count=1 across service / scheduler / config / integration: green - Six new tests under TestBoundedFanOut* + TestSetRenewalConcurrency*: green Audit reference: cowork/issuer-coverage-audit-2026-05-01/RESULTS.md Top-10 fix #9.
107 lines
4.9 KiB
Modula-2
107 lines
4.9 KiB
Modula-2
module github.com/shankar0123/certctl
|
|
|
|
go 1.25.9
|
|
|
|
require (
|
|
github.com/google/uuid v1.6.0
|
|
github.com/lib/pq v1.10.9
|
|
github.com/modelcontextprotocol/go-sdk v1.4.1
|
|
github.com/testcontainers/testcontainers-go v0.35.0
|
|
)
|
|
|
|
require (
|
|
github.com/aws/aws-sdk-go-v2 v1.41.7
|
|
github.com/aws/aws-sdk-go-v2/config v1.32.17
|
|
github.com/aws/aws-sdk-go-v2/service/acmpca v1.46.14
|
|
github.com/leanovate/gopter v0.2.11
|
|
github.com/masterzen/winrm v0.0.0-20250927112105-5f8e6c707321
|
|
github.com/pkg/sftp v1.13.10
|
|
golang.org/x/crypto v0.45.0
|
|
golang.org/x/sync v0.18.0
|
|
software.sslmate.com/src/go-pkcs12 v0.7.0
|
|
)
|
|
|
|
require (
|
|
dario.cat/mergo v1.0.0 // indirect
|
|
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
|
|
github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 // indirect
|
|
github.com/ChrisTrenkamp/goxpath v0.0.0-20210404020558-97928f7e12b6 // indirect
|
|
github.com/Microsoft/go-winio v0.6.2 // indirect
|
|
github.com/aws/aws-sdk-go-v2/credentials v1.19.16 // indirect
|
|
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.23 // indirect
|
|
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.23 // indirect
|
|
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.23 // indirect
|
|
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.24 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.9 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.23 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/signin v1.0.11 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/sso v1.30.17 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.21 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/sts v1.42.1 // indirect
|
|
github.com/aws/smithy-go v1.25.1 // indirect
|
|
github.com/bodgit/ntlmssp v0.0.0-20240506230425-31973bb52d9b // indirect
|
|
github.com/bodgit/windows v1.0.1 // indirect
|
|
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
|
|
github.com/containerd/containerd v1.7.18 // indirect
|
|
github.com/containerd/log v0.1.0 // indirect
|
|
github.com/containerd/platforms v0.2.1 // indirect
|
|
github.com/cpuguy83/dockercfg v0.3.2 // indirect
|
|
github.com/davecgh/go-spew v1.1.1 // indirect
|
|
github.com/distribution/reference v0.6.0 // indirect
|
|
github.com/docker/docker v27.1.1+incompatible // indirect
|
|
github.com/docker/go-connections v0.5.0 // indirect
|
|
github.com/docker/go-units v0.5.0 // indirect
|
|
github.com/felixge/httpsnoop v1.0.4 // indirect
|
|
github.com/go-logr/logr v1.4.1 // indirect
|
|
github.com/go-logr/stdr v1.2.2 // indirect
|
|
github.com/go-ole/go-ole v1.2.6 // indirect
|
|
github.com/gofrs/uuid v4.4.0+incompatible // indirect
|
|
github.com/gogo/protobuf v1.3.2 // indirect
|
|
github.com/google/jsonschema-go v0.4.2 // indirect
|
|
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
|
|
github.com/hashicorp/go-uuid v1.0.3 // indirect
|
|
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
|
|
github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
|
|
github.com/jcmturner/gofork v1.7.6 // indirect
|
|
github.com/jcmturner/goidentity/v6 v6.0.1 // indirect
|
|
github.com/jcmturner/gokrb5/v8 v8.4.4 // indirect
|
|
github.com/jcmturner/rpc/v2 v2.0.3 // indirect
|
|
github.com/klauspost/compress v1.17.4 // indirect
|
|
github.com/kr/fs v0.1.0 // indirect
|
|
github.com/kr/text v0.2.0 // indirect
|
|
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
|
|
github.com/magiconair/properties v1.8.7 // indirect
|
|
github.com/masterzen/simplexml v0.0.0-20190410153822-31eea3082786 // indirect
|
|
github.com/moby/docker-image-spec v1.3.1 // indirect
|
|
github.com/moby/patternmatcher v0.6.0 // indirect
|
|
github.com/moby/sys/sequential v0.5.0 // indirect
|
|
github.com/moby/sys/user v0.1.0 // indirect
|
|
github.com/moby/term v0.5.0 // indirect
|
|
github.com/morikuni/aec v1.0.0 // indirect
|
|
github.com/opencontainers/go-digest v1.0.0 // indirect
|
|
github.com/opencontainers/image-spec v1.1.0 // indirect
|
|
github.com/pkg/errors v0.9.1 // indirect
|
|
github.com/pmezard/go-difflib v1.0.0 // indirect
|
|
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
|
|
github.com/segmentio/asm v1.1.3 // indirect
|
|
github.com/segmentio/encoding v0.5.4 // indirect
|
|
github.com/shirou/gopsutil/v3 v3.23.12 // indirect
|
|
github.com/shoenig/go-m1cpu v0.1.6 // indirect
|
|
github.com/sirupsen/logrus v1.9.3 // indirect
|
|
github.com/stretchr/testify v1.10.0 // indirect
|
|
github.com/tidwall/transform v0.0.0-20201103190739-32f242e2dbde // indirect
|
|
github.com/tklauser/go-sysconf v0.3.12 // indirect
|
|
github.com/tklauser/numcpus v0.6.1 // indirect
|
|
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
|
|
github.com/yusufpapurcu/wmi v1.2.3 // indirect
|
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
|
|
go.opentelemetry.io/otel v1.24.0 // indirect
|
|
go.opentelemetry.io/otel/metric v1.24.0 // indirect
|
|
go.opentelemetry.io/otel/trace v1.24.0 // indirect
|
|
golang.org/x/net v0.47.0 // indirect
|
|
golang.org/x/oauth2 v0.34.0 // indirect
|
|
golang.org/x/sys v0.40.0 // indirect
|
|
golang.org/x/text v0.31.0 // indirect
|
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
|
)
|