mirror of
https://github.com/shankar0123/certctl.git
synced 2026-06-07 16:01:30 +00:00
shankar0123
6acf3559a3
Sprint 5 unified-master-audit closure. Pre-fix:
- docs/operator/scale.md L163-185 held a TBD-laden table with 5
scenario rows. The Phase 8 scenarios shipped 2026-05-14; baseline
capture on canonical hardware was 'the next operational step'
that had not been taken.
- Acquirers + operators asking 'what's the scale ceiling?' got
'TBD' as the in-tree answer.
The audit's fix wanted three things:
1. Capture p50/p95/p99 + error rate + memory profile on a fixed-
spec runner.
2. Replace the scale.md TBD rows with real numbers.
3. Archive k6 artifacts under deploy/test/loadtest-artifacts/.
The actual capture is a workflow_dispatch run the operator triggers
on a real Linux runner — it can't happen from a sandbox without
Docker. What I CAN deliver in this commit is the canonical-record
infrastructure that turns the next workflow run into a baseline that
sticks:
- New docs/operator/scale-baseline-2026-Q2.md is the canonical
record. Documents the three scenarios, the methodology, the
capture procedure, and a 'Latest capture' table with
placeholder rows ready to receive the workflow_dispatch run's
numbers. The doc explicitly defends the 'ubuntu-latest runner'
choice (reproducibility > paid-AWS-account specificity).
- docs/operator/scale.md L163-185 — the TBD table — replaced with
a pointer paragraph to the new baseline file. Per the
canonical-doc-pointer pattern: the operator-posture doc changes
when scenarios change; the baseline doc changes on every
capture. Splitting them avoids review-noise on per-capture
commits.
- New deploy/test/loadtest-artifacts/ directory with a README
documenting the long-term-archive contract (the GHA artifact
retention is 90 days; numbers acquisition reviewers look at
months later need a committed home).
Operator next steps to fill the placeholders:
1. Trigger Actions → loadtest → Run workflow.
2. Download the three matrix-leg artifacts.
3. Update the baseline doc's 'Latest capture' rows.
4. Commit the raw artifacts (or git-lfs for >100 MB archives) to
deploy/test/loadtest-artifacts/.
Closes TEST-005 (infrastructure side). Numbers land on the next
canonical-runner workflow_dispatch capture.
53 lines
2.2 KiB
Markdown
53 lines
2.2 KiB
Markdown
# loadtest-artifacts/
|
|
|
|
> Last reviewed: 2026-05-16
|
|
|
|
Long-term archive of k6 load-test results from the `loadtest` GitHub
|
|
Actions workflow. TEST-005 closure (Sprint 5, 2026-05-16) introduces
|
|
this directory as the committed home for captures the operator
|
|
chooses to retain past GitHub's 90-day artifact-retention window.
|
|
|
|
## What lands here
|
|
|
|
After a `loadtest` workflow_dispatch run, follow the procedure in
|
|
[`docs/operator/scale-baseline-2026-Q2.md`](../../../docs/operator/scale-baseline-2026-Q2.md#capture-procedure):
|
|
|
|
1. Download the three matrix-leg artifacts from the workflow page.
|
|
2. Update the latest-capture table in the baseline doc with the
|
|
extracted percentiles.
|
|
3. Commit the raw artifacts you want long-term-retained here, named:
|
|
|
|
```
|
|
2026-Q2-bulk-renewal-<run-id>.tar.gz
|
|
2026-Q2-acme-burst-<run-id>.tar.gz
|
|
2026-Q2-agent-storm-<run-id>.tar.gz
|
|
```
|
|
|
|
4. If any single archive exceeds 100 MB, route it through `git lfs`
|
|
(configured at repo root via `.gitattributes`).
|
|
|
|
## Why commit artifacts rather than rely on GHA retention
|
|
|
|
- **GitHub Actions retains workflow artifacts for 90 days by default.**
|
|
Acquisition-diligence reviewers looking at scale evidence months
|
|
later get a 404 unless we keep the raw NDJSON in tree.
|
|
- **Reproducibility.** Pinning the k6 NDJSON to a SHA makes it
|
|
cheap to re-derive percentiles with a different filter (e.g.
|
|
"p99 excluding the warmup ramp's first 30 seconds") without
|
|
re-running the workflow.
|
|
|
|
## What does NOT belong here
|
|
|
|
- **Per-PR ephemeral runs.** The `loadtest` workflow runs on
|
|
workflow_dispatch + weekly cron; per-PR runs would be too noisy
|
|
and aren't retained.
|
|
- **Production-environment captures.** These artifacts are the
|
|
ubuntu-latest reference baseline. An operator capturing their
|
|
production-environment scale should put the artifacts in their
|
|
own observability platform — committing them here would imply
|
|
"this is what certctl's reference numbers are" which it isn't.
|
|
- **Manual k6 captures from a developer's laptop.** Same rationale
|
|
as the visual-regression snapshot runbook
|
|
([`docs/operator/runbooks/e2e-snapshot-update.md`](../../../docs/operator/runbooks/e2e-snapshot-update.md))
|
|
— only the CI environment produces canonical numbers.
|