mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-24 12:42:38 +00:00
docs(mcp): attribute the cold start — the app is 0.5s of it, the platform is the rest
The boot marks answered the question on the first production cold start, and the answer overturns the reasoning behind an infrastructure change made earlier today. +244ms node started (startup + module loading) +431ms corpora read from disk (+187ms) +491ms index read, validated, hydrated (+60ms for 3.12 MB) +504ms listening +677ms query embedder warm (background) Against a 31.5s cold start, the application accounts for half a second. The other ~98% is Container Apps activation: KEDA reacting, pod scheduling, mounting the 236 MB image, starting the container. The container also boots faster than a development laptop (504ms vs 1273ms), so nothing here is CPU-starved. Consequences recorded rather than quietly dropped: - Raising the container to 0.5 vCPU was justified by the embedding model load on the cold-start path. Prebaked, that load is 173ms and runs in the background where only the first search could wait on it. The justification does not survive measurement. The setting stays — it plausibly helps concurrent requests, and at scale-to-zero the cost difference is negligible — but the Bicep comment now says what is true instead of what was assumed, so nobody re-derives the wrong reason. - Index size is irrelevant to startup (60ms for 3.12 MB), so the corpus can grow freely, and the embedding rounding earns its keep on file size alone. - The only lever that would actually remove the cold start is minReplicas: 1, at roughly 7x the monthly free consumption grant. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+22
-17
@@ -215,29 +215,34 @@ sequence collapses to a single instant however it is queried. The server therefo
|
||||
times itself — `[boot +Nms]` marks from `process.uptime()`, which includes Node's own
|
||||
startup and module loading.
|
||||
|
||||
On a development machine the whole app boot is ~1.3 s:
|
||||
Measured in production on a real cold start (31.5 s to first byte that time):
|
||||
|
||||
```
|
||||
+860ms node started (startup + module loading)
|
||||
+1199ms corpora read from disk (+339ms)
|
||||
+1242ms index read, validated, hydrated (+43ms <- for 3.12 MB)
|
||||
+1273ms listening (+31ms)
|
||||
+1497ms query embedder warm (+224ms, background)
|
||||
+244ms node started (startup + module loading)
|
||||
+431ms corpora read from disk (+187ms)
|
||||
+491ms index read, validated, hydrated (+60ms <- for 3.12 MB)
|
||||
+504ms listening
|
||||
+677ms query embedder warm (background)
|
||||
```
|
||||
|
||||
Two things follow, and both contradict plausible assumptions:
|
||||
**The application accounts for half a second of it.** The other ~31 s — 98% — is
|
||||
Container Apps activation: KEDA reacting to the request, scheduling a pod, mounting
|
||||
the 236 MB image, starting the container. The container also boots *faster* than a
|
||||
development laptop (504 ms against 1273 ms), so nothing here is CPU-starved.
|
||||
|
||||
- **Index size is not the cold-start cost.** Parsing and hydrating 3.12 MB takes 43 ms.
|
||||
Growing the corpus further is cheap.
|
||||
- **The prebaked embedder is not either** — 224 ms, and it loads in the background, so
|
||||
only the first *search* could ever wait on it.
|
||||
Three conclusions, and two of them overturn assumptions this file previously stated:
|
||||
|
||||
The dominant local cost is Node startup and module loading. Whether that, or platform
|
||||
activation (pod scheduling, container start), dominates the production 24.5 s is what
|
||||
the `[boot]` marks answer on the next cold start. If they show the app listening after
|
||||
a few seconds, the remainder is platform time that no CPU or index change can touch —
|
||||
and `minReplicas: 1` would be the only real lever, at roughly 7× the monthly free
|
||||
consumption grant.
|
||||
- **Index size is irrelevant to startup.** 3.12 MB hydrates in 60 ms. Growing the
|
||||
corpus is effectively free, and the embedding rounding earns its keep on file size
|
||||
alone, not on boot time.
|
||||
- **Raising the container to 0.5 vCPU was not justified by cold start.** The stated
|
||||
reason was the embedding model load; prebaked, that is 173 ms and runs in the
|
||||
background, where only the first *search* could ever wait on it. The extra vCPU may
|
||||
still help concurrent requests, but the cold-start argument does not survive
|
||||
measurement.
|
||||
- **The only real lever is `minReplicas: 1`**, which removes activation entirely at
|
||||
roughly 7× the monthly free consumption grant. Shrinking the image would shave the
|
||||
mount, but that is a fraction of platform time, not the bulk of it.
|
||||
|
||||
# Cross-repo coupling
|
||||
|
||||
|
||||
+15
-7
@@ -192,13 +192,21 @@ resource containerApp 'Microsoft.App/containerApps@2024-03-01' = {
|
||||
{
|
||||
name: 'mcp-server'
|
||||
image: '${acr.properties.loginServer}/${containerAppName}:${imageTag}'
|
||||
// 0.5 vCPU is deliberate, not a default. The corpus embeddings are
|
||||
// precomputed at image build time, so steady-state search is cheap —
|
||||
// but a scale-to-zero cold start still loads the ~23 MB ONNX embedding
|
||||
// model to embed the incoming QUERY, and that is seconds of pure CPU on
|
||||
// the critical path of the first search. Doubling the vCPU roughly
|
||||
// halves it. Container Apps only accepts fixed CPU/memory pairs, so the
|
||||
// memory moves with it. Do not "optimize" this back to 0.25.
|
||||
// 0.5 vCPU, but NOT for the reason originally given. It was raised from
|
||||
// 0.25 to shorten the scale-to-zero cold start, on the assumption that
|
||||
// loading the ONNX embedding model dominated it. Measurement disproved
|
||||
// that: the app boots in 504 ms in production (see the [boot] marks in
|
||||
// the container log and docs/mcp/docs-corpus.md), of which the prebaked
|
||||
// embedder is 173 ms in the background. Of a 31 s cold start, ~98% is
|
||||
// Container Apps activation — pod scheduling, image mount, container
|
||||
// start — which no CPU setting touches.
|
||||
//
|
||||
// It stays at 0.5 because it plausibly helps CONCURRENT requests
|
||||
// (maxReplicas is 1, so one replica serves up to 10 in-flight calls) and
|
||||
// because at scale-to-zero the container bills only while active, making
|
||||
// the difference negligible. If cold start ever needs to actually go
|
||||
// away, the only lever is minReplicas: 1. Container Apps accepts fixed
|
||||
// CPU/memory pairs, so memory moves with the CPU.
|
||||
resources: {
|
||||
cpu: json('0.5')
|
||||
memory: '1.0Gi'
|
||||
|
||||
Reference in New Issue
Block a user