mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-10-25 21:20:25 +00:00
ca90588e4e
* fix(gitops): bound GitOps history with a retention window `gitops_history` is insert-only and the reconcile tick appends to it continuously, so it grew for the life of an installation. A new `gitops_history_retention_days` setting (default 30, range 1 to 365) seeds with the other GitOps settings and prunes history from the monitor's cleanup pass. Drained outbox rows are pruned with the history they announced; an undrained row gets a grace period past the window because the boot repair can still drain it from its payload, and a row still undrained after the grace is counted, logged, and dropped so it cannot accumulate for ever. The window is instance-local, like the artifact retry interval beside it: the history being pruned is the one this instance holds, and the monitor that prunes it runs here. * fix(gitops): keep each live application's newest history row through the prune `cleanupOldGitOpsHistory` deleted `gitops_history` purely by age, but `latestTransitionByApplication` reads `MAX(created_at)` per application and feeds the "last activity" fact the portfolio and application sheet render. An application with no history inside the window, because polling is off, a source is suspended, a node is detached, or a Blueprint is idle, lost every row and its activity column silently went empty. The prune now exempts each live application's newest row, so the claim the JSDoc, the PR body and the settings helper text already make is true. Scoped to live applications rather than to every application_id: a tombstone in `gitops_applications` is permanent, so an unscoped exemption would retain one history row per stack ever retired and leave exactly the growth this setting exists to stop. A retired application's rows are now pruned in full. Also folds in three follow-ups from the audit: - The prune is chunked to GITOPS_PRUNE_CHUNK_ROWS rows per table per pass, so a first prune on a long-lived install clears its backlog over several janitor ticks instead of holding SQLite's single writer for the whole delete. The method reports historyRemaining and outboxRemaining so the debug log says whether a pass is still catching up. - The outbox OR-delete is two statements, so the undrained branch uses idx_gitops_settled_outbox_undrained instead of scanning for the drained one. - The retention tests assert exact counts and exact surviving row ids rather than toBeGreaterThanOrEqual over a shared database. New tests pin each guard: a quiet application's newest row survives every window, its older rows are pruned, a tombstoned application clears entirely, an unsettled attempt is still recoverable after a prune both inside the window and outside it, and a backlog clears in bounded passes with the remainder reported. Removing the newest-row exemption turns 10 of the 11 tests red; removing only the liveness clause turns the tombstone test red. Docs: the GitOps portfolio page describes the window and its setting, and now states that the newest entry per application is kept, which is what the code does. * test(gitops): drop the unused db handles from the recovery assertions * fix(gitops): keep a settled history row ahead of its started row A reconcile attempt stamps both rows with the same time, and the prune broke that tie on a random id, so it could delete the settlement and leave startup recovery to reopen the attempt. Recency now follows insertion order, in the exemption and in each chunk. Remaining counts run only while developer mode is on, drained outbox deletes use an age index, and the settings copy says the kept row is the newest one for each live application. * fix(gitops): log the undrained rows a history prune actually deletes A pass is capped, so the warning was reporting the whole backlog. The history-window tests now fail if the permission, the page save, the monitor window, or the getter fallback stops matching the code. * fix(gitops): prune a superseded unsettled attempt past the window An unsettled reservation survives past the retention window only while it is that live application's newest row. Creating applications keep that row; detached applications do not. * fix(gitops): skip the history prune when the retention setting cannot be read A failed settings read used to fall back to 30 days and delete rows the operator had chosen to keep. The getter now throws, and that pass deletes nothing. A missing or unusable stored value still uses 30. * fix(gitops): restore the history window when the retention spec fails The retention spec restored the seeded 30 days as its last statement, not in a finally, so a failed assertion left the shared instance on 14. The CI retry then read 14 and failed the fresh-install assertion, turning one failure into a persistent red. Everything after the first save now runs inside a try whose finally puts the window back, so the spec leaves the instance as it found it whether it passes or fails. * docs(settings): list the GitOps history window in the retention reference The Data Retention table listed every other retention window and not the GitOps one, so an operator checking retention windows did not find it. The portfolio page described the setting in a paragraph, which is not where retention windows are looked up. Adds the row with its default and range, and a line under the table saying the window is set per Sencho instance and where the control lives. * fix(gitops): report an unmeasured prune remainder as unknown The prune reported a remainder of 0 whenever developer mode was off, so the diagnostic log read "none left" for a table it had never counted. It now reports null, and the log line says the remainder was not measured. Three smaller corrections from the same review: - The diagnostics allowlist omitted the history window, so a support bundle could not explain a timeline that had already been pruned. - The GitOps registry entry described every control as belonging to this node, while the retry interval and the history window are instance local. The sentence now names no scope it does not have. * fix(gitops): correct what the history window is documented to govern The copy said the window governs notifications. It does not: the prune deletes history rows and their outbox rows, and the bell entries those outbox rows announce age out under the Notification log window instead. An operator setting 7 days would have expected their bell to follow. It also promised to keep the database small, which nothing delivers. The prune stops old rows from being written; it never reclaims the file. - The helper now names the record behind the GitOps timeline, points bell entries at the Notification log, and says the window stops history from growing. - The settings reference gains the GitOps section it never had, with the window under it rather than under a per-node table that ends in a Save button the control does not use. - The portfolio page stops claiming a shorter window never changes what an application is doing: a webhook redelivery reads this same record, which the same paragraph already said. - A fraction below one day now clamps to a day instead of flooring to zero, which as a cutoff is now and would delete everything not exempt. The only caller passes an integer, so this is a guard, not a reachable fix. * fix(settings): snap a typed number chip to its own step on commit The chip clamped to min and max but never rounded, so typing 1.5 wrote "1.5" and every integer setting rejected it at the route with a bare "Validation failed" toast. Nothing was saved, so no data was at risk, but the control accepted input it could not store. Rounding to the control's step fixes that and leaves a genuinely fractional chip alone: the Docker janitor asks for 0.5 GiB and keeps it. Clamping still runs after rounding, so a fraction below the floor cannot round down under it. The settings reference also listed GitOps last under Infrastructure while the sidebar renders it third. * docs(gitops): replace the unsourced window figure with what the code does The helper and both docs said a window under about 3 days can let a Git host retry a webhook. Nothing in the code picks three days, and the dedupe the sentence describes only applies to the direct webhook path. The text now states the condition instead of a number: a redelivery can run the work again once the record of its first delivery is pruned. A provider endpoint keeps its own delivery record, pruned to a fixed depth, so the window does not affect it, and the sentence says so. * docs(settings): say where a remote instance's window is set The reference explained that the window belongs to an instance rather than a node, but not how to change the window of a remote one, which is on that instance.
719 lines
29 KiB
TypeScript
719 lines
29 KiB
TypeScript
/**
|
|
* DatabaseService executes this at init, then separately seeds
|
|
* gitops_schema_version, adds the gitops_* columns to
|
|
* stack_update_recovery_generations, and adds deployed_generation_id to
|
|
* health_gate_runs. Those three live outside this string because they alter
|
|
* pre-existing tables rather than creating GitOps ones.
|
|
*/
|
|
|
|
export const GITOPS_SCHEMA_SQL = `
|
|
CREATE TABLE IF NOT EXISTS gitops_migration_checkpoints (
|
|
scope TEXT PRIMARY KEY,
|
|
schema_version INTEGER NOT NULL,
|
|
fingerprint TEXT NOT NULL,
|
|
migrated_at INTEGER NOT NULL
|
|
);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_create_checkpoints (
|
|
application_id TEXT PRIMARY KEY,
|
|
stack_name TEXT NOT NULL,
|
|
phase TEXT NOT NULL CHECK (phase IN (
|
|
'pre_stack','stack_created','promoting','manifest_committed','pointers_committed'
|
|
)),
|
|
generation_id TEXT NULL,
|
|
operation_id TEXT NOT NULL,
|
|
repo_url TEXT NOT NULL,
|
|
branch TEXT NOT NULL,
|
|
compose_path TEXT NOT NULL,
|
|
compose_paths_json TEXT NOT NULL,
|
|
context_dir TEXT NULL,
|
|
sync_env INTEGER NOT NULL DEFAULT 0,
|
|
env_path TEXT NULL,
|
|
auth_type TEXT NOT NULL,
|
|
encrypted_token TEXT NULL,
|
|
encrypted_deploy_key TEXT NULL,
|
|
ssh_known_hosts_entry TEXT NULL,
|
|
ssh_host_key_fingerprint TEXT NULL,
|
|
encrypted_ca_bundle TEXT NULL,
|
|
auto_apply_on_webhook INTEGER NOT NULL DEFAULT 0,
|
|
auto_deploy_on_apply INTEGER NOT NULL DEFAULT 0,
|
|
commit_sha TEXT NULL,
|
|
applied_spec_json TEXT NULL,
|
|
created_managed_root INTEGER NOT NULL DEFAULT 0,
|
|
created_at INTEGER NOT NULL,
|
|
updated_at INTEGER NOT NULL
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_create_ck_stack
|
|
ON gitops_create_checkpoints(stack_name);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_create_ck_phase
|
|
ON gitops_create_checkpoints(phase);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_applications (
|
|
id TEXT PRIMARY KEY,
|
|
lifecycle_key TEXT NOT NULL,
|
|
lifecycle_status TEXT NOT NULL CHECK (lifecycle_status IN (
|
|
'active','creating','detached','deleted'
|
|
)),
|
|
target_mode TEXT NOT NULL CHECK (target_mode IN ('direct','inline_blueprint','blueprint')),
|
|
stack_name TEXT NULL,
|
|
configured_source_stack_name TEXT NULL,
|
|
blueprint_id INTEGER NULL,
|
|
configured_repo_url TEXT NULL,
|
|
repo_identity_json TEXT NULL,
|
|
configured_ref TEXT NULL,
|
|
compose_paths_json TEXT NULL,
|
|
context_dir TEXT NULL,
|
|
sync_env INTEGER NULL,
|
|
env_path TEXT NULL,
|
|
materialization_fingerprint TEXT NULL,
|
|
desired_commit_sha TEXT NULL,
|
|
fetched_commit_sha TEXT NULL,
|
|
fetched_resolved_ref_kind TEXT NULL CHECK (
|
|
fetched_resolved_ref_kind IS NULL OR fetched_resolved_ref_kind IN ('branch','tag','sha')
|
|
),
|
|
candidate_generation_id TEXT NULL,
|
|
accepted_generation_id TEXT NULL,
|
|
candidate_plan_blocked INTEGER NOT NULL DEFAULT 0,
|
|
review_required INTEGER NOT NULL DEFAULT 0,
|
|
review_block_reason TEXT NULL,
|
|
artifact_set_id TEXT NULL,
|
|
latest_artifact_set_id TEXT NULL,
|
|
intent_revision_id TEXT NULL,
|
|
rollout_candidate_id TEXT NULL,
|
|
rollout_generation_id TEXT NULL,
|
|
source_acceptance_ref TEXT NULL,
|
|
placement_approval_ref TEXT NULL,
|
|
rollout_authorization_ref TEXT NULL,
|
|
legacy_combined_approval_ref TEXT NULL,
|
|
preflight_fingerprint TEXT NULL,
|
|
-- Latest registry/capability preflight evaluation for derive. Independent of
|
|
-- the fingerprint frozen into a live rollout_authorization.
|
|
latest_preflight_evidence_json TEXT NULL,
|
|
latest_operation_id TEXT NULL,
|
|
active_operation_id TEXT NULL,
|
|
active_operation_stage TEXT NULL CHECK (
|
|
active_operation_stage IS NULL OR active_operation_stage IN (
|
|
'fetch_started','apply_started','deploy_started','recovery_started'
|
|
)
|
|
),
|
|
active_operation_at INTEGER NULL,
|
|
active_generation_id TEXT NULL,
|
|
pause_at INTEGER NULL,
|
|
pause_reason TEXT NULL,
|
|
-- Who placed the pause. A system hold (a refused dispatch) belongs to the
|
|
-- rollout it was holding, so a source acceptance clears it when it supersedes
|
|
-- that rollout; an operator pause and a health-policy hold survive.
|
|
pause_origin TEXT NOT NULL DEFAULT 'operator' CHECK (pause_origin IN ('operator','system','health')),
|
|
-- Distinct from pause_reason: sourceSuspended/sourceUnsuspended write this
|
|
-- field, not the one rolloutPaused/rolloutUnpaused share across app and
|
|
-- target rows, so suspending a source can never clobber an unrelated
|
|
-- rollout pause reason (or the reverse).
|
|
source_suspended_reason TEXT NULL,
|
|
-- Controller-owned bookkeeping. NULL poll_interval_secs inherits the
|
|
-- global default; 0 disables polling for this application. next_poll_at
|
|
-- is the durable scheduling cursor. attempt_seq is allocated
|
|
-- transactionally per submission lacking a stable external delivery id.
|
|
source_policy TEXT NOT NULL DEFAULT 'manual' CHECK (
|
|
source_policy IN ('manual','review','automatic')
|
|
),
|
|
-- Placement policy, independent of source policy. operator is the only safe
|
|
-- default: automatic placement moves workloads on nodes, and an install that
|
|
-- has never asked to be automated must not start.
|
|
placement_policy TEXT NOT NULL DEFAULT 'operator' CHECK (
|
|
placement_policy IN ('operator','bounded_auto')
|
|
),
|
|
-- Whether rollout authorization is minted by policy or waits for an operator.
|
|
-- New installs wait, because an install with no rollout history has no
|
|
-- evidence that any target can take the generation. Existing installations
|
|
-- are a separate case, handled by the one-time backfill in the migration,
|
|
-- because they already authorized themselves.
|
|
rollout_authorization_policy TEXT NOT NULL DEFAULT 'manual' CHECK (
|
|
rollout_authorization_policy IN ('manual','automatic')
|
|
),
|
|
-- Why bounded automatic placement last declined, and when. Null when no
|
|
-- bounded-auto evaluation has declined, which is the common case and includes
|
|
-- every application still on the operator policy.
|
|
--
|
|
-- Recorded because the decision is otherwise invisible once the producer
|
|
-- returns. A placement left waiting for review under an automatic policy with
|
|
-- no stated reason is indistinguishable from a policy that does nothing, and
|
|
-- that is exactly how an automatic path reads when it is broken.
|
|
--
|
|
-- The vocabulary is the closed decision vocabulary and not a free string, so
|
|
-- a reason that no decision can produce cannot be stored or projected. Cleared
|
|
-- when the review is resolved, and when the policy changes, because a reason
|
|
-- that described the old policy describes nothing now.
|
|
placement_policy_refusal_reason TEXT NULL CHECK (
|
|
placement_policy_refusal_reason IS NULL OR placement_policy_refusal_reason IN (
|
|
'policy_is_operator','no_placement_change','stateless_addition','stateless_removal',
|
|
'mixed_add_and_remove','multiple_additions','multiple_removals','stateful_workload',
|
|
'unknown_workload','first_multi_node_placement','pin_driven_placement','cordon_override','cordon_driven_removal',
|
|
'stale_node','unknown_connectivity','missing_evidence','malformed_evidence',
|
|
'destructive_effect','conflicting_operation'
|
|
)
|
|
),
|
|
placement_policy_refused_at INTEGER NULL,
|
|
poll_interval_secs INTEGER NULL,
|
|
next_poll_at INTEGER NULL,
|
|
attempt_seq INTEGER NOT NULL DEFAULT 0,
|
|
partial_json TEXT NULL,
|
|
failure_stage TEXT NULL CHECK (
|
|
failure_stage IS NULL OR failure_stage IN (
|
|
'fetch','validation','apply','create','recovery'
|
|
)
|
|
),
|
|
failure_class TEXT NULL,
|
|
failure_at INTEGER NULL,
|
|
retry_at INTEGER NULL,
|
|
retry_count INTEGER NOT NULL DEFAULT 0,
|
|
suspended_at INTEGER NULL,
|
|
recovery_ref TEXT NULL,
|
|
recovery_phase TEXT NULL CHECK (
|
|
recovery_phase IS NULL OR recovery_phase IN (
|
|
'capturing','restoring','compensating','complete','failed'
|
|
)
|
|
),
|
|
interruption_stage TEXT NULL CHECK (
|
|
interruption_stage IS NULL OR interruption_stage IN (
|
|
'fetch_started','apply_started','deploy_started','recovery_started'
|
|
)
|
|
),
|
|
interruption_at INTEGER NULL,
|
|
interruption_operation_id TEXT NULL,
|
|
interruption_generation_id TEXT NULL,
|
|
evidence_fresh_at INTEGER NULL,
|
|
-- Why this row could not prove something, recorded at write time. Read-time
|
|
-- limitations are derived; these are the ones only the writer knows.
|
|
evidence_limitations_json TEXT NULL,
|
|
created_at INTEGER NOT NULL,
|
|
updated_at INTEGER NOT NULL,
|
|
CHECK (target_mode != 'direct' OR (stack_name IS NOT NULL AND blueprint_id IS NULL)),
|
|
CHECK (target_mode != 'inline_blueprint' OR (blueprint_id IS NOT NULL AND stack_name IS NULL AND configured_repo_url IS NULL)),
|
|
CHECK (target_mode != 'blueprint' OR (blueprint_id IS NOT NULL AND stack_name IS NULL AND configured_repo_url IS NOT NULL))
|
|
);
|
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_gitops_app_active_direct
|
|
ON gitops_applications(stack_name)
|
|
WHERE lifecycle_status IN ('active','creating') AND target_mode = 'direct';
|
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_gitops_app_active_blueprint_any
|
|
ON gitops_applications(blueprint_id)
|
|
WHERE lifecycle_status IN ('active','creating')
|
|
AND target_mode IN ('inline_blueprint','blueprint');
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_app_lifecycle_key
|
|
ON gitops_applications(lifecycle_key);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_app_status
|
|
ON gitops_applications(lifecycle_status);
|
|
-- The two unique indexes above are partial on the live rows, so neither serves
|
|
-- a lookup for a detached one. Without this the drift route's fallback is a
|
|
-- full scan and a sort. Direct only: Blueprint retirement writes 'deleted',
|
|
-- never 'detached', so the Blueprint equivalent would index an empty set.
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_app_detached_direct
|
|
ON gitops_applications(stack_name, updated_at DESC)
|
|
WHERE lifecycle_status = 'detached' AND target_mode = 'direct';
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_generations (
|
|
id TEXT PRIMARY KEY,
|
|
application_id TEXT NOT NULL,
|
|
commit_sha TEXT NOT NULL,
|
|
repo_url TEXT NOT NULL,
|
|
configured_ref TEXT NOT NULL,
|
|
resolved_ref_kind TEXT NULL CHECK (
|
|
resolved_ref_kind IS NULL OR resolved_ref_kind IN ('branch','tag','sha')
|
|
),
|
|
repo_identity_json TEXT NOT NULL,
|
|
manifest_version INTEGER NOT NULL,
|
|
candidate_dir TEXT NOT NULL,
|
|
applied_dir TEXT NOT NULL,
|
|
expected_invocation_json TEXT NOT NULL,
|
|
materialization_fingerprint TEXT NOT NULL,
|
|
validation_ok INTEGER NOT NULL,
|
|
plan_blocked INTEGER NOT NULL DEFAULT 0,
|
|
change_plan_fingerprint TEXT NULL,
|
|
operation_id TEXT NOT NULL,
|
|
trigger TEXT NOT NULL,
|
|
actor TEXT NULL,
|
|
previous_generation_id TEXT NULL,
|
|
redacted_limitations_json TEXT NOT NULL DEFAULT '[]',
|
|
-- Portable accepted-generation contract (content only: no node id, local
|
|
-- path, target mode, or secret value). Additive and nullable so existing
|
|
-- rows decode as an explicit limitation rather than invented evidence; a
|
|
-- legacy pending candidate lacking these must be re-evaluated before it
|
|
-- can be accepted or dispatched.
|
|
portable_manifest_json TEXT NULL,
|
|
compose_inputs_json TEXT NULL,
|
|
source_policy_evidence_json TEXT NULL,
|
|
security_policy_evidence_json TEXT NULL,
|
|
support_requirements_json TEXT NULL,
|
|
compatibility_requirements_json TEXT NULL,
|
|
secret_capability_json TEXT NULL,
|
|
created_at INTEGER NOT NULL
|
|
);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_sops_identities (
|
|
id TEXT PRIMARY KEY,
|
|
application_id TEXT NOT NULL,
|
|
stack_name TEXT NOT NULL,
|
|
recipient TEXT NOT NULL,
|
|
encrypted_identity TEXT NOT NULL,
|
|
label TEXT NULL,
|
|
created_at INTEGER NOT NULL,
|
|
rotated_at INTEGER NULL
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_sops_id_app_stack
|
|
ON gitops_sops_identities(application_id, stack_name);
|
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_gitops_sops_id_recipient
|
|
ON gitops_sops_identities(application_id, stack_name, recipient);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_gen_app_created
|
|
ON gitops_generations(application_id, created_at);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_gen_sha
|
|
ON gitops_generations(commit_sha);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_gen_op
|
|
ON gitops_generations(operation_id);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_gen_repo_ref
|
|
ON gitops_generations(repo_url, configured_ref);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_artifact_sets (
|
|
id TEXT PRIMARY KEY,
|
|
generation_id TEXT NOT NULL,
|
|
evidence_version INTEGER NOT NULL,
|
|
authoritative INTEGER NOT NULL DEFAULT 0,
|
|
qualification TEXT NOT NULL CHECK (qualification IN (
|
|
'unresolved','exact','qualified','stale','unavailable','local_build_unverified'
|
|
)),
|
|
evidence_json TEXT NOT NULL DEFAULT '{}',
|
|
created_at INTEGER NOT NULL,
|
|
UNIQUE (generation_id, evidence_version)
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_artifact_gen
|
|
ON gitops_artifact_sets(generation_id, evidence_version);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_intent_revisions (
|
|
id TEXT PRIMARY KEY,
|
|
application_id TEXT NOT NULL,
|
|
blueprint_id INTEGER NOT NULL,
|
|
compose_content_sha256 TEXT NOT NULL,
|
|
blueprint_revision INTEGER NOT NULL,
|
|
deploy_stack_name TEXT NOT NULL,
|
|
selector_json TEXT NOT NULL,
|
|
pinned_node_id INTEGER NULL,
|
|
cordon_implications_json TEXT NOT NULL DEFAULT '[]',
|
|
rollout_strategy_json TEXT NOT NULL DEFAULT '{}',
|
|
runtime_drift_policy TEXT NULL,
|
|
stateful_policy_json TEXT NULL,
|
|
health_failure_rollback_policy_json TEXT NULL,
|
|
operation_id TEXT NOT NULL,
|
|
actor TEXT NULL,
|
|
created_at INTEGER NOT NULL
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_intent_app_created
|
|
ON gitops_intent_revisions(application_id, created_at);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_intent_blueprint
|
|
ON gitops_intent_revisions(blueprint_id);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_intent_content
|
|
ON gitops_intent_revisions(compose_content_sha256);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_rollout_candidates (
|
|
id TEXT PRIMARY KEY,
|
|
application_id TEXT NOT NULL,
|
|
intent_revision_id TEXT NOT NULL,
|
|
compose_content_sha256 TEXT NOT NULL,
|
|
accepted_generation_id TEXT NULL,
|
|
artifact_set_id TEXT NULL,
|
|
required_targets_json TEXT NOT NULL,
|
|
authoritative INTEGER NOT NULL DEFAULT 0,
|
|
provenance TEXT NOT NULL CHECK (provenance IN (
|
|
'intent_change','roster_change','legacy_inline'
|
|
)),
|
|
operation_id TEXT NOT NULL,
|
|
created_at INTEGER NOT NULL
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_rollout_app
|
|
ON gitops_rollout_candidates(application_id, created_at);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_rollout_generations (
|
|
id TEXT PRIMARY KEY,
|
|
application_id TEXT NOT NULL,
|
|
intent_revision_id TEXT NOT NULL,
|
|
rollout_candidate_id TEXT NOT NULL,
|
|
accepted_generation_id TEXT NULL,
|
|
artifact_set_id TEXT NULL,
|
|
placement_approval_ref TEXT NULL,
|
|
source_acceptance_ref TEXT NULL,
|
|
rollout_authorization_ref TEXT NULL,
|
|
required_targets_json TEXT NOT NULL,
|
|
preflight_fingerprint TEXT NULL,
|
|
preflight_evidence_json TEXT NULL,
|
|
rollout_strategy_json TEXT NOT NULL DEFAULT '{}',
|
|
-- The policy snapshot this generation executes, frozen at the moment the
|
|
-- generation opened. Nullable with no default on purpose: a row that predates
|
|
-- the column is SQL NULL, which the decoder reconstructs as the legacy
|
|
-- behavior those rollouts actually had. An empty-object default would make
|
|
-- absence impossible and turn every existing generation into a decode failure.
|
|
policy_snapshot_json TEXT NULL,
|
|
provenance TEXT NOT NULL CHECK (provenance IN (
|
|
'legacy_inline','placement_approval','rollout_authorization'
|
|
)),
|
|
supersedes_generation_id TEXT NULL,
|
|
superseded_at INTEGER NULL,
|
|
-- Set only by the operator's Supersede or Rollback. A system supersede (a
|
|
-- preflight drift, a placement invalidation, a re-authorization) leaves this
|
|
-- null, which is what lets the automatic policy tell a withdrawal from a
|
|
-- supersede it may re-mint.
|
|
withdrawn_at INTEGER NULL,
|
|
operation_id TEXT NOT NULL,
|
|
actor TEXT NULL,
|
|
trigger TEXT NOT NULL,
|
|
created_at INTEGER NOT NULL
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_rollout_gen_app
|
|
ON gitops_rollout_generations(application_id, created_at);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_approvals (
|
|
id TEXT PRIMARY KEY,
|
|
kind TEXT NOT NULL CHECK (kind IN (
|
|
'source_acceptance','placement_approval','rollout_authorization','legacy_combined'
|
|
)),
|
|
authority TEXT NOT NULL CHECK (authority IN (
|
|
'operator','configured_policy','legacy_combined'
|
|
)),
|
|
authoritative INTEGER NOT NULL DEFAULT 0,
|
|
application_id TEXT NOT NULL,
|
|
generation_id TEXT NULL,
|
|
intent_revision_id TEXT NULL,
|
|
artifact_set_id TEXT NULL,
|
|
rollout_candidate_id TEXT NULL,
|
|
rollout_generation_id TEXT NULL,
|
|
source_acceptance_ref TEXT NULL,
|
|
placement_approval_ref TEXT NULL,
|
|
required_targets_json TEXT NULL,
|
|
preflight_fingerprint TEXT NULL,
|
|
fingerprint TEXT NULL,
|
|
blast_json TEXT NULL,
|
|
policy_provenance_json TEXT NULL,
|
|
actor TEXT NULL,
|
|
created_at INTEGER NOT NULL,
|
|
CHECK (
|
|
kind != 'source_acceptance' OR (
|
|
authoritative = 1
|
|
AND authority IN ('operator','configured_policy')
|
|
AND generation_id IS NOT NULL
|
|
)
|
|
),
|
|
CHECK (
|
|
kind != 'placement_approval' OR (
|
|
authoritative = 1
|
|
AND authority IN ('operator','configured_policy')
|
|
AND intent_revision_id IS NOT NULL
|
|
AND blast_json IS NOT NULL
|
|
)
|
|
),
|
|
CHECK (
|
|
kind != 'rollout_authorization' OR (
|
|
authoritative = 1
|
|
AND authority IN ('operator','configured_policy')
|
|
AND generation_id IS NOT NULL
|
|
AND artifact_set_id IS NOT NULL
|
|
AND intent_revision_id IS NOT NULL
|
|
AND rollout_candidate_id IS NOT NULL
|
|
AND source_acceptance_ref IS NOT NULL
|
|
AND placement_approval_ref IS NOT NULL
|
|
AND required_targets_json IS NOT NULL
|
|
AND preflight_fingerprint IS NOT NULL
|
|
)
|
|
),
|
|
CHECK (
|
|
kind != 'legacy_combined' OR (
|
|
authoritative = 0
|
|
AND authority = 'legacy_combined'
|
|
)
|
|
)
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_approval_app
|
|
ON gitops_approvals(application_id, created_at);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_target_current (
|
|
application_id TEXT NOT NULL,
|
|
node_id INTEGER NOT NULL,
|
|
target_status TEXT NOT NULL CHECK (target_status IN ('active','tombstoned')),
|
|
desired_generation_id TEXT NULL,
|
|
candidate_generation_id TEXT NULL,
|
|
applied_generation_id TEXT NULL,
|
|
deployed_generation_id TEXT NULL,
|
|
healthy_generation_id TEXT NULL,
|
|
-- The last stack-scope verdict, bound to the generation it judged. Kept so a
|
|
-- failed check stays visible as a failure instead of collapsing back into
|
|
-- "not checked yet" once its promotion is withdrawn.
|
|
last_health_status TEXT NULL CHECK (
|
|
last_health_status IS NULL OR last_health_status IN ('passed','failed','unknown')
|
|
),
|
|
last_health_generation_id TEXT NULL,
|
|
last_health_run_id TEXT NULL,
|
|
-- The health run this target is currently awaiting a verdict from, allocated
|
|
-- before its apply was dispatched. A non-null value is the queue's sole
|
|
-- idempotency key: only that run may consume the target or mutate its health
|
|
-- state, so a duplicate or out-of-order report records evidence and nothing
|
|
-- else. Every policy allocates one, observe included: the run is what makes
|
|
-- 'record the outcome' mean anything, and observe's own decision is a no-op,
|
|
-- so it changes no other state.
|
|
pending_health_run_id TEXT NULL,
|
|
-- How many times this target has been retried under the same rollout
|
|
-- generation. Reset when rollout_generation_id changes, which is what makes
|
|
-- retry_once exactly once per target and generation.
|
|
health_attempts INTEGER NOT NULL DEFAULT 0,
|
|
-- Why this rollout stopped advancing, recorded before any external restore is
|
|
-- attempted. A restart that lands mid-rollback reads this and holds the
|
|
-- rollout rather than resuming the queue and dispatching the targets that came
|
|
-- after the failed one. It fences the unfinished restore; it does not replay
|
|
-- it, so the restore has to be finished or undone through the manual route.
|
|
health_stop_reason TEXT NULL CHECK (
|
|
health_stop_reason IS NULL OR health_stop_reason IN (
|
|
'health_failed','health_unknown','health_retry_exhausted','rollout_stopped',
|
|
'rollback_unavailable','rollback_completed','rollback_pending','stop_acknowledged','health_passed','health_retried'
|
|
)
|
|
),
|
|
lkg_generation_id TEXT NULL,
|
|
lkg_artifact_set_id TEXT NULL,
|
|
lkg_unavailable_at INTEGER NULL,
|
|
lkg_unavailable_reason TEXT NULL CHECK (
|
|
lkg_unavailable_reason IS NULL OR lkg_unavailable_reason IN (
|
|
'generation_missing','recovery_unretainable'
|
|
)
|
|
),
|
|
expected_artifact_set_id TEXT NULL,
|
|
latest_artifact_set_id TEXT NULL,
|
|
observed_artifact_identity_json TEXT NULL,
|
|
-- What Compose was actually invoked with on this node, read back off the
|
|
-- running project. NULL means Sencho has not looked, never a value written
|
|
-- to stand in for a node that could not be reached. See json.ts.
|
|
observed_invocation_json TEXT NULL,
|
|
intent_revision_id TEXT NULL,
|
|
rollout_candidate_id TEXT NULL,
|
|
rollout_generation_id TEXT NULL,
|
|
source_acceptance_ref TEXT NULL,
|
|
placement_approval_ref TEXT NULL,
|
|
rollout_authorization_ref TEXT NULL,
|
|
legacy_combined_approval_ref TEXT NULL,
|
|
legacy_applied_revision INTEGER NULL,
|
|
connectivity TEXT NULL CHECK (
|
|
connectivity IS NULL OR connectivity IN ('unknown','reachable','unreachable','stale')
|
|
),
|
|
latest_stage TEXT NULL,
|
|
active_operation_id TEXT NULL,
|
|
active_operation_stage TEXT NULL CHECK (
|
|
active_operation_stage IS NULL OR active_operation_stage IN (
|
|
'deploy_started','blueprint_deploy_started','blueprint_withdraw_started','recovery_started'
|
|
)
|
|
),
|
|
active_operation_at INTEGER NULL,
|
|
active_generation_id TEXT NULL,
|
|
active_intent_revision_id TEXT NULL,
|
|
active_rollout_candidate_id TEXT NULL,
|
|
failure_stage TEXT NULL CHECK (
|
|
failure_stage IS NULL OR failure_stage IN (
|
|
'deploy','recovery','blueprint_deploy','blueprint_withdraw'
|
|
)
|
|
),
|
|
failure_class TEXT NULL,
|
|
failure_at INTEGER NULL,
|
|
recovery_ref TEXT NULL,
|
|
recovery_generation_id TEXT NULL,
|
|
recovery_phase TEXT NULL CHECK (
|
|
recovery_phase IS NULL OR recovery_phase IN (
|
|
'capturing','restoring','compensating','complete','failed'
|
|
)
|
|
),
|
|
-- The class and time of the recovery failure this row still claims.
|
|
--
|
|
-- failure_class is shared with deploy and withdraw failures, so a claim that
|
|
-- may have moved the target would lose its class and its timestamp the moment
|
|
-- a later failure wrote the shared slot, and the claim would read as a refusal
|
|
-- that moved nothing. These columns keep the claim's own record, so a failure
|
|
-- can record itself beside it without retiring it, and only success evidence
|
|
-- (an acknowledgement or a bind) or a claim that moved nothing is allowed to
|
|
-- drop it.
|
|
recovery_failure_class TEXT NULL,
|
|
recovery_failure_at INTEGER NULL,
|
|
interruption_stage TEXT NULL CHECK (
|
|
interruption_stage IS NULL OR interruption_stage IN (
|
|
'deploy_started','blueprint_deploy_started','blueprint_withdraw_started','recovery_started'
|
|
)
|
|
),
|
|
interruption_at INTEGER NULL,
|
|
interruption_operation_id TEXT NULL,
|
|
interruption_generation_id TEXT NULL,
|
|
interruption_intent_revision_id TEXT NULL,
|
|
interruption_rollout_candidate_id TEXT NULL,
|
|
pause_at INTEGER NULL,
|
|
pause_reason TEXT NULL,
|
|
retry_at INTEGER NULL,
|
|
suspended_at INTEGER NULL,
|
|
partial_json TEXT NULL,
|
|
-- Why this target could not prove something, recorded at write time.
|
|
evidence_limitations_json TEXT NULL,
|
|
updated_at INTEGER NOT NULL,
|
|
PRIMARY KEY (application_id, node_id),
|
|
CHECK (
|
|
(lkg_unavailable_at IS NULL AND lkg_unavailable_reason IS NULL)
|
|
OR (lkg_unavailable_at IS NOT NULL AND lkg_unavailable_reason IS NOT NULL)
|
|
),
|
|
CHECK (
|
|
lkg_unavailable_at IS NULL
|
|
OR (lkg_generation_id IS NULL AND lkg_artifact_set_id IS NULL)
|
|
)
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_target_status
|
|
ON gitops_target_current(target_status);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_target_node
|
|
ON gitops_target_current(node_id);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_history (
|
|
id TEXT PRIMARY KEY,
|
|
created_at INTEGER NOT NULL,
|
|
application_id TEXT NOT NULL,
|
|
target_mode TEXT NOT NULL,
|
|
lifecycle_key TEXT NOT NULL,
|
|
stack_name TEXT NULL,
|
|
blueprint_id INTEGER NULL,
|
|
node_id INTEGER NULL,
|
|
dedupe_target TEXT NOT NULL,
|
|
repo_url TEXT NULL,
|
|
configured_ref TEXT NULL,
|
|
repo_identity_json TEXT NULL,
|
|
commit_sha TEXT NULL,
|
|
generation_id TEXT NULL,
|
|
artifact_set_id TEXT NULL,
|
|
intent_revision_id TEXT NULL,
|
|
rollout_candidate_id TEXT NULL,
|
|
rollout_generation_id TEXT NULL,
|
|
source_acceptance_ref TEXT NULL,
|
|
placement_approval_ref TEXT NULL,
|
|
rollout_authorization_ref TEXT NULL,
|
|
legacy_combined_approval_ref TEXT NULL,
|
|
operation_id TEXT NOT NULL,
|
|
stage TEXT NOT NULL,
|
|
outcome TEXT NOT NULL CHECK (outcome IN (
|
|
'committed','failed','skipped','superseded','recovered','unknown'
|
|
)),
|
|
trigger TEXT NOT NULL,
|
|
actor TEXT NULL,
|
|
before_json TEXT NOT NULL,
|
|
after_json TEXT NOT NULL,
|
|
required_targets_json TEXT NULL,
|
|
validation_json TEXT NULL,
|
|
per_target_results_json TEXT NULL,
|
|
health_run_id TEXT NULL,
|
|
health_snapshot_json TEXT NULL,
|
|
invocation_observed_json TEXT NULL,
|
|
recovery_ref TEXT NULL,
|
|
redacted_reason_class TEXT NULL
|
|
);
|
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_gitops_history_dedupe
|
|
ON gitops_history(application_id, operation_id, stage, dedupe_target);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_app_created
|
|
ON gitops_history(application_id, created_at DESC, id DESC);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_sha ON gitops_history(commit_sha);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_gen ON gitops_history(generation_id);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_artifact ON gitops_history(artifact_set_id);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_blueprint ON gitops_history(blueprint_id);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_rollout ON gitops_history(rollout_candidate_id);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_rollout_gen ON gitops_history(rollout_generation_id);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_node ON gitops_history(node_id);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_trigger ON gitops_history(trigger);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_actor ON gitops_history(actor);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_outcome ON gitops_history(outcome);
|
|
-- listUnsettledReconcileAttempts filters on stage and orders by created_at;
|
|
-- without this, that query (run on every startup, ahead of the server
|
|
-- listening) scans and sorts the whole table.
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_stage_created ON gitops_history(stage, created_at);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_repo_ref
|
|
ON gitops_history(repo_url, configured_ref);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_stack_created
|
|
ON gitops_history(stack_name, created_at DESC, id DESC);
|
|
-- Serves the cross-stack history page, whose ordering and cursor are on
|
|
-- (created_at, id) with no other filter. Every other index here leads with a
|
|
-- different column, so without this one that route sorts the whole table.
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_history_created
|
|
ON gitops_history(created_at DESC, id DESC);
|
|
|
|
CREATE TABLE IF NOT EXISTS gitops_settled_outbox (
|
|
settled_history_id TEXT PRIMARY KEY,
|
|
payload_json TEXT NOT NULL,
|
|
payload_version INTEGER NOT NULL,
|
|
created_at INTEGER NOT NULL,
|
|
updated_at INTEGER NOT NULL,
|
|
drained_at INTEGER NULL
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_settled_outbox_undrained
|
|
ON gitops_settled_outbox(created_at)
|
|
WHERE drained_at IS NULL;
|
|
-- Drained rows are the steady-state majority. The prune deletes them by age on
|
|
-- the monitor tick, and the undrained partial index above cannot serve that scan.
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_settled_outbox_drained
|
|
ON gitops_settled_outbox(created_at)
|
|
WHERE drained_at IS NOT NULL;
|
|
|
|
CREATE TABLE IF NOT EXISTS git_provider_endpoints (
|
|
id TEXT PRIMARY KEY,
|
|
stack_name TEXT NOT NULL,
|
|
provider TEXT NOT NULL CHECK (provider IN (
|
|
'github', 'gitlab', 'gitea', 'forgejo', 'bitbucket_cloud'
|
|
)),
|
|
encrypted_secret TEXT NOT NULL,
|
|
encrypted_secret_previous TEXT NULL,
|
|
previous_secret_expires_at INTEGER NULL,
|
|
enabled INTEGER NOT NULL DEFAULT 1,
|
|
event_scope TEXT NOT NULL DEFAULT 'configured_ref' CHECK (event_scope IN (
|
|
'configured_ref', 'configured_ref_and_prs'
|
|
)),
|
|
created_at INTEGER NOT NULL,
|
|
updated_at INTEGER NOT NULL,
|
|
UNIQUE(stack_name, provider)
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_git_provider_endpoints_stack
|
|
ON git_provider_endpoints(stack_name);
|
|
|
|
CREATE TABLE IF NOT EXISTS git_provider_deliveries (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
endpoint_id TEXT NOT NULL,
|
|
delivery_id TEXT NOT NULL,
|
|
state TEXT NOT NULL,
|
|
event_type TEXT NULL,
|
|
event_action TEXT NULL,
|
|
ref TEXT NULL,
|
|
candidate_sha TEXT NULL,
|
|
outcome_class TEXT NULL,
|
|
received_at INTEGER NOT NULL,
|
|
updated_at INTEGER NOT NULL,
|
|
FOREIGN KEY(endpoint_id) REFERENCES git_provider_endpoints(id) ON DELETE CASCADE,
|
|
UNIQUE(endpoint_id, delivery_id)
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_git_provider_deliveries_endpoint
|
|
ON git_provider_deliveries(endpoint_id, received_at DESC);
|
|
`;
|
|
|
|
/**
|
|
* Due-scan indexes that mention `configured_source_stack_name`. They cannot
|
|
* live in GITOPS_SCHEMA_SQL: CREATE TABLE IF NOT EXISTS is a no-op on an
|
|
* existing applications table, so a CREATE INDEX in that same blob would
|
|
* run before the additive column exists. DatabaseService execs this after
|
|
* maybeAddCol. Both DROPs precede their CREATEs because CREATE INDEX IF
|
|
* NOT EXISTS never updates an index that already exists under the same name.
|
|
*/
|
|
const GITOPS_DUE_INDEX_WHERE = `(target_mode = 'direct' OR (target_mode = 'blueprint' AND configured_source_stack_name IS NOT NULL))
|
|
AND lifecycle_status = 'active'
|
|
AND suspended_at IS NULL
|
|
AND active_operation_stage IS NULL`;
|
|
|
|
export const GITOPS_DUE_INDEX_SQL = `
|
|
DROP INDEX IF EXISTS idx_gitops_app_poll_due;
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_app_poll_due
|
|
ON gitops_applications(next_poll_at)
|
|
WHERE ${GITOPS_DUE_INDEX_WHERE}
|
|
AND next_poll_at IS NOT NULL
|
|
AND retry_at IS NULL;
|
|
DROP INDEX IF EXISTS idx_gitops_app_retry_due;
|
|
CREATE INDEX IF NOT EXISTS idx_gitops_app_retry_due
|
|
ON gitops_applications(retry_at)
|
|
WHERE ${GITOPS_DUE_INDEX_WHERE}
|
|
AND retry_at IS NOT NULL;
|
|
`;
|