mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-21 19:56:33 +00:00
ae6fcfe7a1
* chore(ansible): add dev-status playbook (read-only health doctor)
New `dev-status` role + playbook + Makefile target. Read-only snapshot of
spinifex.target, failed units, port listeners, OVN/OVS state, gateway
round-trip, AMI/keypair/volume/instance counts, and build-vs-install drift
for spx + nbdkit plugin + microvm artifacts. Pairs with dev-preflight as
the post-install / running-system diagnostic.
Plan: docs/development/improvements/ansible-dev-extensions.md (mulga).
Bead: mulga-siv-86.1.
* chore(ansible): add dev-logs playbook (bug-report bundle)
New `logs` role + playbook + Makefile target. Captures journalctl for
spinifex-* / ovn-* / openvswitch-switch / qemu / nbdkit, copies
/var/log/spinifex/, snapshots systemctl status + networking (ip/ss/ovs/
ovn show) + spx/plugin/microvm version manifest + dmesg, then bundles
into /tmp/spinifex-logs-<host>-<ts>.tar.gz owned by the invoking user.
Tail mode deliberately omitted — devs should `journalctl -u 'spinifex-*' -f`
directly; ansible captures stdout and is the wrong tool for follow.
Overrides: -e logs_since=10min, -e logs_output_dir=/tmp,
-e logs_include_dmesg=false.
Plan: docs/development/improvements/ansible-dev-extensions.md (mulga).
Bead: mulga-siv-86.2.
* chore(ansible): add dev-snapshot / dev-restore (state checkpoint)
New snapshot + restore roles + playbooks + Makefile targets.
dev-snapshot quiesces the full spinifex+OVN+OVS stack in dependency
order (spinifex.target → ovn-controller → ovn-central →
openvswitch-switch), kills lingering QEMU VMs, then tars /etc/spinifex,
/var/lib/spinifex (viperblock volumes included), /var/lib/ovn (NB+SB
DBs), and /etc/openvswitch/conf.db into /var/lib/spinifex-snapshots/
<name>.tar.gz with a sibling .manifest, restarts services in reverse,
and waits for the AWS gateway to come back.
dev-restore inverses: stop services, wipe the same path set, untar the
named bundle, copy /etc/spinifex/ca.pem back into the system trust
store and rebuild the CA bundle (the snapshot doesn't include
/usr/local/share/ca-certificates/spinifex-ca.crt), daemon-reload,
restart in reverse order.
Both refuse to run without -e snapshot_name=<name>, validate the name
matches [A-Za-z0-9._-]+, and produce honest summaries when the gateway
fails to come back (does not mask other ongoing issues).
Tested on dev box: snapshot/restore of 2.4 GiB state (large viperblock
volumes) completes cleanly with no data drift; services all return to
active.
Plan: docs/development/improvements/ansible-dev-extensions.md (mulga).
Bead: mulga-siv-86.3.
* fix(ansible): wrap aws/spx calls in sg spinifex to fix group inheritance
Adds scripts/ansible/scripts/aws-as-spinifex.sh and spx-as-spinifex.sh
helpers that re-enter the command under `sg spinifex -c '...'` with
AWS_PROFILE=spinifex preset. Swaps every AWS / spx admin task across
the status, init, deploy, smoketest, snapshot, and restore roles to
use these via spinifex_aws_cli / spinifex_spx_cli vars.
Root cause (mulga-siv-92): ansible's `become_user` is a no-op when the
target user equals the calling user, so sudo never runs and
supplementary groups are inherited from the operator's invoking shell
rather than re-initialised via initgroups(3). On shells that predate
setup.sh's spinifex group add, the spinifex group is missing and
/etc/spinifex/ca.pem (drwxr-x--- root:spinifex, the AWS CLI's
ca_bundle) is unreadable — aws cli fails with SSL Errno 13. The
helpers re-enter the command in a shell that explicitly assumes the
spinifex group via sg(1), bypassing the issue regardless of how
ansible-playbook is invoked.
Side benefit: drops 50 lines of repeated `become: true / become_user /
environment: AWS_PROFILE` boilerplate across the roles.
Closes mulga-siv-92.
* chore(ansible): add dev-version playbook (build vs install drift)
New `version` role + playbook + Makefile target. Compares sha256 of
built artifacts (bin/spx, viperblock/lib/nbdkit-viperblock-plugin.so,
build/microvm/{vmlinuz,initramfs.cpio.gz}) against their installed
counterparts (/usr/local/bin/spx, plugindir, /usr/share/spinifex/
microvm/*). Prints per-artifact drift verdict plus sub-repo HEAD
short-revs (spinifex/viperblock/predastore) for context. Closes with
either "DRIFT detected — run: make ansible-dev-deploy" or "No drift".
Pure read-only. Faster than dev-status (no AWS/OVN/port probing).
Plan: docs/development/improvements/ansible-dev-extensions.md (mulga).
Bead: mulga-siv-86.9.
* chore(ansible): add missing roles/deploy/meta/main.yml
All other roles have meta/main.yml; deploy was the lone outlier.
Consistency fix. Closes mulga-siv-86.10.
* chore(ansible): add dev-vpc playbook (idempotent dev VPC fixture)
New `vpc` role + playbook + Makefile target. Idempotent
present/absent dispatch creates a deterministic spinifex-dev-vpc with
public subnet (10.99.1.0/24, map-public-ip-on-launch), internet
gateway, route table, and default route. Hooks into smoketest's launch
path — smoketest's existing
Name=map-public-ip-on-launch,Values=true filter now picks it up
instead of falling through to the "no subnet → skip launch" branch.
Idempotency: each step describe-then-create, looking up by Name tag
where possible. Re-running on a converged VPC is changed=0, skipped=5
(the create tasks).
Workaround for mulga-siv-94 (gateway silently drops
--tag-specifications and no-ops create-tags for route tables): role
falls back to "RT associated with our subnet" lookup so tag absence
doesn't break idempotency. create-tags task is kept as a no-op-now
forward-looking step for when the gateway gap is closed.
Teardown (-e dev_vpc_state=absent) enumerates ALL non-main route
tables in the VPC (not just the tagged one), so orphans from prior
runs that landed before the fallback was added get cleaned up too.
Tested end-to-end against josh-banksia dev gateway: create / re-run
(idempotent) / teardown all converge cleanly.
Plan: docs/development/improvements/ansible-dev-extensions.md (mulga).
Bead: mulga-siv-86.4.
162 lines
6.7 KiB
YAML
162 lines
6.7 KiB
YAML
---
|
|
# Detailed build-vs-install drift report for the four artifacts that
|
|
# `ansible-dev-deploy` swaps: spx, the nbdkit viperblock plugin, and the
|
|
# two microvm artifacts (kernel + initramfs). Pure read-only; faster
|
|
# than dev-status (no AWS / OVN / port probing).
|
|
|
|
- name: version — built spx
|
|
ansible.builtin.stat:
|
|
path: "{{ spinifex_root }}/bin/spx"
|
|
checksum_algorithm: sha256
|
|
register: ver_spx_built
|
|
|
|
- name: version — installed spx
|
|
become: true
|
|
ansible.builtin.stat:
|
|
path: /usr/local/bin/spx
|
|
checksum_algorithm: sha256
|
|
register: ver_spx_installed
|
|
|
|
- name: version — installed spx version string
|
|
ansible.builtin.shell: |
|
|
/usr/local/bin/spx version 2>/dev/null | grep -vE '^[0-9]{4}/' | head -1
|
|
register: ver_spx_version
|
|
changed_when: false
|
|
failed_when: false
|
|
when: ver_spx_installed.stat.exists
|
|
|
|
- name: version — built plugin
|
|
ansible.builtin.stat:
|
|
path: "{{ mulga_root }}/viperblock/lib/nbdkit-viperblock-plugin.so"
|
|
checksum_algorithm: sha256
|
|
register: ver_plugin_built
|
|
|
|
- name: version — nbdkit plugin dir
|
|
ansible.builtin.shell: |
|
|
nbdkit --dump-config 2>/dev/null | awk -F= '/^plugindir=/{print $2; exit}'
|
|
register: ver_plugindir
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: version — installed plugin
|
|
become: true
|
|
ansible.builtin.stat:
|
|
path: "{{ ver_plugindir.stdout | trim }}/nbdkit-viperblock-plugin.so"
|
|
checksum_algorithm: sha256
|
|
register: ver_plugin_installed
|
|
when: ver_plugindir.stdout | trim | length > 0
|
|
|
|
- name: version — built microvm artifacts
|
|
ansible.builtin.stat:
|
|
path: "{{ spinifex_root }}/build/microvm/{{ item }}"
|
|
checksum_algorithm: sha256
|
|
register: ver_microvm_built
|
|
loop:
|
|
- vmlinuz
|
|
- initramfs.cpio.gz
|
|
|
|
- name: version — installed microvm artifacts
|
|
become: true
|
|
ansible.builtin.stat:
|
|
path: /usr/share/spinifex/microvm/{{ item }}
|
|
checksum_algorithm: sha256
|
|
register: ver_microvm_installed
|
|
loop:
|
|
- vmlinuz
|
|
- initramfs.cpio.gz
|
|
|
|
- name: version — sub-repo HEAD short-rev (spinifex)
|
|
ansible.builtin.command: git -C {{ spinifex_root | quote }} rev-parse --short HEAD
|
|
register: ver_spinifex_head
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: version — sub-repo HEAD short-rev (viperblock)
|
|
ansible.builtin.command: git -C {{ mulga_root }}/viperblock rev-parse --short HEAD
|
|
register: ver_viperblock_head
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: version — sub-repo HEAD short-rev (predastore)
|
|
ansible.builtin.command: git -C {{ mulga_root }}/predastore rev-parse --short HEAD
|
|
register: ver_predastore_head
|
|
changed_when: false
|
|
failed_when: false
|
|
|
|
- name: version — assemble per-artifact rows
|
|
ansible.builtin.set_fact:
|
|
ver_artifact_rows: >-
|
|
{{
|
|
[
|
|
('%-18s built=%s installed=%s drift=%s' | format(
|
|
'spx',
|
|
(ver_spx_built.stat.checksum | default('-') | truncate(13, true, ''))
|
|
if ver_spx_built.stat.exists else 'MISSING',
|
|
(ver_spx_installed.stat.checksum | default('-') | truncate(13, true, ''))
|
|
if ver_spx_installed.stat.exists else 'MISSING',
|
|
('YES' if ver_spx_built.stat.exists and ver_spx_installed.stat.exists
|
|
and ver_spx_built.stat.checksum != ver_spx_installed.stat.checksum
|
|
else 'no')
|
|
)),
|
|
('%-18s built=%s installed=%s drift=%s' | format(
|
|
'plugin (nbdkit)',
|
|
(ver_plugin_built.stat.checksum | default('-') | truncate(13, true, ''))
|
|
if ver_plugin_built.stat.exists else 'MISSING',
|
|
(ver_plugin_installed.stat.checksum | default('-') | truncate(13, true, ''))
|
|
if (ver_plugin_installed.stat | default({})).get('exists', false) else 'MISSING',
|
|
('YES' if ver_plugin_built.stat.exists
|
|
and (ver_plugin_installed.stat | default({})).get('exists', false)
|
|
and ver_plugin_built.stat.checksum != ver_plugin_installed.stat.checksum
|
|
else 'no')
|
|
)),
|
|
('%-18s built=%s installed=%s drift=%s' | format(
|
|
'microvm vmlinuz',
|
|
(ver_microvm_built.results[0].stat.checksum | default('-') | truncate(13, true, ''))
|
|
if ver_microvm_built.results[0].stat.exists else 'MISSING',
|
|
(ver_microvm_installed.results[0].stat.checksum | default('-') | truncate(13, true, ''))
|
|
if ver_microvm_installed.results[0].stat.exists else 'MISSING',
|
|
('YES' if ver_microvm_built.results[0].stat.exists
|
|
and ver_microvm_installed.results[0].stat.exists
|
|
and ver_microvm_built.results[0].stat.checksum
|
|
!= ver_microvm_installed.results[0].stat.checksum
|
|
else 'no')
|
|
)),
|
|
('%-18s built=%s installed=%s drift=%s' | format(
|
|
'microvm initramfs',
|
|
(ver_microvm_built.results[1].stat.checksum | default('-') | truncate(13, true, ''))
|
|
if ver_microvm_built.results[1].stat.exists else 'MISSING',
|
|
(ver_microvm_installed.results[1].stat.checksum | default('-') | truncate(13, true, ''))
|
|
if ver_microvm_installed.results[1].stat.exists else 'MISSING',
|
|
('YES' if ver_microvm_built.results[1].stat.exists
|
|
and ver_microvm_installed.results[1].stat.exists
|
|
and ver_microvm_built.results[1].stat.checksum
|
|
!= ver_microvm_installed.results[1].stat.checksum
|
|
else 'no')
|
|
))
|
|
]
|
|
}}
|
|
|
|
- name: version — compute any-drift flag
|
|
ansible.builtin.set_fact:
|
|
ver_any_drift: "{{ 'YES' in (ver_artifact_rows | join(' ')) }}"
|
|
|
|
- name: version — summary
|
|
ansible.builtin.debug:
|
|
msg: >-
|
|
{{
|
|
[
|
|
'── Spinifex build vs install ─────────────────────────────────────────────',
|
|
'spx (installed): ' ~ ((ver_spx_version.stdout | default('') | trim) or '(not installed)')
|
|
]
|
|
+ ver_artifact_rows
|
|
+ [
|
|
'── Sub-repo HEADs ──',
|
|
'spinifex : ' ~ ((ver_spinifex_head.stdout | default('') | trim) or '(not a git repo)'),
|
|
'viperblock : ' ~ ((ver_viperblock_head.stdout | default('') | trim) or '(missing)'),
|
|
'predastore : ' ~ ((ver_predastore_head.stdout | default('') | trim) or '(missing)'),
|
|
'──────────────────────────────────────────────────────────────────────────',
|
|
('DRIFT detected — run: make ansible-dev-deploy' if ver_any_drift
|
|
else 'No drift; built = installed for all artifacts.')
|
|
]
|
|
}}
|