Commit Graph

203 Commits

Author SHA1 Message Date
Julian Sommer deae613a75 refactor: split daemon_handlers.go into per-domain files
Split the monolithic daemon_handlers.go into focused files:
  - daemon_handlers_account.go
  - daemon_handlers_console.go
  - daemon_handlers_igw.go
  - daemon_handlers_image.go
  - daemon_handlers_instance.go
  - daemon_handlers_key.go
  - daemon_handlers_snapshot.go
  - daemon_handlers_tag.go
  - daemon_handlers_volume.go
  - daemon_handlers_vpc.go

Add tests for ModifyVolume success, TerminateStoppedInstance with volumes,
DescribeInstanceTypes capacity filter, StartStoppedInstance type-not-available,
AttachVolume/DetachVolume validation paths, RunInstances error paths, and
DescribeInstances malformed ID.

Fix diff-coverage.sh to use git rename/copy detection (-M -C) so file
splits are not incorrectly flagged as all-new code.
2026-02-24 14:28:24 +10:00
Julian Sommer 8188707c91 feat: VPC networking v1 — OVN-based virtual networking for EC2 instances 2026-02-24 14:13:57 +10:00
Julian Sommer b9c528ce4c fix frontend vuln 2026-02-24 11:56:17 +10:00
Julian Sommer 5992d3f056 test: general test coverage improvements across hive packages
Coverage improvements:
- awsec2query: parser byte slice, invalid input, locationName fallback (73% → 83%)
- config: EC2StartInstancesResponse.Respond NATS round-trip, NodeBaseDir, ShardWAL
- daemon: 14 NATS delegate handlers, device allocation, canAllocate (count=0 boundary),
  resource management, shutdown ACK/progress, JetStream WriteServiceManifest,
  fix smallestAllocType to score by both vCPU and memory
- gateway: GenerateEC2ErrorResponse XML validation (fixed no-op assertion),
  DiscoverActiveNodes (5 scenarios), ParseAWSQueryArgs, ParseArgsToStruct (41.6% → 52.1%)
- utils: ConnectNATS (success, token, wrong token, bad address), NATSRequest
- vm: ResetNodeLocalState, PIDFile/QMP, NoGraphic, Serial+Console partial configs,
  NetDevs, ARM64, KVM, full config (68.1% → 91.7%)
2026-02-20 15:09:33 +10:00
Julian Sommer 6bc36455fc test coverage improvements: instance, hiveui, health, utils
Total hive coverage: 54.3% → 55.8% (+1.5%)

- instance/service_impl.go: new test file (generateHostname, RunInstance,
  cloud-init template, constructor)
- hiveui.go: new test file (security headers, gzip middleware, New(),
  Status(), Reload())
- daemon/health.go: extend existing tests (handleInstanceCrash core flow,
  first crash time, unknown instance type, maybeRestart scheduling)
- utils/utils.go: extend existing tests (pidPath all branches, RuntimeDir,
  extractDiskImagePath, WritePidFile error path)
- fix: use smallestAllocType in health tests to avoid CI resource failures
2026-02-20 13:46:31 +10:00
Julian Sommer db45c028be test: e2e + unit coverage for ModifyInstanceAttribute and key service
- Add Phase 7b to single-node E2E: stop instance, modify type (nano →
  xlarge), start, SSH in, verify vCPU count (nproc) and memory
  (/proc/meminfo) match the new instance type
- Add Phase 8q negative test: modify-instance-attribute on running
  instance expects InvalidInstanceID.NotFound
- Improve key service integration test coverage (mulga-580)
- Pin viperblock/predastore CI checkout to main (fixes feature branch CI)
- Update TEST_COVERAGE.md
2026-02-20 12:01:03 +10:00
Julian Sommer ff6ca8a057 feat: modify-instance-attribute (#48) 2026-02-20 12:19:20 +11:00
Julian Sommer df0e8a6459 ci: add race detection, fix test race conditions 2026-02-20 09:40:02 +10:00
Julian Sommer eba702b1f2 test: add 10 viperblockd coverage tests, fix daemon test sleep
viperblockd: add snapshot handler, sync/unmount/delete with VB instance,
invalid JSON, dual-publish, generic topic routing, and auxiliary volume
suffix tests. Coverage 43.1% → 53.6%.

daemon: write clean shutdown marker in restore tests to skip the 3s
crash-recovery sleep (no QEMU runs in tests). Suite 36s → 6s.
2026-02-20 08:42:30 +10:00
Julian Sommer b4dcd3ead9 gofix: modernize codebase with go fix analyzers
Apply safe modernizations across the codebase:
- minmax: replace if/else clamping with min/max builtins
- fmtappendf: replace []byte(fmt.Sprintf) with fmt.Appendf
- forvar: remove redundant loop variable re-declarations
- inline: rename reflect.Ptr to reflect.Pointer
- mapsloop: replace explicit map loops with maps package
- omitzero: remove ineffective omitempty on time.Time fields
- rangeint: replace 3-clause for loops with range-over-int
- slicescontains: replace manual loops with slices.Contains
2026-02-19 14:40:16 +10:00
Julian Sommer 17c011e9db fix 2026-02-19 13:17:20 +10:00
Julian Sommer a330e35d5f feat: account settings console access (#47) 2026-02-19 13:05:11 +10:00
Julian Sommer a5ff1ca5a7 feat: more awserrors (#46) 2026-02-19 12:15:45 +11:00
Julian Sommer 8573800914 fix: move instances with unavailable type to stopped on recovery (#45) 2026-02-19 10:31:13 +11:00
Julian Sommer ca14ecf2bc fix: port splitting (#44) 2026-02-18 15:20:40 +11:00
Julian Sommer 0c65d75371 feat: add coordinated cluster shutdown to multi-node E2E tests (#42) 2026-02-18 13:22:12 +11:00
Ben Duncan 4986e3fb87 fix: disable sharded WAL by default, bump viperblock v1.0.14
Sharded WAL causes incomplete write errors in E2E tests. Disable by
default in hive.toml template, service.go fallback, and viperblock
constructor. Can be re-enabled via hive.toml shardwal=true.
2026-02-18 07:08:11 +10:00
Ben Duncan 20e67051b0 feat: EC2 GetConsoleOutput via QEMU socket chardev + logfile capture
Replace -serial pty with socket chardev + logfile for serial output:
- Console log written to {RuntimeDir}/console-{instanceID}.log
- Serial socket at {RuntimeDir}/serial-{instanceID}.sock for admin access
- Eliminates PTS extraction deadlock (mulga-lfo) — no more ptsChan
- GetConsoleOutput API handler reads log, returns base64-encoded output
- Per-instance NATS topic ec2.{instanceID}.GetConsoleOutput (no fan-out)
- Admin debugging via: socat -,raw,echo=0 UNIX-CONNECT:serial-*.sock
2026-02-18 06:25:43 +10:00
Ben Duncan 5f370ab75e fix: bump viperblock v1.0.13, add local Docker E2E test docs
Update viperblock dependency to v1.0.13 which fixes CreateSnapshot
skipping WAL flush/consolidate when the VB instance doesn't own the
WAL files (viperblockd snapshot scenario).

Add local Docker E2E test instructions to TESTING.md.
2026-02-18 06:25:20 +10:00
Ben Duncan bb1fe5e485 feat: add configurable shard WAL toggle, bump viperblock v1.0.11
Add [viperblock] section to hive.toml with shardwal config (default
true). The setting flows from config through viperblockd to nbdkit
plugin args, allowing operators to disable sharded WAL if needed.

Bumps viperblock dependency from v1.0.10 to v1.0.11 which includes
the sharded WAL implementation (UseShardedWAL/OpenShardedWAL), fixing
CI build failures.
2026-02-17 21:48:44 +10:00
Ben Duncan 1b431b29f7 Merge branch 'main' of github.com:mulgadc/hive 2026-02-17 16:14:49 +10:00
Ben Duncan c645fc46b1 fix: dispatch chunk WAL opening for EFI and cloud-init volumes
With UseShardedWAL now defaulting to true, the EFI and cloud-init
volume preparation must open sharded WAL files instead of always
opening the legacy WAL. Without this, Flush() dispatches to the
sharded path but shard DB files are nil, causing zero-byte writes.
2026-02-17 16:14:00 +10:00
Julian Sommer 83b1435c99 fix: refactor instance type detection (#41) 2026-02-17 15:54:45 +11:00
Julian Sommer 7128a22fdb feat: detect CPU generation instead of vendor for instance types (#40) 2026-02-17 14:59:29 +11:00
Julian Sommer b7396b97c0 fix: use available instance types in allocation test to prevent flaky CI
The FilterUnavailableTypesAfterAllocation test picked a random 2-vCPU
type from the raw instance type map, which could include memory-hungry
types (e.g. r5a.large at 16 GB) that exceed host RAM on CI runners
(~15.6 GB). Now picks from the already-filtered available types list.
2026-02-17 13:30:44 +10:00
Julian Sommer 1735a46f11 fix: e2e 2026-02-17 12:32:40 +10:00
Julian Sommer 8d9da69abb fix: e2e 2026-02-17 11:52:31 +10:00
Julian Sommer c6b89e873a feat: more testing 2026-02-17 11:47:56 +10:00
Julian Sommer aeb802448b feat: more instance types 2026-02-17 10:39:59 +10:00
Julian Sommer b648fe7390 fix: merge 2026-02-17 10:32:35 +10:00
Ben Duncan 24924be941 Cluster heartbeat support, improved cluster start/stop, coordinated shutdown in multi-server mode 2026-02-14 16:59:32 +10:00
Ben Duncan c5e561abbf Improved cluster stats and INSTALL.md doc 2026-02-14 16:19:25 +10:00
Ben Duncan 774655eff0 hive top, hive get cmds for cluster health and stats. improved docs for iac, cluster, multi-server restart 2026-02-14 15:38:03 +10:00
Ben Duncan 5863aeed13 Phase 0/1 multi-server-restart enhancements 2026-02-14 13:23:24 +10:00
Ben Duncan febeae8c47 preflight fixes for gh errors 2026-02-14 09:23:42 +10:00
Ben Duncan 3095a90090 e2e tests, multi server improvements, doc updates for workflow 2026-02-14 08:50:24 +10:00
Ben Duncan d80dd0996a Fixed gosec and added go skills 2026-02-14 06:54:00 +10:00
Ben Duncan 6df221309d Merge branch 'main' of github.com:mulgadc/hive 2026-02-13 20:32:08 +10:00
Ben Duncan db0d46ca5b Improved multi-server node testing and tweaks 2026-02-13 20:31:45 +10:00
Julian Sommer 8774fb1e95 feat: distributed start instances (#37) 2026-02-12 15:15:05 +11:00
Julian Sommer fb361dfe49 fix: update instance status BEFORE calling it 2026-02-12 11:11:59 +10:00
Julian Sommer 815e9a8c36 fix: duplicate subscription 2026-02-12 09:03:24 +10:00
Julian Sommer c7d832a823 fix: nats topic 2026-02-12 08:30:23 +10:00
Julian Sommer 32e2f7bf78 fix: daemon reboot 2026-02-11 16:05:21 +10:00
Julian Sommer 202fffc8db chore: utils tests 2026-02-11 15:21:29 +10:00
Julian Sommer 72709dda12 linter fix 2026-02-11 15:15:30 +10:00
Julian Sommer 0ff03be0dd fix 2026-02-11 15:10:55 +10:00
Julian Sommer 998649cbab feat: createImage (#34) 2026-02-11 16:07:19 +11:00
Julian Sommer efe07a9ed4 chore: bump go version 2026-02-11 12:59:32 +10:00
Julian Sommer 27530ba270 chore: daemon tests 2026-02-11 12:48:21 +10:00