222 Commits

Author SHA1 Message Date
Julian Sommer 94ec56bb59 fix: extract shared type 2026-03-06 09:45:34 +10:00
Julian Sommer 2d304528b8 chore: bump deps 2026-03-06 08:15:48 +10:00
Julian Sommer 35f2104325 feat: more tests 2026-03-05 15:09:24 +10:00
Julian Sommer c4027a0380 test: add gateway core routing, IAM auth, and handler test coverage
Squash of dev commits: gateway core routing/policy/middleware tests,
IAM auth and handler tests, awserrors tests, respondWithJSON helper,
utils refactor into focused files, and daemon test speedups.
2026-03-05 14:20:00 +10:00
Julian Sommer 77c6d05042 refactor: replace NYTimes/gziphandler with stdlib compress/gzip
Drop the external gziphandler dependency in hiveui. The replacement
uses a simple http.ResponseWriter wrapper that checks Content-Type
against an allowlist before enabling gzip compression. Same behavior,
zero external deps.
2026-03-05 12:53:46 +10:00
Julian Sommer 8af3d2e5fa feat: migrate off fiber (#62) 2026-03-05 12:53:46 +10:00
Julian Sommer 49c5807ef1 refactor: accounts 2026-03-05 11:41:51 +10:00
Julian Sommer 5fad038b87 feat: IAM account scoping (#61) 2026-03-05 09:06:35 +10:00
Julian Sommer 371193a832 feat: IAM v1 (#59) 2026-03-03 15:23:19 +11:00
Julian Sommer 9d66ef8233 fix: sanitize error codes to prevent leaking internal errors to clients
Add awserrors.ValidErrorCode() to validate error strings against
ErrorLookup before sending to clients. Replace hardcoded
"ValidationError" strings in utils.go with awserrors constants.
2026-02-27 08:49:18 +10:00
Julian Sommer 992111e809 fix: race condition + frontend refetching 2026-02-27 08:14:50 +10:00
Julian Sommer 3720a5ba90 feat: blockdevicemappings fix (#55) 2026-02-26 11:47:37 +11:00
Julian Sommer 6e3fe148af feat: frontend features 2026-02-26 08:22:55 +10:00
Julian Sommer 61e8eb9d91 fix: reset LaunchTime on instance relaunch to prevent watchdog termination
During cluster restart, restoreInstances resets running instances (with
dead QEMU) to pending for relaunch, but didn't reset LaunchTime. The
pending watchdog (5min timeout) saw the stale LaunchTime from the
original launch and immediately marked instances as failed/terminated,
preventing any relaunch from succeeding.
2026-02-25 10:48:16 +10:00
Julian Sommer 1453beca6b Merge remote-tracking branch 'origin/main' into dev 2026-02-25 10:45:45 +10:00
Julian Sommer d0056ee925 fix: daemon-level retry with backoff for NATS/JetStream startup (#52) 2026-02-25 11:43:10 +11:00
Julian Sommer 55486490aa fix: kv stream recovery (#54) 2026-02-25 11:41:28 +11:00
Julian Sommer 6b9fc7b600 refactor: daemon 2026-02-25 08:22:46 +10:00
Julian Sommer c2e66e1ab4 refactor: extract respondWithError; fix nil panic in RunInstance
Extract duplicated respondWithError closures into a single package-level
function. Guard against nil ImageId in handleEC2RunInstances to prevent
panic under race conditions. Fix diff-coverage base ref for main branch.
2026-02-24 15:02:57 +10:00
Julian Sommer deae613a75 refactor: split daemon_handlers.go into per-domain files
Split the monolithic daemon_handlers.go into focused files:
  - daemon_handlers_account.go
  - daemon_handlers_console.go
  - daemon_handlers_igw.go
  - daemon_handlers_image.go
  - daemon_handlers_instance.go
  - daemon_handlers_key.go
  - daemon_handlers_snapshot.go
  - daemon_handlers_tag.go
  - daemon_handlers_volume.go
  - daemon_handlers_vpc.go

Add tests for ModifyVolume success, TerminateStoppedInstance with volumes,
DescribeInstanceTypes capacity filter, StartStoppedInstance type-not-available,
AttachVolume/DetachVolume validation paths, RunInstances error paths, and
DescribeInstances malformed ID.

Fix diff-coverage.sh to use git rename/copy detection (-M -C) so file
splits are not incorrectly flagged as all-new code.
2026-02-24 14:28:24 +10:00
Julian Sommer 8188707c91 feat: VPC networking v1 — OVN-based virtual networking for EC2 instances 2026-02-24 14:13:57 +10:00
Julian Sommer b9c528ce4c fix frontend vuln 2026-02-24 11:56:17 +10:00
Julian Sommer 5992d3f056 test: general test coverage improvements across hive packages
Coverage improvements:
- awsec2query: parser byte slice, invalid input, locationName fallback (73% → 83%)
- config: EC2StartInstancesResponse.Respond NATS round-trip, NodeBaseDir, ShardWAL
- daemon: 14 NATS delegate handlers, device allocation, canAllocate (count=0 boundary),
  resource management, shutdown ACK/progress, JetStream WriteServiceManifest,
  fix smallestAllocType to score by both vCPU and memory
- gateway: GenerateEC2ErrorResponse XML validation (fixed no-op assertion),
  DiscoverActiveNodes (5 scenarios), ParseAWSQueryArgs, ParseArgsToStruct (41.6% → 52.1%)
- utils: ConnectNATS (success, token, wrong token, bad address), NATSRequest
- vm: ResetNodeLocalState, PIDFile/QMP, NoGraphic, Serial+Console partial configs,
  NetDevs, ARM64, KVM, full config (68.1% → 91.7%)
2026-02-20 15:09:33 +10:00
Julian Sommer 6bc36455fc test coverage improvements: instance, hiveui, health, utils
Total hive coverage: 54.3% → 55.8% (+1.5%)

- instance/service_impl.go: new test file (generateHostname, RunInstance,
  cloud-init template, constructor)
- hiveui.go: new test file (security headers, gzip middleware, New(),
  Status(), Reload())
- daemon/health.go: extend existing tests (handleInstanceCrash core flow,
  first crash time, unknown instance type, maybeRestart scheduling)
- utils/utils.go: extend existing tests (pidPath all branches, RuntimeDir,
  extractDiskImagePath, WritePidFile error path)
- fix: use smallestAllocType in health tests to avoid CI resource failures
2026-02-20 13:46:31 +10:00
Julian Sommer db45c028be test: e2e + unit coverage for ModifyInstanceAttribute and key service
- Add Phase 7b to single-node E2E: stop instance, modify type (nano →
  xlarge), start, SSH in, verify vCPU count (nproc) and memory
  (/proc/meminfo) match the new instance type
- Add Phase 8q negative test: modify-instance-attribute on running
  instance expects InvalidInstanceID.NotFound
- Improve key service integration test coverage (mulga-580)
- Pin viperblock/predastore CI checkout to main (fixes feature branch CI)
- Update TEST_COVERAGE.md
2026-02-20 12:01:03 +10:00
Julian Sommer ff6ca8a057 feat: modify-instance-attribute (#48) 2026-02-20 12:19:20 +11:00
Julian Sommer df0e8a6459 ci: add race detection, fix test race conditions 2026-02-20 09:40:02 +10:00
Julian Sommer eba702b1f2 test: add 10 viperblockd coverage tests, fix daemon test sleep
viperblockd: add snapshot handler, sync/unmount/delete with VB instance,
invalid JSON, dual-publish, generic topic routing, and auxiliary volume
suffix tests. Coverage 43.1% → 53.6%.

daemon: write clean shutdown marker in restore tests to skip the 3s
crash-recovery sleep (no QEMU runs in tests). Suite 36s → 6s.
2026-02-20 08:42:30 +10:00
Julian Sommer b4dcd3ead9 gofix: modernize codebase with go fix analyzers
Apply safe modernizations across the codebase:
- minmax: replace if/else clamping with min/max builtins
- fmtappendf: replace []byte(fmt.Sprintf) with fmt.Appendf
- forvar: remove redundant loop variable re-declarations
- inline: rename reflect.Ptr to reflect.Pointer
- mapsloop: replace explicit map loops with maps package
- omitzero: remove ineffective omitempty on time.Time fields
- rangeint: replace 3-clause for loops with range-over-int
- slicescontains: replace manual loops with slices.Contains
2026-02-19 14:40:16 +10:00
Julian Sommer 17c011e9db fix 2026-02-19 13:17:20 +10:00
Julian Sommer a330e35d5f feat: account settings console access (#47) 2026-02-19 13:05:11 +10:00
Julian Sommer a5ff1ca5a7 feat: more awserrors (#46) 2026-02-19 12:15:45 +11:00
Julian Sommer 8573800914 fix: move instances with unavailable type to stopped on recovery (#45) 2026-02-19 10:31:13 +11:00
Julian Sommer ca14ecf2bc fix: port splitting (#44) 2026-02-18 15:20:40 +11:00
Julian Sommer 0c65d75371 feat: add coordinated cluster shutdown to multi-node E2E tests (#42) 2026-02-18 13:22:12 +11:00
Ben Duncan 4986e3fb87 fix: disable sharded WAL by default, bump viperblock v1.0.14
Sharded WAL causes incomplete write errors in E2E tests. Disable by
default in hive.toml template, service.go fallback, and viperblock
constructor. Can be re-enabled via hive.toml shardwal=true.
2026-02-18 07:08:11 +10:00
Ben Duncan 20e67051b0 feat: EC2 GetConsoleOutput via QEMU socket chardev + logfile capture
Replace -serial pty with socket chardev + logfile for serial output:
- Console log written to {RuntimeDir}/console-{instanceID}.log
- Serial socket at {RuntimeDir}/serial-{instanceID}.sock for admin access
- Eliminates PTS extraction deadlock (mulga-lfo) — no more ptsChan
- GetConsoleOutput API handler reads log, returns base64-encoded output
- Per-instance NATS topic ec2.{instanceID}.GetConsoleOutput (no fan-out)
- Admin debugging via: socat -,raw,echo=0 UNIX-CONNECT:serial-*.sock
2026-02-18 06:25:43 +10:00
Ben Duncan 5f370ab75e fix: bump viperblock v1.0.13, add local Docker E2E test docs
Update viperblock dependency to v1.0.13 which fixes CreateSnapshot
skipping WAL flush/consolidate when the VB instance doesn't own the
WAL files (viperblockd snapshot scenario).

Add local Docker E2E test instructions to TESTING.md.
2026-02-18 06:25:20 +10:00
Ben Duncan bb1fe5e485 feat: add configurable shard WAL toggle, bump viperblock v1.0.11
Add [viperblock] section to hive.toml with shardwal config (default
true). The setting flows from config through viperblockd to nbdkit
plugin args, allowing operators to disable sharded WAL if needed.

Bumps viperblock dependency from v1.0.10 to v1.0.11 which includes
the sharded WAL implementation (UseShardedWAL/OpenShardedWAL), fixing
CI build failures.
2026-02-17 21:48:44 +10:00
Ben Duncan 1b431b29f7 Merge branch 'main' of github.com:mulgadc/hive 2026-02-17 16:14:49 +10:00
Ben Duncan c645fc46b1 fix: dispatch chunk WAL opening for EFI and cloud-init volumes
With UseShardedWAL now defaulting to true, the EFI and cloud-init
volume preparation must open sharded WAL files instead of always
opening the legacy WAL. Without this, Flush() dispatches to the
sharded path but shard DB files are nil, causing zero-byte writes.
2026-02-17 16:14:00 +10:00
Julian Sommer 83b1435c99 fix: refactor instance type detection (#41) 2026-02-17 15:54:45 +11:00
Julian Sommer 7128a22fdb feat: detect CPU generation instead of vendor for instance types (#40) 2026-02-17 14:59:29 +11:00
Julian Sommer b7396b97c0 fix: use available instance types in allocation test to prevent flaky CI
The FilterUnavailableTypesAfterAllocation test picked a random 2-vCPU
type from the raw instance type map, which could include memory-hungry
types (e.g. r5a.large at 16 GB) that exceed host RAM on CI runners
(~15.6 GB). Now picks from the already-filtered available types list.
2026-02-17 13:30:44 +10:00
Julian Sommer 1735a46f11 fix: e2e 2026-02-17 12:32:40 +10:00
Julian Sommer 8d9da69abb fix: e2e 2026-02-17 11:52:31 +10:00
Julian Sommer c6b89e873a feat: more testing 2026-02-17 11:47:56 +10:00
Julian Sommer aeb802448b feat: more instance types 2026-02-17 10:39:59 +10:00
Julian Sommer b648fe7390 fix: merge 2026-02-17 10:32:35 +10:00
Ben Duncan 24924be941 Cluster heartbeat support, improved cluster start/stop, coordinated shutdown in multi-server mode 2026-02-14 16:59:32 +10:00