21 Commits

Author SHA1 Message Date
Anand 51ff999b8d AI assistant: reasoning effort that works locally, usage stats, perf and UX fixes
- Map reasoning effort to chat_template_kwargs.enable_thinking alongside
  reasoning_effort, so the selector actually toggles thinking on Qwen3-class
  local runtimes (LM Studio/vLLM/Ollama), which ignore reasoning_effort
  entirely. Falls back cleanly on hosted APIs that reject the extra field.
- Track and render per-message token usage (prompt/completion/tokens-per-sec)
  from the ferrum_usage SSE envelope, with an estimated fallback.
- Persist reasoningEffort and createdAt on conversations/messages.
- Memoize Markdown on text so streaming a response no longer re-parses every
  already-committed message on each token.
- AI provider settings: auto-add all discovered models on first save instead
  of requiring a second click per model.
- SSE event stream: clear the write deadline so long-lived connections don't
  get cut mid-stream.
2026-09-11 09:22:49 +05:30
Anand d63c284fbf Fix build errors and a batch of UI/data bugs found in review
Frontend build:
- combobox.tsx: DropdownMenuContent doesn't expose onOpenAutoFocus (Radix
  Popper primitive, unlike Dialog/Popover) — tsc failed. Focus the search
  input from an effect on `open` instead.

Backend data bugs (JSON parsing / validation):
- pve.JournalEntry: "n" is sometimes sent quoted by PVE ("n":"1"), which
  failed the whole unmarshal and blanked the node Journal tab. Decoded
  loosely, same as the existing cpuinfo.mhz string/number quirk.
- pve.ClusterConfigNode: pve_addr was typed int but PVE always sends a
  string (an IP) — populated it 502'd Cluster & SDN > Members. Fixed to
  string.
- ClusterPage: PVE also errors /cluster/config/nodes outright on a
  standalone (non-clustered) node — that hit the same 502 error card
  instead of the existing "not part of a cluster" message.
- ai_providers.go: updateAIProvider rejected the built-in Needle 2
  provider's "needle://local" baseUrl on every save (missing the
  needle.IsBuiltin bypass createAIProvider already had), so it could only
  ever be disabled, never edited. Deleting it only lasted until the next
  restart (the seeder always re-created it) — now tracked via a
  "dismissed" flag in the settings table so a delete sticks.

UI polish:
- Added amber/rose/teal accent color presets (theme.tsx, AppearanceCard,
  index.css, backend whitelist).
- StatusDot's glow was clipped on one side wherever it sat inside a
  `truncate` (overflow-hidden) flex row — moved truncation to just the
  text sibling (ConnectionsPage, AlertActivityWidget).
- ResourceAreaChart: a wide Y-axis tick ("47.68 MB/s") wraps onto two
  lines in recharts, and the chart only had 8px of top margin — the first
  line rendered off the top edge. Widened the axis gutter and margin.
- AlertActivityWidget required *both* of two independent queries to fail
  before showing an error, so one broken endpoint alone rendered stale
  counts instead of the error state.
- DonutChart: the hover tooltip followed the cursor by default, which on
  a compact ring collided with the centered value/label text. Pinned it
  below the ring instead.
- NodeDetailPage storage rows: a variable number of badges before the
  usage bar made every row's bar start at a different x. Switched to a
  fixed-width grid, same pattern already used for two other bar lists.
- RunningTasksWidget: a failed task's PVE status is a full sentence, not
  a short word — stuffing it into a badge blew the row's layout up.
  Collapsed to "Failed" with the full text as a hover title.
- Alert value/threshold formatting assumed every metric is a 0-100
  percent; three built-in alerts (cert_expiry/connection_stale in days,
  storage_orphan_disk in bytes) printed nonsense like "67108864.0%".
  Added a shared formatAlertValue() and used it everywhere alert
  instances are rendered (AlertsPage, OverviewPage, NotificationBell).

README: added a Features section summarizing the fleet management,
automation, integration, and access-control capabilities.
2026-09-10 19:40:00 +05:30
Anand fadc1f24e3 Wave 3 (3/3): scheduled fleet health digests + Terraform/Ansible export
- internal/digest: fleet summary builder + scheduler, delivered via the existing notify.Notifier (SMTP/Gotify), admin settings + send-now endpoint
- internal/export: Terraform (proxmox_vm_qemu/proxmox_lxc) and Ansible YAML inventory generators from live fleet inventory, downloadable via /connections/{id}/export/{terraform,ansible}
- Migration renumbered 00031 to avoid colliding with 00026-00030 already on this branch
2026-09-10 15:08:22 +05:30
Anand 542a9b2eb5 Wave 3 (2/3): config drift detection + guest lifecycle policies
- internal/api/drift.go: per-guest config/firewall baseline capture + diff, fleet-wide drift summary
- internal/poller/lifecycle.go + internal/api/lifecycle.go: snapshot retention sweep (tag or fleet default, dry-run until explicitly enforced) and orphaned-disk detection (report-only, surfaced as alerts)
- Migrations renumbered 00029/00030 to avoid colliding with 00026-00028 already on this branch
2026-09-10 15:04:30 +05:30
Anand 2c8e114aeb Wave 3 (1/3): capacity forecasting + fleet health score
- internal/api/forecast.go: OLS trend fit over node RRD history, per-node and fleet-wide capacity-warning endpoints
- internal/api/health.go: 0-100 health score (reachability, alerts, node/guest status, backup reliability) per-connection and fleet-wide
- web: CapacityForecastWidget, HealthScoreWidget dashboard widgets, registered in the widget catalog
2026-09-10 15:02:59 +05:30
Anand cafa22f714 Wire wave-2 features into navigation and live UI
- App.tsx/AppShell/CommandPalette: register Bulk Operations and Webhooks routes + nav entries
- WebhooksPage.tsx: full CRUD UI for outgoing webhook subscriptions, event-type picker, delivery log, secret-reveal-once dialog
- CommandPalette: guest search now backed by GET /search once typing (ranked, cross-connection) instead of only ever filtering the locally-cached inventory fetch — fixes the DOM-mounting-at-scale concern the file already flagged in its own comments
- search.go: added id to searchResult so CommandPalette's focusGuestId highlight works the same for server results as local ones
- NotificationBell: subscribes to the SSE event stream (alert.triggered/resolved) to invalidate the alert queries immediately and toast new critical alerts, instead of waiting out the 30s poll
2026-09-10 14:43:34 +05:30
Anand 4dfeb9848e Wave 2: real-time events/webhooks, cross-remote tags/search/bulk-ops, cert & session monitoring
- internal/events: in-process pub/sub bus; GET /api/v1/events SSE stream
- internal/notify/webhooks.go: signed outgoing webhook dispatcher w/ retry + delivery log
- internal/api/tags.go, search.go, bulk.go: cross-remote tag aggregation, global search, fleet-wide bulk guest actions
- internal/auth/sessions.go: session listing/revocation (self-service + admin), extends the existing stateful session table
- internal/poller/certificates.go: cert-expiry + connection-staleness alerting via the existing alert_rules pipeline
- web: useEventStream hook, GlobalSearch + BulkOperationsPage (not yet wired into nav), SessionsCard on ProfilePage
2026-09-10 11:49:11 +05:30
Anand ad43bec538 Merge PBS remotes + cross-cluster migration (backend) and UI modernization pass (frontend)
- internal/pbs: PBS client, datastores/namespaces/snapshots/prune/GC/sync/verify jobs
- connections: type column (pve|pbs), PBS-aware client resolution
- pve/guests: RemoteMigrateGuest + POST .../remote-migrate endpoint for true cross-cluster live migration
- web: shared ListSearch/Textarea primitives, DataTable search modernized
2026-09-10 11:31:57 +05:30
Anand 9b3d093eb8 Expand Needle 2 tool catalog, fix diagnostics, and publish Docker images to GHCR
- Add 48 new MCP/AI-assistant tools covering guest lifecycle, node
  operations, firewall/security, and backup/replication/HA/storage/SDN
  management. Every mutating tool is admin-gated the same way
  guest_power_action already is; migrate/resize/move-disk, node
  reboot/shutdown, disk wipe, cert revocation, and cluster-node removal are
  deliberately left out as being as destructive as a delete.
- ai_chat.go: when a provider (chiefly Needle, a pure tool-router with no
  narrative output of its own) finishes calling tools but returns nothing to
  say, render the tool results themselves as the answer instead of the
  misleading "ran out of tool calls" message.
- needle.go: serialize every request against the shared Needle subprocess
  (it handles one request at a time) to stop concurrent callers from racing
  it, and surface the subprocess's captured output when a request fails
  because it died mid-response, instead of a bare network error.
- Dockerfile: switch the final stage from distroless "static" to "base" —
  the bundled Needle CLI is a dynamically-linked glibc binary and cannot run
  in an image with no libc at all.
- .github/workflows/release.yml: build and push a multi-arch (amd64/arm64)
  Docker image to ghcr.io on every version tag, tagged with the version and
  "latest".
- Dockerfile/README: add OCI image labels and document the published GHCR
  image as the primary Docker install path.
2026-09-09 23:03:54 +05:30
Anand ae352951ad Distinguish mid-stream AI provider errors from unreachable-provider errors
- callChatCompletion previously prefixed every failure, including a
  provider crashing/erroring mid-response, with "could not reach AI
  provider" -- misleading once some content had already streamed live.
  parseChatCompletionStream now wraps stream-phase errors in a new
  streamError type; aiChat checks for it and reports "AI provider
  error: ..." instead.
- parseChatCompletionStream never checked scanner.Err() after its read
  loop, so a dropped connection or an over-long line (bufio.Scanner's
  1MB cap) looked identical to a clean [DONE] and silently returned a
  truncated answer as if nothing had gone wrong. Now surfaced as a
  streamError too.
- Added tests for both cases.
2026-09-09 20:46:22 +05:30
Anand ae165397fe Fixed AI Chat Context issue 2026-09-08 21:50:21 +05:30
Anand 846cb8ce63 Bundle Needle 2 as a built-in, default AI assistant
Ships the official Needle 2 CLI binary (Apache-2.0) baked into the ferrum
binary itself for windows/amd64, linux/amd64, linux/arm64, and
darwin/arm64 via go:embed behind per-platform build tags. No download,
no FERRUM_NEEDLE_BIN, no manual provider setup on those platforms.

- internal/needle: resolveBinPath prefers an explicit FERRUM_NEEDLE_BIN,
  otherwise extracts the embedded binary to a cache file on first use.
- Fixed a port collision: Needle's --serve defaulted to :8080, the same
  default as ferrum's own server; it now runs on a dedicated port.
- Fixed the real 'assistant times out' bug: Needle's tool-retrieval does
  a one-time embedding pass on its first request once the tool catalog
  exceeds 5 tools (ferrum declares 10), which routinely took longer than
  the old readiness check's 500ms-per-attempt retry loop allowed. Each
  cancelled attempt kept occupying Needle's single-threaded request
  loop, so retries piled up and never let a real response through.
  Replaced with two phases: cheap/retryable raw TCP dials until the
  socket accepts a connection, then exactly one real request allowed to
  run for the full startup budget.
  Subprocess stdout/stderr are now captured so a future failure surfaces
  a real reason instead of a bare timeout.
- seedBuiltinNeedleProvider now runs on every startup (not just fresh
  installs), idempotently upserting the built-in provider and always
  reasserting its model as the global default AI assistant.
- .gitignore: carved out an exception for the bundled Windows binary,
  which the blanket *.exe rule would otherwise have silently excluded.
2026-09-06 20:26:00 +05:30
Anand 22c1dd382c Add API keys, MCP server, admin AI providers, and a built-in local LLM option
- User-scoped API keys (Profile > API Keys) for 3rd-party REST API access
  and MCP clients, each locked to one scope at creation, with expiry,
  revocation, and last-used tracking.
- A hand-rolled MCP (Model Context Protocol) server exposing the fleet
  (connections, nodes, guests, storage, pools, alerts, cluster status) as
  read tools plus one admin-gated power-action tool, so Claude Code/Desktop
  or any other MCP client can query and operate the fleet directly.
- Both the REST API and MCP are off by default and toggleable instance-wide
  from Settings > API & MCP, enforced live on every request.
- Admin-managed AI providers (any OpenAI-chat-completions-compatible
  endpoint) backing the AI Assistant's tool-calling loop, replacing the
  single hardcoded provider.
- A built-in, zero-config, no-API-key local provider backed by Needle 2
  (internal/needle) for fully offline tool-calling, wired in as a one-click
  preset. Requires the operator to separately download the Needle 2 binary
  and point FERRUM_NEEDLE_BIN at it -- Ferrum never fetches executable
  content from the network itself; see README "Built-in LLM (Needle 2)".
- System settings (CORS allow-list, instance-wide toggles) moved to the
  admin Settings UI; environment variables are now scoped to true
  bootstrap-level config only (listen address, TLS, DB connection, secret,
  optional Needle binary path).
- Fixed: node Journal tab 502'ing with "unexpected end of JSON input" on an
  empty response, and separately with a decode error on PVE versions that
  return a bare-string journal line instead of the documented {n,t} object.
- Fixed: bottom content padding disappearing on every page except the AI
  Assistant (an unconditional h-full on the content wrapper let overflowing
  content bleed through where the padding should render).
- Fixed: Profile page felt cramped despite a wide viewport (stray max-w-2xl
  cap not present on the equivalent Settings page).
- Test coverage added for the previously-untested MCP package and the new
  Needle adapter (20 new Go tests), plus a regression test for the journal
  decode fix.
2026-09-06 13:26:30 +05:30
Anand 66fc770ecc Show LXC container IPs without needing a guest agent
Unlike a QEMU VM, an LXC container shares the host kernel, so the host can
read its network namespace directly — Proxmox's own
/nodes/{node}/lxc/{vmid}/interfaces exposes exactly that, no in-guest
agent required at all. Wired up via the same guestAgentNetwork endpoint
and AgentNetworkInterface shape the QEMU path already used, so the guest
detail dialog's Network tab now shows live IPs for containers too instead
of a static "containers don't need this" message that never actually
fetched anything.
2026-09-04 00:31:33 +05:30
Anand 7295a28ec9 Opt-in SSO single logout, nice chart scales, storage usage donut, topology export/snap, disk SMART + temperature
- OIDC: RP-Initiated Logout is now opt-in (default off) via a new
  'single_logout' setting, with the exact post-logout redirect URL shown
  in Settings for the admin to register at their provider. Fixes the
  regression from last time: enabling it unconditionally broke sign-out
  for anyone whose IdP hadn't been told to trust the redirect yet
  (Keycloak's invalid_redirect_uri, browser stuck on a stale page).
- formatBytes shows up to 2 decimals (was an adaptive 0-or-1 rule); every
  bytes/rate chart now computes a rounded 'nice' axis scale (0/5/10/15/20
  GB, the standard Heckbert algorithm) and locks every tick + the tooltip
  to one consistent unit derived from the axis's own max.
- Storage page: the capacity donut is now sized by used bytes + a free
  remainder instead of by total capacity share, which was always 100%
  the moment there was only one pool — completely disconnected from the
  "224 GB used" text next to it.
- Topology: Export SVG (fits the full diagram regardless of current pan/
  zoom) and a Snap-to-grid toggle.
- New Disks tab on the node detail page: every physical disk with model/
  serial/size/type, PASSED/FAILED health, SSD/NVMe wearout %, and a
  per-drive temperature read from SMART, plus a full SMART attribute
  table per disk. Backed by new /nodes/{node}/disks and
  /nodes/{node}/disks/smart endpoints.
- CPU/GPU temperature is not exposed by Proxmox's own API (no built-in
  lm-sensors/nvidia-smi integration) and isn't something this can add
  without a node-side agent Proxmox doesn't ship — disk temperature via
  SMART is the thermal data actually available.
2026-09-04 00:22:53 +05:30
Anand d04b769670 Fix storage metrics double-counting, sign-out, SMTP plaintext, and OIDC gaps
- Fleet Overview's Storage KPI and per-connection totals (internal/api/
  overview.go) were summing shared storage once per node that mounts it,
  since PVE's own Shared flag is unreliable for storage added per-node
  without ticking it. New dedupeSharedStorage collapses those by an exact
  (name, total, used) match within a connection; local-only plugin types
  are never touched. Mirrored on the Storage page, which also now
  disambiguates same-named local pools (e.g. every node's "local-lvm") by
  node so a bar chart's category axis never gets duplicate labels.
- Sign-out: React Query keeps a query's last-successful data through a
  failed refetch, so the cached signed-in user survived the post-logout
  401 and the app never noticed. signOut() now pins ["auth","me"] to null
  directly instead of invalidating-and-hoping a refetch lands in time.
- SMTP: net/smtp.SendMail opportunistically attempts STARTTLS whenever the
  server advertises it regardless of the admin's "Use STARTTLS" toggle, so
  an internal relay with a bad cert failed sends the admin explicitly asked
  to be plaintext. A dedicated sendPlain path never attempts TLS.
- OIDC: added an "Auto-create new accounts" toggle — when off, a
  first-time SSO login for an unrecognized identity is refused instead of
  silently provisioning one. Also added RP-Initiated Logout: signing out of
  an SSO session now also ends the session at the identity provider when it
  advertises support for it (end_session_endpoint), via a stored id_token
  on OIDC sessions (new sessions.oidc_id_token column).
- ResourceAreaChart tooltips now fall back to the same yTickFormatter given
  to the axis when a series has no formatter of its own — fixes raw
  unformatted numbers in Fleet Trend and every RRD chart built on it
  (Node/Guest detail) that only ever set the axis formatter.
2026-09-03 23:17:38 +05:30
Anand 61ee806af6 Add realtime polling defaults, Gotify/SMTP notifications, UI-configurable OIDC, and expanded admin settings
- Default all queries to a 20s poll + refetch-on-focus (main.tsx) instead of
  a per-page opt-in, so every page/widget stays live without manual tuning.
- New internal/notify package: Gotify and SMTP (stdlib net/smtp, STARTTLS
  and implicit-TLS-on-465) notifications, each independently optional. Fires
  from the alert evaluator on new alert triggers; admin-configurable from
  Settings with a send-test-notification action per channel.
- OIDC/SSO moved from config.yaml-only to a DB-backed, admin-editable
  Settings card — swaps the live client with no restart. config.yaml is
  used to seed the database once on first boot after upgrading.
- New Security settings: session TTL, login lockout policy, and a real
  "require 2FA for admins" enforcement (requireTOTPEnrolled middleware)
  that blocks non-enrolled admins from everything but /profile and logout.
- New org-wide default preferences (theme/accent/look/landing page) for
  brand-new accounts, plus a personal landing-page picker and an
  email-me-alerts opt-in on Profile.
- Storage page: separate Local vs Shared/External storage tables and
  capacity donuts, fixing shared-storage totals that were being summed once
  per node that mounts them (e.g. a 2TB NFS share on 4 nodes read as 8TB).
- RankedBarChart: stop the longest bar's value label wrapping onto two
  lines (recharts auto-wraps LabelList when space is tight).
2026-09-03 22:00:41 +05:30
Anand 29f0c18dac Expand looks to 9 and add structural per-look chrome
- Add glassmorphism, neumorphism, and brutalist looks (backend validation,
  theme.tsx types, AppearanceCard picker) alongside the existing six.
- Give looks real structural knobs, not just color: sidebar width/border,
  header height/blur are now CSS vars AppShell reads, and card shadow/hover
  states are fully token-driven instead of a hardcoded dark-mode override.
- Sidebar nav active state drops its hard box border on glass/neumorphic
  looks (glow pill / inset shadow) instead of always drawing a box.
- Fix RankedBarChart's value label wrapping onto two lines on the longest
  bar (recharts auto-wraps LabelList when space gets tight) by rendering
  it as a plain unconstrained <text> with a wider right margin.
2026-09-03 15:44:24 +05:30
Anand 29a9519ed7 Fix contaminated default theme and expand to 6 distinct looks
The GitHub merge of feature/glass-flight-deck and feature/theme-selector
left :root/.dark (the "no [data-look] override" baseline) set to Glass
Flight Deck's tokens instead of the plain Enterprise baseline, so every
look inherited its Oswald type, tight radii, persistent caution bar, and
corner brackets regardless of which one was selected — the themes were
indistinguishable because they were all secretly the same rejected design
underneath. Restores :root/.dark to a clean Enterprise baseline and moves
every Glass-Flight-Deck-specific token into its own explicit
[data-look="glassFlightDeck"] block, same as the others.

Also makes .panel-label's tracked-caps transform and .corner-frame's
visible border look-gated (glassFlightDeck only) instead of global, and
MasterCautionBar look-aware (persistent nameplate only in Glass Flight
Deck; a conventional reactive banner — hidden when nominal — everywhere
else), so no look can leak another look's structural signature.

Expands the six-value Look enum/backend whitelist with two new designs:
- Midnight: modern dev-tool dark (Nord/GitHub-dark-adjacent), cool
  blue-slate ground with a visible glow instead of a hard shadow.
- Paper: ultra-minimal and warm, off-white ground, zero shadow, hairline
  borders only, with a genuinely warm (not blue-black) dark variant.

All six verified live end-to-end (build/typecheck clean, each rendered and
screenshotted in both themes where applicable).
2026-09-03 14:52:13 +05:30
Anand b6d15bc5f6 Add a Look & Feel selector: Enterprise / Proxmox-native / Terminal
Adds a whole-app visual-register switch, orthogonal to the light/dark
toggle and accent color, via a new [data-look] token layer in index.css:

- Enterprise (default): unchanged clean SaaS baseline.
- Proxmox-native: utilitarian and dense — plain system font, smaller root
  scale, zero radius, no shadows, flat bordered panels.
- Terminal: quiet and precise — the whole UI (not just data) sets in
  monospace, no display face, flat near-zero-radius panels.

Persisted server-side alongside theme/accent (new `look` column on
user_preferences, migration 00008) with a live-scoped preview per option
in Settings > Appearance.
2026-09-03 12:28:50 +05:30
Anand 9ab6b4340a Initial commit 2026-09-03 08:36:04 +05:30