gsadmin/pulse
1
0
Fork 0
mirror of https://github.com/rcourtman/Pulse.git synced 2026-09-22 19:23:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(pbs): Resolve PBS data display issues and update docs - Corrects PBS API interaction logic, retains logging/fetch improvements, updates README/.env.example based on testing.

Browse Source
This commit is contained in:
courtmanr@gmail.com
2025-04-29 09:45:25 +01:00
parent e1fb099357
commit 4fed85dbf0
3 changed files with 40 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+5 -3
View File
@@ -28,14 +28,16 @@ PROXMOX_TOKEN_SECRET=your-api-token-secret-uuid
# --- Proxmox Backup Server (PBS) Integration (Optional) ---
# Only API Token authentication is supported.
# Use consecutive numbers (_2, _3, ...) for additional PBS instances
# Use consecutive numbers (_2, _3, ...) for additional PBS instances (Not currently supported by Pulse)
# PBS_HOST=your-pbs-ip-or-hostname
# PBS_TOKEN_ID=your-pbs-token-id@pbs!my-token
# PBS_TOKEN_SECRET=your-pbs-token-secret
# Optional: Specify custom port if not default 8007
# PBS_PORT=8007
# Optional: Provide a display name for this PBS instance
# PBS_NODE_NAME=My Primary PBS
# Required (Unless Token has Sys.Audit): Internal hostname of the PBS server.
# Found using 'hostname' command on the PBS server via SSH.
# See README for details on why this is usually required with API tokens.
# PBS_NODE_NAME=your-pbs-internal-hostname
# Optional: Set to true to allow self-signed certificates (default: true)
# PBS_ALLOW_SELF_SIGNED_CERTS=true
README.md
+12 -12
View File
@@ -33,13 +33,13 @@ A lightweight monitoring application for Proxmox VE that displays real-time stat
### Environment Variables
1. **Copy Example File:** This application requires environment variables for configuration. Copy the example environment file from `server/.env.example` to `server/.env`.
1. **Copy Example File:** This application requires environment variables for configuration. Copy the example environment file `.env.example` to `.env`.
```bash
cp server/.env.example server/.env
cp .env.example .env
```
2. **Edit `.env`:** Open `server/.env` in a text editor and update the values for your Proxmox environment, including the Host, Token ID, and Token Secret obtained below.
2. **Edit `.env`:** Open `.env` in a text editor and update the values for your Proxmox environment, including the Host, Token ID, and Token Secret obtained below.
The following variables are available:
- `PROXMOX_HOST`: URL of your Proxmox server (e.g., `https://your-proxmox-ip:8006`).
@@ -70,7 +70,7 @@ A lightweight monitoring application for Proxmox VE that displays real-time stat
- `PBS_HOST`: URL of your Proxmox Backup Server (e.g., `https://your-pbs-ip-or-hostname:8007`).
- `PBS_TOKEN_ID`: Your PBS API Token ID (e.g., `user@pam!tokenid`). Create this in the PBS UI (see below).
- `PBS_TOKEN_SECRET`: Your PBS API Token Secret.
- `PBS_NODE_NAME`: (Potentially Required) The internal hostname of your PBS server. **Crucially, this might be different from the hostname used in `PBS_HOST`**. See detailed explanation below.
- `PBS_NODE_NAME`: **Required (Unless Token has Sys.Audit)** The internal hostname of your PBS server (e.g., the output of `hostname` on the PBS server). **This is generally required when using API tokens**, as the endpoint used for automatic node discovery (`/api2/json/nodes`) is typically restricted for tokens (see below). Crucially, this might be different from the hostname used in `PBS_HOST`.
- `PBS_ALLOW_SELF_SIGNED_CERTS`: (Optional) Set to `true` if your PBS server uses self-signed SSL certificates. Defaults to `false`.
- `PBS_PORT`: (Optional) Port for the PBS API. Defaults to `8007`.
@@ -78,9 +78,9 @@ A lightweight monitoring application for Proxmox VE that displays real-time stat
**Why `PBS_NODE_NAME` is Important:**
Pulse needs to query task lists specific to the PBS node (e.g., `/api2/json/nodes/{nodeName}/tasks`). While Pulse attempts to discover this node name automatically, this can fail due to API limitations or specific permission issues (like accessing `/api2/json/nodes`, which is not a standard PBS management endpoint).
Pulse needs to query task lists specific to the PBS node (e.g., `/api2/json/nodes/{nodeName}/tasks`). It attempts to discover this node name automatically by querying the `/api2/json/nodes` endpoint first. However, **this endpoint is typically restricted for API tokens** (returning a 403 Forbidden error), even for tokens with high privileges, unless the `Sys.Audit` permission is explicitly granted on the root path (`/`).
If Pulse cannot fetch task data correctly (which might result in recent backups showing as 'stale'), you likely need to set `PBS_NODE_NAME` manually.
Therefore, **setting `PBS_NODE_NAME` in your `.env` file is the standard and recommended way** to ensure Pulse can correctly query the task endpoints for your PBS instance when using API token authentication. If it's not set and automatic discovery fails due to permissions, Pulse will be unable to fetch task data.
**How to find your PBS Node Name:**
1. **SSH:** Log into your PBS server via SSH and run the command `hostname`. The output is the value needed for `PBS_NODE_NAME`.
@@ -132,7 +132,7 @@ An API token is recommended for connecting Pulse to Proxmox.
*Note: Assigning the `PVEAuditor` role at the root path (`/`) with `Propagate` checked is crucial for Pulse to discover and monitor all nodes, VMs, containers, and storage in your cluster.*
5. **Update your `server/.env` file** with the `Token ID` (which looks like `user@realm!tokenid`, e.g., `pulse-monitor@pam!pulse`) and the `Secret` you saved.
5. **Update your `.env` file** with the `Token ID` (which looks like `user@realm!tokenid`, e.g., `pulse-monitor@pam!pulse`) and the `Secret` you saved.
### Creating a Proxmox Backup Server API Token
@@ -159,10 +159,10 @@ If you are configuring PBS monitoring, you need a separate API token created wit
* **Role:** `Audit` (This role provides necessary read-only access, including system status and task history).
* Ensure `Propagate` is checked.
* Click `Add`.
*
* **Note on Permissions & API Behavior:** While minimal roles like `DatastoreAudit` might seem sufficient, testing (on PBS v3.3.4) revealed specific API behavior with tokens. GET requests to `/tasks` failed with a `400 Bad Request ("value does not match the regex pattern")` if the request included a `Content-Type` header (which some HTTP clients add by default), whereas requests omitting this header succeeded. Pulse now includes a workaround for this client-side. However, using the broader `Audit` role on the root path `/` also ensures Pulse can reliably access all necessary data (datastores, snapshots, task history) for full monitoring functionality. This role is still read-only.
5. **Update your `server/.env` file** with the PBS `Token ID` (`PBS_TOKEN_ID`) and the `Secret` (`PBS_TOKEN_SECRET`).
* **Note on Permissions:** The `Audit` role granted on the root path (`/`) provides sufficient read-only access for Pulse to monitor datastores, snapshots, and task history.
5. **Update your `.env` file** with the PBS `Token ID` (`PBS_TOKEN_ID`) and the `Secret` (`PBS_TOKEN_SECRET`).
### Required Permissions
@@ -224,7 +224,7 @@ Using Docker Compose is the recommended way to run the application in a containe
**Steps:**
1. **Configure Environment:** Ensure you have created and configured your `server/.env` file as described in the [Environment Variables](#environment-variables) section above.
1. **Configure Environment:** Ensure you have created and configured your `.env` file as described in the [Environment Variables](#environment-variables) section above.
2. **Run:** Navigate to the project root directory in your terminal and run:
```bash
@@ -243,7 +243,7 @@ To stop the container(s) defined in the `docker-compose.yml` file, run:
docker compose down
```
*Note: If you modify the `server/.env` file after the container is already running, you may need to restart the container for the changes to take effect. You can do this by running `docker compose down` followed by `docker compose up -d`, or by using `docker compose up -d --force-recreate`.*
*Note: If you modify the `.env` file after the container is already running, you may need to restart the container for the changes to take effect. You can do this by running `docker compose down` followed by `docker compose up -d`, or by using `docker compose up -d --force-recreate`.*
### Alternative: Quick Start with Inline Variables
server/index.js
+23 -53
View File
@@ -280,37 +280,12 @@ async function initializeAllPbsClients() {
httpsAgent: new https.Agent({
rejectUnauthorized: !config.allowSelfSignedCerts
}),
// REMOVED default headers here
// headers: { 'Content-Type': 'application/json' }
headers: { 'Content-Type': 'application/json' }
});
pbsAxiosInstance.interceptors.request.use(reqConfig => {
// Correct PBS format: PBSAPIToken=TOKENID:TOKENSECRET
reqConfig.headers.Authorization = `PBSAPIToken=${config.tokenId}:${config.tokenSecret}`;
// ---> WORKAROUND for PBS API Bug <---
// PBS API (tested on 3.3.4) incorrectly returns 400 Bad Request ("value does not match regex pattern")
// on GET requests (e.g., /tasks) if a 'Content-Type' header is present when using API Token authentication.
// Therefore, explicitly remove 'Content-Type' for GET requests.
// It seems fine/required for POST/PUT requests, so add it back for non-GET.
// See Bugzilla #6365 for related details.
if (reqConfig.method && reqConfig.method.toLowerCase() !== 'get') {
reqConfig.headers['Content-Type'] = 'application/json';
} else {
// Ensure Content-Type is removed for GET requests
delete reqConfig.headers['Content-Type'];
}
// ---> END WORKAROUND <---
// ---> ADD: Set default Accept header (optional but good practice)
if (!reqConfig.headers['Accept']) {
reqConfig.headers['Accept'] = 'application/json, text/plain, */*';
}
// ---> END ADD
// ---> DEBUG: Log outgoing request headers (can be removed later)
// console.log(`DEBUG: Axios Request Headers for ${config.name}:`, JSON.stringify(reqConfig.headers)); // REMOVED
// ---> END DEBUG
return reqConfig;
});
@@ -326,7 +301,7 @@ async function initializeAllPbsClients() {
});
clientData = { client: pbsAxiosInstance, config: config };
console.log(`INFO: Successfully initialized client for instance '${config.name}' (Token Auth)`);
console.log(`INFO: [PBS Init] Successfully initialized client for instance '${config.name}' (Token Auth)`);
} else {
// This case should not be reachable anymore if loadPbsConfig only creates 'token' authMethod configs
console.error(`ERROR: Unexpected authMethod '${config.authMethod}' found during PBS client initialization for: ${config.name}`);
@@ -341,7 +316,7 @@ async function initializeAllPbsClients() {
});
await Promise.allSettled(initPromises);
console.log(`INFO: Finished initialization. ${Object.keys(pbsApiClients).length} / ${pbsConfigs.length} PBS clients initialized successfully.`);
console.log(`INFO: [PBS Init] Finished initialization. ${Object.keys(pbsApiClients).length} / ${pbsConfigs.length} PBS clients initialized successfully.`);
}
if (Object.keys(apiClients).length === 0 && pbsConfigs.length === 0) {
@@ -1250,19 +1225,16 @@ async function fetchAllPbsTasksForProcessing(pbsClient, nodeName) {
return { tasks: null, error: true };
}
try {
const sinceTimestamp = Math.floor((Date.now() - 7 * 24 * 60 * 60 * 1000) / 1000); // RESTORED
const requestUrl = `/nodes/${nodeName}/tasks`;
const requestParams = {
since: sinceTimestamp, // RESTORED
limit: 1000, // Kept (was 1000 after increasing from 20)
errors: 1, // RESTORED
};
const paramsToSend = Object.keys(requestParams).length > 0 ? { params: requestParams } : {};
// ---> DEBUG: Log outgoing request URL and params
// console.log(`DEBUG: Axios GET Request to ${pbsClient.config.name} - URL: ${requestUrl}, ParamsObj: ${JSON.stringify(paramsToSend)}`); // REMOVED
// ---> END DEBUG
const response = await pbsClient.client.get(requestUrl, paramsToSend);
const sinceTimestamp = Math.floor((Date.now() - 7 * 24 * 60 * 60 * 1000) / 1000);
const response = await pbsClient.client.get(`/nodes/${nodeName}/tasks`, {
params: {
since: sinceTimestamp,
limit: 1000, // Fetch a larger number to cover 7 days of various tasks
errors: 1,
}
});
const allTasks = response.data?.data ?? [];
console.log(`INFO: Fetched ${allTasks.length} tasks from PBS for processing.`);
return { tasks: allTasks, error: false };
} catch (error) {
console.error(`ERROR: Failed to fetch PBS task list for node ${nodeName} (${pbsClient.config.name}): ${error.message}`, error.stack);
@@ -1334,10 +1306,10 @@ function processPbsTasks(allTasks) {
// Process and sort recent tasks for each category
const sortTasksDesc = (a, b) => (b.startTime || 0) - (a.startTime || 0);
const recentBackupTasks = taskResults.backup.list.map(createDetailedTask).sort(sortTasksDesc).slice(0, 100);
const recentVerifyTasks = taskResults.verify.list.map(createDetailedTask).sort(sortTasksDesc).slice(0, 100);
const recentSyncTasks = taskResults.sync.list.map(createDetailedTask).sort(sortTasksDesc).slice(0, 100);
const recentPruneGcTasks = taskResults.pruneGc.list.map(createDetailedTask).sort(sortTasksDesc).slice(0, 100);
const recentBackupTasks = taskResults.backup.list.map(createDetailedTask).sort(sortTasksDesc).slice(0, 20);
const recentVerifyTasks = taskResults.verify.list.map(createDetailedTask).sort(sortTasksDesc).slice(0, 20);
const recentSyncTasks = taskResults.sync.list.map(createDetailedTask).sort(sortTasksDesc).slice(0, 20);
const recentPruneGcTasks = taskResults.pruneGc.list.map(createDetailedTask).sort(sortTasksDesc).slice(0, 20);
console.log(`INFO: Processed PBS Tasks - Backup: ${taskResults.backup.list.length} (OK: ${taskResults.backup.ok}, Fail: ${taskResults.backup.failed}), Verify: ${taskResults.verify.list.length} (OK: ${taskResults.verify.ok}, Fail: ${taskResults.verify.failed}), Sync: ${taskResults.sync.list.length} (OK: ${taskResults.sync.ok}, Fail: ${taskResults.sync.failed}), Prune/GC: ${taskResults.pruneGc.list.length} (OK: ${taskResults.pruneGc.ok}, Fail: ${taskResults.pruneGc.failed})`);
@@ -1395,13 +1367,11 @@ async function fetchPbsTaskSummaryByType(pbsClient, nodeName, taskTypes) {
async function fetchPbsDatastoreData(pbsClient) {
// Fetches datastore usage details from PBS using the /status/datastore-usage endpoint
console.log(`INFO: Fetching PBS datastore data for ${pbsClient.config.name}...`);
console.log("INFO: Fetching PBS datastore data...");
let datastores = [];
const primaryUrl = '/status/datastore-usage';
const fallbackUrl = '/config/datastore';
try {
// Fetch usage stats for all datastores at once
const usageResponse = await pbsClient.client.get(primaryUrl);
const usageResponse = await pbsClient.client.get('/status/datastore-usage');
const usageData = usageResponse.data?.data ?? [];
if (usageData.length > 0) {
@@ -1422,15 +1392,14 @@ async function fetchPbsDatastoreData(pbsClient) {
}
} catch (usageError) {
console.error(`ERROR: Failed to fetch PBS datastore usage via ${primaryUrl} for ${pbsClient.config.name}: ${usageError.message}. Trying fallback ${fallbackUrl}.`, usageError.stack);
console.error(`ERROR: Failed to fetch PBS datastore usage via /status/datastore-usage for ${pbsClient.config.name}: ${usageError.message}. Trying fallback /config/datastore.`, usageError.stack);
// --- Fallback Logic ---
try {
const configResponse = await pbsClient.client.get(fallbackUrl);
const configResponse = await pbsClient.client.get('/config/datastore');
const datastoresConfig = configResponse.data?.data ?? [];
if (datastoresConfig.length > 0) {
console.log(`INFO: Fetched config for ${datastoresConfig.length} PBS datastores (fallback). Status unavailable.`);
// Map the received data to the expected format
datastores = datastoresConfig.map(dsConfig => ({
datastores = datastoresConfig.map(dsConfig => ({
name: dsConfig.name,
path: dsConfig.path,
total: null, // Usage/Status info unavailable from config
@@ -1442,12 +1411,13 @@ async function fetchPbsDatastoreData(pbsClient) {
console.warn("WARN: Fallback fetch of PBS datastore config also returned empty data.");
}
} catch (configError) {
console.error(`ERROR: Fallback fetch of PBS datastore config (${fallbackUrl}) for ${pbsClient.config.name}: ${configError.message}`, configError.stack);
console.error(`ERROR: Fallback fetch of PBS datastore config (/config/datastore) for ${pbsClient.config.name}: ${configError.message}`, configError.stack);
if (configError.response) { // Log more detail if available
console.error(`Fallback error details: Status=${configError.response.status}, Data=${JSON.stringify(configError.response.data)}`);
}
// Keep datastores as empty array if both primary and fallback attempts fail
}
// --- End Fallback ---
}
console.log(`INFO: Finished fetching PBS datastore data. Found ${datastores.length} datastores.`);