gsadmin/pulse
1
0
Fork 0
mirror of https://github.com/rcourtman/Pulse.git synced 2026-09-23 11:46:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: improve Docker entrypoint script and clean up test files

Browse Source 
Docker improvements:
- Fixed entrypoint script to properly handle UID/GID changes
- Simplified user/group recreation logic to avoid conflicts
- Properly handles switching between different UID/GID values

Cleanup:
- Removed temporary test scripts and files
- Removed PROXMOX_ENDPOINTS.md documentation
- Cleaned up various test Python and shell scripts

Docker testing confirmed:
- Data persistence working across restarts
- UID/GID configuration working correctly
- Volume backup/restore functioning properly
This commit is contained in:
Pulse Monitor
2025-08-16 17:54:45 +00:00
parent 2983c09a4a
commit e518f69f2c
10 changed files with 11 additions and 535 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PROXMOX_ENDPOINTS.md
-68
View File
@@ -1,68 +0,0 @@
# Proxmox API Endpoint Documentation
## Update Frequencies and Use Cases
Based on empirical testing against Proxmox VE, here's what each endpoint provides:
### Node Metrics
| Endpoint | Update Frequency | Use Case | Data Freshness |
|----------|-----------------|----------|----------------|
| `/nodes` | ~10 seconds | Node list, basic status | Cached/aggregated |
| `/nodes/{node}/status` | **1 second** | Real-time node metrics | Real-time |
| `/cluster/resources?type=node` | ~10 seconds | Cluster overview | Cached |
### VM/Container Metrics
| Endpoint | Update Frequency | Use Case | Data Freshness |
|----------|-----------------|----------|----------------|
| `/nodes/{node}/qemu` | On state change | VM list | Current |
| `/nodes/{node}/qemu/{vmid}/status/current` | **1 second** | Real-time VM metrics | Real-time |
| `/nodes/{node}/lxc` | On state change | Container list | Current |
| `/nodes/{node}/lxc/{vmid}/status/current` | **1 second** | Real-time container metrics | Real-time |
### Storage & Backup
| Endpoint | Update Frequency | Use Case | Data Freshness |
|----------|-----------------|----------|----------------|
| `/nodes/{node}/storage` | ~30 seconds | Storage overview | Cached |
| `/nodes/{node}/storage/{storage}/content` | On change | Backup listings | Current |
| `/nodes/{node}/tasks` | On change | Task status | Current |
## Key Findings
1. **Real-time endpoints** (`/status` and `/status/current`) update every second
2. **List endpoints** (`/nodes`, `/qemu`, `/lxc`) are cached/aggregated
3. **pvestatd** updates different endpoints at different rates
4. The commonly cited "10 second update interval" only applies to aggregated endpoints
## Recommended Polling Strategy
For real-time monitoring:
- **Node metrics**: Poll `/nodes/{node}/status` every 1-2 seconds
- **VM/Container metrics**: Poll `/status/current` endpoints every 1-2 seconds
- **Storage**: Poll every 30-60 seconds (changes less frequently)
- **Backup tasks**: Poll every 30-60 seconds or on-demand
## Test Results
### Test 1: /nodes endpoint
- Update interval: ~10 seconds
- Shows aggregated data across cluster
### Test 2: /nodes/{node}/status endpoint
- Update interval: **1 second**
- Provides real-time CPU, memory, disk, uptime
- This is the endpoint to use for live monitoring
### Test 3: /cluster/resources endpoint
- Update interval: ~10 seconds
- Similar to /nodes but includes VMs/containers
## Implementation Notes
Current Pulse implementation uses:
- `GetNodes()` - Uses `/nodes` (slow, cached)
- `GetNodeStatus()` - Uses `/nodes/{node}/status` (real-time)
We should prioritize GetNodeStatus() data over GetNodes() data for metrics that need to be real-time.
docker-entrypoint.sh
+11 -20
View File
@@ -17,28 +17,19 @@ if [ "$(id -u)" = "0" ]; then
exec "$@"
fi
# Create group if it doesn't exist
if ! getent group pulse >/dev/null 2>&1; then
addgroup -g "$PGID" pulse
else
# Modify existing group GID if different
current_gid=$(getent group pulse | cut -d: -f3)
if [ "$current_gid" != "$PGID" ]; then
delgroup pulse 2>/dev/null || true
addgroup -g "$PGID" pulse
fi
fi
# Check if we need to modify the user/group
current_uid=$(id -u pulse 2>/dev/null || echo "")
current_gid=$(getent group pulse 2>/dev/null | cut -d: -f3 || echo "")
# Create user if it doesn't exist
if ! id -u pulse >/dev/null 2>&1; then
# If user/group don't match, recreate them
if [ "$current_uid" != "$PUID" ] || [ "$current_gid" != "$PGID" ]; then
# Remove existing user and group
deluser pulse 2>/dev/null || true
delgroup pulse 2>/dev/null || true
# Create new group and user with desired IDs
addgroup -g "$PGID" pulse
adduser -D -u "$PUID" -G pulse pulse
else
# Modify existing user UID if different
current_uid=$(id -u pulse)
if [ "$current_uid" != "$PUID" ]; then
deluser pulse 2>/dev/null || true
adduser -D -u "$PUID" -G pulse pulse
fi
fi
# Fix ownership of data directory
test-60s.sh
-30
View File
@@ -1,30 +0,0 @@
#!/bin/bash
TOKEN="pulse-monitor@pam!test-token=a0c05119-0e04-4918-ac94-1fa604259bf1"
URL="https://192.168.0.5:8006/api2/json/nodes"
echo "Testing for 60 seconds..."
last_cpu=""
changes=()
for i in {1..60}; do
response=$(curl -sk -H "Authorization: PVEAPIToken=$TOKEN" "$URL")
cpu=$(echo "$response" | jq -r '.data[] | select(.node=="delly") | .cpu')
if [ "$i" -eq 1 ]; then
echo "Second $i: CPU=$cpu (initial)"
elif [ "$cpu" != "$last_cpu" ]; then
echo "Second $i: CPU changed"
changes+=($i)
fi
last_cpu=$cpu
sleep 1
done
echo ""
echo "Changes at seconds: ${changes[@]}"
echo "Total changes: ${#changes[@]} in 60 seconds"
if [ ${#changes[@]} -gt 0 ]; then
echo "Average interval: $((60 / ${#changes[@]})) seconds"
fi
test-all-endpoints.sh
-50
View File
@@ -1,50 +0,0 @@
#!/bin/bash
TOKEN="pulse-monitor@pam!test-token=a0c05119-0e04-4918-ac94-1fa604259bf1"
AUTH="Authorization: PVEAPIToken=$TOKEN"
BASE="https://192.168.0.5:8006/api2/json"
echo "Testing different Proxmox endpoints for CPU data..."
echo "================================================"
# Test 1: /nodes endpoint
echo -e "\n1. Testing /nodes endpoint (10 samples):"
last=""
for i in {1..10}; do
cpu=$(curl -sk -H "$AUTH" "$BASE/nodes" | jq -r '.data[] | select(.node=="delly") | .cpu')
if [ "$cpu" != "$last" ]; then
echo " Sample $i: CPU=$cpu (changed)"
else
echo " Sample $i: CPU=$cpu"
fi
last=$cpu
sleep 1
done
# Test 2: /nodes/delly/status endpoint
echo -e "\n2. Testing /nodes/delly/status endpoint (10 samples):"
last=""
for i in {1..10}; do
cpu=$(curl -sk -H "$AUTH" "$BASE/nodes/delly/status" | jq -r '.data.cpu // 0')
if [ "$cpu" != "$last" ]; then
echo " Sample $i: CPU=$cpu (changed)"
else
echo " Sample $i: CPU=$cpu"
fi
last=$cpu
sleep 1
done
# Test 3: /cluster/resources endpoint
echo -e "\n3. Testing /cluster/resources endpoint (10 samples):"
last=""
for i in {1..10}; do
cpu=$(curl -sk -H "$AUTH" "$BASE/cluster/resources?type=node" | jq -r '.data[] | select(.node=="delly") | .cpu // 0')
if [ "$cpu" != "$last" ]; then
echo " Sample $i: CPU=$cpu (changed)"
else
echo " Sample $i: CPU=$cpu"
fi
last=$cpu
sleep 1
done
test-api.sh
-7
View File
@@ -1,7 +0,0 @@
#!/bin/bash
for i in {1..10}; do
timestamp=$(date +"%H:%M:%S")
cpu=$(curl -s -H "X-API-Token: 0999c3bdf6d98647da81c00643ea5c4fe4560aaefed9519e" http://localhost:7655/api/state | jq -r '.nodes[] | select(.name=="delly") | .cpu')
echo "$timestamp: $cpu"
sleep 2
done
test-hash.go
-23
View File
@@ -1,23 +0,0 @@
package main
import (
"fmt"
"golang.org/x/crypto/bcrypt"
"os"
)
func main() {
if len(os.Args) < 2 {
fmt.Println("Usage: go run test-hash.go <password>")
os.Exit(1)
}
password := os.Args[1]
hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
fmt.Printf("Error: %v\n", err)
os.Exit(1)
}
fmt.Println(string(hash))
}
test-nodes-endpoint.py
-84
View File
@@ -1,84 +0,0 @@
#!/usr/bin/env python3
"""
Test /nodes endpoint to see update frequency
"""
import time
import json
import subprocess
from datetime import datetime
def get_nodes_data():
"""Get all nodes data"""
try:
result = subprocess.run(
['ssh', 'root@delly', 'pvesh', 'get', '/nodes', '--output-format', 'json'],
capture_output=True,
text=True,
timeout=5
)
if result.returncode == 0:
return json.loads(result.stdout)
except Exception as e:
print(f"Error: {e}")
return None
def main():
print("Testing /nodes endpoint - tracking delly specifically")
print("Polling every 1 second for 30 seconds")
print("-" * 80)
last_cpu = None
last_mem = None
cpu_changes = []
mem_changes = []
for i in range(30):
current_time = datetime.now().strftime('%H:%M:%S')
nodes = get_nodes_data()
if nodes is None:
print(f"{current_time} - Failed to get data")
time.sleep(1)
continue
# Find delly
delly = None
for node in nodes:
if node.get('node') == 'delly':
delly = node
break
if delly is None:
print(f"{current_time} - Delly not found")
time.sleep(1)
continue
cpu = delly.get('cpu', 0)
mem = delly.get('mem', 0)
if last_cpu is not None:
if cpu != last_cpu:
print(f"{current_time} - CPU changed: {last_cpu:.10f} -> {cpu:.10f}")
cpu_changes.append(i)
if mem != last_mem:
delta_mb = (mem - last_mem) / (1024*1024)
print(f"{current_time} - Mem changed: {delta_mb:+.1f} MB")
mem_changes.append(i)
else:
print(f"{current_time} - Initial: CPU={cpu:.10f}, Mem={mem/(1024*1024*1024):.2f} GB")
last_cpu = cpu
last_mem = mem
time.sleep(1)
print(f"\nCPU changes at seconds: {cpu_changes}")
print(f"Memory changes at seconds: {mem_changes}")
if len(cpu_changes) > 1:
intervals = [cpu_changes[i+1] - cpu_changes[i] for i in range(len(cpu_changes)-1)]
print(f"CPU change intervals: {intervals}")
print(f"Average CPU update interval: {sum(intervals)/len(intervals):.1f} seconds")
if __name__ == "__main__":
main()
test-proxmox-fast.py
-78
View File
@@ -1,78 +0,0 @@
#!/usr/bin/env python3
"""
Test with specific node endpoint instead of /nodes
"""
import time
import json
import subprocess
from datetime import datetime
def get_node_stats_specific():
"""Get delly stats from specific node endpoint"""
try:
# Use the specific node endpoint
result = subprocess.run(
['ssh', 'root@delly', 'pvesh', 'get', '/nodes/delly/status', '--output-format', 'json'],
capture_output=True,
text=True,
timeout=5
)
if result.returncode == 0:
node = json.loads(result.stdout)
return {
'cpu': node.get('cpu', 0),
'wait': node.get('wait', 0),
'load': node.get('loadavg', [0])[0] if 'loadavg' in node else 0,
'mem_used': node.get('memory', {}).get('used', 0),
'mem_total': node.get('memory', {}).get('total', 0),
'uptime': node.get('uptime', 0)
}
except Exception as e:
print(f"Error: {e}")
return None
def main():
print("Testing /nodes/delly/status endpoint specifically")
print("Polling every 1 second for 30 seconds")
print("-" * 80)
last_stats = None
changes_at = []
for i in range(30):
current_time = datetime.now().strftime('%H:%M:%S')
stats = get_node_stats_specific()
if stats is None:
print(f"{current_time} - Failed to get stats")
time.sleep(1)
continue
if last_stats is not None:
# Check if CPU changed
if stats['cpu'] != last_stats['cpu']:
delta = stats['cpu'] - last_stats['cpu']
print(f"{current_time} - CPU changed: {last_stats['cpu']:.10f} -> {stats['cpu']:.10f} (delta: {delta:+.10f})")
changes_at.append(i)
# Check memory
if stats['mem_used'] != last_stats['mem_used']:
delta_mb = (stats['mem_used'] - last_stats['mem_used']) / (1024*1024)
print(f"{current_time} - Memory changed: {delta_mb:+.1f} MB")
else:
print(f"{current_time} - Initial CPU: {stats['cpu']:.10f}, Mem: {stats['mem_used']/(1024*1024*1024):.2f} GB")
last_stats = stats
time.sleep(1)
if len(changes_at) > 1:
intervals = [changes_at[i+1] - changes_at[i] for i in range(len(changes_at)-1)]
avg_interval = sum(intervals) / len(intervals) if intervals else 0
print(f"\nChanges detected at seconds: {changes_at}")
print(f"Intervals between changes: {intervals}")
print(f"Average interval: {avg_interval:.1f} seconds")
else:
print(f"\nOnly {len(changes_at)} changes detected in 30 seconds")
if __name__ == "__main__":
main()
test-proxmox-polling.py
-126
View File
@@ -1,126 +0,0 @@
#!/usr/bin/env python3
"""
Test script to monitor how frequently Proxmox API values actually change
"""
import time
import json
import subprocess
from datetime import datetime
def get_node_stats():
"""Get node stats directly from Proxmox API using pvesh"""
try:
result = subprocess.run(
['ssh', 'root@delly', 'pvesh', 'get', '/nodes', '--output-format', 'json'],
capture_output=True,
text=True,
timeout=5
)
if result.returncode == 0:
data = json.loads(result.stdout)
# Find delly specifically
node = None
for n in data:
if n.get('node') == 'delly':
node = n
break
if node is None:
return None
return {
'cpu': node.get('cpu', 0),
'mem': node.get('mem', 0),
'maxmem': node.get('maxmem', 0),
'disk': node.get('disk', 0),
'maxdisk': node.get('maxdisk', 0),
'uptime': node.get('uptime', 0)
}
except Exception as e:
print(f"Error: {e}")
return None
def main():
print("Monitoring Proxmox API for value changes...")
print("Polling every 0.5 seconds to catch any changes")
print("-" * 80)
last_stats = None
last_change_time = None
poll_count = 0
change_count = 0
# Track when each metric last changed
last_changes = {}
# Run for 60 seconds
start_time = time.time()
duration = 60
while time.time() - start_time < duration:
poll_count += 1
current_time = datetime.now().strftime('%H:%M:%S.%f')[:-3]
stats = get_node_stats()
if stats is None:
print(f"{current_time} - Failed to get stats")
time.sleep(0.5)
continue
if last_stats is None:
# First poll
print(f"{current_time} - Initial values:")
print(f" CPU: {stats['cpu']:.10f}")
print(f" Memory: {stats['mem']} / {stats['maxmem']}")
print(f" Disk: {stats['disk']} / {stats['maxdisk']}")
print(f" Uptime: {stats['uptime']}")
for key in stats:
last_changes[key] = current_time
else:
# Check what changed
changes = []
for key in stats:
if stats[key] != last_stats[key]:
time_since_last = None
if key in last_changes:
# Calculate seconds since last change
try:
prev_time = datetime.strptime(last_changes[key], '%H:%M:%S.%f')
curr_time = datetime.strptime(current_time, '%H:%M:%S.%f')
delta = (curr_time - prev_time).total_seconds()
time_since_last = f"{delta:.1f}s"
except:
pass
if key == 'cpu':
changes.append(f"CPU: {last_stats[key]:.10f} -> {stats[key]:.10f} (after {time_since_last})")
elif key in ['mem', 'disk']:
changes.append(f"{key.upper()}: {last_stats[key]} -> {stats[key]} (after {time_since_last})")
elif key == 'uptime':
changes.append(f"Uptime: +{stats[key] - last_stats[key]}s (after {time_since_last})")
last_changes[key] = current_time
if changes:
change_count += 1
print(f"{current_time} - CHANGES DETECTED (poll #{poll_count}):")
for change in changes:
print(f" {change}")
last_change_time = current_time
last_stats = stats
time.sleep(0.5) # Poll every 500ms to catch any changes
# Summary
print("\n" + "=" * 80)
print("SUMMARY:")
print(f"Total polls: {poll_count}")
print(f"Total changes detected: {change_count}")
print(f"Average time between changes: {duration/change_count if change_count > 0 else 0:.1f} seconds")
print("\nTime between changes for each metric:")
# This is approximate based on change count
if change_count > 0:
avg_interval = poll_count / change_count * 0.5
print(f"Estimated update interval: ~{avg_interval:.1f} seconds")
if __name__ == "__main__":
main()
test-raw-curl.sh
-49
View File
@@ -1,49 +0,0 @@
#!/bin/bash
# Simple curl test to check Proxmox update frequency
TOKEN="pulse-monitor@pam!test-token=a0c05119-0e04-4918-ac94-1fa604259bf1"
URL="https://192.168.0.5:8006/api2/json/nodes"
echo "Testing Proxmox API update frequency with raw curl"
echo "Polling every 1 second for 30 seconds"
echo "================================================"
last_cpu=""
last_mem=""
count=0
for i in {1..30}; do
# Get current time
timestamp=$(date +"%H:%M:%S")
# Make API call
response=$(curl -sk -H "Authorization: PVEAPIToken=$TOKEN" "$URL")
# Extract CPU and memory for delly node
cpu=$(echo "$response" | jq -r '.data[] | select(.node=="delly") | .cpu')
mem=$(echo "$response" | jq -r '.data[] | select(.node=="delly") | .mem')
# Check if values changed
if [ "$i" -eq 1 ]; then
echo "$timestamp - Initial: CPU=$cpu, Mem=$((mem / 1024 / 1024 / 1024)) GB"
else
if [ "$cpu" != "$last_cpu" ]; then
echo "$timestamp - CPU CHANGED: $last_cpu -> $cpu"
((count++))
fi
if [ "$mem" != "$last_mem" ]; then
mem_diff=$(( (mem - last_mem) / 1024 / 1024 ))
if [ "$mem_diff" -ne 0 ]; then
echo "$timestamp - MEM CHANGED: ${mem_diff:+}${mem_diff} MB"
fi
fi
fi
last_cpu=$cpu
last_mem=$mem
sleep 1
done
echo ""
echo "Total CPU changes detected: $count in 30 seconds"