Optimized Observability for Proxmox VE:
The Looking Glass Architecture
A self-hosted, lightweight telemetry console for Proxmox VE clusters, utilizing VictoriaMetrics and VictoriaLogs to optimize resource utilization.
Resource Optimization Profile
Looking Glass is a self-hosted telemetry console. By deploying the ShiftMon stack (VictoriaMetrics + VictoriaLogs) instead of traditional, resource-intensive TSDB architectures, it maintains real-time instrumentation with highly optimized storage and memory usage.
Operating a Proxmox VE cluster requires continuous monitoring, yet traditional solutions present an operational trade-off.
Standard monitoring patterns based on full-scale Prometheus and Grafana deployments introduce significant overhead. These systems often consume substantial CPU, memory, and NVMe write endurance simply to collect and store the telemetry of the virtual machines and LXC containers they are intended to observe.
Looking Glass addresses this problem by acting as a streamlined, self-hosted frontend designed specifically for Proxmox metric collections, drastically reducing system footprints in both enterprise environments and home infrastructure.
System Architecture: Under the Hood
Rather than utilizing persistent, high-frequency local scraping agents on the hypervisor, Looking Glass interfaces directly with Proxmox VE's built-in metric servers.
Proxmox VE Cluster Node │ ├── Native InfluxDB Push Metric Protocol ──► VictoriaMetrics (TSDB Engine) └── Telegraf Syslog Integration (RFC 5424) ──► VictoriaLogs (Log Processor)
The software utilizes a lightweight backend coupled with a modern web administration console. It queries the data endpoints directly from VictoriaMetrics and VictoriaLogs. In standard production testing, this architecture demonstrated up to a 92% reduction in memory footprint compared to baseline Prometheus installations, while maintaining sub-second syslog ingestion latency.
Interactive Console: Looking Glass Simulator
A functional replica of the Looking Glass interface. Switch logical workspaces, adjust telemetry limits, and inspect the runbook diagnostic tool inside the "Active Alerts" tab.
Target: PVEHostHighCPU
Diagnosis: Automated cross-referencing of metrics indicates that the hypervisor resource exhaustion is caused by an I/O wait lock from lxc-database-04 (ID: 301).
Remediation Steps:
- Establish SSH connection to hypervisor node pve-node-04.
- Halt the target container with
pct stop 301. - Inspect guest system storage limits and database transactions before restarting.
Key Technical Advantages
1. Logical Environment Isolation
Standard monitoring solutions require complex dashboard filters or distinct server endpoints to segregate systems. Looking Glass natively supports Logical Environments directly within its primary interface.
Hypervisor nodes are grouped into custom environments, such as Production, Staging, or Lab. This system environment filter actively segments telemetry dashboards, centralized syslog query pipes, and active triggers. The configuration state is cached client-side in the browser, eliminating unnecessary remote schema calls.
2. Local Runbook Diagnostics
Real-time diagnostic procedures should accompany system alerts. Looking Glass maps alerts triggered by vmalert or Alertmanager to local runbook guidelines.
When a resource utilization limit is exceeded or a storage volume approaches capacity, the system cross-references active events with local operational runbooks. This design minimizes troubleshooting latency, providing operators with actionable restoration guidelines without requiring immediate external database lookups.
3. Strategic Platform Interoperability
Looking Glass serves as a highly consolidated administrative console. It visualizes general cluster resource health, guest container statistics, and active alerts while avoiding UI clutter.
If deep inspection of the raw dataset is required during troubleshooting, the system includes native Platform Links. These deep links map existing queries directly to advanced visualization tools (Grafana, VictoriaMetrics VMUI, or VictoriaLogs engine), streamlining the pivot to deeper analysis.
Configuration Limits & Sizing Guidelines
Looking Glass is a self-hosted platform. It utilizes environment variable definitions (such as SAAS_USER_TIER) to enforce scaling limitations and optimize system memory allocation:
| Profile Name | Active Series Cap | Recommended Nodes | Data Retention |
|---|---|---|---|
| Standard (Free) | 10,000 | 2 Nodes | 7 Days |
| Pro | 100,000 | 10 Nodes | 30 Days |
| Enterprise | Unlimited | Unlimited | 90 Days |
To prevent server resource issues, the internal telemetry monitoring system (managed by the UpsellHook system service) tracks active series metrics, displaying system-state diagnostics before limits are reached.
Deployment Troubleshooting Guidelines
Review these standard troubleshooting pathways if you encounter configuration discrepancies during installation:
Indicates that the system backend service cannot establish connectivity with the target TSDB database. Verify the status of the VictoriaMetrics daemon and check the VM_CLOUD_URL variable configured within the deployment environment.
The console retrieves active telemetry series by polling the /api/v1/status/tsdb administrative endpoint. Confirm that administrative API routes are enabled and accessible on your VictoriaMetrics server.
Metrics and logs use distinct ingest pipelines. Verify that your Proxmox hosts are executing Telegraf and that rsyslog configurations are correctly routing data stream entries to the VictoriaLogs ingress port.
Deployment Blueprint & Configuration
Deploy Looking Glass into your local infrastructure using the sample Docker Compose specification and environment variables provided below.
version: "3.8"
services:
backend:
image: cramd/looking-glass-backend:latest
ports:
- "5000:5000"
env_file:
- .env
restart: unless-stopped
frontend:
image: cramd/looking-glass-frontend:latest
ports:
- "3000:3000"
environment:
- NEXT_PUBLIC_API_URL=http://localhost:5000
restart: unless-stopped
Source and deployment docs on GitHub: cramd/Looking-Glass-. Container images: cramd/lookingglass on Docker Hub.