Compute (Hypervisor)

The Compute module (id hypervisor, v1.0.0) connects FreeSDN to one or more Proxmox VE clusters. It gives you a unified view of cluster health, node resources, virtual machine and container lifecycle, snapshots, scheduled backups, storage pools, SDN zones, and per-guest firewall rules - all under the same 5-tier RBAC and staged-write safety contract that governs every other FreeSDN adapter.

The Proxmox adapter implements staged apply gates, a circuit breaker, central secret redaction, tenant scoping, SSRF protection, and role gates. See Supported Vendors for the complete contract matrix.

Choose the staged path for review and durable tracking

The gateway-proxmox endpoints stage changes for explicit review and apply. VM/CT power, cloning, same-cluster migration (including bulk power/migration), manual backup and snapshot create/delete/rollback now create and apply durable records immediately. Other direct /api/v1/hypervisor/… writes can still bypass those records when the deployment permits writes. Use the gateway-proxmox staged-write flow for durable operation tracking. Staged apply requires ADAPTER_READ_ONLY=false and force=true, plus confirmation for destructive actions.

Connecting a Proxmox cluster

1. Create an API token in Proxmox VE

In the Proxmox web UI navigate to Datacenter → Permissions → API Tokens. Create a token for a user with PVEAdmin or an equivalent privilege set. Note the token ID (format user@realm!tokenname) and the token secret - the secret is shown only once.

API token authentication is preferred over username/password. If you use username/password, the adapter caches the PVE ticket for 90 minutes (tickets are valid 2 hours; the cache is deliberately shorter to avoid serving an expiring ticket). On a 401 from an idempotent request it drops the cached ticket and retries once.

2. Register the controller in FreeSDN

POST /api/v1/controllers
Content-Type: application/json

{
  "name": "pve-cluster-1",
  "controller_type": "proxmox",
  "host": "pve.lan",
  "port": 8006,
  "use_ssl": true,
  "verify_ssl": false,
  "config": {
    "token_id": "root@pam!freesdn",
    "token_secret": "<token-secret>"
  },
  "site_id": "<site-uuid>"
}

Default connection parameters: port 8006, SSL on, certificate verification off (Proxmox nodes ship with self-signed certs). Set verify_ssl: true only if your cluster uses a trusted CA.

3. Trigger an initial sync

POST /api/v1/discovery/controllers/<controller-uuid>

The sync loop runs automatically every 120 seconds (configurable). Each Proxmox node becomes a ProxmoxNode record in the unified device inventory. Individual VMs and containers are tracked in hypervisor.virtual_machines.

SAVEPOINT isolation per controller

During each sync cycle, each Proxmox controller runs in its own database SAVEPOINT (session.begin_nested()). If one cluster is unreachable, only that cluster’s sync fails - other controllers in the same org continue normally.

Module settings

Setting	Default	Range	Description
sync_interval	120	30-3600 s	How often FreeSDN polls the cluster for node and VM state
show_templates	false	bool	Show VM templates in the Virtual Machines list

Configure via Hypervisor → Settings in the UI or:

PUT /api/v1/modules/org/{organization_id}/hypervisor/settings
Content-Type: application/json

{ "sync_interval": 60, "show_templates": false }

Feature domains

The adapter covers 12 feature domains:

Domain	What you get
Cluster	Status, quorum, resources, log, replication jobs, cluster-wide options
Nodes	Inventory, CPU/memory/disk metrics, RRD history, sensors, services, physical disks, syslog, APT updates, certificates, subscription
VMs (QEMU)	List, config, create, power ops, clone, migrate, resize, template conversion, pending config diff, CloudInit, console proxy, bulk ops
Containers (LXC)	Same lifecycle as VMs; remote-migrate included
Snapshots	Create (with optional RAM state), rollback, delete per VM/CT
Backups	Scheduled job CRUD, manual run, prune, restore from PBS or local storage, backup age report
Storage pools	Browse pools and content by type, ISO/template upload (4 GB cap), volume delete, prune preview
Tasks	List recent tasks, status detail, log tail, stop a running task
HA	Resource and group CRUD
SDN	Read-only configured/running/pending inspection and per-node dry-run summaries; writes unavailable until the reviewed native-lock workflow is qualified
Ceph	Cluster status and detail (404 if Ceph is not deployed on the node)
Firewall	Cluster, node, and per-guest firewall rule CRUD

Key API endpoints

All endpoints mount under /api/v1/hypervisor/. Reads require a valid session (viewer+). Writes require site_admin minimum role. Browse the full surface at /api/v1/docs (enable ENABLE_DOCS=true in non-production environments).

Cluster and fleet

Method	Path	Purpose
GET	/controllers/{id}/dashboard	Cluster dashboard summary
GET	/controllers/{id}/cluster/status	Quorum state, node count, PVE version
GET	/controllers/{id}/cluster/resources	All resources; ?type=node|qemu|lxc|storage|sdn
GET	/controllers/{id}/cluster/log	Cluster log; ?max_entries=1..5000 (default 50)
GET	/fleet/dashboard	Cross-cluster summary across all Proxmox controllers; ?site_id
GET	/fleet/task-statistics	Cross-cluster task statistics; ?site_id

Nodes

Method	Path	Purpose
GET	/controllers/{id}/nodes	List nodes
GET	/controllers/{id}/nodes/{node}	Node detail (CPU, memory, disk, PVE version)
GET	/controllers/{id}/nodes/{node}/services	Node service list
GET	/controllers/{id}/nodes/{node}/disks	Physical disks
GET	/controllers/{id}/nodes/{node}/disks/smart	SMART data; ?disk=/dev/sda
GET	/controllers/{id}/nodes/{node}/syslog	Syslog tail; ?limit=1..500
GET	/controllers/{id}/nodes/{node}/sensors	Sensor/temperature readings
GET	/controllers/{id}/nodes/{node}/rrd	RRD history (LTTB-downsampled); ?timeframe=hour|day|week|month|year&max_points=10..5000
POST	/controllers/{id}/nodes/{node}/reboot	Reboot node (site_admin)
POST	/controllers/{id}/nodes/{node}/shutdown	Shut down node (site_admin)

The node path parameter is validated against ^[a-zA-Z0-9._-]+$ (max 63 chars).

VMs and containers

Method	Path	Purpose
GET	/controllers/{id}/vms	All VMs across nodes; ?type=qemu|lxc
GET	/controllers/{id}/nodes/{node}/vms	VMs on a specific node
GET	/controllers/{id}/nodes/{node}/containers	LXC containers on a node
GET	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/config	VM/CT config (secrets redacted)
POST	/controllers/{id}/vms	Create QEMU VM (site_admin)
POST	/controllers/{id}/containers	Create LXC container (site_admin)
POST	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/action	Power action: start|stop|shutdown|reboot|suspend|resume (site_admin)
PUT	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/config	Update VM/CT config (site_admin)
POST	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/clone	Clone to new VM (site_admin)
POST	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/migrate	Migrate to another node (site_admin)
PUT	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/resize	Resize disk (site_admin)
DELETE	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}	Delete VM/CT - irreversible (site_admin)
POST	/controllers/{id}/bulk-action	Run an action on multiple VMs/CTs (site_admin)
POST	/controllers/{id}/bulk-migrate	Migrate multiple VMs to a target node (site_admin)

vmid is validated as an integer in the range 100-999,999,999. vm_type accepts only qemu or lxc.

VM delete is irreversible

DELETE …/{vm_type}/{vmid} is unconditionally force-gated in the adapter (CAN-016 hardening). Even with ADAPTER_READ_ONLY=false set in the environment, a direct call to this endpoint without an explicit force=true from the staging applier will be refused. The only safe path to delete a VM is via the staged dual-gate - stage the delete operation, have a site_admin review it, then apply.

Snapshots

Method	Path	Purpose
GET	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/snapshots	List snapshots
POST	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/snapshots	Create snapshot; body: request_id, force, snapname (alphanum/_-, ≤40; not current), description (≤255), vmstate bool (QEMU only)
POST	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/snapshots/{snapname}/rollback	Roll back to snapshot; body: request_id, force; query: confirmed=true (site_admin)
DELETE	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/snapshots/{snapname}	Delete snapshot; query: request_id, force=true, confirmed=true (site_admin)

Storage pools and content

Method	Path	Purpose
GET	/controllers/{id}/nodes/{node}/storage	List storage pools with usage stats
GET	/controllers/{id}/nodes/{node}/storage/{storage}/content	Browse content; ?content=images|iso|backup|rootdir|vztmpl|snippets&vmid=
POST	/controllers/{id}/nodes/{node}/storage/{storage}/upload	Upload ISO or template (multipart, 4 GB cap)
GET	/controllers/{id}/nodes/{node}/storage/{storage}/prune-preview	Preview what a prune would remove
POST	/controllers/{id}/nodes/{node}/storage/{storage}/prune	Execute prune with retention policy (site_admin)
DELETE	/controllers/{id}/nodes/{node}/storage/{storage}/content/{volume}	Delete a storage volume (site_admin)

Upload streams through a 1 MB-chunk temp file then posts to PVE. The temp file is removed in a finally block. Uploads beyond 4 GB receive HTTP 413.

Backups

Method	Path	Purpose
GET	/controllers/{id}/backup/jobs	List scheduled backup jobs
POST	/controllers/{id}/backup/jobs	Create backup job (site_admin)
PUT	/controllers/{id}/backup/jobs/{job_id}	Update backup job (site_admin)
DELETE	/controllers/{id}/backup/jobs/{job_id}	Delete backup job (site_admin)
POST	/controllers/{id}/nodes/{node}/{vm_type}/{vmid}/backup	Run manual backup; body: request_id, force, storage, mode, compress (site_admin)
POST	/controllers/{id}/backup/restore	Direct restore is refused; use reviewed proxmox.backup.restore staging below
GET	/controllers/{id}/backup/age-report	Age report; ?threshold_hours=1..8760 (default 24)

SDN zones and VNets

Method	Path	Purpose
GET	/controllers/{id}/sdn/zones	Visible configured zones
GET	/controllers/{id}/sdn/vnets	Visible configured VNets
GET	/controllers/{id}/sdn/controllers	Visible configured SDN controllers, without secret fields
GET	/controllers/{id}/sdn/inspection	Separate configured/running/pending observations, read status, visibility limits and permission metadata
GET	/controllers/{id}/sdn/nodes/{node}/dry-run	Node interface/FRR diff counts; raw content withheld, no network mutation

The former five direct SDN write routes are removed. New SDN staging and legacy replay are rejected with HTTP 409. The SDN tab is an inspection surface, not a complete apply review: it does not inspect every object family, establish atomicity, or verify node and guest connectivity. Use native Proxmox for changes until the durable reviewed lock/apply workflow is qualified. See SDN management for the compatibility boundary and recovery requirements.

HA resources and groups

Method	Path	Purpose
GET	/controllers/{id}/ha/resources	List HA resources
POST	/controllers/{id}/ha/resources	Add VM/CT to HA; sid format (vm|ct):\d+ (site_admin)
DELETE	/controllers/{id}/ha/resources/{sid}	Remove from HA (site_admin)
GET	/controllers/{id}/ha/groups	List HA groups
POST	/controllers/{id}/ha/groups	Create HA group (site_admin)
DELETE	/controllers/{id}/ha/groups/{group}	Delete HA group (site_admin)

Guest agent (QEMU)

Force-gated - extra sensitive

Guest-agent exec and file operations run commands and write files inside your guest OS. They require both the QEMU guest agent running in the VM and the staged dual-gate: ADAPTER_READ_ONLY=false plus operator sign-off. All output (stdout, file contents) is scrubbed through the central redact_secrets filter (~90 sensitive key patterns, camelCase-aware) before it is returned to the caller. Never use exec as a shortcut to bypass staged change review.

Method	Path	Purpose
GET	/controllers/{id}/nodes/{node}/qemu/{vmid}/agent/info	Guest network interfaces (502 if agent unavailable)
POST	/controllers/{id}/nodes/{node}/qemu/{vmid}/agent/exec	Execute command in guest; body: command, input_data - output redacted (site_admin)
GET	/controllers/{id}/nodes/{node}/qemu/{vmid}/agent/exec-status/{pid}	Poll exec stdout/status - redacted (site_admin)
POST	/controllers/{id}/nodes/{node}/qemu/{vmid}/agent/file-read	Read file from guest filesystem - redacted (site_admin)
POST	/controllers/{id}/nodes/{node}/qemu/{vmid}/agent/file-write	Write file into guest filesystem (site_admin)

These endpoints accept site_admin minimum role. The QEMU guest agent package must be installed and running inside the VM.

Staged writes

FreeSDN’s staged Proxmox apply path uses a dual-gate safety contract. The gate has two independent conditions; both must be cleared before a write reaches the cluster:

1. Environment gate: ADAPTER_READ_ONLY must be false on the api container. The shipped Docker Compose stack runs read-WRITE (ADAPTER_READ_ONLY=false) so FreeSDN manages your gear out of the box. Set ADAPTER_READ_ONLY=true in .env for a monitor-only deployment, which keeps all writes refused. This is a single, platform-wide flag: the legacy per-vendor OMADA_READ_ONLY is no longer OR’d in.
2. Explicit apply gate: the operator must submit force=true to apply the staged record. After the environment and permission checks pass, the staging applier passes that intent to the adapter. Destructive actions also require the applicable confirmation.

What this means in practice

With the shipped settings (ADAPTER_READ_ONLY=false), some permitted direct hypervisor actions can reach the cluster without a staged record. Immediate power, clone, same-cluster migration, manual backup and snapshot lifecycle operations now share durable records and task observation; other direct actions retain their own role, confirmation and catastrophic-operation guards. Backup restore, backup prune and storage volume deletion are examples that require the staged path. Use staged changes when review, dispatch history and provider-task observation are required.

In a monitor-only deployment (ADAPTER_READ_ONLY=true), write endpoints in /api/v1/hypervisor/… are refused by their service or adapter read-only gate - the request does not record a pending change and does not touch the cluster.

To stage mutations, use the gateway-proxmox staging endpoints described below, which create PendingChange records and queue them for operator review. An operator with site_admin+ role reviews and applies from the Hypervisor UI (Pending Changes tab) or via:

POST /api/v1/gateway-vpn/changes/{change-id}/apply

The staging endpoints live under /api/v1/gateway-proxmox-{vm,container,snapshot,storage,backup,cluster}/ (gated at include time by enforce_catastrophic_stage_role). You do not call them directly from user-facing code - the UI and Fabric executor drive them.

Staging flow for a VM operation

1. Operator authors a change in the UI (e.g., stop a VM for maintenance).
2. FreeSDN creates a PendingChange record with feature proxmox.vm.stop, stores the payload, and returns a PendingChangeResponse.
3. A site_admin reviews the pending change in the UI.
4. On apply: the staging applier calls the adapter after the gates pass. An acknowledgement marks dispatch as applied; an asynchronous task must still finish.
5. For a returned Proxmox task ID, FreeSDN persists provider_task and observes its progress with read-only checks. Bulk apply pauses at an unresolved task.

Catastrophic-op stage-time role check

The stage queue checks the site_admin role at stage time, not just at apply time. This prevents a lower-tier hypervisor:write caller from planting a destructive operation (VM delete, node shutdown, storage wipe, certificate upload) in the queue for a site_admin to unknowingly apply later (WP-08 hardening).

Immediate power requests and safe transport retries

The single-guest power endpoint and bulk start, stop, shutdown and reboot now use the shared staging claim and provider-task observer. QEMU suspend/resume are also covered. These buttons submit immediately after the existing operator action; they do not add a second manual staging step. The service checks the site-admin role, tenant and site scope, active controller/site, controller sync enablement, ADAPTER_READ_ONLY=false and explicit force=true before dispatch. It rechecks controller ownership at dispatch. The Pending Changes drawer holds the resulting operation record, actor, acknowledgement and provider task.

API compatibility change: old power bodies containing only action are rejected. Upgrade API clients alongside the backend and frontend. Send a fresh UUID for each intended operation, and preserve that UUID when retrying the same request after a transport failure:

POST /api/v1/hypervisor/controllers/{controller_id}/nodes/pve1/qemu/101/action
Content-Type: application/json

{
  "action": "reboot",
  "request_id": "6ad272e1-771d-4aa7-820e-e5b974d00a61",
  "force": true
}

The response is an operation receipt with request_id, change_id, dispatch status, and separate provider_task metadata. HTTP 202 does not certify provider acceptance or completion: inspect status. applied records the acknowledgement; only an observed provider task state of succeeded reports completion. applying, failed and reconciliation_required must not be treated as successful completion. If the task ID is missing, its outcome remains unknown.

Repeating an identical request ID, under the same actor and organization, returns the saved receipt without redispatching any operation already attempted. Reusing that ID for a different controller, guest, node or action returns 409. A durable pending request that has not been claimed may still dispatch on retry. This is at-most-once dispatch per retained intent record, not a guarantee that a remote operation executes exactly once or completes successfully. A new ID is a new intent. Database rollback to an older backup or deletion of the ledger can remove replay protection; reconcile provider state before resuming writes after recovery.

For bulk power, provide force: true and a distinct request_id on each target. The same guest cannot appear twice in one bulk power request. Targets commit independently and dispatch serially; a partial HTTP response is not a transaction rollback. Retrying the original target list and IDs returns existing receipts for attempted targets and may dispatch previously unattempted ones. Preserve the IDs outside the request if building an integration. The browser does not automatically retry a failed power mutation; inspect Pending Changes before issuing another intent. Bulk result success means acknowledgement, not task completion.

This coverage does not include direct deletion, remote migration, configuration, HA/firewall changes, or other direct writes. Their existing confirmation and read-only rules remain unchanged. Use the staged endpoints for durable tracking of those operations. This change needs the provider-task schema from migration 015, but adds no new migration and does not invent history for earlier writes.

Cloning and same-cluster migration

The single-guest /clone and /migrate endpoints now use the same durable claim as power operations. They require request_id (UUID) and force: true, an unscoped site-admin-or-higher principal with access to an active controller/site, and ADAPTER_READ_ONLY=false. Older bodies without IDs receive HTTP 422; omitting explicit force is refused. Successful receipt responses are HTTP 202. These are API compatibility changes: update external clients with the server.

{
  "request_id": "e345a0ee-1a36-437a-81d2-fb1aa50e6f52",
  "force": true,
  "newid": 202,
  "name": "application-copy",
  "full": true
}

That clone body is submitted against the source guest’s /clone endpoint. All clone options, including target node, storage and description, bind to the ID. For a QEMU VM’s /migrate endpoint:

{
  "request_id": "e9e28e67-40ed-4d16-a24b-c14495307921",
  "force": true,
  "target": "pve2",
  "online": true
}

Single-guest migration now defaults online to false: explicitly request QEMU live migration. For LXC, online: true is rejected before dispatch. A stopped container can migrate with online: false, restart: false. A running container requires explicit restart: true, which permits shutdown, migration and restart with downtime. FreeSDN does not silently convert a live request into a restart. Proxmox still decides whether storage, CPU, privileges, HA state and other prerequisites allow the operation. See the upstream container migration implementation.

Bulk migration requires a unique request ID on every target and force: true at the batch level. online controls QEMU guests only; restart controls LXC containers only and defaults false. Repeated guests or IDs in a batch are refused. Each target commits its own receipt. If a response is lost or a batch is partial, resubmit the same IDs and options, including for targets already accepted. The existing receipt returns without another provider write. Reusing a power request ID for cloning or migration also conflicts. Changed options require a new, explicitly intended operation after reviewing the previous result.

The response’s status describes dispatch. Inspect provider_task in Pending Changes for completion: acceptance is not a finished clone/migration. Missing task IDs stay unknown; interrupted/ambiguous dispatches require reconciliation and are not automatically retried. UI dialogs disable automatic mutation retry and preserve IDs for manual resubmission while the same dialog remains open. Closing and reopening a dialog starts a new intent. IDs are not retained across browser reloads; inspect Pending Changes before submitting after a reload.

This protection depends on retained staging records. A new ID is a new intent; there is no global per-guest lock and no exactly-once claim across database restore, ledger deletion or direct operations outside FreeSDN. Remote migration, deletion, resize, configuration and other direct writes still need separate coverage. No new database migration is added; schema migration 015 and the provider-task observer remain prerequisites for durable completion tracking.

Manual backups, snapshots and reviewed restore

Manual backup and snapshot create, rollback and delete use the same durable claim and HTTP 202 receipt as power and migration. They require an unscoped site-admin-or-higher principal, access to an active controller/site, ADAPTER_READ_ONLY=false, explicit force=true and a UUID request_id. Snapshot rollback and deletion additionally require confirmed=true; permission to write does not replace destructive confirmation.

API compatibility change: send request_id and force in the JSON body for manual backup and snapshot creation. Rollback now requires a JSON body with those two fields plus a separate confirmed=true query parameter. Snapshot deletion uses query parameters for all three values, with no DELETE body. Missing IDs receive HTTP 422. Update API clients together with the server and frontend.

The request ID binds the guest type, node, controller, action and all operation options. Reusing it with changed snapshot names, descriptions, memory state, backup storage, mode or compression returns HTTP 409. Identical retries return the recorded outcome without repeating an already claimed write. The browser disables automatic mutation retry and keeps create/backup/restore IDs while the dialog remains open. Closing/reopening a dialog, confirming a new action or reloading the page can create a new intent; inspect Pending Changes after a lost response before doing so. External clients must persist their own IDs.

Restore is still stage, review, then apply. It never runs as a side effect of opening the dialog or retrying staging. The UI uses this request shape:

POST /api/v1/gateway-proxmox-backup/{controller_id}/changes/proxmox.backup.restore?operation=create
Content-Type: application/json

{
  "request_id": "05e9b07b-003a-4e1d-991a-0219de2a5b01",
  "target_id": "202",
  "payload": {
    "node": "pve1",
    "vm_type": "lxc",
    "archive": "pbs:backup/ct/101/2026-09-16T03:04:05Z",
    "vmid": 202,
    "start": false,
    "unique": true
  }
}

Keyed restore staging uses the same unscoped site-admin and active-target checks, works in read-only mode, and returns the existing record for an identical ID, even after that record has been applied. It rejects unknown payload fields, string booleans and unsafe archive references. request_id is optional on the legacy gateway staging API for compatibility; calls without it create independent records and have no staging replay protection. It is supported here only for proxmox.backup.restore with operation create.

Review the pending record, then apply it separately at POST /api/v1/gateway-vpn/changes/{change_id}/apply with {"force": true, "confirmed": true}. Staging a flag in the payload cannot authorize apply. The shared gate forwards the apply-time decision to Proxmox’s second preflight through a transient view; it is not persisted or sent in provider requests. Restore rechecks controller/site activity and ownership after claiming and before using credentials. A claimed or failed restore is never automatically replayed. Other vendors retain their confirmation-free applier payloads.

Canonical PBS volume IDs include UTC timestamp colons. FreeSDN accepts those backup/vm/... and backup/ct/... forms as well as safe file-backed archive IDs, while rejecting traversal, URLs, encoded paths and invalid PBS dates. The UI recognizes pbs-ct and PBS container paths as LXC archives. This matches the upstream PBS storage format.

unique controls MAC address regeneration, not permission to overwrite an existing guest. FreeSDN’s force opens its write gate; it does not send Proxmox’s separate overwrite option. Use an appropriate unused target ID; the provider may refuse an existing guest. See the upstream QEMU and LXC restore options. Treat restore as destructive regardless of this restriction.

An acknowledged backup or an observed successful provider task does not prove that the archive is recoverable. A full restore drill, application validation, measured RPO/RTO and independent failure alerts remain necessary. These changes add no migration beyond the existing task-tracking prerequisite, migration 015.

Task completion and interrupted operations

The Pending Changes drawer distinguishes request acceptance from provider completion. status=applied remains the dispatch status for compatibility. When provider_task is present, its state is the latest provider observation:

State	Meaning
pending	Accepted task ID; no observation yet
running	Task is still running
succeeded	Provider reported successful termination
warnings	Provider finished with warnings; review its task log
failed	Provider reported failure; inspect partial changes
unknown	Outcome is unverified; reconcile before repeating the operation

A lost write acknowledgement becomes reconciliation_required, not a definite rejection. Observer interruption only causes another status read; it never replays the mutation. Returned task IDs are checked against the provider’s response identity. Task IDs for API-token users are supported.

Upgrade: apply migration 015_proxmox_task_tracking before starting the new API and workers. Existing history is preserved but is not retroactively certified. Celery beat dispatches hypervisor.poll_tasks every 30 seconds to the sync queue; an active worker consuming it is required. The task observes at most 20 due records per batch. Running tasks become due after 30 seconds; unavailable observations back off five minutes. Queue load adds delay.

The observer skips deleted, disabled or reassigned targets and checks that the controller’s site and organization match the staged change. The configured credentials still need permission to read the task. An invalid task ID is not used in a URL. After seven days, an unavailable outcome requires manual verification and automatic observation stops; a task still observed running continues to be polled.

checked_at records the latest attempt, including unsuccessful attempts. finished_at records when FreeSDN observed termination, not the provider’s end time. Bulk apply does not resume automatically. The existing controller.change.applied event describes acknowledgement; it must not be used as a provider-completion trigger. Guaranteed completion-event delivery, automatic compensation and durable tracking of direct hypervisor writes are not part of this observation path.

Fabric integration

The hypervisor module exposes five Fabric operation targets. All are staged writes - an operator must sign off before they execute.

Operation id	Required inputs	Permission
hypervisor.vm.snapshot	controller_id, node, vmid, snapname	hypervisor.manage_snapshots
hypervisor.vm.start	controller_id, node, vmid (+ vm_type qemu/lxc)	hypervisor.manage_vms
hypervisor.vm.stop	same	hypervisor.manage_vms
hypervisor.vm.shutdown	same	hypervisor.manage_vms
hypervisor.vm.reboot	same	hypervisor.manage_vms

Example wiring: OPNsense firewall rule applied → snapshot affected VMs. Author this as a Fabric Connection targeting hypervisor.vm.snapshot and wire it to the controller.change.applied event from your firewall controller. See Fabric for wiring syntax and Connection authoring.

Permissions

Permission code	Minimum role	Covers
hypervisor.view	viewer	Read-only access to all cluster, node, VM, and storage data
hypervisor.manage_vms	site_admin	VM/CT power operations, console, guest agent, bulk ops
hypervisor.manage_snapshots	site_admin	Create, rollback, and delete snapshots
hypervisor.manage_backups	site_admin	Create, update, and trigger backup jobs
hypervisor.manage_nodes	site_admin	Node-level operations (reboot, shutdown, services)

Permission codes vs enforced role checks

The module manifest declares the hypervisor.* permission codes shown above. The /api/v1/hypervisor/… endpoints enforce these through minimum-role checks (require_min_role("site_admin") for writes, session-only for reads). The gateway-proxmox staging endpoints use hypervisor:read / hypervisor:write permission strings. Both paths resolve to the same effective access tiers.

Frontend: the Hypervisor page

Navigate to Hypervisor in the left sidebar (route /hypervisor). Choose a controller from the dropdown when you have multiple Proxmox clusters registered.

With no controller selected - the page shows a fleet dashboard: clusters online/total, total nodes, VMs, containers, and aggregate CPU/memory/storage utilization drawn from /fleet/dashboard.

With a controller selected - tabs include:

Tab	Contents
Dashboard	Cluster health, quorum state, HA active count, per-node resource bars
Nodes	Node list with CPU/memory/disk sparklines; click a node for a detail drawer with sub-tabs: Overview · VMs · Containers · Services · Disks · Network · Sensors
Virtual Machines	VM list with status, vCPU, memory; power actions; bulk action bar
Containers	LXC container list; same operations as VMs
Storage	Pool browser with content-type filter (all/ISO/templates/backup/disk images/snippets); upload and restore dialogs
Tasks	Recent task list with status and log tail
Backup	Scheduled job list; manual backup trigger; backup age report
Firewall	Cluster and per-guest firewall rule tables
HA	HA resource and group management
Pools	Resource pool list

Additional component tabs available in the drawer and via navigation: Ceph, Replication, PBS (Proxmox Backup Server), Certificates, SDN, Monitoring (RRD charts), Updates (APT), Subscriptions, Templates (when show_templates=true), Cluster Log, Kiosk Mode.

Adapter internals

Authentication and connection

The Proxmox client (ProxmoxClientConfig) connects to {host}:{port} (default 8006) over HTTPS. It supports two auth modes:

• API token (preferred): token_id (user@realm!tokenname) + token_secret. Tokens are Fernet-decrypted at runtime; they never appear in logs or error messages.
• Ticket auth: username/password/realm. The client caches the ticket for 90 minutes (PVE tickets are valid 2 hours); on a 401 from an idempotent request it drops the cached ticket and retries once.

Safety mechanisms

• Read-only gate: every POST, PUT, PATCH, and DELETE request checks _is_adapter_read_only() before proceeding. If the gate is closed, the request is recorded as read_only_blocked in metrics and an AdapterError is raised.
• Circuit breaker: 5 consecutive failures open the breaker for 60 seconds. Idempotent timeouts retry with jittered backoff.
• Path-traversal guard: _validate_path(path) runs at every _request chokepoint.
• Rate limiter: 120 requests/minute, 10 concurrent connections. A dedicated 2-slot semaphore handles large uploads so a 4 GB ISO transfer does not starve API calls.
• Response size cap: check_response_size(resp) bounds device response bodies.
• Secret redaction: redact_secrets (central, ~90 sensitive key patterns, camelCase-aware) is applied to every adapter read. Full VM/CT config responses (GET …/config) go through this broader central filter. _SENSITIVE_CONFIG_KEYS = {cipassword, sshkeys, args, hookscript} are stripped from pending-config (GET …/pending) responses only. CloudInit cipassword/sshkeys/ipconfigN are redacted. PVE ticket fragments and URLs are stripped from error messages.

RRD downsampling

All RRD endpoints use LTTB (Largest-Triangle-Three-Buckets) downsampling. The max_points parameter (10-5000, default 500) controls output resolution. This keeps chart queries fast even for year-range timeframes.

Gotchas and limitations

• Proxmox VE only. Proxmox Mail Gateway (PMG) and Proxmox Backup Server (PBS, as a standalone appliance) are not managed here. The adapter only talks to PVE clusters.
• Cluster membership changes require shell access. The Proxmox REST API does not expose adding or removing cluster nodes. Use the Proxmox UI or SSH for those operations.
• Node status may lag by up to sync_interval seconds. If a node goes offline between sync cycles, FreeSDN’s status field reflects the last successful poll, not real-time state.
• Templates hidden by default. VM templates do not appear in the Virtual Machines list unless you set show_templates: true in module settings.
• Ceph tab returns 404 when Ceph is not deployed. This is expected - the adapter passes the 404 through cleanly rather than raising an error.
• SDN is currently read only in FreeSDN. Removed direct write routes cannot be used; new staging and historical replay are rejected. Inspect visible configuration here and use native Proxmox management with independent recovery access for changes.
• Upload cap is 4 GB. Uploading ISOs or templates larger than 4 GB returns HTTP 413. Split or pre-download large images directly on the PVE node.
• Remote-migrate requires a separate Proxmox cluster as the target. Both source and target clusters must be reachable from the FreeSDN API container.

FreeSDN’s Proxmox adapter vs. direct Proxmox UI

FreeSDN adds multi-tenant org/site scoping, RBAC role gates, the staged-write queue, secret redaction, and Fabric wiring on top of the PVE API. It does not replace the Proxmox UI for cluster-topology changes, Ceph provisioning, or storage backend configuration - those still require direct Proxmox access.

Next steps

• Supported Vendors - Proxmox adapter contract, maturity tier, and known limitations.
• Staged Changes - how the pending-change queue works across all adapters.
• Fabric - wire hypervisor operations to events from other modules.
• Roles and Permissions - full 5-tier role hierarchy and how site grants interact with module permissions.
• Storage (TrueNAS) - companion module for ZFS pool health and staged blob writes.