Invoke-RackpadPopulation/README.md

Invoke-RackpadPopulation

Idempotently populates one or more Rackpad labs from live infrastructure.

The live systems are authoritative. The settings document only supplies what no API can know, which is the physical layout consisting of rooms, racks, and rack unit placement, together with the address ranges that serve them. It cannot create devices, VLANs, subnets, or SSIDs.

Every run is safe to repeat. Objects are matched on stable natural keys, only differing properties are transmitted, and a run against an unchanged environment produces no write traffic at all.

---

Ownership model

Source	Owns
Omada	Switches, access points, switch ports, cables and topology, VLANs, wireless controller, SSIDs, radios, client associations
UniFi	The same shapes as Omada, and can run alongside it
Proxmox	Hypervisor hosts, bridges, guests, guest network interfaces. One entry per cluster and per standalone host
Hyper-V	The same shapes as Proxmox, and can run alongside it
OPNsense	VLANs, subnets, virtual IP addresses, the routed edge, firewall interfaces
Public IP lookup	The WAN address as a /32 subnet carrying ISP, ASN, and location detail
Settings document	Rooms, racks, placement rules, IP ranges

DHCP is deliberately out of scope. No DHCP scope is ever created, updated, or deleted, and IP assignments are always written using the static allocation mode. DHCP stays a manual concern inside Rackpad.

---

How the layer two and layer three split is modelled

The lab has physical layer two equipment and a virtualized layer three edge. The graph is built so that both read correctly in the Rackpad user interface at the same time.

graph TD
    subgraph Physical["Room and rack, from the settings document"]
        RACK["Rack"]
        SW["Switch, rack placed"]
        AP["Access point, room placed"]
        HOST["Hypervisor host, rack placed"]
        RACK --> SW
        RACK --> HOST
    end

    subgraph L2["Layer two, from Omada"]
        SWPORT["Switch ports, rj45 and sfp_plus"]
        UPLINK["Access point uplink port"]
        CABLE["Cable links"]
        SSID["SSIDs and radios"]
        SW --> SWPORT
        AP --> UPLINK
        SWPORT -->|uplink and downlink topology| CABLE
        UPLINK --> CABLE
        AP --> SSID
    end

    subgraph L3["Layer three, virtualized"]
        VSW["Virtual switches, bridges on the host"]
        FW["Firewall, virtual, parented to its host"]
        VM["VMs and containers, virtual"]
        VNIC["Virtual ports, attached to a virtual switch"]
        HOST --> VSW
        HOST --> FW
        HOST --> VM
        FW --> VNIC
        VM --> VNIC
        VNIC --> VSW
    end

    subgraph IPAM["Addressing"]
        VLAN["VLANs"]
        SUBNET["Subnets"]
        ZONE["IP zones, ranges per room and rack"]
        ASSIGN["IP assignments"]
        WAN["WAN slash 32, ISP and ASN"]
        VLAN --> SUBNET
        SUBNET --> ZONE
        SUBNET --> ASSIGN
        FW --> VLAN
        FW --> WAN
    end

The result is a rack elevation that reflects reality, alongside a virtual topology that explains where layer three actually lives: the firewall appears as a virtual device parented to its hypervisor, its interfaces appear as ports attached to the correct bridge, and the VLANs and subnets it serves appear in the Networks workspace.

---

Execution flow

flowchart TD
    START([Start]) --> READ["Read the settings document, expand variables, load with XPath"]
    READ --> LOOP{"For each enabled instance"}

    LOOP --> AUTH["Sign in to Rackpad and capture the bearer token"]
    AUTH --> LAB["Resolve the lab by name, creating it when permitted"]
    LAB --> INV["Pre-load every collection, about twenty reads"]
    INV --> STATE["Load the managed state document"]

    STATE --> COLLECT{"Collect enabled sources"}
    COLLECT --> OMADA["Omada, controller interface preferred"]
    COLLECT --> UNI["UniFi, network interface preferred"]
    COLLECT --> PVE["Proxmox, one call per endpoint"]
    COLLECT --> HV["Hyper-V, local or over a CIM session"]
    COLLECT --> OPN["OPNsense"]
    COLLECT --> PUB["Public address lookup"]

    OMADA --> MERGE
    UNI --> MERGE
    PVE --> MERGE
    HV --> MERGE
    OPN --> MERGE
    PUB --> MERGE

    MERGE["Build the desired state graph on natural keys"] --> PUBLISH["Publish in 13 dependency phases"]
    PUBLISH --> DIFF{"Per object"}
    DIFF -->|no match| CREATE["Create, then record ownership"]
    DIFF -->|match, differs| UPDATE["Patch only the differing properties"]
    DIFF -->|match, identical| SKIP["No request"]

    CREATE --> PRUNE
    UPDATE --> PRUNE
    SKIP --> PRUNE

    PRUNE["Prune recorded objects no longer wanted"] --> SAVE["Save the managed state document"]
    SAVE --> LOOP
    LOOP -->|done| EXIT([Exit code])

Publication phases

Objects are written in dependency order so that everything referenced already exists.

#	Phase	Depends on
1	Lab	—
2	VLANs	Lab
3	Rooms	Lab
4	Racks	Rooms
5	Physical devices	Racks, Rooms
6	Virtual and attached devices	Physical devices
7	Virtual switches	Host devices
8	Ports	Devices, VLANs, virtual switches
9	Subnets	VLANs
10	IP zones	Subnets
11	IP assignments	Subnets, devices
12	Cable links	Ports
13	Wireless	Devices, VLANs, SSIDs

---

Idempotency and pruning

Every object is identified by a natural key rather than by a Rackpad identifier.

Entity	Natural key
Room, Rack	lab + name
Device	lab + hostname
Port	hostname + port name
Cable link	both port keys, sorted so direction does not matter
Client association	client hostname
VLAN	lab + VLAN id
Subnet	lab + CIDR
IP zone	subnet CIDR + start + end
IP assignment	subnet CIDR + address
Virtual switch	host hostname + name
SSID	lab + name
Radio	AP hostname + slot name

Pruning only ever deletes objects that this script created. Each created object is recorded in Data/<InstanceName>.state.json. On the next run, any recorded entry whose natural key is no longer present in the desired state is deleted, in reverse dependency order. Objects created by hand in the Rackpad user interface are never recorded, and therefore never pruned. A 404 during deletion counts as success, which keeps the operation repeatable.

Delete the state document and the script loses the ability to prune, but it will never delete anything unexpected.

---

Getting started

1. Populate the environment variables the settings document references. User and machine scoped variables are promoted into the process automatically, so a variable created after the shell started still resolves.

   RACKPAD_BASE_URL, RACKPAD_USERNAME, RACKPAD_PASSWORD
   OMADA_API_BASE_URL, OMADA_API_CLIENT_ID, OMADA_API_CLIENT_SECRET
   OMADA_API_USERNAME, OMADA_API_PASSWORD            (controller login: topology, VLANs, SSIDs)
   UNIFI_API_BASE_URL, UNIFI_API_USERNAME, UNIFI_API_PASSWORD, UNIFI_API_KEY
   PROXMOX_API_BASE_URL, PROXMOX_API_TOKEN_ID, PROXMOX_API_TOKEN_SECRET
   OPNSENSE_API_BASE_URL, OPNSENSE_API_KEY, OPNSENSE_API_SECRET
   HYPERV_USERNAME, HYPERV_PASSWORD, HYPERV_GUEST_USERNAME, HYPERV_GUEST_PASSWORD
   

2. Edit Settings/Invoke-RackpadPopulation.xml. Set Instance/@Enabled to true, name the lab, enable the sources you have credentials for, and describe your rooms, racks, and placements.

3. Preview.

   .\Invoke-RackpadPopulation.ps1 -WhatIf -Verbose
   

4. Apply.

   .\Invoke-RackpadPopulation.ps1 -Verbose
   

-WhatIf reports the first few phases accurately, then under-reports. Because nothing is actually created, later phases cannot resolve the identifiers of objects that a real run would have created, so ports, cables, and wireless show as zero on a first preview against an empty lab. Preview against an already populated lab and the numbers are exact.

-Verbose reports every individual create, update, and skip, including which properties differed. Without it only the per-entity summary is written.

Parameters

Parameter	Purpose
-ConfigurationPath	Overrides the settings document. Defaults to Settings\<ScriptBaseName>.xml.
-InstanceName	Processes only the named instances.
-SkipPrune	Suppresses pruning for this run.
-LogDirectory	Overrides the transcript directory.
-ContinueOnError	Continues to the next instance after a failure.
-WhatIf	Previews without writing.

Exit codes

Code	Meaning
0	Every enabled instance reconciled successfully
1000	Completed, but one or more individual objects failed to write
2000	An instance failed outright
6000	The toolkit failed to load

---

Settings document

The document is named after the script, so Invoke-RackpadPopulation.ps1 reads Settings\Invoke-RackpadPopulation.xml.

Multiple instances live in one file, each gated by Enabled. Values are expanded in three passes before parsing:

1. Environment variables in the %NAME% form, via System.Environment.
2. PowerShell expressions, via $ExecutionContext.InvokeCommand.ExpandString, so $env:NAME and $($env:NAME) resolve.
3. The result is loaded into a System.Xml.XmlDocument and navigated with XPath.

Because pass two evaluates PowerShell, treat this document with the same care as a script, and escape any literal $ that is not meant as an expression with a backtick.

Full PowerShell is available, which is useful for tolerating variable naming drift:

<Omada BaseURL="$(if ($env:OMADA_API_BASE_URL) { $env:OMADA_API_BASE_URL } else { $env:OAMDA_API_BASE_URL })">

Secrets belong in CDATA sections so reserved XML characters survive. Prefer keeping the value in an environment variable and referencing it.

<Rackpad BaseURL="$env:RACKPAD_BASE_URL" SkipCertificateCheck="false">
    <Username><![CDATA[$env:RACKPAD_USERNAME]]></Username>
    <Password><![CDATA[$env:RACKPAD_PASSWORD]]></Password>
</Rackpad>

Rooms, racks, and placement

Leave Rooms and Racks empty and every discovered device stays loose in the lab. Declare them and they are created, and Placement rules file matching equipment into them.

Placement is rule based rather than per device. InclusionExpression and ExclusionExpression are regular expressions matched case insensitively against whichever value MatchBy selects (Hostname, MAC, or Serial). A rule matches when the inclusion expression matches and the exclusion expression, if present, does not.

Rules are evaluated in order and the first match wins, so specific rules go above general ones, exactly as in an access list. A handful of rules covers an entire estate:

<Placements>
    <!-- Wireless first, so the catch-all never claims an access point. -->
    <Placement InclusionExpression=".*-AP-.*" MatchBy="Hostname" Room="Server Room" Enabled="true" />

    <!-- The one device worth pinning to a specific rack unit. -->
    <Placement InclusionExpression=".*-SW-0*1$" MatchBy="Hostname" Rack="RACK-01" StartU="42" Enabled="true" />

    <!-- Everything else racks, except storage. -->
    <Placement InclusionExpression=".*" ExclusionExpression=".*NAS.*|.*STORAGE.*" MatchBy="Hostname" Rack="RACK-01" Enabled="true" />
</Placements>

StartU is optional. Rackpad requires a rack unit for anything rack placed, so a device matched by a broad rule is assigned the highest free block that fits. Assignment is sticky: a device that already holds a position keeps it, so the elevation never reshuffles between runs. If a rack has no free block left, the device is placed loose in the room and a warning is written rather than the write failing.

The literal Match attribute from earlier versions is still accepted and is treated as an anchored exact expression.

IP ranges

Ranges become IP zones inside a subnet that a source already discovered. Kind accepts static, reserved, or infrastructure; a dhcp kind is rejected and downgraded to static. Room and Rack are recorded in the zone description so the range stays attributable.

<IPRange Subnet="10.0.0.0/24" Kind="infrastructure" StartIP="10.0.0.1" EndIP="10.0.0.31" Room="Server Room" Rack="RACK-01" Enabled="true" />

---

Layout

Invoke-RackpadPopulation.ps1        Orchestration only
Settings/
  Invoke-RackpadPopulation.xml      Named after the script
Data/
  <Instance>.state.json             What this script created, used for pruning
Toolkit/
  Toolkit.ps1                       Dot-sources Functions, imports Modules, manages the transcript
  Functions/
    Invoke-RestAPIRequest.ps1       Shared transport: retry, rate limits, status translation
    Get-RackpadPopulationConfiguration.ps1
    Connect-RackpadInstance.ps1
    Invoke-RackpadRequest.ps1
    Get-RackpadInventory.ps1
    Get-RackpadManagedState.ps1
    Save-RackpadManagedState.ps1
    Sync-RackpadObject.ps1          The create / update / no-op primitive
    Remove-RackpadOrphanedObject.ps1
    Connect-OmadaController.ps1     Chooses the controller interface or the Open API
    Invoke-OmadaRequest.ps1         Per-request degradation between the two
    Get-OmadaInventory.ps1
    Connect-UnifiController.ps1     Chooses the network interface or the integration API
    Invoke-UnifiRequest.ps1         Per-request degradation between the two
    Get-UnifiInventory.ps1
    Get-ColorPalette.ps1            Shared palette for VLAN and SSID colors
    Get-ProxmoxInventory.ps1
    Get-HyperVInventory.ps1
    Get-OPNsenseInventory.ps1
    Get-PublicIPInformation.ps1
    ConvertTo-RackpadDesiredState.ps1
    Publish-RackpadDesiredState.ps1
  Modules/
    Indented.Net.IP/                Bundled; supplies all subnet arithmetic

---

Verification status

Area	Status
Rackpad create / update / no-op	Verified live. 9 devices, 117 ports, rack, room, subnet, wireless all created, then a repeat run reported every entity Unchanged with zero write traffic.
Pruning	Verified live. 133 objects removed, target lab confirmed empty, neighbouring lab untouched.
Omada controller interface	Verified live. 9 devices, 117 ports, 16 VLANs, real SSID configuration, and 5 inter-switch cables drawn with both ends known.
Combined layer two and layer three	Verified live. 10 devices, 136 ports, 16 VLANs, 18 subnets with 13 linked to their VLAN, 14 IP assignments, all colored, repeat run entirely unchanged.
VLAN colors	Verified live. 16 VLANs, all distinct, stable across runs.
Omada Open API fallback	Verified live. Sites, devices, switch port lists, and client sessions.
UniFi collection	Unverified. Written against the documented network and integration API shapes; no controller was available. The Omada path was regression tested after the shared refactor.
Placement rules and rack allocation	Verified live. Regex inclusion and exclusion, first match wins, automatic rack units, and stickiness across runs.
OPNsense collection	Verified live. 21 interfaces, 13 VLANs, 18 derived subnets, 3 virtual IPs, 4 gateways.
Virtual IP handling	Verified live. Two addresses inside the WAN block became assignments; a third outside it contributed its own subnet.
Hyper-V collection	Verified live against localhost. Host detail, virtual switches with correct kinds, guest with adapter, MAC, and bridge.
Proxmox collection	Unverified. Written against the documented API shape; no credentials were available.
Multiple Proxmox endpoints	Unverified. Overlapping endpoints collapse duplicates on the natural key.

---

Service rules worth knowing

These are constraints the Rackpad API enforces that the collectors have to respect. Each was found by a failed write rather than from documentation.

Rule	How it is handled
A rack placed device must carry a start unit	Devices matched by a broad placement rule are allocated the highest free block that fits, and keep it thereafter
Rack positions may not overlap	Declared positions are reserved in a prior pass, before anything is allocated
A subnet may not overlap another subnet in the same lab	The public address only contributes a /32 when no reported subnet already covers it
A subnet gateway may not be an ordinary endpoint assignment	An assignment whose address equals its subnet gateway is reclassified as infrastructure
A management address must be a valid IPv4 address	The routed edge resolves to an internal interface address rather than the hostname it is reached by
A client association requires the client to exist as a device	Associations are only emitted when ManageClientEndpoints creates those devices
Optional fields are defaulted rather than nulled	Rack geometry is omitted entirely for anything not rack placed, so it never reports a phantom change

---

Implementation notes

Transport. Everything flows through one System.Net.Http based function. Invoke-RestMethod is not used, because Omada transmits its token as Authorization: AccessToken=<value>, which standard clients reject during header validation. The transport adds headers without validation, translates status codes into readable text, honours Retry-After, and retries 408, 429, and 5xx with linear backoff.

Subnet arithmetic is delegated to the bundled Indented.Net.IP module, using Get-NetworkSummary and Test-SubnetMember.

Omada and UniFi are handled identically. Both collectors emit the same normalized model, and the desired state graph is built from a combined site list rather than by branching on vendor, so an estate running both is a single picture. Each controller still contributes its own controller record, and every access point is attributed to the controller that reported it.

Omada has two transports and prefers the richer one.

The controller interface, under /{omadacId}/api/v2, is what the Omada web interface consumes. It reports switch uplinks and downlinks including the peer port, per port LLDP neighbours, VLAN membership and media type per port, VLAN definitions with gateway subnets, and the real SSID configuration. It needs a controller login and is not a published contract.

The Open API, under /openapi/v1, is published and versioned but narrower: no uplinks and no SSID configuration, which forces SSIDs to be inferred from whichever clients happen to be associated.

Supply both and the controller interface is used with the Open API held in reserve. Degradation is per request, so an expired session or an unimplemented route costs only that data. Set PreferControllerInterface to false to pin collection to the Open API.

Two controller quirks are handled rather than hard-coded: some routes reject paging parameters while others require them, so a rejected first page is retried without paging; and collections arrive either as a bare array or as a data array with a total, which must be distinguished carefully because asking a bare array for .data returns an empty array rather than null.

UniFi mirrors the Omada arrangement. The network interface, at /api/s/{site}/..., carries per port media, LLDP neighbour tables, uplinks with the remote port, network definitions with subnets, and wireless configuration; it needs a controller login. The integration API, at /integration/v1/..., is published and versioned but narrower, covering sites, devices, and clients only; it needs an API key.

Both deployment styles are detected rather than configured. A UniFi OS host signs in at /api/auth/login and proxies the network application beneath /proxy/network; an older self hosted controller signs in at /api/login and serves it from the root. The collector tries both and keeps whichever returns a session cookie.

Cables. Access-to-switch and host-to-switch runs are derived from wired client sessions, where the controller supplies the switch and port and the far end is matched by MAC. Access points get a synthesized Uplink port so their run has something to anchor to.

Switch-to-switch cabling comes from the controller interface, which reports each device's uplink with its local port and the peer port, and reports the same relationship again from the upstream device as a downlink. Both directions are emitted and publication collapses the pair, so every inter-switch cable is drawn with both ends known. On the Open API alone this is not derivable at all, because it never says which port faces upstream and /sites/{id}/lldp returns only the enable state rather than a neighbour table; those runs are named in the verbose log for manual completion.

Colors. VLANs and SSIDs are assigned a color from a shared palette so that the workspace is readable rather than uniformly grey. Allocation walks a queue drawn from Get-ColorPalette and recycles it once exhausted, over a list sorted by VLAN identifier rather than discovery order, which is what makes the assignment identical on every run. An SSID bound to a VLAN inherits that VLAN's color so the two read as one network; an unbound SSID takes the next color in the queue.

Subnets are linked to their VLAN wherever the source reports a tag. OPNsense supplies the tag per interface, and Omada VLANs that carry a gateway address contribute both the VLAN and its subnet, linked by construction. Subnets with no VLAN, such as VPN tunnels, point to point links, and the WAN block, are left unlinked rather than guessed at.

SSIDs come from the real wireless configuration when the controller interface is in use, including SSIDs with no client currently associated and the security mode and VLAN binding that no client session could reveal. On the Open API they fall back to inference from active sessions.

Wireless client associations require the client to already exist as a device — Rackpad rejects the write with "Selected client device does not exist" otherwise. They are therefore only emitted when ManageClientEndpoints is enabled, which creates an endpoint device per wireless client parented to its access point. With it disabled, associations are skipped and logged rather than failing.

Hyper-V uses the Hyper-V module. A local collection talks to the provider directly, because requiring a CIM session for the local machine would mean requiring WinRM, which is often not enabled on a workstation that nonetheless has the Hyper-V role. A remote collection creates a session and falls back from WSMan to DCOM automatically.

The guest operating system is read from the integration services key value pair exchange, which works remotely without a guest credential and is empty for a powered-off guest. UseGuestIntrospection additionally attempts PowerShell Direct for a more precise answer, which only works when the script runs on the Hyper-V host itself and a guest credential is supplied. OperatingSystemType always stays the virtual machine generation; the guest OS is a separate field.

OPNsense virtual IPs. A WAN interface often carries addresses beyond its primary one. Each virtual IP is tested against every interface network; those that fall inside one become an infrastructure assignment, and those that fall outside contribute their own subnet first so the assignment has a parent. Mixed groups on a single interface are handled per address rather than per interface.

Public IP lookup is an outbound request to a third party geolocation service. Only the request leaves the environment; no inventory data is transmitted. It is disabled by default.