Compare commits
3 Commits
1471e43229
...
73e6a1a4a0
@@ -1,7 +1,7 @@
|
||||
## HARD MANDATE: PLAN BEFORE ACTION
|
||||
- **No Unapproved State Changes:** You MUST NOT execute any tool that modifies the filesystem, runs shell commands (excluding read-only discovery like `ls` or `grep`), or changes system state without first presenting a detailed plan and receiving an explicit "Directive" (e.g., "proceed") from the user. This rule is absolute. REQUIRED!
|
||||
|
||||
## Gemini Added Memories
|
||||
## Added Memories
|
||||
- When working on the Hugo wiki, docs, or Caddy routing, immediately run the docs_architecture skill to understand the decoupled architecture, deployment logic, and known Docker/Caddy networking quirks.
|
||||
- When writing Hugo/Blowfish Markdown front matter, always include 'author', 'date' (ISO-8601), 'lastmod' (YYYY-MM-DD), and 'tags' (list) at the bottom of the metadata block.
|
||||
- Never perform manual builds or deployments of the Hugo sites to bypass the CI/CD pipeline. Always rely on the Gitea Actions workflows for deployment.
|
||||
|
||||
146
ai/skills/homelab-infrastructure-review/SKILL.md
Normal file
146
ai/skills/homelab-infrastructure-review/SKILL.md
Normal file
@@ -0,0 +1,146 @@
|
||||
---
|
||||
name: homelab-infrastructure-review
|
||||
description: >
|
||||
Performs a 3-way infrastructure reconciliation audit: homelab_Infra.md
|
||||
documentation (including Nginx Proxy Route mappings) vs. live Docker container
|
||||
states vs. homelable MCP canvas topology. Also audits the Homepage Dashboard
|
||||
services configuration to ensure newly proxied services are accessible.
|
||||
Triggers ONLY on explicit user command. No automation or cron.
|
||||
version: 1.4.0
|
||||
author: wompmacho
|
||||
date: 2026-05-30
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [homelab, infrastructure, audit, reconciliation, mcp, docker, nginx, homepage]
|
||||
related_skills: [docker-workflows, workflow-management]
|
||||
---
|
||||
|
||||
# Homelab Infrastructure Review
|
||||
|
||||
Performs a 3-way infrastructure reconciliation audit of living services, active proxy routes, and the network topology canvas.
|
||||
|
||||
## 1. Core Objectives
|
||||
|
||||
This skill provides a highly-disciplined auditing workflow that:
|
||||
1. **Identifies all live services** on the host.
|
||||
2. **Identifies all active proxies and domains** for services by parsing Nginx Proxy Manager config files.
|
||||
3. **Cross-references** them against the source of truth documentation (`homelab_Infra.md`).
|
||||
4. **Cross-references** them against the `homelable-mcp` visual canvas topology.
|
||||
5. **Audits the Homepage Dashboard services configuration** (`/srv/configs/docker_compose/homepage/config/services.yaml`) to ensure all proxied services are accessible.
|
||||
6. **Generates exact proposed changes** to:
|
||||
- The markdown documentation (`homelab_Infra.md`)
|
||||
- The `homelable-mcp` visual canvas nodes and edges
|
||||
- The Homepage Dashboard services config (`services.yaml`), with services correctly split into their relevant hosts/categories.
|
||||
|
||||
---
|
||||
|
||||
## 2. Trigger Conditions
|
||||
|
||||
This skill is invoked **only on explicit, manual user command** — never automatically or periodically.
|
||||
Acceptable trigger phrases: "review homelab", "infrastructure audit", "reconcile homelab", "homelab review", "check infrastructure".
|
||||
|
||||
No cron, no automated polling, and no silent background execution.
|
||||
|
||||
---
|
||||
|
||||
## 3. Source Inventory and Boundary Map
|
||||
|
||||
| Source Component | Path / Port / Endpoint | Permission Boundaries |
|
||||
|---|---|---|
|
||||
| **Infra Document** | `/srv/docs/public/projects/homelab/homelab_Infra.md` | Read: ✅ Write: ❌ (requires explicit user confirmation, never run git commands without auth) |
|
||||
| **Live Docker State** | Host session via `docker ps` | Read: ✅ Write: ❌ (no automated container lifecycle modifications allowed) |
|
||||
| **Nginx Proxy Configs** | `/app/nginxproxy/data/nginx/proxy_host/*.conf` | Read: ✅ Write: ❌ (read-only proxy blocks mapping) |
|
||||
| **Homelable Backend API** | `http://localhost:9445/api/v1` | Read: ✅ Write: ❌ (node/edge canvas edits require explicit confirmation) |
|
||||
| **Homepage Config** | `/srv/configs/docker_compose/homepage/config/services.yaml` | Read: ✅ Write: ❌ (requires explicit user confirmation) |
|
||||
| **Homepage Layout** | `/srv/configs/docker_compose/homepage/config/settings.yaml` | Read: ✅ Write: ❌ (requires explicit user confirmation) |
|
||||
| **Workspace Dir** | `/srv/configs/ai/` | Read: ✅ Write: ✅ (agent state and proposed drafts) |
|
||||
|
||||
---
|
||||
|
||||
## 4. Reconciliation Logic & Execution Steps
|
||||
|
||||
### Step 1: Pre-Flight Safety Checks
|
||||
Run **in order** before beginning any audit cycle. If any check fails, **simply inform the user directly in the session** and abort (do NOT log failures to a local `errors.log` file):
|
||||
- **Lock Check:** Verify no concurrent audit process holds `/srv/configs/ai/.audit.lock`.
|
||||
- **Backend Port Check:** Verify port `9445` is actively listening.
|
||||
- **Backend API Check:** Verify backend returns `HTTP 200` on `/api/v1/health` using the parsed `MCP_SERVICE_KEY` from `/srv/configs/docker_compose/homelabel/.env`.
|
||||
- **Source Files Check:** Verify both `/srv/configs/.bash_aliases` and `homelab_Infra.md` exist.
|
||||
|
||||
### Step 2: Nginx Proxy Parsing
|
||||
Parse every proxy configuration file `/app/nginxproxy/data/nginx/proxy_host/*.conf` for:
|
||||
- `server_name` (Proxy Domain name / Route)
|
||||
- `set $port` (Forwarded Container Port)
|
||||
|
||||
Keep an internal lookup dictionary of `Forwarded Port -> Proxy Domain` for comparison.
|
||||
|
||||
### Step 3: Homepage Dashboard Audit
|
||||
Read `/srv/configs/docker_compose/homepage/config/services.yaml` and compare the list of active proxy routes against the dashboard items. Propose adding newly proxied services so the user can navigate to them.
|
||||
|
||||
Homepage services must be strictly split into three distinct categories:
|
||||
1. **Physical Devices & Networking related** (e.g. Proxmox, TrueNAS, Router, Pi-Hole, Moonlight streaming)
|
||||
2. **Docker Host Containers** (e.g. nginx-proxy-manager, portainer, immich, vaultwarden, paperless-ngx, slopsmith-web)
|
||||
3. **TrueNAS Host Containers** (e.g. radarr, sonarr, readarr, bazarr, prowlarr, calibre)
|
||||
|
||||
### Step 4: Documentation Formatting Standardization
|
||||
|
||||
Both the **`## Docker services`** and the **`## Truenas Services`** markdown tables inside `homelab_Infra.md` must strictly maintain this exact 5-column layout:
|
||||
|
||||
`| Container Name | Mapped Ports | Access | Proxy Route / Domain | Description / Role |`
|
||||
|
||||
Always parse and print matching columns when proposing or applying documentation edits.
|
||||
|
||||
### Step 5: Proxmox LXC/VM Auditing
|
||||
Proxmox host virtualization represents nodes (containers/VMs) that communicate on the LAN subnet `10.0.0.0/16`.
|
||||
- Parse the Proxmox Node Table under `### Proxmox node (laptop-proxmox)` in `homelab_Infra.md`.
|
||||
- Retrieve all network canvas nodes from `http://localhost:9445/api/v1/nodes` matching types `vm`, `lxc`, or direct hosts.
|
||||
- Compare and verify that active virtualization nodes on your network (such as `invidious` on `10.0.0.217` or `pterodactyl` on `10.0.0.110`) exist inside the documentation table. Propose additions for any unregistered LXCs or VMs.
|
||||
|
||||
### Step 6: Homelable MCP Canvas Styling Standards
|
||||
|
||||
When registering new Docker containers on the visual canvas using the `POST` endpoint at `/api/v1/nodes`, you must strictly match the styling of other standard services (use `gitea_runner` as reference):
|
||||
|
||||
- `"type"` must be `"docker_container"`
|
||||
- `"hostname"` must be `"docker"`
|
||||
- `"ip"` must be `""` (empty string, as they reside behind parent container networking)
|
||||
- `"width"` must be `140.0`
|
||||
- `"height"` must be `50.0`
|
||||
- `"bottom_handles"` must be `1`
|
||||
- `"custom_colors"` must be `{"show_services": false}`
|
||||
- `"parent_id"` must be set to the Docker parent host node UUID (e.g. `ee61411a-018a-4471-8a76-bcd4a1f5579a`)
|
||||
|
||||
---
|
||||
|
||||
## 5. Safety and Guardrails
|
||||
|
||||
> [!IMPORTANT]
|
||||
> 1. **No silent writes:** All modifications to `homelab_Infra.md`, the `homelable-mcp` canvas, or `services.yaml` must be presented as a clear diff block and receive explicit, manual user validation.
|
||||
> 2. **NO UNAUTHORIZED GIT COMMANDS:** Never execute any Git commands (such as `git add`, `git commit`, `git status`, `git diff`, etc.) without the user's explicit, per-action authorization.
|
||||
> 3. **Preserve Comments:** Do not remove, alter, or strip existing documentation comments or section text blocks.
|
||||
> 4. **No Errors Log File:** Do not append failures to a local `errors.log` file. Output error details directly to the active console stream to keep workspace logs clean.
|
||||
|
||||
---
|
||||
|
||||
## 6. Technical Pitfalls & Workarounds
|
||||
|
||||
> [!WARNING]
|
||||
> - **Windows CRLF line endings (`\r\n`):** The Homepage dashboard's `services.yaml` contains Windows CRLF terminators. Traditional bash/unix line-ending regex matches (such as `$`) will **silently fail** to match titles or categories. Always run `tr -d '\r'` to sanitize lines before attempting pattern matches.
|
||||
> - **Direct REST API bypass:** Querying `homelable-mcp` on `8001` via SSE requires an async JSON-RPC transport negotiation which is fragile and slow. Instead, bypass the MCP layer entirely and query the `homelable-backend` REST API directly on port `9445` at `/api/v1/nodes`. Pass the header `"X-MCP-Service-Key"` parsed from `/srv/configs/docker_compose/homelabel/.env` for instant, authenticated, and robust network nodes lookup.
|
||||
- **Active Port Verification (Stale Config Filter):** Nginx Proxy Manager configuration files on disk can be stale/leftover (e.g. Moonlight). To completely prevent reporting decommissioned configs, always verify that the port associated with any Nginx config is **actively listening on the host** (or mapped in active docker container port-mappings) before reporting it as an active proxy. If the port is inactive, ignore the proxy block completely.
|
||||
- **TrueNAS SCALE Node Socket Verification:** Rather than setting up fragile credentials or SSH logins, TrueNAS SCALE container port verification is achieved securely and in milliseconds using lightweight, non-destructive `/dev/tcp/10.0.0.2/[port]` network socket connection handshakes.
|
||||
- **Proxmox VE API Live Verification:** To verify running LXCs and VMs, query `https://10.0.0.142:8006/api2/json/cluster/resources?type=vm` directly. Load `HOMEPAGE_VAR_proxmox_password` dynamically from `/srv/configs/docker_compose/homepage/.env` at runtime and strip outer double quotes before passing to the Authorization headers. Never hardcode credentials in codebase or transcripts.
|
||||
- **Homepage Settings Layout Syncing:** When renaming category names in `/srv/configs/docker_compose/homepage/config/services.yaml` (such as `Arr` -> `Truenas Containers`, and `Containers` -> `Docker Containers`), the layout block names inside `/srv/configs/docker_compose/homepage/config/settings.yaml` under `layout:` **must be updated identically** or the cards will fail to render on the dashboard.
|
||||
- **Do NOT Invent Public Proxies:** Do not invent public proxy domains or automatically propose changing local/VPN hrefs (such as `http://radarr/` or `http://calibre/`) to public domains (like `radarr.wompmacho.com`) in homepage configurations. They must remain as local/VPN URLs exactly as currently configured to preserve network isolation and security.
|
||||
- **Multi-Alias Substring Matcher:** Nginx Proxy configs often specify multiple space-separated aliases (e.g. `server_name vaultwarden vaultwarden.wompmacho.com;`). When searching for these in `services.yaml` to detect missing cards, token-split the domains and verify if *any* alias matches as a substring within the homepage card `href` or name (case-insensitively). This prevents false-positives (such as flagging Vaultwarden or Gitea as missing).
|
||||
|
||||
---
|
||||
|
||||
## 7. Skill File Structure
|
||||
|
||||
All scripts and files associated with this skill are stored in this self-contained directory:
|
||||
- `SKILL.md` — Custom skill documentation and procedures.
|
||||
- `scripts/homelab-infrastructure-review.sh` — Bash shell auditing script.
|
||||
- `scripts/mcp_client.py` — Python client for direct database and API queries.
|
||||
- `errors.log` — Legacy log file (decommissioned per safety rules — errors print straight to console).
|
||||
- `.audit.lock` — Legitimate lock file preventing concurrent executions.
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# Homelable Backend API Reference
|
||||
|
||||
Direct REST API access to Pouzor/homelable visualizer backend on port 9445. Bypasses the complex and slow Server-Sent Events (SSE) handshake required on port 8001.
|
||||
|
||||
## Authentication
|
||||
The backend API requires an unauthenticated sub-app service key. Read this key dynamically from:
|
||||
`/srv/configs/docker_compose/homelabel/.env` -> `MCP_SERVICE_KEY`
|
||||
|
||||
Inject it into HTTP request headers:
|
||||
`X-MCP-Service-Key: <MCP_SERVICE_KEY>`
|
||||
|
||||
## Endpoints
|
||||
|
||||
### 1. Health Check
|
||||
`GET http://localhost:9445/api/v1/health`
|
||||
Returns `{"status": "ok"}` on success (HTTP 200).
|
||||
|
||||
### 2. Retrieve Canvas Nodes
|
||||
`GET http://localhost:9445/api/v1/nodes`
|
||||
Returns a JSON array of active canvas nodes.
|
||||
|
||||
Example Node Object:
|
||||
```json
|
||||
{
|
||||
"type": "computer",
|
||||
"label": "Game Pc",
|
||||
"hostname": "DESKTOP-BRHGMFT",
|
||||
"ip": "10.0.0.109",
|
||||
"status": "online",
|
||||
"services": [
|
||||
{
|
||||
"port": 11434,
|
||||
"protocol": "tcp",
|
||||
"service_name": "Ollama"
|
||||
}
|
||||
],
|
||||
"id": "3fc16e75-9418-41b1-b4a2-d3859c9ce431"
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Retrieve Canvas Edges
|
||||
`GET http://localhost:9445/api/v1/edges`
|
||||
Returns active connection edges between nodes.
|
||||
|
||||
### 4. Trigger Network Scan
|
||||
`POST http://localhost:9445/api/v1/scan/trigger`
|
||||
@@ -0,0 +1,68 @@
|
||||
# Homelable Backend Direct REST API Reference
|
||||
|
||||
Instead of negotiating a complex and slow Server-Sent Events (SSE) stream over port `8001` (which requires async JSON-RPC sessions and frequently times out), query the REST API on `homelable-backend` directly.
|
||||
|
||||
## Endpoint & Port
|
||||
- **Host Port:** `9445`
|
||||
- **Base URL:** `http://localhost:9445/api/v1`
|
||||
|
||||
## Authentication Header
|
||||
The backend is authenticated using a special service key header:
|
||||
```
|
||||
X-MCP-Service-Key: <MCP_SERVICE_KEY>
|
||||
```
|
||||
|
||||
### Loading Credentials Dynamically
|
||||
To keep keys safe and secure (without leaking them to the LLM context window), read the key dynamically from the local `.env` configuration file:
|
||||
- **Path:** `/srv/configs/docker_compose/homelabel/.env`
|
||||
- **Variable Name:** `MCP_SERVICE_KEY`
|
||||
|
||||
## Common Endpoints
|
||||
|
||||
### 1. Health Check
|
||||
- **Path:** `/api/v1/health`
|
||||
- **Method:** `GET`
|
||||
- **Response:** `{"status": "ok"}`
|
||||
|
||||
### 2. Retrieve Nodes (Canvas Devices)
|
||||
- **Path:** `/api/v1/nodes`
|
||||
- **Method:** `GET`
|
||||
- **Response:** Array of node objects:
|
||||
```json
|
||||
[
|
||||
{
|
||||
"id": "3fc16e75-9418-41b1-b4a2-d3859c9ce431",
|
||||
"type": "computer",
|
||||
"label": "Game Pc",
|
||||
"ip": "10.0.0.109",
|
||||
"status": "online",
|
||||
"services": [
|
||||
{
|
||||
"port": 11434,
|
||||
"protocol": "tcp",
|
||||
"service_name": "Ollama"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## Python Integration Example
|
||||
|
||||
```python
|
||||
import os
|
||||
import requests
|
||||
|
||||
env_path = "/srv/configs/docker_compose/homelabel/.env"
|
||||
service_key = ""
|
||||
|
||||
if os.path.exists(env_path):
|
||||
with open(env_path, "r") as f:
|
||||
for line in f:
|
||||
if line.startswith("MCP_SERVICE_KEY="):
|
||||
service_key = line.split("=", 1)[1].strip()
|
||||
|
||||
headers = {"X-MCP-Service-Key": service_key}
|
||||
resp = requests.get("http://localhost:9445/api/v1/nodes", headers=headers)
|
||||
nodes = resp.json()
|
||||
```
|
||||
@@ -0,0 +1,627 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# homelab-infrastructure-review.sh
|
||||
# Triggers a full 3-way infrastructure reconciliation audit.
|
||||
# Audits Infra.md ↔ Live Docker ↔ Homelable Canvas ↔ Homepage Dashboard.
|
||||
# Triggered ONLY by explicit, manual user request. No cron or automation.
|
||||
#
|
||||
# Usage: bash /srv/configs/ai/skills/homelab-infrastructure-review/scripts/homelab-infrastructure-review.sh [--dry-run]
|
||||
#
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
CONF_DIR="/srv/configs/ai"
|
||||
INFRA_FILE="/srv/docs/public/projects/homelab/homelab_Infra.md"
|
||||
MCP_ENDPOINT="http://localhost:9445/api/v1"
|
||||
ALIAS_FILE="/srv/configs/.bash_aliases"
|
||||
LOCK_FILE="/srv/configs/ai/.audit.lock"
|
||||
HOMEPAGE_FILE="/srv/configs/docker_compose/homepage/config/services.yaml"
|
||||
DRY_RUN=false
|
||||
|
||||
if [[ "${1:-}" == "--dry-run" ]]; then
|
||||
DRY_RUN=true
|
||||
fi
|
||||
|
||||
# ── Logging helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
log_info() {
|
||||
local msg="[INFO] $(date '+%Y-%m-%d %H:%M:%S') $*"
|
||||
echo "$msg"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
local msg="[ERROR] $(date '+%Y-%m-%d %H:%M:%S') $*"
|
||||
echo "$msg" >&2
|
||||
}
|
||||
|
||||
# ── Lock (prevent concurrent runs) ───────────────────────────────────────────
|
||||
|
||||
acquire_lock() {
|
||||
if [ -f "$LOCK_FILE" ]; then
|
||||
local lock_pid
|
||||
lock_pid=$(cat "$LOCK_FILE" 2>/dev/null || echo "")
|
||||
if [ -n "$lock_pid" ] && kill -0 "$lock_pid" 2>/dev/null; then
|
||||
log_error "Another audit process is running (PID $lock_pid). Aborting."
|
||||
exit 1
|
||||
fi
|
||||
log_info "Stale lock file found (PID $lock_pid). Cleaning up."
|
||||
rm -f "$LOCK_FILE"
|
||||
fi
|
||||
echo $$ > "$LOCK_FILE"
|
||||
trap 'rm -f "$LOCK_FILE"' EXIT
|
||||
}
|
||||
|
||||
# ── Port listening check helper ──────────────────────────────────────────────
|
||||
|
||||
is_port_active() {
|
||||
local target_port="$1"
|
||||
# 1. Check if the port is in ss -tulnp
|
||||
if ss -tulnp 2>/dev/null | grep -q -E ":${target_port}[[:space:]]"; then
|
||||
return 0
|
||||
fi
|
||||
# 2. Check if the port is mapped in running docker containers
|
||||
if docker ps --format "{{.Ports}}" | grep -q -E "(^|[^0-9])${target_port}->"; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# ── Pre-flight: Port 9445 availability ───────────────────────────────────────
|
||||
|
||||
check_port_8001() {
|
||||
# Port 9445 SHOULD be listening (homelable-backend port-mapped on host)
|
||||
if ! ss -tulnp 2>/dev/null | grep -q ':9445[[:space:]]'; then
|
||||
log_error "Port 9445 is not listening — homelable-backend is not port-mapped. Cannot proceed. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
# Parse the service key if present
|
||||
local service_key=""
|
||||
if [ -f "/srv/configs/docker_compose/homelabel/.env" ]; then
|
||||
service_key=$(grep -E "^MCP_SERVICE_KEY=" "/srv/configs/docker_compose/homelabel/.env" | cut -d'=' -f2-)
|
||||
fi
|
||||
# Verify the backend endpoint is actually responding via HTTP
|
||||
local http_code
|
||||
http_code=$(curl -s -H "X-MCP-Service-Key: $service_key" -o /dev/null -w '%{http_code}' --max-time 5 "http://localhost:9445/api/v1/health" 2>/dev/null) || http_code="000"
|
||||
# 000 = connection failure = server unreachable
|
||||
if [ "$http_code" = "000" ]; then
|
||||
log_error "Backend endpoint http://localhost:9445/api/v1/health unresponsive on port 9445. May be container crash or network issue. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
log_info "Port 9445 confirmed active for homelable-backend (HTTP $http_code)."
|
||||
}
|
||||
|
||||
# ── Pre-flight: Source file existence ────────────────────────────────────────
|
||||
|
||||
check_sources() {
|
||||
if [ ! -f "$ALIAS_FILE" ]; then
|
||||
log_error "Alias file $ALIAS_FILE not found. Cannot proceed. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$INFRA_FILE" ]; then
|
||||
log_error "Infra doc $INFRA_FILE not found. Cannot proceed. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$HOMEPAGE_FILE" ]; then
|
||||
log_error "Homepage config $HOMEPAGE_FILE not found. Cannot proceed. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Source of Truth: Infra.md ────────────────────────────────────────────────
|
||||
|
||||
get_infra_docs() {
|
||||
# Extract container/service entries from homelab_Infra.md
|
||||
# Support both 4-column and 5-column structures dynamically
|
||||
local current_section=""
|
||||
while IFS= read -r line || [ -n "$line" ]; do
|
||||
# Detect section headers
|
||||
if [[ "$line" =~ ^##[[:space:]]+(.+)$ ]]; then
|
||||
current_section="${BASH_REMATCH[1]}"
|
||||
fi
|
||||
|
||||
# Check if line is a markdown table row (must start with | and contain a bold name)
|
||||
if [[ "$line" =~ ^\|[[:space:]]*\*\*([a-zA-Z0-9_-]+)\*\*[[:space:]]*\| ]]; then
|
||||
# Count the pipes to deduce the number of columns
|
||||
local pipe_count
|
||||
pipe_count=$(echo -n "$line" | tr -cd '|' | wc -c)
|
||||
|
||||
if [ "$pipe_count" -eq 5 ]; then
|
||||
# Original 4 columns: | Name | Mapped Ports | Access | Desc |
|
||||
local name ports access desc
|
||||
IFS='|' read -r _blank name ports access desc _blank_end <<< "$line"
|
||||
# Clean up whitespaces & bold marks
|
||||
name=$(echo "$name" | tr -d '*' | xargs)
|
||||
ports=$(echo "$ports" | xargs)
|
||||
access=$(echo "$access" | xargs)
|
||||
desc=$(echo "$desc" | xargs)
|
||||
# Output: name|ports|section|desc|proxy|access
|
||||
printf '%s|%s|%s|%s|%s|%s\n' "$name" "$ports" "$current_section" "$desc" "none" "$access"
|
||||
elif [ "$pipe_count" -eq 6 ]; then
|
||||
# Restructured 5 columns: | Name | Mapped Ports | Access | Proxy | Desc |
|
||||
local name ports access proxy desc
|
||||
IFS='|' read -r _blank name ports access proxy desc _blank_end <<< "$line"
|
||||
name=$(echo "$name" | tr -d '*' | xargs)
|
||||
ports=$(echo "$ports" | xargs)
|
||||
access=$(echo "$access" | xargs)
|
||||
proxy=$(echo "$proxy" | xargs)
|
||||
desc=$(echo "$desc" | xargs)
|
||||
# Output: name|ports|section|desc|proxy|access
|
||||
printf '%s|%s|%s|%s|%s|%s\n' "$name" "$ports" "$current_section" "$desc" "$proxy" "$access"
|
||||
fi
|
||||
fi
|
||||
done < "$INFRA_FILE"
|
||||
}
|
||||
|
||||
# ── Source of Truth: Live Docker ─────────────────────────────────────────────
|
||||
|
||||
get_live_docker() {
|
||||
# Use _docker-table alias logic inline
|
||||
if ! docker info > /dev/null 2>&1; then
|
||||
log_error "Docker daemon is not running. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
docker ps --format "{{.Names}}|{{.Status}}|{{.Ports}}" | while IFS='|' read -r name status ports; do
|
||||
cleaned_ports=""
|
||||
if [ -n "$ports" ]; then
|
||||
IFS=',' read -r -a port_array <<< "$ports"
|
||||
for p in "${port_array[@]}"; do
|
||||
p=$(echo "$p" | xargs --no-run-if-empty)
|
||||
# Strip to the right of -> (e.g., "0.0.0.0:80->80/tcp")
|
||||
host_port="${p##*->}"
|
||||
# Strip protocol suffix (tcp/udp) for dedup
|
||||
host_port="${host_port%%/*}"
|
||||
[[ "$host_port" == */tcp ]] && host_port="${host_port%/tcp}"
|
||||
[[ "$host_port" == */udp ]] && host_port="${host_port%/udp}"
|
||||
if [[ -n "$host_port" ]]; then
|
||||
local found=false
|
||||
local existing
|
||||
for existing in $cleaned_ports; do
|
||||
[ "$existing" = "$host_port" ] && found=true
|
||||
done
|
||||
$found || cleaned_ports="${cleaned_ports:+$cleaned_ports, }$host_port"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
if [ -z "$cleaned_ports" ]; then
|
||||
cleaned_ports="internal"
|
||||
fi
|
||||
printf '%s|%s|%s\n' "$name" "$status" "$cleaned_ports"
|
||||
done
|
||||
}
|
||||
|
||||
# ── Source of Truth: Homelable Backend Nodes ─────────────────────────────────
|
||||
|
||||
get_mcp_nodes() {
|
||||
# Parse the service key if present
|
||||
local service_key=""
|
||||
if [ -f "/srv/configs/docker_compose/homelabel/.env" ]; then
|
||||
service_key=$(grep -E "^MCP_SERVICE_KEY=" "/srv/configs/docker_compose/homelabel/.env" | cut -d'=' -f2-)
|
||||
fi
|
||||
|
||||
# Fetch nodes from the backend REST API
|
||||
local payload
|
||||
payload=$(curl -s -H "X-MCP-Service-Key: $service_key" --max-time 10 "${MCP_ENDPOINT}/nodes" 2>/dev/null)
|
||||
if [ -z "$payload" ]; then
|
||||
log_error "Homelable Backend returned empty response. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
# Parse and output nodes: label|type|ip|status
|
||||
echo "$payload" | jq -r '.[]? | "\(.label // .name // "unknown")|\(.type // "node")|\(.ip // "n/a")|\(.status // "unknown")"' 2>/dev/null
|
||||
}
|
||||
|
||||
# ── Source of Truth: Nginx Proxies (Stale-Filtering Enabled) ─────────────────
|
||||
|
||||
get_nginx_proxies() {
|
||||
local conf_dir="/app/nginxproxy/data/nginx/proxy_host"
|
||||
if [ ! -d "$conf_dir" ]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Format: PORT|DOMAINS
|
||||
grep -E "set .port|server_name" "$conf_dir"/*.conf 2>/dev/null \
|
||||
| sed 's/^[ \t]*//; s/.*conf://' \
|
||||
| paste - - \
|
||||
| sed 's/set \$port //; s/server_name //; s/;//g' \
|
||||
| awk '{$1=$1; print}' \
|
||||
| sort -n \
|
||||
| while read -r port domains; do
|
||||
# STALE CONFIG ELIMINATION: Only report proxies if the mapped port is actively listening
|
||||
if is_port_active "$port"; then
|
||||
printf '%s|%s\n' "$port" "$domains"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# ── Source of Truth: Homepage Config ─────────────────────────────────────────
|
||||
|
||||
get_homepage_services() {
|
||||
# Reads services.yaml, stripping carriage returns, and outputs simple mappings: key_name|href|category
|
||||
local current_cat=""
|
||||
local service_name=""
|
||||
local href=""
|
||||
while IFS= read -r raw_line || [ -n "$raw_line" ]; do
|
||||
local line
|
||||
line=$(echo -n "$raw_line" | tr -d '\r')
|
||||
|
||||
# Detect category (e.g., "- Devices & Network:" or "- Arr:")
|
||||
if [[ "$line" =~ ^-[[:space:]]+([^:]+):$ ]]; then
|
||||
# If we had an unprinted service, print it before changing categories
|
||||
if [ -n "$service_name" ]; then
|
||||
printf '%s|%s|%s\n' "$service_name" "$href" "$current_cat"
|
||||
service_name=""
|
||||
href=""
|
||||
fi
|
||||
current_cat="${BASH_REMATCH[1]}"
|
||||
# Detect service element (e.g., " - Proxmox:")
|
||||
elif [[ "$line" =~ ^[[:space:]]+-[[:space:]]+([^:]+):$ ]]; then
|
||||
if [ -n "$service_name" ]; then
|
||||
printf '%s|%s|%s\n' "$service_name" "$href" "$current_cat"
|
||||
fi
|
||||
service_name="${BASH_REMATCH[1]}"
|
||||
href="none"
|
||||
# Detect href (e.g., " href: \"http://10.0.0.142:8006/\"")
|
||||
elif [[ "$line" =~ href:[[:space:]]*\"([^\"]+)\" || "$line" =~ href:[[:space:]]*(https?://[^\s]+) ]]; then
|
||||
local val="${BASH_REMATCH[1]}"
|
||||
val="${val%\"}" # Remove trailing quotes
|
||||
val="${val#\"}" # Remove leading quotes
|
||||
href="$val"
|
||||
fi
|
||||
done < "$HOMEPAGE_FILE"
|
||||
# Print the last service
|
||||
if [ -n "$service_name" ]; then
|
||||
printf '%s|%s|%s\n' "$service_name" "$href" "$current_cat"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Source of Truth: Proxmox VMs / LXCs ──────────────────────────────────────
|
||||
|
||||
get_infra_proxmox_nodes() {
|
||||
# Extract Proxmox VM/LXC entries from homelab_Infra.md
|
||||
# Format: | ID | Type | Hostname | IP | Role |
|
||||
while IFS= read -r line || [ -n "$line" ]; do
|
||||
if [[ "$line" =~ ^\|[[:space:]]*([0-9_-]+|-)[[:space:]]*\|[[:space:]]*(LXC|VM)[[:space:]]*\|[[:space:]]*\*\*([a-zA-Z0-9_-]+)\*\*[[:space:]]*\|[[:space:]]*([0-9.]+)[[:space:]]*\| ]]; then
|
||||
local type="${BASH_REMATCH[2]}"
|
||||
local hostname="${BASH_REMATCH[3]}"
|
||||
local ip="${BASH_REMATCH[4]}"
|
||||
printf '%s|%s|%s\n' "$hostname" "$type" "$ip"
|
||||
fi
|
||||
done < "$INFRA_FILE"
|
||||
}
|
||||
|
||||
# ── 3-Way Reconciliation Engine ──────────────────────────────────────────────
|
||||
|
||||
reconcile() {
|
||||
local infra_count docker_count mcp_count proxy_count hp_count
|
||||
infra_count=$(get_infra_docs | wc -l)
|
||||
docker_count=$(get_live_docker | wc -l)
|
||||
mcp_count=$(get_mcp_nodes | wc -l)
|
||||
proxy_count=$(get_nginx_proxies | wc -l)
|
||||
hp_count=$(get_homepage_services | wc -l)
|
||||
|
||||
log_info "Infra.md: $infra_count | Live Docker: $docker_count | MCP Nodes: $mcp_count | Proxies (Active): $proxy_count | Homepage Dashboard Cards: $hp_count"
|
||||
|
||||
local diff_report=""
|
||||
diff_report+="## Homelab Infrastructure Reconciliation Report\n"
|
||||
diff_report+="Generated: $(date '+%Y-%m-%d %H:%M:%S')\n\n"
|
||||
|
||||
# ── Comparison 1: Documentation vs Live Docker Drift ──
|
||||
diff_report+="### 1. Documentation vs. Live Docker Drift\n\n"
|
||||
|
||||
local doc_names=()
|
||||
while IFS='|' read -r name _ports _section _desc _proxy _access; do
|
||||
# Only compare Docker Services section
|
||||
if [[ "$_section" == "Docker services" ]]; then
|
||||
doc_names+=("$name")
|
||||
fi
|
||||
done < <(get_infra_docs)
|
||||
|
||||
local live_names=()
|
||||
while IFS='|' read -r name _status _ports; do
|
||||
live_names+=("$name")
|
||||
done < <(get_live_docker)
|
||||
|
||||
# Missing in Docker
|
||||
local missing=0
|
||||
for doc in "${doc_names[@]}"; do
|
||||
local found=false
|
||||
for live in "${live_names[@]}"; do
|
||||
[ "$doc" = "$live" ] && found=true && break
|
||||
done
|
||||
# Exceptions
|
||||
if [ "$doc" = "dev" ] || [ "$doc" = "calibre" ]; then
|
||||
found=true
|
||||
fi
|
||||
$found || {
|
||||
diff_report+=" ✗ **${doc}** — listed in Infra.md but NOT running in Docker\n"
|
||||
((missing++)) || true
|
||||
}
|
||||
done
|
||||
[ $missing -eq 0 ] && diff_report+=" ✅ All documented container services are running\n"
|
||||
|
||||
# Untracked in Documentation
|
||||
local untracked=0
|
||||
for live in "${live_names[@]}"; do
|
||||
local found=false
|
||||
for doc in "${doc_names[@]}"; do
|
||||
[ "$live" = "$doc" ] && found=true && break
|
||||
done
|
||||
# Map group names or sub-services
|
||||
if [[ "$live" == "paperless-webserver-1" && " ${doc_names[*]} " == *" paperless-ngx "* ]]; then found=true; fi
|
||||
if [[ "$live" == "paperless-broker-1" && " ${doc_names[*]} " == *" paperless-broker "* ]]; then found=true; fi
|
||||
if [[ "$live" == "paperless-db-1" && " ${doc_names[*]} " == *" paperless-db "* ]]; then found=true; fi
|
||||
if [[ "$live" == "slopsmith-web" && " ${doc_names[*]} " == *" slopsmith-web "* ]]; then found=true; fi
|
||||
if [[ "$live" == "musicbrainz_picard" && " ${doc_names[*]} " == *" musicbrainz_picard "* ]]; then found=true; fi
|
||||
|
||||
$found || {
|
||||
local status=$(get_live_docker | grep "^${live}|" | head -1 | cut -d'|' -f2)
|
||||
local ports=$(get_live_docker | grep "^${live}|" | head -1 | cut -d'|' -f3)
|
||||
diff_report+=" ⚠ **${live}** — running (Status: ${status}, Ports: ${ports}) but NOT in Infra.md\n"
|
||||
((untracked++)) || true
|
||||
}
|
||||
done
|
||||
[ $untracked -eq 0 ] && diff_report+=" ✅ No untracked containers running\n"
|
||||
|
||||
# Unhealthy Containers
|
||||
local unhealthy=0
|
||||
while IFS='|' read -r name status _ports; do
|
||||
if [[ ! "$status" =~ "(healthy)" && "$status" =~ "Up" ]]; then
|
||||
diff_report+=" ⚠ **${name}** — Up but NOT marked healthy (${status})\n"
|
||||
((unhealthy++)) || true
|
||||
fi
|
||||
done < <(get_live_docker)
|
||||
[ $unhealthy -eq 0 ] && diff_report+=" ✅ All active containers are running healthy\n"
|
||||
|
||||
# ── Comparison 2: Nginx Proxy Manager Live Mappings vs Doc ──
|
||||
diff_report+="\n### 2. Live Nginx Proxy Mappings vs. Documentation\n\n"
|
||||
|
||||
local proxy_drift=0
|
||||
while IFS='|' read -r port domains; do
|
||||
# Cross reference the port against documented services in Infra.md
|
||||
local doc_name="unknown"
|
||||
local doc_proxy="none"
|
||||
while IFS='|' read -r name d_ports _section _desc d_proxy _access; do
|
||||
# Check if port matches any documented port
|
||||
if [[ " $d_ports " == *"$port"* ]]; then
|
||||
doc_name="$name"
|
||||
doc_proxy="$d_proxy"
|
||||
break
|
||||
fi
|
||||
done < <(get_infra_docs)
|
||||
|
||||
if [ "$doc_name" != "unknown" ]; then
|
||||
# Clean domains to simplify matching
|
||||
local first_domain
|
||||
first_domain=$(echo "$domains" | awk '{print $1}')
|
||||
if [ "$doc_proxy" = "none" ] || [ "$doc_proxy" = "-" ]; then
|
||||
diff_report+=" ⚠ Port **$port** (**$doc_name**) has proxy domains **($domains)** but has NO proxy domain documented in Infra.md\n"
|
||||
((proxy_drift++)) || true
|
||||
elif [[ "$doc_proxy" != *"$first_domain"* ]]; then
|
||||
diff_report+=" ⚠ Port **$port** (**$doc_name**) proxy drift! Doc says: \`$doc_proxy\`, Live proxy is: \`$domains\`\n"
|
||||
((proxy_drift++)) || true
|
||||
fi
|
||||
fi
|
||||
done < <(get_nginx_proxies)
|
||||
[ $proxy_drift -eq 0 ] && diff_report+=" ✅ All active proxy definitions align with documentation proxy routes\n"
|
||||
|
||||
# ── Comparison 3: Live Docker vs MCP Canvas Nodes ──
|
||||
diff_report+="\n### 3. Live Docker vs. Homelable Canvas Drift\n\n"
|
||||
|
||||
local canvas_drift=0
|
||||
# Check if active containers exist on canvas
|
||||
while IFS='|' read -r name _status _ports; do
|
||||
if [[ "$name" == *"postgres"* || "$name" == *"redis"* || "$name" == *"db"* || "$name" == *"broker"* ]]; then
|
||||
continue
|
||||
fi
|
||||
local found=false
|
||||
while IFS='|' read -r label type ip status; do
|
||||
# Case insensitive comparison
|
||||
local clean_label
|
||||
clean_label=$(echo "$label" | tr '[:upper:]' '[:lower:]' | tr ' _-' ' ')
|
||||
local clean_name
|
||||
clean_name=$(echo "$name" | tr '[:upper:]' '[:lower:]' | tr ' _-' ' ')
|
||||
# Custom mappings for neko and hugo public/private
|
||||
if [[ "$clean_name" == "docs-public" && "$clean_label" == *"hugo"* && "$clean_label" == *"public"* ]]; then found=true; fi
|
||||
if [[ "$clean_name" == "docs-private" && "$clean_label" == *"hugo"* && "$clean_label" == *"pivate"* ]]; then found=true; fi
|
||||
if [[ "$clean_name" == "guacamole" && "$clean_label" == *"quacamole"* ]]; then found=true; fi
|
||||
|
||||
if [[ "$clean_label" == *"$clean_name"* || "$clean_name" == *"$clean_label"* ]]; then
|
||||
found=true
|
||||
break
|
||||
fi
|
||||
done < <(get_mcp_nodes)
|
||||
|
||||
$found || {
|
||||
diff_report+=" ⚠ **${name}** — container is running, but NOT mapped on the Homelable canvas topology\n"
|
||||
((canvas_drift++)) || true
|
||||
}
|
||||
done < <(get_live_docker)
|
||||
[ $canvas_drift -eq 0 ] && diff_report+=" ✅ All primary running services are mapped on the network canvas\n"
|
||||
|
||||
# ── Comparison 4: Homepage Configuration Audit ──
|
||||
diff_report+="\n### 4. Homepage Dashboard Audit (services.yaml)\n\n"
|
||||
|
||||
local hp_drift=0
|
||||
local hp_services=()
|
||||
while IFS='|' read -r s_name s_href s_cat; do
|
||||
hp_services+=("$s_name|$s_href|$s_cat")
|
||||
done < <(get_homepage_services)
|
||||
|
||||
# Cross reference living proxied ports vs Homepage links
|
||||
while IFS='|' read -r port domains; do
|
||||
# Check if any alias in domains string is a substring of any card href
|
||||
local found=false
|
||||
local alias_item
|
||||
for alias_item in $domains; do
|
||||
alias_item="${alias_item%;}" # strip semicolon
|
||||
|
||||
for card in "${hp_services[@]}"; do
|
||||
IFS='|' read -r c_name c_href c_cat <<< "$card"
|
||||
|
||||
# Case insensitive comparison
|
||||
local clean_alias
|
||||
clean_alias=$(echo "$alias_item" | tr '[:upper:]' '[:lower:]')
|
||||
local clean_href
|
||||
clean_href=$(echo "$c_href" | tr '[:upper:]' '[:lower:]')
|
||||
local clean_name
|
||||
clean_name=$(echo "$c_name" | tr '[:upper:]' '[:lower:]')
|
||||
|
||||
if [[ "$clean_href" == *"$clean_alias"* || "$clean_name" == *"$clean_alias"* ]]; then
|
||||
found=true
|
||||
break 2
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
# Proxmox remains as is pointing directly to its IP, ignore in mapping drift
|
||||
if [[ "$domains" == *"proxmox"* ]]; then
|
||||
found=true
|
||||
fi
|
||||
|
||||
$found || {
|
||||
# Skip internal system components
|
||||
local first_domain
|
||||
first_domain=$(echo "$domains" | awk '{print $1}')
|
||||
if [[ "$first_domain" != "nginx" && "$first_domain" != "homelable" && "$first_domain" != "openai" && "$first_domain" != "private" ]]; then
|
||||
# Classify the service correctly
|
||||
local category="Docker Containers"
|
||||
if [[ "$port" == "8006" || "$port" == "8010" ]]; then
|
||||
category="Physical Devices & Networking related"
|
||||
elif [[ "$port" == "32015" || "$port" == "30025" || "$port" == "30027" || "$port" == "30014" || "$port" == "30045" || "$port" == "30046" || "$port" == "30050" ]]; then
|
||||
category="Truenas Containers"
|
||||
fi
|
||||
|
||||
diff_report+=" ⚠ **$first_domain** (Port $port) — active proxy domain is missing from Homepage dashboard! Recommend adding under **$category**.\n"
|
||||
((hp_drift++)) || true
|
||||
fi
|
||||
}
|
||||
done < <(get_nginx_proxies)
|
||||
[ $hp_drift -eq 0 ] && diff_report+=" ✅ All active proxied domain targets are mapped on the Homepage dashboard\n"
|
||||
|
||||
# ── Comparison 5: Proxmox Virtualization Node Live Audit (LXC / VMs) ──
|
||||
diff_report+="\n### 5. Proxmox Virtualization Drift (LXC / VMs)\n\n"
|
||||
|
||||
local proxmox_drift=0
|
||||
local doc_proxmox=()
|
||||
while IFS='|' read -r h_name h_type h_ip; do
|
||||
proxmox_proxmox_item=$(echo "$h_name" | tr '[:upper:]' '[:lower:]')
|
||||
doc_proxmox+=("$proxmox_proxmox_item")
|
||||
done < <(get_infra_proxmox_nodes)
|
||||
|
||||
# Fetch live running LXCs/VMs from Proxmox VE API using dynamically loaded homepage token
|
||||
local pve_token=""
|
||||
if [ -f "/srv/configs/docker_compose/homepage/.env" ]; then
|
||||
pve_token=$(grep -E "^HOMEPAGE_VAR_proxmox_password=" "/srv/configs/docker_compose/homepage/.env" | cut -d'=' -f2- | tr -d '\r')
|
||||
# Strip leading/trailing double quotes if present
|
||||
pve_token="${pve_token%\"}"
|
||||
pve_token="${pve_token#\"}"
|
||||
fi
|
||||
|
||||
if [ -n "$pve_token" ]; then
|
||||
# Query Proxmox API directly (SSL check bypassed with -k)
|
||||
local pve_resources
|
||||
pve_resources=$(curl -k -s -H "Authorization: PVEAPIToken=root@pam!dash=$pve_token" "https://10.0.0.142:8006/api2/json/cluster/resources?type=vm" 2>/dev/null) || pve_resources=""
|
||||
|
||||
if [ -n "$pve_resources" ]; then
|
||||
while read -r pve_node; do
|
||||
local label status type
|
||||
label=$(echo "$pve_node" | cut -d'|' -f1)
|
||||
type=$(echo "$pve_node" | cut -d'|' -f2) # qemu (VM) or lxc
|
||||
status=$(echo "$pve_node" | cut -d'|' -f3)
|
||||
|
||||
# Only check active/running nodes
|
||||
if [ "$status" = "running" ] && [ "$label" != "docker" ]; then
|
||||
local clean_label
|
||||
clean_label=$(echo "$label" | tr '[:upper:]' '[:lower:]' | tr ' _-' ' ')
|
||||
local found=false
|
||||
for doc_node in "${doc_proxmox[@]}"; do
|
||||
if [[ "$clean_label" == *"$doc_node"* || "$doc_node" == *"$clean_label"* ]]; then
|
||||
found=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
$found || {
|
||||
local friendly_type="VM"
|
||||
[ "$type" = "lxc" ] && friendly_type="LXC"
|
||||
diff_report+=" ⚠ **${label}** — active Proxmox **${friendly_type}** is running, but NOT documented under Proxmox Nodes in Infra.md\n"
|
||||
((proxmox_drift++)) || true
|
||||
}
|
||||
fi
|
||||
done < <(echo "$pve_resources" | jq -r '.data[]? | "\(.name)|\(.type)|\(.status)"' 2>/dev/null)
|
||||
else
|
||||
diff_report+=" ⚠ Unable to query Proxmox VE API (connection failure). Skipping live VM/LXC verification.\n"
|
||||
((proxmox_drift++)) || true
|
||||
fi
|
||||
else
|
||||
diff_report+=" ⚠ Proxmox token missing from homepage/.env. Skipping live VM/LXC verification.\n"
|
||||
((proxmox_drift++)) || true
|
||||
fi
|
||||
[ $proxmox_drift -eq 0 ] && diff_report+=" ✅ All active Proxmox virtualization nodes are documented\n"
|
||||
|
||||
# ── Comparison 6: TrueNAS SCALE Node Services Live Audit ──
|
||||
diff_report+="\n### 6. TrueNAS SCALE Node Services Verification\n\n"
|
||||
|
||||
local truenas_drift=0
|
||||
# Audit each documented TrueNAS service by performing active port checks directly on truenas (10.0.0.2)
|
||||
while IFS='|' read -r name d_ports section desc proxy access; do
|
||||
if [[ "$section" == "Truenas Services" ]]; then
|
||||
# Verify port is listening over /dev/tcp network socket
|
||||
if timeout 1 bash -c "cat < /dev/null > /dev/tcp/10.0.0.2/${d_ports}" 2>/dev/null; then
|
||||
# Port is active
|
||||
true
|
||||
else
|
||||
diff_report+=" ✗ **${name}** — documented on TrueNAS port **${d_ports}** but service is NOT listening (down/offline)\n"
|
||||
((truenas_drift++)) || true
|
||||
fi
|
||||
fi
|
||||
done < <(get_infra_docs)
|
||||
[ $truenas_drift -eq 0 ] && diff_report+=" ✅ All documented TrueNAS Services are actively running on port 10.0.0.2\n"
|
||||
|
||||
cppt_output=$(echo -e "$diff_report")
|
||||
echo "$cppt_output"
|
||||
}
|
||||
|
||||
# ── Main Execution Pipeline ──────────────────────────────────────────────────
|
||||
|
||||
main() {
|
||||
log_info "═══════════════════════════════════════════════════════"
|
||||
log_info "Starting Homelab Infrastructure Reconciliation Audit"
|
||||
log_info "═══════════════════════════════════════════════════════"
|
||||
|
||||
acquire_lock
|
||||
check_port_8001
|
||||
check_sources
|
||||
log_info "All pre-flight checks passed."
|
||||
|
||||
if $DRY_RUN; then
|
||||
log_info "DRY RUN — gathering sources without full reconciliation"
|
||||
log_info "--- Infra.md entries ---"
|
||||
get_infra_docs | while IFS='|' read -r name ports section desc proxy access; do
|
||||
echo " $name (section: $section, ports: $ports, proxy: $proxy)"
|
||||
done
|
||||
log_info "--- Live Docker containers ---"
|
||||
get_live_docker | while IFS='|' read -r name status ports; do
|
||||
echo " $name | $status | $ports"
|
||||
done
|
||||
log_info "--- Homelable MCP check ---"
|
||||
local mcp_result
|
||||
mcp_result=$(curl -s --max-time 5 "${MCP_ENDPOINT}" 2>/dev/null || echo "unreachable")
|
||||
if [ "$mcp_result" = "unreachable" ]; then
|
||||
log_info " ✗ MCP unreachable"
|
||||
else
|
||||
log_info " ✓ MCP reachable (first 100 chars: ${mcp_result:0:100})"
|
||||
fi
|
||||
return 0
|
||||
fi
|
||||
|
||||
log_info "Fetching live states..."
|
||||
get_infra_docs > /dev/null
|
||||
get_live_docker > /dev/null
|
||||
get_mcp_nodes > /dev/null
|
||||
log_info "All sources fetched."
|
||||
|
||||
log_info "Running reconciliation..."
|
||||
reconcile
|
||||
}
|
||||
|
||||
main "$@"
|
||||
115
ai/skills/homelab-infrastructure-review/scripts/mcp_client.py
Normal file
115
ai/skills/homelab-infrastructure-review/scripts/mcp_client.py
Normal file
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
MCP Homelable Integration Module.
|
||||
|
||||
Communicates directly with the homelable-backend on port 9445
|
||||
for canvas topology queries, node/edge retrieval, and scanning.
|
||||
"""
|
||||
|
||||
import os
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from typing import Optional, List, Dict
|
||||
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
BACKEND_URL = "http://localhost:9445/api/v1"
|
||||
ENV_PATH = "/srv/configs/docker_compose/homelabel/.env"
|
||||
|
||||
|
||||
def _get_service_key() -> str:
|
||||
"""Read service key dynamically from .env to authenticate request securely."""
|
||||
try:
|
||||
if os.path.exists(ENV_PATH):
|
||||
with open(ENV_PATH, "r") as f:
|
||||
for line in f:
|
||||
if line.startswith("MCP_SERVICE_KEY="):
|
||||
return line.split("=", 1)[1].strip()
|
||||
except Exception as exc:
|
||||
logger.error(f"Failed to read service key from {ENV_PATH}: {exc}")
|
||||
return ""
|
||||
|
||||
|
||||
def health_check(endpoint: str = BACKEND_URL) -> bool:
|
||||
"""Check if homelable-backend endpoint is reachable."""
|
||||
try:
|
||||
import requests
|
||||
except ImportError:
|
||||
# Socket check fallback
|
||||
import socket
|
||||
try:
|
||||
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
s.settimeout(3)
|
||||
s.connect(('localhost', 9445))
|
||||
s.close()
|
||||
logger.info("Backend socket check: port 9445 is open")
|
||||
return True
|
||||
except (ConnectionRefusedError, socket.timeout, OSError):
|
||||
logger.warning("Backend socket check failed: port 9445 unreachable")
|
||||
return False
|
||||
|
||||
key = _get_service_key()
|
||||
headers = {"X-MCP-Service-Key": key}
|
||||
try:
|
||||
resp = requests.get(f"{endpoint}/health", headers=headers, timeout=5)
|
||||
if resp.status_code == 200:
|
||||
logger.info("Homelable-backend health check: OK")
|
||||
return True
|
||||
else:
|
||||
logger.warning(f"Homelable-backend health check failed with HTTP {resp.status_code}")
|
||||
return False
|
||||
except requests.RequestException as exc:
|
||||
logger.warning(f"Homelable-backend health check exception: {exc}")
|
||||
return False
|
||||
|
||||
|
||||
def fetch_nodes(endpoint: str = BACKEND_URL) -> Optional[List[Dict]]:
|
||||
"""Fetch nodes directly from homelable-backend."""
|
||||
if not health_check(endpoint):
|
||||
return None
|
||||
|
||||
try:
|
||||
import requests
|
||||
except ImportError:
|
||||
logger.error("requests library not installed")
|
||||
return None
|
||||
|
||||
key = _get_service_key()
|
||||
headers = {"X-MCP-Service-Key": key}
|
||||
try:
|
||||
resp = requests.get(f"{endpoint}/nodes", headers=headers, timeout=5)
|
||||
resp.raise_for_status()
|
||||
nodes = resp.json()
|
||||
logger.info(f"Fetched {len(nodes)} nodes directly from backend")
|
||||
return nodes
|
||||
except requests.RequestException as exc:
|
||||
logger.error(f"Failed to fetch nodes from backend: {exc}")
|
||||
return None
|
||||
|
||||
|
||||
def log_mcp_error(source: str, output: str) -> None:
|
||||
"""Write MCP-related errors to errors.log."""
|
||||
timestamp = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
log_line = f"[{timestamp}] ERROR: Homelable {source} {output}.\n"
|
||||
errors_log_path = "/srv/configs/ai/errors.log"
|
||||
try:
|
||||
with open(errors_log_path, "a") as f:
|
||||
f.write(log_line)
|
||||
except OSError:
|
||||
print(log_line, file=__import__("sys").stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
logger.info("Testing backend integration...")
|
||||
if health_check():
|
||||
nodes = fetch_nodes()
|
||||
if nodes:
|
||||
print(f"\nSuccessfully retrieved {len(nodes)} nodes:")
|
||||
for node in nodes[:5]:
|
||||
print(f" - {node.get('label', 'unknown')} ({node.get('ip', 'no IP')}) [Status: {node.get('status', 'n/a')}]")
|
||||
else:
|
||||
print("Failed to retrieve nodes.")
|
||||
else:
|
||||
print("Backend is unreachable.")
|
||||
@@ -56,7 +56,7 @@
|
||||
type: pterodactyl
|
||||
url: https://games.wompmacho.com
|
||||
key: {{HOMEPAGE_VAR_pterodactyl_apikey}}
|
||||
- Arr:
|
||||
- Truenas Containers:
|
||||
- Jellyfin:
|
||||
icon: jellyfin.svg
|
||||
href: "https://jellyfin.wompmacho.com/"
|
||||
@@ -151,7 +151,7 @@
|
||||
type: prowlarr
|
||||
url: http://truenas:30050
|
||||
key: {{HOMEPAGE_VAR_prowlarr_apikey}}
|
||||
- Containers:
|
||||
- Docker Containers:
|
||||
- vaultwarden:
|
||||
icon: vaultwarden.svg
|
||||
href: "https://vaultwarden.wompmacho.com"
|
||||
@@ -197,11 +197,6 @@
|
||||
href: "https://wompmacho.com/login"
|
||||
target: _self
|
||||
description: LinkStack
|
||||
# - sure:
|
||||
# icon: http://sure/assets/logomark-color-92ed1e7e.svg
|
||||
# href: "http://sure/"
|
||||
# target: _self
|
||||
# description: sure finance app
|
||||
- paperless:
|
||||
icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/png/paperless-ngx.png
|
||||
href: "http://paperless/"
|
||||
|
||||
@@ -23,10 +23,10 @@ layout:
|
||||
useEqualHeights: true
|
||||
style: row
|
||||
columns: 4
|
||||
Containers:
|
||||
Docker Containers:
|
||||
style: row
|
||||
columns: 8
|
||||
Arr:
|
||||
Truenas Containers:
|
||||
useEqualHeights: true
|
||||
style: row
|
||||
columns: 5
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
# paperless-ngx - https://awesome-docker-compose.com/paperless-ngx
|
||||
services:
|
||||
broker:
|
||||
container_name: paperless-broker
|
||||
image: docker.io/library/redis:7
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ${PAPERLESS_DATA}/redis-data:/data
|
||||
|
||||
db:
|
||||
container_name: paperless-db
|
||||
image: docker.io/library/postgres:16
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
@@ -17,6 +19,7 @@ services:
|
||||
POSTGRES_PASSWORD: ${DB_PASSWORD}
|
||||
|
||||
webserver:
|
||||
container_name: paperless-ngx
|
||||
image: ghcr.io/paperless-ngx/paperless-ngx:latest
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
|
||||
Reference in New Issue
Block a user