Compare commits
12 Commits
73e6a1a4a0
...
main
@@ -11,7 +11,7 @@ grep -E "set .port|server_name" /app/nginxproxy/data/nginx/proxy_host/*.conf \
|
||||
alias nginx='_ports'
|
||||
|
||||
# tmp rebuild for hugo development
|
||||
alias hugo-rebuild='rm -rf public/ && hugo server --appendPort=false --baseURL="/" --ignoreCache'
|
||||
alias hugo-rebuild='cd /srv/dev/hugo/wiki && ./preview.sh public'
|
||||
|
||||
# helpers for quickly editing aliases
|
||||
alias src='source ~/.bash_aliases'
|
||||
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
@@ -40,3 +40,6 @@ dump.rdb
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
|
||||
# Desktop Services Store
|
||||
.DS_Store
|
||||
@@ -14,6 +14,11 @@ Specialized configuration files (JSON) for [Pterodactyl Panel](https://pterodact
|
||||
|
||||
For more details on how to import and use these, see the [Eggs README](./eggs/README.md).
|
||||
|
||||
### Configuration Files
|
||||
|
||||
This repository also tracks global configuration files that maintain system state and environment consistency:
|
||||
- `.bash_aliases`: Tracks persistent command-line aliases for custom environment workflows.
|
||||
|
||||
## Future Scope
|
||||
|
||||
This repository is intended to expand into a comprehensive hub for all configuration types beyond Docker and Pterodactyl, including:
|
||||
|
||||
114
ai/plan.md
114
ai/plan.md
@@ -1,114 +0,0 @@
|
||||
# AI Agent Infrastructure Strategy: Homelab Management
|
||||
|
||||
This document establishes the architecture, workflow rules, and skill definitions for an AI agent to interface with the self-hosted environment using the Model Context Protocol (MCP) and local system aliases.
|
||||
|
||||
---
|
||||
|
||||
## 1. Primary AI Workspace and Persistence Directive
|
||||
|
||||
The agent must treat the `/srv/configs/ai` directory as its absolute source of truth for its own configuration, execution memory, and custom skill scripts.
|
||||
|
||||
### Operational Rules
|
||||
|
||||
* **Storage Location:** All agent workflows, system prompts, automated rules, markdown runbooks, and custom skill definitions *must* be stored in `/srv/configs/ai/`.
|
||||
* **State Management:** Before executing any systemic infrastructure review, the agent must read the current context files in this directory to ensure it is operating under the latest user definitions.
|
||||
* **No Side Effects:** The agent is strictly prohibited from writing or modifying configuration files outside of this path unless explicitly authorized by the user during an interactive session.
|
||||
|
||||
---
|
||||
|
||||
## 2. Homelable MCP Server Integration
|
||||
|
||||
The agent leverages the Model Context Protocol to interface directly with the `homelable-mcp` container running locally. This allows the agent to bind physical network topologies to visual infrastructure diagrams.
|
||||
|
||||
### Connection Architecture
|
||||
|
||||
* **Location:** Co-located on the same local host node.
|
||||
* **Transport Method:** Server-Sent Events (SSE) or local port binding.
|
||||
* **API Endpoint:** `http://localhost:8001`
|
||||
|
||||
### Capabilities Matrix
|
||||
|
||||
| Permission | Action | Operational Boundaries |
|
||||
| ---------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- |
|
||||
| **Read** | List all nodes, edges, full canvas, pending devices, scan history. | Executed proactively during an audit cycle. |
|
||||
| **Write** | Add/update/delete nodes and edges, trigger network scans, approve/hide discovered devices. | **DISABLED BY DEFAULT.** Requires explicit, per-action user confirmation. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Host System Execution Environment & Aliases
|
||||
|
||||
The agent interacts with the host infrastructure by executing commands directly within the host shell session. It relies on system aliases defined in the central configuration file.
|
||||
|
||||
* **Alias Configuration File:** `/srv/configs/.bash_aliases`
|
||||
|
||||
### Required Core Aliases
|
||||
|
||||
The agent must parse and respect the following explicit wrappers found in the alias file:
|
||||
|
||||
* `docker-service`: Used to query container states, image updates, and running stacks.
|
||||
* `nginx`: Used to inspect reverse proxy server blocks, active upstreams, and active domain routings.
|
||||
|
||||
---
|
||||
|
||||
## 4. Skill Definition: Infrastructure Reconciliation & Audit
|
||||
|
||||
### Skill Name
|
||||
|
||||
`review_homelab_infrastructure`
|
||||
|
||||
### Objective
|
||||
|
||||
Cross-reference the active compute and reverse-proxy states against the `homelable` canvas topology and the primary markdown infrastructure log.
|
||||
|
||||
### Source of Truth Inventory Documentation
|
||||
|
||||
* **Target File:** `/srv/docs/public/projects/homelab/homelab_Infra.md`
|
||||
* **Policy:** The agent must maintain and preserve the existing structural formatting of this file when suggesting updates.
|
||||
|
||||
### Execution Workflow
|
||||
|
||||
```
|
||||
[Trigger: Audit Command]
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ 1. Parse aliases via `/srv/configs/.bash_aliases` │
|
||||
│ 2. Read active inventory from `homelab_Infra.md` │
|
||||
│ 3. Fetch live container states via `docker-service` │
|
||||
│ 4. Fetch proxy mappings via host `nginx` commands │
|
||||
│ 5. Pull active topology via Homelable MCP (Port 8001) │
|
||||
└─────────────────────────┬──────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ Perform 3-Way Reconciliation │
|
||||
│ (homelab_Infra.md vs. Live States vs. MCP Canvas) │
|
||||
└─────────────────────────┬──────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ Generate Diff Report & Proposed Changes (Markdown) │
|
||||
└─────────────────────────┬──────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
{ User Explicitly Approves? }
|
||||
/ \
|
||||
(Approved) (Denied)
|
||||
/ \
|
||||
▼ ▼
|
||||
┌─────────────────────────┐ ┌─────────────────────────┐
|
||||
│ - Update MCP Canvas │ │ Abort & Log Audit │
|
||||
│ - Append/Modify │ │ Failure in Workspace │
|
||||
│ homelab_Infra.md │ │ (/srv/configs/ai/) │
|
||||
└─────────────────────────┘ └─────────────────────────┘
|
||||
|
||||
```
|
||||
|
||||
### Safety and Guardrails
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **Strict Read-Only Default:** The agent is explicitly forbidden from making automatic writes or commits.
|
||||
> * **No silent canvas updates:** Every node change, edge modification, or scan approval on the Homelable MCP requires manual confirmation.
|
||||
> * **No silent documentation updates:** Under no circumstances should the agent modify `/srv/docs/public/projects/homelab/homelab_Infra.md` without presenting a clear diff block and receiving confirmation first.
|
||||
>
|
||||
>
|
||||
BIN
ai/skills/.DS_Store
vendored
BIN
ai/skills/.DS_Store
vendored
Binary file not shown.
@@ -1,33 +1,42 @@
|
||||
---
|
||||
name: docs_architecture
|
||||
description: Exact technical architecture, routing, and deployment workflow for WompMacho's decoupled Hugo/Gitea/Nginx Wiki infrastructure.
|
||||
---
|
||||
|
||||
# Docs Architecture Skill
|
||||
|
||||
This skill documents the exact technical architecture, routing, and deployment workflow for WompMacho's decoupled Hugo/Gitea/Caddy Wiki infrastructure.
|
||||
This skill documents the exact technical architecture, routing, and deployment workflow for WompMacho's decoupled Hugo/Gitea/Nginx Wiki infrastructure.
|
||||
|
||||
## 🏗️ 1. Core Architecture (Decoupled Repositories)
|
||||
|
||||
The system is split into three independent Git repositories hosted on `git.wompmacho.com`:
|
||||
|
||||
1. **Framework (`/srv/dev/hugo/wiki`)**: Repository `hugo-framework`
|
||||
* **Purpose**: The central Hugo engine. Contains the Blowfish theme as a submodule (`themes/blowfish`) and the master `config/_default/params.toml`.
|
||||
* **Rule**: NEVER place content (`.md` files) here. NEVER place environment-specific overrides (like hardcoded `editURLs` or top-level menus) in the global `params.toml`.
|
||||
* **Purpose**: The central Hugo engine. The Blowfish theme is fully **decoupled/vendored** directly into the project's root folders (`layouts/`, `assets/`, `static/`, `data/`, `i18n/`, `archetypes/`). No theme submodules are used, protecting the site from upstream updates breaking the build.
|
||||
* **Rule**: NEVER place content (`.md` files) here. NEVER place environment-specific overrides (like hardcoded `editURLs` or top-level menus) in the global config.
|
||||
|
||||
2. **Public Wiki (`/srv/docs/public`)**: Repository `docs-public`
|
||||
* **Purpose**: Public-facing markdown notes, assets (`/static`), and its own top-bar menu configuration (`config/_default/menus.en.toml`).
|
||||
* **Deployment Target**: `/srv/caddy/sites/wiki`
|
||||
* **Deployment Target**: `/srv/www/docs-public`
|
||||
* **Live URL**: `https://wiki.wompmacho.com` (proxied by NPM).
|
||||
|
||||
3. **Private Wiki (`/srv/docs/private`)**: Repository `docs-private`
|
||||
* **Purpose**: Personal, private markdown notes, assets (`/static`), and its own top-bar menu configuration (`config/_default/menus.en.toml`).
|
||||
* **Deployment Target**: `/srv/caddy/sites/private-wiki`
|
||||
* **Deployment Target**: `/srv/www/docs-private`
|
||||
* **Live URL**: `http://10.0.0.190:9897` (or `http://private` via NPM).
|
||||
|
||||
## 💻 2. Local Development ("Live Link")
|
||||
## 💻 2. Local Development ("Live Link" Automation)
|
||||
|
||||
When working on content, the user uses **Code-Server**. To see real-time updates without deploying, the `hugo-framework` repository uses symbolic links.
|
||||
When working on content, the user uses **Code-Server**. To see real-time updates without deploying, the framework repository uses automated symlinking and preview scripts.
|
||||
|
||||
* `content -> /srv/docs/public`
|
||||
* `static -> /srv/docs/public/static`
|
||||
|
||||
**Command to start local preview**: `cd /srv/dev/hugo/wiki && hugo server --bind 0.0.0.0 --appendPort=false --baseURL="/"`
|
||||
* **`preview.sh [public|private]`**:
|
||||
* Creates symlinks: `content -> /srv/docs/[public|private]` and `static -> /srv/docs/[public|private]/static`.
|
||||
* Overwrites or symlinks specific config overrides (like `menus.en.toml`).
|
||||
* Executes `run-dev.sh` to start the local preview server.
|
||||
* Traps exits (Ctrl+C, normal exit) to cleanly restore standard framework defaults and remove preview symlinks automatically.
|
||||
* **`run-dev.sh`**:
|
||||
* Launches the server bound to `0.0.0.0:1313` with code-server proxy parameters.
|
||||
* Passes `--renderToMemory` to prevent local watch loops and file write latency.
|
||||
|
||||
## 🚀 3. CI/CD Pipeline (Gitea Actions)
|
||||
|
||||
@@ -35,7 +44,7 @@ Both `docs-public` and `docs-private` have a `.gitea/workflows/deploy.yaml` acti
|
||||
|
||||
### The Build Process
|
||||
1. **Checkout Content**: The runner checks out the markdown content.
|
||||
2. **Clone Framework**: The runner natively clones the `hugo-framework` (including submodules) using `git clone --recurse-submodules https://git.wompmacho.com/wompmacho/hugo-framework.git`.
|
||||
2. **Clone Framework**: The runner clones the central `hugo-framework` repository directly. Because the Blowfish theme is fully decoupled, there is no need for submodule initialization.
|
||||
3. **Inject Content**: Markdown and static files are copied into the framework's workspace.
|
||||
4. **Inject Dynamic Menus**: The `config/_default/menus.en.toml` is injected, automatically overriding the framework's top navigation bar for that specific site.
|
||||
5. **Inject Edit Links (CRITICAL RULE)**: The "Edit this page" link on the bottom of posts MUST NOT be configured via a `params.toml` override, as Hugo replaces the entire file instead of merging it, destroying the Blowfish styling.
|
||||
@@ -45,17 +54,18 @@ Both `docs-public` and `docs-private` have a `.gitea/workflows/deploy.yaml` acti
|
||||
|
||||
## 🌐 4. Web Server Routing (Nginx)
|
||||
|
||||
The system relies on two extremely lightweight `nginx:alpine` containers defined in `gitea/docker-compose.yaml` to serve static files. They act as "dumb" file servers behind Nginx Proxy Manager (NPM), bypassing the complex Host header validation and 0-byte security issues inherent to Caddy.
|
||||
The system relies on two extremely lightweight `nginx:alpine` containers defined in `gitea/docker-compose.yaml` to serve static files. They act as file servers behind Nginx Proxy Manager (NPM).
|
||||
|
||||
1. **Public (`public_wiki` - Port 9896)**:
|
||||
* Mounts `/srv/caddy/sites/wiki` as read-only.
|
||||
* Mounts `/srv/www/docs-public` as read-only.
|
||||
* NPM routes `wiki.wompmacho.com` to Docker Host IP on port `9896`.
|
||||
2. **Private (`private_wiki` - Port 9897)**:
|
||||
* Mounts `/srv/caddy/sites/private-wiki` as read-only.
|
||||
* Mounts `/srv/www/docs-private` as read-only.
|
||||
* NPM routes `http://private` to Docker Host IP on port `9897`.
|
||||
|
||||
## 🛠️ 5. Known Quirks and Formatting Rules
|
||||
|
||||
* **Submodules**: Always use `git submodule add -b main <url>` and ensure the commit pointer is tracked. Do not copy theme files physically.
|
||||
* **Decoupled Theme**: The Blowfish theme is committed directly in the main codebase. Do not use Git submodules to track the theme.
|
||||
* **Blowfish Homepage**: If the homepage is blank, it means there are no recent posts. By default, Blowfish's `home` layout only lists files with `type: "article"` or `type: "posts"`. For generic folders, set `type: "page"` and `layout: "list"` in `_index.md`.
|
||||
* **Frontmatter**: Blowfish expects standard Hugo frontmatter (`date:` as a string, `authors: ["Name"]` as an array). MkDocs-style nested YAML is incompatible and will crash the build.
|
||||
* **Troubleshooting**: See `references/troubleshooting.md` for handling theme updates, build loops, and TOC configuration.
|
||||
24
ai/skills/docs_architecture/references/troubleshooting.md
Normal file
24
ai/skills/docs_architecture/references/troubleshooting.md
Normal file
@@ -0,0 +1,24 @@
|
||||
# Hugo/Blowfish Troubleshooting & Pitfalls
|
||||
|
||||
## 1. Infinite Rebuild Loops
|
||||
If Hugo triggers a rebuild loop:
|
||||
* **Cause**: Hugo watches the `resources/` directory, and the generation of cached/busted assets (like `admonitions.scss_...json`) triggers a new rebuild event.
|
||||
* **Fix**: Add the following to `config/_default/hugo.toml`:
|
||||
```toml
|
||||
[watch]
|
||||
ignore = ["resources/**"]
|
||||
```
|
||||
* **Tool Usage**: If the loop persists, start the server with polling to reduce sensitivity:
|
||||
`hugo server --poll 1s`
|
||||
|
||||
## 2. Table of Contents (TOC) Issues
|
||||
Blowfish manages TOC natively via `params.toml`. Avoid using manual `[TOC]` or `{{< toc >}}` in your Markdown unless explicitly required by a specific layout.
|
||||
* **Standard Setup**: Ensure `showTableOfContents = true` is set in the `[article]` and `[list]` sections of your `config/_default/params.toml`.
|
||||
* **Custom Layouts**: If TOC is rendering in the wrong place, it is likely due to an overridden `layouts/_default/single.html`. Revert to the native theme structure (`container` and `prose` classes) so the Blowfish partials can handle the placement.
|
||||
* **TOC ScrollSpy Highlighting**: If the TOC fails to highlight active headings when scrolling, verify that `smartTOC = true` is enabled in `params.toml`. If it is nested inside the `[article]` section, ensure that the `layouts/partials/toc.html` checks:
|
||||
`{{ if or .Site.Params.smartTOC .Site.Params.article.smartTOC }}`
|
||||
to correctly resolve the nested parameter.
|
||||
|
||||
## 3. Theme Updates & Customization
|
||||
* **Version mismatch**: When Hugo updates, theme templates often break (e.g., `can't evaluate field Locale`).
|
||||
* **Decoupled Theme Installation**: The theme is fully decoupled/vendored in the root folders. If you need to update Blowfish, you will copy/vendor the new templates into `layouts/` and assets into `assets/`, but you must review custom changes to templates such as `layouts/_default/single.html` and `layouts/partials/toc.html` to make sure custom styles and scripts (like multiple TOC elements or active highlight overrides) are preserved.
|
||||
@@ -18,7 +18,7 @@ metadata:
|
||||
|
||||
# Homelab Infrastructure Review
|
||||
|
||||
Performs a 3-way infrastructure reconciliation audit of living services, active proxy routes, and the network topology canvas.
|
||||
This skill has been refactored into an intelligent, context-aware auditing engine. It uses a YAML configuration (`scripts/audit_config.yaml`) to understand architectural conventions (aliases, ignored containers) and employs a robust, incremental development workflow. **Always test changes in small, atomic steps** (`syntax check -> run`) to avoid complex debugging cycles. The core logic now uses a Python helper (`scripts/load_config.py`) for configuration and `docker inspect` for accurate health checks.
|
||||
|
||||
## 1. Core Objectives
|
||||
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
# Teach the skill how to map running containers to their documented names
|
||||
# This resolves false positives for services running behind a single proxy container.
|
||||
service_aliases:
|
||||
nicotine: gluetun
|
||||
torrent: gluetun
|
||||
paperless-db: paperless-ngx
|
||||
paperless-broker: paperless-ngx
|
||||
guac-postgresql: guacamole
|
||||
gitea-db-1: gitea
|
||||
npm-sync: nginx-proxy-manager
|
||||
homelable-mcp: homelable
|
||||
homelable-backend: homelable
|
||||
homelable-frontend: homelable
|
||||
|
||||
# Teach the skill how to map container names to their visual canvas labels
|
||||
# This resolves false positives for services with user-friendly canvas names.
|
||||
canvas_aliases:
|
||||
docs-public: hugo (public)
|
||||
docs-private: hugo (private)
|
||||
|
||||
# A simple list of containers to ALWAYS ignore during the audit.
|
||||
# This is for transient, dev, or intentionally undocumented containers.
|
||||
ignore_containers:
|
||||
- openai
|
||||
- slopsmith-web
|
||||
- pihole-dns-shim # Managed by pihole LXC, not documented as a separate service.
|
||||
- immich_machine_learning # Sub-service of Immich
|
||||
- immich_postgres # Sub-service of Immich
|
||||
- immich_redis # Sub-service of Immich
|
||||
- guac-postgresql # Sub-service of Guacamole
|
||||
- nginx-proxy-manager # Infrastructure, not a user-facing service
|
||||
- npm-sync # Sub-service of Nginx Proxy Manager
|
||||
- cloudflare-ddns # Infrastructure, not a user-facing service
|
||||
@@ -1,625 +1,125 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# homelab-infrastructure-review.sh
|
||||
# Triggers a full 3-way infrastructure reconciliation audit.
|
||||
# Audits Infra.md ↔ Live Docker ↔ Homelable Canvas ↔ Homepage Dashboard.
|
||||
# Triggered ONLY by explicit, manual user request. No cron or automation.
|
||||
#
|
||||
# Usage: bash /srv/configs/ai/skills/homelab-infrastructure-review/scripts/homelab-infrastructure-review.sh [--dry-run]
|
||||
# Triggers a full, intelligent, context-aware reconciliation audit.
|
||||
#
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
CONF_DIR="/srv/configs/ai"
|
||||
# ── Configuration & Logging ──────────────────────────────────────────────────
|
||||
AUDIT_CONFIG_FILE="/srv/configs/ai/skills/homelab-infrastructure-review/scripts/audit_config.yaml"
|
||||
INFRA_FILE="/srv/docs/public/projects/homelab/homelab_Infra.md"
|
||||
MCP_ENDPOINT="http://localhost:9445/api/v1"
|
||||
ALIAS_FILE="/srv/configs/.bash_aliases"
|
||||
LOCK_FILE="/srv/configs/ai/.audit.lock"
|
||||
HOMEPAGE_FILE="/srv/configs/docker_compose/homepage/config/services.yaml"
|
||||
DRY_RUN=false
|
||||
|
||||
if [[ "${1:-}" == "--dry-run" ]]; then
|
||||
DRY_RUN=true
|
||||
fi
|
||||
|
||||
# ── Logging helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
log_info() {
|
||||
local msg="[INFO] $(date '+%Y-%m-%d %H:%M:%S') $*"
|
||||
echo "$msg"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
local msg="[ERROR] $(date '+%Y-%m-%d %H:%M:%S') $*"
|
||||
echo "$msg" >&2
|
||||
}
|
||||
|
||||
# ── Lock (prevent concurrent runs) ───────────────────────────────────────────
|
||||
log_info() { echo "[INFO] $(date '+%Y-%m-%d %H:%M:%S') $*"; }
|
||||
log_error() { echo "[ERROR] $(date '+%Y-%m-%d %H:%M:%S') $*" >&2; }
|
||||
|
||||
# ── Pre-flight & Lock ────────────────────────────────────────────────────────
|
||||
acquire_lock() {
|
||||
if [ -f "$LOCK_FILE" ]; then
|
||||
local lock_pid
|
||||
lock_pid=$(cat "$LOCK_FILE" 2>/dev/null || echo "")
|
||||
if [ -n "$lock_pid" ] && kill -0 "$lock_pid" 2>/dev/null; then
|
||||
log_error "Another audit process is running (PID $lock_pid). Aborting."
|
||||
exit 1
|
||||
local pid; pid=$(cat "$LOCK_FILE" 2>/dev/null)
|
||||
if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then
|
||||
log_error "Audit already running (PID $pid). Aborting."; exit 1
|
||||
fi
|
||||
log_info "Stale lock file found (PID $lock_pid). Cleaning up."
|
||||
rm -f "$LOCK_FILE"
|
||||
log_info "Stale lock file found. Cleaning up."; rm -f "$LOCK_FILE"
|
||||
fi
|
||||
echo $$ > "$LOCK_FILE"
|
||||
trap 'rm -f "$LOCK_FILE"' EXIT
|
||||
echo $$ > "$LOCK_FILE"; trap 'rm -f "$LOCK_FILE"' EXIT
|
||||
}
|
||||
preflight_checks() {
|
||||
if [ ! -f "$AUDIT_CONFIG_FILE" ]; then log_error "audit_config.yaml not found. Aborting."; exit 1; fi
|
||||
log_info "All pre-flight checks passed."
|
||||
}
|
||||
|
||||
# ── Port listening check helper ──────────────────────────────────────────────
|
||||
|
||||
is_port_active() {
|
||||
local target_port="$1"
|
||||
# 1. Check if the port is in ss -tulnp
|
||||
if ss -tulnp 2>/dev/null | grep -q -E ":${target_port}[[:space:]]"; then
|
||||
return 0
|
||||
fi
|
||||
# 2. Check if the port is mapped in running docker containers
|
||||
if docker ps --format "{{.Ports}}" | grep -q -E "(^|[^0-9])${target_port}->"; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# ── Pre-flight: Port 9445 availability ───────────────────────────────────────
|
||||
|
||||
check_port_8001() {
|
||||
# Port 9445 SHOULD be listening (homelable-backend port-mapped on host)
|
||||
if ! ss -tulnp 2>/dev/null | grep -q ':9445[[:space:]]'; then
|
||||
log_error "Port 9445 is not listening — homelable-backend is not port-mapped. Cannot proceed. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
# Parse the service key if present
|
||||
local service_key=""
|
||||
if [ -f "/srv/configs/docker_compose/homelabel/.env" ]; then
|
||||
service_key=$(grep -E "^MCP_SERVICE_KEY=" "/srv/configs/docker_compose/homelabel/.env" | cut -d'=' -f2-)
|
||||
fi
|
||||
# Verify the backend endpoint is actually responding via HTTP
|
||||
local http_code
|
||||
http_code=$(curl -s -H "X-MCP-Service-Key: $service_key" -o /dev/null -w '%{http_code}' --max-time 5 "http://localhost:9445/api/v1/health" 2>/dev/null) || http_code="000"
|
||||
# 000 = connection failure = server unreachable
|
||||
if [ "$http_code" = "000" ]; then
|
||||
log_error "Backend endpoint http://localhost:9445/api/v1/health unresponsive on port 9445. May be container crash or network issue. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
log_info "Port 9445 confirmed active for homelable-backend (HTTP $http_code)."
|
||||
}
|
||||
|
||||
# ── Pre-flight: Source file existence ────────────────────────────────────────
|
||||
|
||||
check_sources() {
|
||||
if [ ! -f "$ALIAS_FILE" ]; then
|
||||
log_error "Alias file $ALIAS_FILE not found. Cannot proceed. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$INFRA_FILE" ]; then
|
||||
log_error "Infra doc $INFRA_FILE not found. Cannot proceed. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$HOMEPAGE_FILE" ]; then
|
||||
log_error "Homepage config $HOMEPAGE_FILE not found. Cannot proceed. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Source of Truth: Infra.md ────────────────────────────────────────────────
|
||||
|
||||
# ── Data Sources ─────────────────────────────────────────────────────────────
|
||||
get_infra_docs() {
|
||||
# Extract container/service entries from homelab_Infra.md
|
||||
# Support both 4-column and 5-column structures dynamically
|
||||
local current_section=""
|
||||
local in_services_section=false
|
||||
while IFS= read -r line || [ -n "$line" ]; do
|
||||
# Detect section headers
|
||||
if [[ "$line" =~ ^##[[:space:]]+(.+)$ ]]; then
|
||||
current_section="${BASH_REMATCH[1]}"
|
||||
if [[ "$line" =~ ^##[[:space:]]+(Docker|Truenas)[[:space:]]+services ]]; then
|
||||
in_services_section=true
|
||||
elif [[ "$line" =~ ^##[[:space:]] && ! "$line" =~ ^##[[:space:]]+(Docker|Truenas)[[:space:]]+services ]]; then
|
||||
in_services_section=false
|
||||
fi
|
||||
|
||||
# Check if line is a markdown table row (must start with | and contain a bold name)
|
||||
if [[ "$line" =~ ^\|[[:space:]]*\*\*([a-zA-Z0-9_-]+)\*\*[[:space:]]*\| ]]; then
|
||||
# Count the pipes to deduce the number of columns
|
||||
local pipe_count
|
||||
pipe_count=$(echo -n "$line" | tr -cd '|' | wc -c)
|
||||
|
||||
if [ "$pipe_count" -eq 5 ]; then
|
||||
# Original 4 columns: | Name | Mapped Ports | Access | Desc |
|
||||
local name ports access desc
|
||||
IFS='|' read -r _blank name ports access desc _blank_end <<< "$line"
|
||||
# Clean up whitespaces & bold marks
|
||||
name=$(echo "$name" | tr -d '*' | xargs)
|
||||
ports=$(echo "$ports" | xargs)
|
||||
access=$(echo "$access" | xargs)
|
||||
desc=$(echo "$desc" | xargs)
|
||||
# Output: name|ports|section|desc|proxy|access
|
||||
printf '%s|%s|%s|%s|%s|%s\n' "$name" "$ports" "$current_section" "$desc" "none" "$access"
|
||||
elif [ "$pipe_count" -eq 6 ]; then
|
||||
# Restructured 5 columns: | Name | Mapped Ports | Access | Proxy | Desc |
|
||||
local name ports access proxy desc
|
||||
IFS='|' read -r _blank name ports access proxy desc _blank_end <<< "$line"
|
||||
name=$(echo "$name" | tr -d '*' | xargs)
|
||||
ports=$(echo "$ports" | xargs)
|
||||
access=$(echo "$access" | xargs)
|
||||
proxy=$(echo "$proxy" | xargs)
|
||||
desc=$(echo "$desc" | xargs)
|
||||
# Output: name|ports|section|desc|proxy|access
|
||||
printf '%s|%s|%s|%s|%s|%s\n' "$name" "$ports" "$current_section" "$desc" "$proxy" "$access"
|
||||
fi
|
||||
if $in_services_section && [[ "$line" =~ ^\|[[:space:]]*\*\*([a-zA-Z0-9_-]+)\*\* ]]; then
|
||||
IFS='|' read -r _ name _ <<< "$line"; printf '%s\n' "$(echo "$name" | tr -d '*' | xargs)"
|
||||
fi
|
||||
done < "$INFRA_FILE"
|
||||
}
|
||||
|
||||
# ── Source of Truth: Live Docker ─────────────────────────────────────────────
|
||||
|
||||
get_live_docker() {
|
||||
# Use _docker-table alias logic inline
|
||||
if ! docker info > /dev/null 2>&1; then
|
||||
log_error "Docker daemon is not running. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
docker ps --format "{{.Names}}|{{.Status}}|{{.Ports}}" | while IFS='|' read -r name status ports; do
|
||||
cleaned_ports=""
|
||||
if [ -n "$ports" ]; then
|
||||
IFS=',' read -r -a port_array <<< "$ports"
|
||||
for p in "${port_array[@]}"; do
|
||||
p=$(echo "$p" | xargs --no-run-if-empty)
|
||||
# Strip to the right of -> (e.g., "0.0.0.0:80->80/tcp")
|
||||
host_port="${p##*->}"
|
||||
# Strip protocol suffix (tcp/udp) for dedup
|
||||
host_port="${host_port%%/*}"
|
||||
[[ "$host_port" == */tcp ]] && host_port="${host_port%/tcp}"
|
||||
[[ "$host_port" == */udp ]] && host_port="${host_port%/udp}"
|
||||
if [[ -n "$host_port" ]]; then
|
||||
local found=false
|
||||
local existing
|
||||
for existing in $cleaned_ports; do
|
||||
[ "$existing" = "$host_port" ] && found=true
|
||||
done
|
||||
$found || cleaned_ports="${cleaned_ports:+$cleaned_ports, }$host_port"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
if [ -z "$cleaned_ports" ]; then
|
||||
cleaned_ports="internal"
|
||||
fi
|
||||
printf '%s|%s|%s\n' "$name" "$status" "$cleaned_ports"
|
||||
done
|
||||
get_live_docker_containers() {
|
||||
while IFS= read -r line; do
|
||||
local name; name=$(echo "$line" | jq -r '.Names')
|
||||
local health; health=$(docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}no_health_check{{end}}' "$name" 2>/dev/null || echo "error")
|
||||
local status;
|
||||
case "$health" in
|
||||
healthy) status="healthy";;
|
||||
unhealthy) status="UNHEALTHY";;
|
||||
*) status="no_health_check";;
|
||||
esac
|
||||
printf '%s|%s\n' "$name" "$status"
|
||||
done <<< "$(docker ps --format '{{json .}}')"
|
||||
}
|
||||
|
||||
# ── Source of Truth: Homelable Backend Nodes ─────────────────────────────────
|
||||
|
||||
get_mcp_nodes() {
|
||||
# Parse the service key if present
|
||||
local service_key=""
|
||||
if [ -f "/srv/configs/docker_compose/homelabel/.env" ]; then
|
||||
service_key=$(grep -E "^MCP_SERVICE_KEY=" "/srv/configs/docker_compose/homelabel/.env" | cut -d'=' -f2-)
|
||||
fi
|
||||
|
||||
# Fetch nodes from the backend REST API
|
||||
local payload
|
||||
payload=$(curl -s -H "X-MCP-Service-Key: $service_key" --max-time 10 "${MCP_ENDPOINT}/nodes" 2>/dev/null)
|
||||
if [ -z "$payload" ]; then
|
||||
log_error "Homelable Backend returned empty response. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
# Parse and output nodes: label|type|ip|status
|
||||
echo "$payload" | jq -r '.[]? | "\(.label // .name // "unknown")|\(.type // "node")|\(.ip // "n/a")|\(.status // "unknown")"' 2>/dev/null
|
||||
local service_key=""; if [ -f "/srv/configs/docker_compose/homelabel/.env" ]; then service_key=$(grep -E "^MCP_SERVICE_KEY=" "/srv/configs/docker_compose/homelabel/.env" | cut -d'=' -f2-); fi
|
||||
curl -s -H "X-MCP-Service-Key: $service_key" "http://localhost:9445/api/v1/nodes" | jq -r '.[]? | .label // .name // "unknown"'
|
||||
}
|
||||
|
||||
# ── Source of Truth: Nginx Proxies (Stale-Filtering Enabled) ─────────────────
|
||||
|
||||
get_nginx_proxies() {
|
||||
local conf_dir="/app/nginxproxy/data/nginx/proxy_host"
|
||||
if [ ! -d "$conf_dir" ]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Format: PORT|DOMAINS
|
||||
grep -E "set .port|server_name" "$conf_dir"/*.conf 2>/dev/null \
|
||||
| sed 's/^[ \t]*//; s/.*conf://' \
|
||||
| paste - - \
|
||||
| sed 's/set \$port //; s/server_name //; s/;//g' \
|
||||
| awk '{$1=$1; print}' \
|
||||
| sort -n \
|
||||
| while read -r port domains; do
|
||||
# STALE CONFIG ELIMINATION: Only report proxies if the mapped port is actively listening
|
||||
if is_port_active "$port"; then
|
||||
printf '%s|%s\n' "$port" "$domains"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# ── Source of Truth: Homepage Config ─────────────────────────────────────────
|
||||
|
||||
get_homepage_services() {
|
||||
# Reads services.yaml, stripping carriage returns, and outputs simple mappings: key_name|href|category
|
||||
local current_cat=""
|
||||
local service_name=""
|
||||
local href=""
|
||||
while IFS= read -r raw_line || [ -n "$raw_line" ]; do
|
||||
local line
|
||||
line=$(echo -n "$raw_line" | tr -d '\r')
|
||||
|
||||
# Detect category (e.g., "- Devices & Network:" or "- Arr:")
|
||||
if [[ "$line" =~ ^-[[:space:]]+([^:]+):$ ]]; then
|
||||
# If we had an unprinted service, print it before changing categories
|
||||
if [ -n "$service_name" ]; then
|
||||
printf '%s|%s|%s\n' "$service_name" "$href" "$current_cat"
|
||||
service_name=""
|
||||
href=""
|
||||
fi
|
||||
current_cat="${BASH_REMATCH[1]}"
|
||||
# Detect service element (e.g., " - Proxmox:")
|
||||
elif [[ "$line" =~ ^[[:space:]]+-[[:space:]]+([^:]+):$ ]]; then
|
||||
if [ -n "$service_name" ]; then
|
||||
printf '%s|%s|%s\n' "$service_name" "$href" "$current_cat"
|
||||
fi
|
||||
service_name="${BASH_REMATCH[1]}"
|
||||
href="none"
|
||||
# Detect href (e.g., " href: \"http://10.0.0.142:8006/\"")
|
||||
elif [[ "$line" =~ href:[[:space:]]*\"([^\"]+)\" || "$line" =~ href:[[:space:]]*(https?://[^\s]+) ]]; then
|
||||
local val="${BASH_REMATCH[1]}"
|
||||
val="${val%\"}" # Remove trailing quotes
|
||||
val="${val#\"}" # Remove leading quotes
|
||||
href="$val"
|
||||
fi
|
||||
done < "$HOMEPAGE_FILE"
|
||||
# Print the last service
|
||||
if [ -n "$service_name" ]; then
|
||||
printf '%s|%s|%s\n' "$service_name" "$href" "$current_cat"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Source of Truth: Proxmox VMs / LXCs ──────────────────────────────────────
|
||||
|
||||
get_infra_proxmox_nodes() {
|
||||
# Extract Proxmox VM/LXC entries from homelab_Infra.md
|
||||
# Format: | ID | Type | Hostname | IP | Role |
|
||||
while IFS= read -r line || [ -n "$line" ]; do
|
||||
if [[ "$line" =~ ^\|[[:space:]]*([0-9_-]+|-)[[:space:]]*\|[[:space:]]*(LXC|VM)[[:space:]]*\|[[:space:]]*\*\*([a-zA-Z0-9_-]+)\*\*[[:space:]]*\|[[:space:]]*([0-9.]+)[[:space:]]*\| ]]; then
|
||||
local type="${BASH_REMATCH[2]}"
|
||||
local hostname="${BASH_REMATCH[3]}"
|
||||
local ip="${BASH_REMATCH[4]}"
|
||||
printf '%s|%s|%s\n' "$hostname" "$type" "$ip"
|
||||
fi
|
||||
done < "$INFRA_FILE"
|
||||
}
|
||||
|
||||
# ── 3-Way Reconciliation Engine ──────────────────────────────────────────────
|
||||
# ── Reconciliation Engine ────────────────────────────────────────────────────
|
||||
if [ -f "$AUDIT_CONFIG_FILE" ]; then
|
||||
source <(python3 /srv/configs/ai/skills/homelab-infrastructure-review/scripts/load_config.py "$AUDIT_CONFIG_FILE")
|
||||
fi
|
||||
|
||||
reconcile() {
|
||||
local infra_count docker_count mcp_count proxy_count hp_count
|
||||
infra_count=$(get_infra_docs | wc -l)
|
||||
docker_count=$(get_live_docker | wc -l)
|
||||
mcp_count=$(get_mcp_nodes | wc -l)
|
||||
proxy_count=$(get_nginx_proxies | wc -l)
|
||||
hp_count=$(get_homepage_services | wc -l)
|
||||
local doc_services=(); while IFS= read -r name; do doc_services+=("$(echo "$name"|tr '[:upper:]' '[:lower:]')"); done < <(get_infra_docs)
|
||||
local live_services=(); local untracked_report=""; local unhealthy_report=""
|
||||
|
||||
log_info "Infra.md: $infra_count | Live Docker: $docker_count | MCP Nodes: $mcp_count | Proxies (Active): $proxy_count | Homepage Dashboard Cards: $hp_count"
|
||||
while IFS='|' read -r name status; do
|
||||
local lower_name; lower_name=$(echo "$name" | tr '[:upper:]' '[:lower:]')
|
||||
live_services+=("$lower_name")
|
||||
if [[ " ${IGNORE_CONTAINERS} " == *" ${lower_name} "* ]]; then continue; fi
|
||||
|
||||
local diff_report=""
|
||||
diff_report+="## Homelab Infrastructure Reconciliation Report\n"
|
||||
diff_report+="Generated: $(date '+%Y-%m-%d %H:%M:%S')\n\n"
|
||||
|
||||
# ── Comparison 1: Documentation vs Live Docker Drift ──
|
||||
diff_report+="### 1. Documentation vs. Live Docker Drift\n\n"
|
||||
|
||||
local doc_names=()
|
||||
while IFS='|' read -r name _ports _section _desc _proxy _access; do
|
||||
# Only compare Docker Services section
|
||||
if [[ "$_section" == "Docker services" ]]; then
|
||||
doc_names+=("$name")
|
||||
local found=false; for doc in "${doc_services[@]}"; do [ "$lower_name" = "$doc" ] && found=true && break; done
|
||||
local alias_var="SERVICE_ALIAS_${lower_name//-/_}"
|
||||
if ! $found && [[ -v $alias_var ]]; then
|
||||
local target="${!alias_var}"; for doc in "${doc_services[@]}"; do [ "$target" = "$doc" ] && found=true && break; done
|
||||
fi
|
||||
done < <(get_infra_docs)
|
||||
|
||||
local live_names=()
|
||||
while IFS='|' read -r name _status _ports; do
|
||||
live_names+=("$name")
|
||||
done < <(get_live_docker)
|
||||
if ! $found; then untracked_report+=" ⚠ **$name** is running but not documented.\n"; fi
|
||||
if [[ "$status" == "UNHEALTHY" ]]; then unhealthy_report+=" ✗ **$name** is **UNHEALTHY**.\n"; fi
|
||||
done < <(get_live_docker_containers)
|
||||
|
||||
# Missing in Docker
|
||||
local missing=0
|
||||
for doc in "${doc_names[@]}"; do
|
||||
local found=false
|
||||
for live in "${live_names[@]}"; do
|
||||
[ "$doc" = "$live" ] && found=true && break
|
||||
done
|
||||
# Exceptions
|
||||
if [ "$doc" = "dev" ] || [ "$doc" = "calibre" ]; then
|
||||
found=true
|
||||
fi
|
||||
$found || {
|
||||
diff_report+=" ✗ **${doc}** — listed in Infra.md but NOT running in Docker\n"
|
||||
((missing++)) || true
|
||||
}
|
||||
local missing_report=""
|
||||
for doc in "${doc_services[@]}"; do
|
||||
if [ "$doc" = "dev" ] || [ "$doc" = "calibre" ]; then continue; fi # Exceptions
|
||||
local found=false; for live in "${live_services[@]}"; do [ "$doc" = "$live" ] && found=true && break; done
|
||||
if ! $found; then missing_report+=" ✗ **$doc** is documented but not running.\n"; fi
|
||||
done
|
||||
[ $missing -eq 0 ] && diff_report+=" ✅ All documented container services are running\n"
|
||||
|
||||
# Untracked in Documentation
|
||||
local untracked=0
|
||||
for live in "${live_names[@]}"; do
|
||||
local found=false
|
||||
for doc in "${doc_names[@]}"; do
|
||||
[ "$live" = "$doc" ] && found=true && break
|
||||
done
|
||||
# Map group names or sub-services
|
||||
if [[ "$live" == "paperless-webserver-1" && " ${doc_names[*]} " == *" paperless-ngx "* ]]; then found=true; fi
|
||||
if [[ "$live" == "paperless-broker-1" && " ${doc_names[*]} " == *" paperless-broker "* ]]; then found=true; fi
|
||||
if [[ "$live" == "paperless-db-1" && " ${doc_names[*]} " == *" paperless-db "* ]]; then found=true; fi
|
||||
if [[ "$live" == "slopsmith-web" && " ${doc_names[*]} " == *" slopsmith-web "* ]]; then found=true; fi
|
||||
if [[ "$live" == "musicbrainz_picard" && " ${doc_names[*]} " == *" musicbrainz_picard "* ]]; then found=true; fi
|
||||
|
||||
$found || {
|
||||
local status=$(get_live_docker | grep "^${live}|" | head -1 | cut -d'|' -f2)
|
||||
local ports=$(get_live_docker | grep "^${live}|" | head -1 | cut -d'|' -f3)
|
||||
diff_report+=" ⚠ **${live}** — running (Status: ${status}, Ports: ${ports}) but NOT in Infra.md\n"
|
||||
((untracked++)) || true
|
||||
}
|
||||
local mcp_nodes=(); while IFS= read -r label; do mcp_nodes+=("$(echo "$label"|tr '[:upper:]' '[:lower:]'|tr ' _-' ' ')"); done < <(get_mcp_nodes)
|
||||
local canvas_report=""
|
||||
for name in "${live_services[@]}"; do
|
||||
if [[ " ${IGNORE_CONTAINERS} " == *" ${name} "* ]]; then continue; fi
|
||||
local found=false; for node in "${mcp_nodes[@]}"; do [[ "$node" == *"$name"* || "$name" == *"$node"* ]] && found=true && break; done
|
||||
local alias_var="CANVAS_ALIAS_${name//-/_}"
|
||||
if ! $found && [[ -v $alias_var ]]; then
|
||||
local target="${!alias_var}"; for node in "${mcp_nodes[@]}"; do [ "$target" = "$node" ] && found=true && break; done
|
||||
fi
|
||||
if ! $found; then canvas_report+=" ⚠ **$name** is running but not on the canvas.\n"; fi
|
||||
done
|
||||
[ $untracked -eq 0 ] && diff_report+=" ✅ No untracked containers running\n"
|
||||
|
||||
# Unhealthy Containers
|
||||
local unhealthy=0
|
||||
while IFS='|' read -r name status _ports; do
|
||||
if [[ ! "$status" =~ "(healthy)" && "$status" =~ "Up" ]]; then
|
||||
diff_report+=" ⚠ **${name}** — Up but NOT marked healthy (${status})\n"
|
||||
((unhealthy++)) || true
|
||||
fi
|
||||
done < <(get_live_docker)
|
||||
[ $unhealthy -eq 0 ] && diff_report+=" ✅ All active containers are running healthy\n"
|
||||
|
||||
# ── Comparison 2: Nginx Proxy Manager Live Mappings vs Doc ──
|
||||
diff_report+="\n### 2. Live Nginx Proxy Mappings vs. Documentation\n\n"
|
||||
|
||||
local proxy_drift=0
|
||||
while IFS='|' read -r port domains; do
|
||||
# Cross reference the port against documented services in Infra.md
|
||||
local doc_name="unknown"
|
||||
local doc_proxy="none"
|
||||
while IFS='|' read -r name d_ports _section _desc d_proxy _access; do
|
||||
# Check if port matches any documented port
|
||||
if [[ " $d_ports " == *"$port"* ]]; then
|
||||
doc_name="$name"
|
||||
doc_proxy="$d_proxy"
|
||||
break
|
||||
fi
|
||||
done < <(get_infra_docs)
|
||||
|
||||
if [ "$doc_name" != "unknown" ]; then
|
||||
# Clean domains to simplify matching
|
||||
local first_domain
|
||||
first_domain=$(echo "$domains" | awk '{print $1}')
|
||||
if [ "$doc_proxy" = "none" ] || [ "$doc_proxy" = "-" ]; then
|
||||
diff_report+=" ⚠ Port **$port** (**$doc_name**) has proxy domains **($domains)** but has NO proxy domain documented in Infra.md\n"
|
||||
((proxy_drift++)) || true
|
||||
elif [[ "$doc_proxy" != *"$first_domain"* ]]; then
|
||||
diff_report+=" ⚠ Port **$port** (**$doc_name**) proxy drift! Doc says: \`$doc_proxy\`, Live proxy is: \`$domains\`\n"
|
||||
((proxy_drift++)) || true
|
||||
fi
|
||||
fi
|
||||
done < <(get_nginx_proxies)
|
||||
[ $proxy_drift -eq 0 ] && diff_report+=" ✅ All active proxy definitions align with documentation proxy routes\n"
|
||||
|
||||
# ── Comparison 3: Live Docker vs MCP Canvas Nodes ──
|
||||
diff_report+="\n### 3. Live Docker vs. Homelable Canvas Drift\n\n"
|
||||
|
||||
local canvas_drift=0
|
||||
# Check if active containers exist on canvas
|
||||
while IFS='|' read -r name _status _ports; do
|
||||
if [[ "$name" == *"postgres"* || "$name" == *"redis"* || "$name" == *"db"* || "$name" == *"broker"* ]]; then
|
||||
continue
|
||||
fi
|
||||
local found=false
|
||||
while IFS='|' read -r label type ip status; do
|
||||
# Case insensitive comparison
|
||||
local clean_label
|
||||
clean_label=$(echo "$label" | tr '[:upper:]' '[:lower:]' | tr ' _-' ' ')
|
||||
local clean_name
|
||||
clean_name=$(echo "$name" | tr '[:upper:]' '[:lower:]' | tr ' _-' ' ')
|
||||
# Custom mappings for neko and hugo public/private
|
||||
if [[ "$clean_name" == "docs-public" && "$clean_label" == *"hugo"* && "$clean_label" == *"public"* ]]; then found=true; fi
|
||||
if [[ "$clean_name" == "docs-private" && "$clean_label" == *"hugo"* && "$clean_label" == *"pivate"* ]]; then found=true; fi
|
||||
if [[ "$clean_name" == "guacamole" && "$clean_label" == *"quacamole"* ]]; then found=true; fi
|
||||
|
||||
if [[ "$clean_label" == *"$clean_name"* || "$clean_name" == *"$clean_label"* ]]; then
|
||||
found=true
|
||||
break
|
||||
fi
|
||||
done < <(get_mcp_nodes)
|
||||
|
||||
$found || {
|
||||
diff_report+=" ⚠ **${name}** — container is running, but NOT mapped on the Homelable canvas topology\n"
|
||||
((canvas_drift++)) || true
|
||||
}
|
||||
done < <(get_live_docker)
|
||||
[ $canvas_drift -eq 0 ] && diff_report+=" ✅ All primary running services are mapped on the network canvas\n"
|
||||
|
||||
# ── Comparison 4: Homepage Configuration Audit ──
|
||||
diff_report+="\n### 4. Homepage Dashboard Audit (services.yaml)\n\n"
|
||||
|
||||
local hp_drift=0
|
||||
local hp_services=()
|
||||
while IFS='|' read -r s_name s_href s_cat; do
|
||||
hp_services+=("$s_name|$s_href|$s_cat")
|
||||
done < <(get_homepage_services)
|
||||
|
||||
# Cross reference living proxied ports vs Homepage links
|
||||
while IFS='|' read -r port domains; do
|
||||
# Check if any alias in domains string is a substring of any card href
|
||||
local found=false
|
||||
local alias_item
|
||||
for alias_item in $domains; do
|
||||
alias_item="${alias_item%;}" # strip semicolon
|
||||
|
||||
for card in "${hp_services[@]}"; do
|
||||
IFS='|' read -r c_name c_href c_cat <<< "$card"
|
||||
|
||||
# Case insensitive comparison
|
||||
local clean_alias
|
||||
clean_alias=$(echo "$alias_item" | tr '[:upper:]' '[:lower:]')
|
||||
local clean_href
|
||||
clean_href=$(echo "$c_href" | tr '[:upper:]' '[:lower:]')
|
||||
local clean_name
|
||||
clean_name=$(echo "$c_name" | tr '[:upper:]' '[:lower:]')
|
||||
|
||||
if [[ "$clean_href" == *"$clean_alias"* || "$clean_name" == *"$clean_alias"* ]]; then
|
||||
found=true
|
||||
break 2
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
# Proxmox remains as is pointing directly to its IP, ignore in mapping drift
|
||||
if [[ "$domains" == *"proxmox"* ]]; then
|
||||
found=true
|
||||
fi
|
||||
|
||||
$found || {
|
||||
# Skip internal system components
|
||||
local first_domain
|
||||
first_domain=$(echo "$domains" | awk '{print $1}')
|
||||
if [[ "$first_domain" != "nginx" && "$first_domain" != "homelable" && "$first_domain" != "openai" && "$first_domain" != "private" ]]; then
|
||||
# Classify the service correctly
|
||||
local category="Docker Containers"
|
||||
if [[ "$port" == "8006" || "$port" == "8010" ]]; then
|
||||
category="Physical Devices & Networking related"
|
||||
elif [[ "$port" == "32015" || "$port" == "30025" || "$port" == "30027" || "$port" == "30014" || "$port" == "30045" || "$port" == "30046" || "$port" == "30050" ]]; then
|
||||
category="Truenas Containers"
|
||||
fi
|
||||
|
||||
diff_report+=" ⚠ **$first_domain** (Port $port) — active proxy domain is missing from Homepage dashboard! Recommend adding under **$category**.\n"
|
||||
((hp_drift++)) || true
|
||||
fi
|
||||
}
|
||||
done < <(get_nginx_proxies)
|
||||
[ $hp_drift -eq 0 ] && diff_report+=" ✅ All active proxied domain targets are mapped on the Homepage dashboard\n"
|
||||
|
||||
# ── Comparison 5: Proxmox Virtualization Node Live Audit (LXC / VMs) ──
|
||||
diff_report+="\n### 5. Proxmox Virtualization Drift (LXC / VMs)\n\n"
|
||||
|
||||
local proxmox_drift=0
|
||||
local doc_proxmox=()
|
||||
while IFS='|' read -r h_name h_type h_ip; do
|
||||
proxmox_proxmox_item=$(echo "$h_name" | tr '[:upper:]' '[:lower:]')
|
||||
doc_proxmox+=("$proxmox_proxmox_item")
|
||||
done < <(get_infra_proxmox_nodes)
|
||||
|
||||
# Fetch live running LXCs/VMs from Proxmox VE API using dynamically loaded homepage token
|
||||
local pve_token=""
|
||||
if [ -f "/srv/configs/docker_compose/homepage/.env" ]; then
|
||||
pve_token=$(grep -E "^HOMEPAGE_VAR_proxmox_password=" "/srv/configs/docker_compose/homepage/.env" | cut -d'=' -f2- | tr -d '\r')
|
||||
# Strip leading/trailing double quotes if present
|
||||
pve_token="${pve_token%\"}"
|
||||
pve_token="${pve_token#\"}"
|
||||
fi
|
||||
|
||||
if [ -n "$pve_token" ]; then
|
||||
# Query Proxmox API directly (SSL check bypassed with -k)
|
||||
local pve_resources
|
||||
pve_resources=$(curl -k -s -H "Authorization: PVEAPIToken=root@pam!dash=$pve_token" "https://10.0.0.142:8006/api2/json/cluster/resources?type=vm" 2>/dev/null) || pve_resources=""
|
||||
|
||||
if [ -n "$pve_resources" ]; then
|
||||
while read -r pve_node; do
|
||||
local label status type
|
||||
label=$(echo "$pve_node" | cut -d'|' -f1)
|
||||
type=$(echo "$pve_node" | cut -d'|' -f2) # qemu (VM) or lxc
|
||||
status=$(echo "$pve_node" | cut -d'|' -f3)
|
||||
|
||||
# Only check active/running nodes
|
||||
if [ "$status" = "running" ] && [ "$label" != "docker" ]; then
|
||||
local clean_label
|
||||
clean_label=$(echo "$label" | tr '[:upper:]' '[:lower:]' | tr ' _-' ' ')
|
||||
local found=false
|
||||
for doc_node in "${doc_proxmox[@]}"; do
|
||||
if [[ "$clean_label" == *"$doc_node"* || "$doc_node" == *"$clean_label"* ]]; then
|
||||
found=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
$found || {
|
||||
local friendly_type="VM"
|
||||
[ "$type" = "lxc" ] && friendly_type="LXC"
|
||||
diff_report+=" ⚠ **${label}** — active Proxmox **${friendly_type}** is running, but NOT documented under Proxmox Nodes in Infra.md\n"
|
||||
((proxmox_drift++)) || true
|
||||
}
|
||||
fi
|
||||
done < <(echo "$pve_resources" | jq -r '.data[]? | "\(.name)|\(.type)|\(.status)"' 2>/dev/null)
|
||||
else
|
||||
diff_report+=" ⚠ Unable to query Proxmox VE API (connection failure). Skipping live VM/LXC verification.\n"
|
||||
((proxmox_drift++)) || true
|
||||
fi
|
||||
printf "## Homelab Audit Report\nGenerated: $(date)\n\n"
|
||||
printf "### Docker vs. Documentation\n"
|
||||
if [ -z "$untracked_report$missing_report$unhealthy_report" ]; then
|
||||
printf " ✅ All services are aligned and healthy.\n"
|
||||
else
|
||||
diff_report+=" ⚠ Proxmox token missing from homepage/.env. Skipping live VM/LXC verification.\n"
|
||||
((proxmox_drift++)) || true
|
||||
printf "%s%s%s" "$untracked_report" "$missing_report" "$unhealthy_report"
|
||||
fi
|
||||
[ $proxmox_drift -eq 0 ] && diff_report+=" ✅ All active Proxmox virtualization nodes are documented\n"
|
||||
|
||||
# ── Comparison 6: TrueNAS SCALE Node Services Live Audit ──
|
||||
diff_report+="\n### 6. TrueNAS SCALE Node Services Verification\n\n"
|
||||
|
||||
local truenas_drift=0
|
||||
# Audit each documented TrueNAS service by performing active port checks directly on truenas (10.0.0.2)
|
||||
while IFS='|' read -r name d_ports section desc proxy access; do
|
||||
if [[ "$section" == "Truenas Services" ]]; then
|
||||
# Verify port is listening over /dev/tcp network socket
|
||||
if timeout 1 bash -c "cat < /dev/null > /dev/tcp/10.0.0.2/${d_ports}" 2>/dev/null; then
|
||||
# Port is active
|
||||
true
|
||||
else
|
||||
diff_report+=" ✗ **${name}** — documented on TrueNAS port **${d_ports}** but service is NOT listening (down/offline)\n"
|
||||
((truenas_drift++)) || true
|
||||
fi
|
||||
fi
|
||||
done < <(get_infra_docs)
|
||||
[ $truenas_drift -eq 0 ] && diff_report+=" ✅ All documented TrueNAS Services are actively running on port 10.0.0.2\n"
|
||||
|
||||
cppt_output=$(echo -e "$diff_report")
|
||||
echo "$cppt_output"
|
||||
printf "\n### Docker vs. Canvas\n"
|
||||
if [ -z "$canvas_report" ]; then printf " ✅ All services are mapped.\n"; else printf "%s" "$canvas_report"; fi
|
||||
}
|
||||
|
||||
# ── Main Execution Pipeline ──────────────────────────────────────────────────
|
||||
|
||||
# ── Main Execution ───────────────────────────────────────────────────────────
|
||||
main() {
|
||||
log_info "═══════════════════════════════════════════════════════"
|
||||
log_info "Starting Homelab Infrastructure Reconciliation Audit"
|
||||
log_info "═══════════════════════════════════════════════════════"
|
||||
|
||||
log_info "Starting Homelab Infrastructure Audit"
|
||||
acquire_lock
|
||||
check_port_8001
|
||||
check_sources
|
||||
log_info "All pre-flight checks passed."
|
||||
|
||||
if $DRY_RUN; then
|
||||
log_info "DRY RUN — gathering sources without full reconciliation"
|
||||
log_info "--- Infra.md entries ---"
|
||||
get_infra_docs | while IFS='|' read -r name ports section desc proxy access; do
|
||||
echo " $name (section: $section, ports: $ports, proxy: $proxy)"
|
||||
done
|
||||
log_info "--- Live Docker containers ---"
|
||||
get_live_docker | while IFS='|' read -r name status ports; do
|
||||
echo " $name | $status | $ports"
|
||||
done
|
||||
log_info "--- Homelable MCP check ---"
|
||||
local mcp_result
|
||||
mcp_result=$(curl -s --max-time 5 "${MCP_ENDPOINT}" 2>/dev/null || echo "unreachable")
|
||||
if [ "$mcp_result" = "unreachable" ]; then
|
||||
log_info " ✗ MCP unreachable"
|
||||
else
|
||||
log_info " ✓ MCP reachable (first 100 chars: ${mcp_result:0:100})"
|
||||
fi
|
||||
return 0
|
||||
fi
|
||||
|
||||
log_info "Fetching live states..."
|
||||
get_infra_docs > /dev/null
|
||||
get_live_docker > /dev/null
|
||||
get_mcp_nodes > /dev/null
|
||||
log_info "All sources fetched."
|
||||
|
||||
preflight_checks
|
||||
log_info "Running reconciliation..."
|
||||
reconcile
|
||||
}
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
import yaml
|
||||
import sys
|
||||
|
||||
def main(config_path):
|
||||
try:
|
||||
with open(config_path, 'r') as f:
|
||||
config = yaml.safe_load(f)
|
||||
|
||||
if 'service_aliases' in config and config['service_aliases']:
|
||||
for key, value in config['service_aliases'].items():
|
||||
print(f"export SERVICE_ALIAS_{key.replace('-', '_')}='{value}'")
|
||||
|
||||
if 'canvas_aliases' in config and config['canvas_aliases']:
|
||||
for key, value in config['canvas_aliases'].items():
|
||||
print(f"export CANVAS_ALIAS_{key.replace('-', '_')}='{value}'")
|
||||
|
||||
if 'ignore_containers' in config and config['ignore_containers']:
|
||||
ignore_list = " ".join(config['ignore_containers'])
|
||||
print(f"export IGNORE_CONTAINERS='{ignore_list}'")
|
||||
|
||||
except Exception as e:
|
||||
print(f"# Config load failed: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) > 1:
|
||||
main(sys.argv[1])
|
||||
else:
|
||||
print("# Usage: python load_config.py <path_to_config.yaml>", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
@@ -23,3 +23,12 @@ This skill allows Gemini to efficiently manage the lifecycle of project workflow
|
||||
- **Action Items**:
|
||||
- [ ] [Specific task]
|
||||
- **Verification**: [Public-domain method to verify success]
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls & Best Practices
|
||||
|
||||
- **Avoid Absolute Positioning in Layouts**: When customizing themes, prefer Flexbox or Grid over `position: absolute` for structural elements like Sidebars or TOCs to prevent content overlap and preserve responsive layout integrity.
|
||||
- **Global Theme Settings**: For TOC rendering in Blowfish, rely on `params.toml` configuration (`showTableOfContents = true`) rather than custom Markdown shortcodes (`[TOC]` or `{{< toc >}}`), which are often theme-incompatible.
|
||||
- **Custom CSS vs Theme Overrides**: Always use `assets/css/custom.css` for structural overrides. Never modify theme-internal files (`themes/blowfish/...`), as they will be overwritten during updates.
|
||||
- **Container Widths**: When overriding container widths, use relative units (percentages) and `!important` sparingly to avoid breaking theme-defined responsive breakpoints.
|
||||
|
||||
18
docker_compose/farmOS/.gitignore
vendored
Normal file
18
docker_compose/farmOS/.gitignore
vendored
Normal file
@@ -0,0 +1,18 @@
|
||||
# ------------------------------------------------------------------------------
|
||||
# SECRETS AND CREDENTIALS (NEVER COMMIT THESE)
|
||||
# ------------------------------------------------------------------------------
|
||||
|
||||
# Environment variables (API keys, DB passwords, etc.)
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
|
||||
# Local Development DBs (often contain real-ish data)
|
||||
*.db
|
||||
*.sqlite
|
||||
dump.rdb
|
||||
|
||||
# Data
|
||||
/keys
|
||||
/sites
|
||||
/db
|
||||
40
docker_compose/farmOS/docker-compose.yaml
Normal file
40
docker_compose/farmOS/docker-compose.yaml
Normal file
@@ -0,0 +1,40 @@
|
||||
# farmOS - https://farmos.org/ - https://git.drupalcode.org/project/farm/-/blob/4.x/docker/docker-compose.production.yml?ref_type=heads
|
||||
---
|
||||
services:
|
||||
farmOS-db:
|
||||
container_name: farmOS-db
|
||||
image: postgres:17
|
||||
volumes:
|
||||
- './db:/var/lib/postgresql/data'
|
||||
environment:
|
||||
# Set a strong password and optionally change the user/database name.
|
||||
POSTGRES_USER: ${POSTGRES_USER}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||
POSTGRES_DB: ${POSTGRES_DB}
|
||||
restart: unless-stopped
|
||||
|
||||
farmOS-www:
|
||||
container_name: farmOS-www
|
||||
depends_on:
|
||||
- farmOS-db
|
||||
# Update this to the latest stable version before deploying.
|
||||
image: farmos/farmos:latest
|
||||
volumes:
|
||||
- './sites:/opt/drupal/web/sites'
|
||||
- './keys:/opt/drupal/keys'
|
||||
ports:
|
||||
- '${FARMOS_HTTP_PORT}:80'
|
||||
restart: unless-stopped
|
||||
labels:
|
||||
# Nginx Proxy Manager Automation
|
||||
- "npm.proxy.domains=farm.wompmacho.com"
|
||||
- "npm.proxy.host=${DOCKER_HOST_IP}"
|
||||
- "npm.proxy.port=${FARMOS_HTTP_PORT}"
|
||||
- "npm.proxy.scheme=http"
|
||||
- "npm.proxy.ssl_verify=false"
|
||||
- "npm.proxy.websockets=true"
|
||||
- "npm.proxy.ssl.force=true"
|
||||
- "npm.proxy.ssl.certificate.id=2"
|
||||
#- "npm.proxy.advanced.config=location = / { return 301 /guacamole/; }"
|
||||
# Pi-hole Automation (Point to Docker Host/Proxy IP)
|
||||
- "pihole.custom-record=[[\"farm.wompmacho.com\", \"${DOCKER_HOST_IP}\"]]"
|
||||
@@ -2,7 +2,7 @@
|
||||
---
|
||||
services:
|
||||
backend:
|
||||
image: ghcr.io/pouzor/homelable-backend:latest
|
||||
image: ghcr.io/pouzor/homelable-backend:2.5.1
|
||||
container_name: homelable-backend
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
@@ -18,7 +18,7 @@
|
||||
- "${HOMELABEL_BACK_PORT}:8000"
|
||||
|
||||
mcp:
|
||||
image: ghcr.io/pouzor/homelable-mcp:latest
|
||||
image: ghcr.io/pouzor/homelable-mcp:2.5.1
|
||||
container_name: homelable-mcp
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
@@ -32,7 +32,7 @@
|
||||
|
||||
frontend:
|
||||
#image: ghcr.io/pouzor/homelable-frontend-standalone:latest # standalone version
|
||||
image: ghcr.io/pouzor/homelable-frontend:latest
|
||||
image: ghcr.io/pouzor/homelable-frontend:2.5.1
|
||||
container_name: homelable-frontend
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
@@ -41,12 +41,12 @@
|
||||
- backend
|
||||
labels:
|
||||
# Nginx Proxy Manager Automation
|
||||
- "npm.proxy.domains=homelable"
|
||||
- "npm.proxy.domains=homelable.wompmacho.com"
|
||||
- "npm.proxy.host=${DOCKER_HOST_IP}"
|
||||
- "npm.proxy.port=${HOMELABEL_FRONT_PORT}"
|
||||
- "npm.proxy.scheme=http"
|
||||
#- "npm.proxy.websockets=true"
|
||||
- "npm.proxy.websockets=true"
|
||||
#- "npm.proxy.ssl_verify=false"
|
||||
#- "npm.proxy.ssl.force=true"
|
||||
#- "npm.proxy.ssl.certificate.id=1"
|
||||
- "pihole.custom-record=[[\"homelable\", \"${DOCKER_HOST_IP}\"]]"
|
||||
- "npm.proxy.ssl.force=true"
|
||||
- "npm.proxy.ssl.certificate.id=2"
|
||||
- "pihole.custom-record=[[\"homelable.wompmacho.com\", \"${DOCKER_HOST_IP}\"]]"
|
||||
@@ -243,11 +243,11 @@
|
||||
href: "http://reaper/"
|
||||
target: _self
|
||||
description: reaper in docker
|
||||
- Slopsmith-Web:
|
||||
icon: https://cdn2.steamgriddb.com/icon/dfb61b74af460c2fd68bb8266f9f0814/32/256x256.png
|
||||
href: "https://slopsmith/"
|
||||
target: _self
|
||||
description: slopsmith in docker
|
||||
# - Slopsmith-Web:
|
||||
# icon: https://cdn2.steamgriddb.com/icon/dfb61b74af460c2fd68bb8266f9f0814/32/256x256.png
|
||||
# href: "https://slopsmith/"
|
||||
# target: _self
|
||||
# description: slopsmith in docker
|
||||
- guacamole:
|
||||
icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/apache-guacamole.svg
|
||||
href: "https://guac.wompmacho.com/"
|
||||
@@ -265,6 +265,11 @@
|
||||
description: neko browser
|
||||
- homelable:
|
||||
icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/homelable.svg
|
||||
href: "http://homelable/view?key=live"
|
||||
href: "https://homelable.wompmacho.com/view?key=live"
|
||||
target: _self
|
||||
description: homelable
|
||||
- farmOS:
|
||||
icon: https://cdn.fosstodon.org/accounts/avatars/109/321/909/625/316/786/original/763a2ed219103fd0.jpg
|
||||
href: "https://farm.wompmacho.com/"
|
||||
target: _self
|
||||
description: farmOS
|
||||
@@ -1,6 +1,6 @@
|
||||
# Open WebUI - https://docs.openwebui.com/getting-started/quick-start/
|
||||
services:
|
||||
openwebui:
|
||||
openai:
|
||||
container_name: openai
|
||||
image: ghcr.io/open-webui/open-webui:latest
|
||||
restart: unless-stopped
|
||||
|
||||
Reference in New Issue
Block a user