Building block: shared storage¶
Contract¶
Files that more than one instance must see: plugin attachments (ITSM), user background images,
uploaded themes, the plugin zip cache. Today these are written to the local file system under
STORAGE_DIR (/data/storage) and the image layer (themes).
Options¶
| Option | HA of the layer | Notes | When |
|---|---|---|---|
| Local disk per instance | none | files written on A are not on B — acceptable only while no plugin stores user files | Tier 0, dev |
Shared mount (NFS / CIFS / Kubernetes RWX volume) for /data/storage |
that of the NAS/volume | works today without core changes; the dev test bench mounts the same host directory into both instances | Tier 1 now |
Object store (MinIO, Garage, SeaweedFS, S3) via LYNDRIX_STORAGE_DRIVER=s3 |
yes | planned (ctx.storage abstraction with local/fs/s3 drivers) |
Tier 2 later |
Lyndrix settings¶
STORAGE_DIR, SECURITY_DIR (the key file must be on every instance; a shared secret, not shared
storage). Planned: LYNDRIX_STORAGE_DRIVER, LYNDRIX_STORAGE_S3_*.
Verify¶
The preflight storage check does it for you: every instance writes a marker file
STORAGE_DIR/.cluster/<instance_id> on each heartbeat that names the process (its incarnation)
which wrote it. An instance that can open a peer's marker by name and reads the incarnation
that peer's registry row carries has proof that both look at one directory → ok. The check does
not list the directory and does not look at file times (both are unreliable on NFS: listing and
attribute caches, server clock vs. reader clock).
- Only instances with a role in
LYNDRIX_STORAGE_SHARED_ROLES(defaultapi— they serve user files) have to share. A runner-only instance on another host is mentioned in theokmessage ("not shared with … — fine") and reportsokitself. - An instance with such a role whose marker is not visible →
warn: "/data/storage is NOT shared with docker-auto (api)". That is a real finding, not a glitch: uploads, attachments and uploaded themes written through one of the two do not exist on the other, so a user who is routed to both sees files come and go. Mount the same volume there, or keep that instance out of the user-facing load balancer. - A peer that registered less than two heartbeats ago is not judged yet; a
warnis re-checked two heartbeats later instead of staying for five minutes. With no second file-serving instance alive the check cannot decide and stayswarn. - The mount is only ever touched by one thread of the cluster layer; if it hangs, the check says "… is not responding — a storage job has been stuck for N s".
- The leader removes marker files of instances that have been gone for more than a day.
Then upload a theme or an ITSM attachment through the load balancer and fetch it again a few times — every instance must serve it.
Known limits¶
- No object-store driver yet (
LYNDRIX_STORAGE_DRIVER=s3with MinIO/Garage/S3 is the roadmap item); today "shared" means one mounted directory. - The core writes
/app/logs/lyndrix.logon every instance — do not share that directory; rely on stdout/docker logsinstead. - NiceGUI's per-user storage (
/app/.nicegui) is process-bound; irrelevant withreact.