MrSphay/odysseus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docker: add NVIDIA/AMD GPU overlays via COMPOSE_FILE (#254)

Browse Source 
Opt-in overlays under docker/ that pass the host GPU into the odysseus
container. Pick one in .env:

  COMPOSE_FILE=docker-compose.yml:docker/gpu.nvidia.yml
  COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml

Non-GPU users are unaffected (no default merge). README now points at
the overlays instead of the old ad-hoc `gpus: all` suggestion.

Each overlay header notes that it only exposes the GPU devices — the
slim image still needs vLLM / llama-cpp-python / etc. installed via
Cookbook -> Dependencies before models can serve on GPU.

Tested on Arch + Docker 29.5.1 + RTX 4090:
  docker compose exec odysseus nvidia-smi -L
  GPU 0: NVIDIA GeForce RTX 4090 (UUID: GPU-...)
Cookbook hardware scan reports the 24 GB GPU and recommends GPU-fit
models. `docker compose config` validates cleanly for all three
COMPOSE_FILE variants (base, +nvidia, +amd).

Builds on the structure proposed in #91 by @krllus with the path /
docs fixes from the review on that PR.

Closes #163.

Co-authored-by: krllus <krllus@users.noreply.github.com>
This commit is contained in:
Sirsyorrz
2026-06-01 15:00:09 +10:00
committed by GitHub
parent 2537b80f88
commit 4c0aadbb5e
4 changed files with 78 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
.env.example
View File

@@ -123,3 +123,22 @@ SEARXNG_INSTANCE=http://localhost:8080
# Empty/local/localhost runs scripts on the app host. Set to an SSH host alias
# if you intentionally want scheduled scripts to run remotely.
# ODYSSEUS_SCRIPT_HOST=localhost
# ============================================================
# GPU support (Docker Compose)
# ============================================================
# Pass the host GPU into the odysseus container. Default (unset) = CPU.
# COMPOSE_FILE is a native `docker compose` feature: a colon-separated
# list of files merged left-to-right. Pick ONE GPU line below, or leave
# all commented for CPU.
#
# NVIDIA (requires nvidia-container-toolkit + `nvidia-ctk runtime
# configure --runtime=docker` on the host):
# COMPOSE_FILE=docker-compose.yml:docker/gpu.nvidia.yml
#
# AMD ROCm (requires ROCm drivers on the host):
# COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml
#
# These overlays only expose the GPU devices. The slim Odysseus image
# still needs CUDA/ROCm userspace via Cookbook -> Dependencies (vLLM,
# llama-cpp-python, etc.) before models can actually serve on GPU.