Support extra CA bundle for private-CA LLM providers (#769)
Adding GigaChat (Sber) or an on-premise enterprise LLM gateway as a
model endpoint fails on first probe with
CERTIFICATE_VERIFY_FAILED: self-signed certificate in certificate
chain (_ssl.c:1000)
because their TLS chain is signed by a private root CA (Russian Trusted
Root CA for GigaChat; corporate CA for on-prem) that isn't part of the
default system / certifi trust store. The endpoint shows offline in
the picker even though the URL and API key are correct (issue #722).
The right fix is to extend the trust store, not to weaken verification.
This change:
- src/tls_overrides.py: new module that resolves an opt-in env var
LLM_CA_BUNDLE at import time, builds a shared SSLContext via
ssl.create_default_context() (so the system / certifi bundle is
loaded first) and layers the operator's PEM on top with
load_verify_locations(). Exposes llm_verify() returning a value
suitable for httpx `verify=`. Defaults to True (httpx built-in
trust) when the env var is unset, when the file is missing, or
when the PEM fails to load — verification is never silently
disabled, the warning is logged and we fall back to the safe path.
- src/llm_core.py: thread llm_verify() into the shared AsyncClient
used by stream_llm / streaming completions.
- routes/model_routes.py: thread llm_verify() into the five httpx.get
call sites in _probe_endpoint / _ping_endpoint so adding a
private-CA endpoint goes green on the very first probe and the
picker stops showing it offline.
- .env.example: document LLM_CA_BUNDLE with the GigaChat case as the
concrete example.
Deliberately NOT included: a verify=False knob (global or per-host).
Disabling verification exposes the affected endpoint to MITM, and the
operator-supplied bundle is the correct fix for legitimate private-CA
providers — so the only switch in this PR is the safe one.
Closes #722.
This commit is contained in:
10
.env.example
10
.env.example
@@ -27,6 +27,16 @@ LLM_HOST=localhost
|
||||
# Research service LLM endpoint
|
||||
# RESEARCH_LLM_ENDPOINT=http://localhost:8000/v1/chat/completions
|
||||
|
||||
# Extra CA bundle for LLM providers whose TLS chain isn't in the default
|
||||
# trust store. Layered ON TOP of the system / certifi bundle — verification
|
||||
# stays on for every host, the trust set just gets larger. Useful for:
|
||||
# - GigaChat / Sber (Russian Trusted Root CA): without this the endpoint
|
||||
# shows offline with CERTIFICATE_VERIFY_FAILED — self-signed certificate
|
||||
# in certificate chain.
|
||||
# - On-premise / corporate LLM gateways with an internal CA.
|
||||
# Point at a PEM file containing the missing root(s).
|
||||
# LLM_CA_BUNDLE=/etc/odysseus/ca/extra-roots.pem
|
||||
|
||||
# ============================================================
|
||||
# Search & Web
|
||||
# ============================================================
|
||||
|
||||
Reference in New Issue
Block a user