nubs 19a3fc59c9 fix(model-context): key context-window cache by (endpoint, model) (#2614) ...

get_context_length() cached the resolved context window by model id alone,
so two different remote endpoints serving the same model id (e.g. a capped
proxy at 8k vs. the full provider at 200k) collided: the first to resolve
won process-wide and the other endpoint was served the wrong window. That
silently over-trims conversations on the larger-window endpoint (it feeds
context_compactor) or overflows the smaller one (provider 400s).

Key the cache on (endpoint_url, model). Local endpoints already always
re-query, so they are unaffected.

Fixes #2603

2026-06-05 02:50:56 +02:00

1.8 KiB

Raw Permalink Blame History

View Raw