pewdiepie-archdaemon 4074e77d93 Cookbook: auto-set KV cache to fp8 for DeepSeek V3/V4/R1 MoE families ...

These models OOM on --kv-cache-dtype auto (≈bf16) at any usable
context with current tensor-parallel layouts. _detectModelOptimizations
now seeds opts.kvCacheDtype='fp8' for them, and the serve panel's KV
Cache select picks that up as the default unless the user has a
saved override on this skill.

2026-06-14 08:57:29 +09:00

129 KiB

Raw Blame History

View Raw