Allow serve-profile generation for models whose trained context window is below 8192 while preserving the 8K shrink floor for larger models.
5.2 KiB
5.2 KiB
Allow serve-profile generation for models whose trained context window is below 8192 while preserving the 8K shrink floor for larger models.