MrSphay/odysseus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Surface silent model fallback instead of masking it (#868)

Browse Source 
When the selected model fails before producing output, stream_llm_with_fallback
quietly switches to the next candidate and the reply is shown under the
originally selected model's name, so a misconfigured provider looks like it
works. (Concretely: a Bedrock gateway that 400s every Anthropic/Claude request
appears fine because another model silently answers under the Claude label.)

Emit a `fallback` SSE event ({selected_model, answered_by, reason}) the first
time a non-primary candidate produces output, forward it through the agent loop
and both chat-route paths, stamp the response metrics with the model that
actually answered, and show a notice + relabel the reply in the UI.

Tested: python -m pytest tests/test_llm_core_fallback.py (3 pass);
python -m py_compile src/llm_core.py src/agent_loop.py routes/chat_routes.py;
node --check static/js/chat.js.
This commit is contained in:
James Arslan
2026-06-02 04:37:25 +02:00
committed by GitHub
parent 2d6b777799
commit 6776c7d691
5 changed files with 135 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
static/js/chat.js
View File

@@ -1771,6 +1771,26 @@ import createResearchSynapse from './researchSynapse.js';
if (tsSpan) roleEl.appendChild(tsSpan);
}
}
} else if (json.type === 'fallback') {
// The selected model failed and another provider answered. Make
// it visible so a misconfigured provider is never silently
// masked under the selected model's name.
if (!_isBg) {
var _selM = _shortModel(json.selected_model || '');
var _ansM = _shortModel(json.answered_by || '');
uiModule.showToast('⚠ ' + _selM + ' failed — answered by ' + _ansM, 6000);
if (holder) {
var _rEl = holder.querySelector('.role');
if (_rEl) {
var _tsS = _rEl.querySelector('.role-timestamp');
_rEl.textContent = _ansM + ' (fallback) ';
_rEl.title = (json.selected_model || '') + ' failed' +
(json.reason ? ': ' + json.reason : '') + ' — answered by ' + (json.answered_by || '');
_applyModelColor(_rEl, json.answered_by);
if (_tsS) _rEl.appendChild(_tsS);
}
}
}
} else if (json.type === 'attachments') {
if (_isBg) continue;
// Update user bubble — replace file chips with image previews