get_context_length() cached the resolved context window by model id alone,
so two different remote endpoints serving the same model id (e.g. a capped
proxy at 8k vs. the full provider at 200k) collided: the first to resolve
won process-wide and the other endpoint was served the wrong window. That
silently over-trims conversations on the larger-window endpoint (it feeds
context_compactor) or overflows the smaller one (provider 400s).
Key the cache on (endpoint_url, model). Local endpoints already always
re-query, so they are unaffected.
Fixes#2603
The CalDAV pull prunes events in the synced calendar+window whose UID the
server didn't just return, to propagate upstream deletions. But CalendarEvent
had no field distinguishing a server-pulled row from a locally-created one, so
the prune also deleted events that were never on the server: events created by
the agent / email triage (which never write back to the server) and UI events
whose best-effort write-back failed. Result: silent, unrecoverable loss of the
user's appointments (hard db.delete, no soft-delete).
Add an 'origin' column to calendar_events (lightweight idempotent migration,
mirroring _migrate_add_calendar_is_utc), set origin='caldav' on rows the sync
inserts/updates, and gate the prune on origin == 'caldav'. Locally-created
events carry origin NULL and are never pruned. On the first sync after the
migration nothing is pruned (all rows NULL until re-marked), erring toward
keeping data.
Fixes#2704
* feat(mcp): add Streamable HTTP transport with OAuth 2.0
Odysseus could only reach MCP servers over stdio and SSE, so modern
remote servers like https://mcp.higgsfield.ai/mcp (Streamable HTTP,
gated behind OAuth) could not be connected.
Add an `http` transport that connects via the SDK's
streamablehttp_client and authenticates with the SDK's
OAuthClientProvider: RFC 9728 protected-resource discovery, RFC 8414
authorization-server metadata, Dynamic Client Registration,
authorization-code + PKCE, and token refresh. A small bridge
(src/mcp_oauth.py) connects the SDK's blocking callback to the existing
web callback route via an asyncio.Future keyed by the OAuth `state`,
and the dynamic client registration plus tokens persist per-server in a
new encrypted `oauth_tokens` column.
The connect runs as a bounded background task so the "Add server"
request returns immediately; redirect_handler publishes needs_auth +
auth_url to connection state as soon as discovery/DCR completes (which
can exceed the bounded wait), and the UI polls until connected. Remote
users finish via the existing paste-back flow. The Google OAuth path is
left unchanged.
- core/database.py: encrypted oauth_tokens column + migration
- src/mcp_oauth.py: OAuth provider, DB-backed TokenStorage, state registry
- src/mcp_manager.py: http dispatch, background connect, _connect_http
- routes/mcp_routes.py: http validation, needs_auth/auth_url, callback bridge
- static/js/settings.js: Streamable HTTP option + OAuth flow with polling
- tests: 5 new unit tests (transport dispatch, registry, token storage)
Verified against the live Higgsfield server: discovery, DCR (client_id
issued), loopback redirect accepted, and a PKCE authorization URL with
needs_auth status. No regressions (full suite delta is only the 5 added
passing tests).
* fix(mcp): address PR #1033 review feedback
- mcp_oauth: derive redirect URI from OAUTH_REDIRECT_BASE_URL/APP_PUBLIC_URL
(default http://localhost:7000) instead of hardcoding the port
- mcp_oauth: leave OAuth scope unset so the SDK derives it from the server's
WWW-Authenticate/protected-resource metadata; hardcoding an OIDC scope broke
non-OpenID MCP servers (verified: Higgsfield still gets its server-derived
scope)
- mcp_oauth: prune abandoned OAuth flows (_prune_stale + _pending_ts) so the
module-level registries can't grow unbounded
- mcp_oauth: persist tokens/client-info in a single DB session/commit
(_update) instead of a load+save double round-trip
- mcp_manager: cancel and drop the background connect task in
disconnect_server so a deleted server stops publishing status
- database: document why the oauth_tokens migration uses TEXT while the model
declares EncryptedText (encryption is applied at the Python layer)
- settings.js: surface persistent OAuth-poll failures and an explicit timeout
message instead of silently swallowing errors
- tests: cover the stale-flow pruning
* static/js/settings.js now shows an in-flight loading state on the buttons that fire requests:
Chat models often emit GitHub/Slack-style :shortcode: text (e.g. 😊,
🎤) instead of the actual emoji. The renderer only converted real
Unicode emoji to the monochrome line icons, so shortcodes rendered as literal
text.
Add a pure, browser-free shortcode->Unicode map (emojiShortcodes.js) and run it
inside svgifyEmoji ahead of the existing Unicode->SVG pass, skipping <code>/<pre>
so code stays literal. Covers ~430 common shortcodes plus common aliases
(+1/thumbsup, etc.).
Keep the conversion from touching anything it shouldn't:
* Scope it to chat. mdToHtml/svgifyEmoji take a { shortcodes } option (default
on); document and email body rendering (compose, export, preview) pass it as
false so author-typed :shortcode: text stays literal. The Unicode->SVG pass
still runs there exactly as before.
* Only convert a :shortcode: that stands on its own. A word-boundary guard
leaves embedded colon runs alone, so "1:100:2", "10:30:45", "16:9" and
host:fire:port are never rewritten.
Tests: extend the node-driven unit test with the boundary/false-positive cases,
and fix the markdown-rendering test loader to resolve the new emojiShortcodes
import.
Makes the cookbook venv fallback-chain test deterministic by simulating the inside-venv shell state directly instead of depending on the GitHub runner Python environment. Final focused #2580 CI-baseline cleanup.
Calls execute_tool_block through the live src.tool_execution module in the edit-file admin-gate test so the monkeypatched _owner_is_admin seam and the called function belong to the same module object. Fixes the scoped #2580 CI-order edit-file failure. Remaining Python failure is the unrelated cookbook fallback-chain environment test.
* feat: Add workspace: confine agent tools to a folder
Pick a server folder as the agent's workspace so its file/shell tools work
there and don't touch files outside it. File tools are hard-confined; bash/
python run with cwd set to the folder.
Includes a slash command: `/workspace` (alias `/ws`) — show / `set <path>` /
`clear` / `pick` (open the directory browser).
- routes/workspace_routes.py: GET /api/workspace/browse (admin-only).
- src/tool_execution.py: hard path confinement for read_file/write_file;
bash/python cwd. Threaded route → stream_agent_loop → execute_tool_block.
- src/agent_loop.py: workspace note prepended to the system prompt.
- static/: overflow menu item, input-bar pill, directory-browser modal, and
the /workspace slash command.
- tests/test_workspace_confine.py.
* Wire workspace confinement into tools that landed after this PR
edit_file (#1239) and grep/glob/ls (#1670) merged after workspace-confine was
written, so they bypassed the workspace boundary. Thread the workspace through:
- edit_file: _do_edit_file resolves via _resolve_tool_path_in_workspace
- grep/glob/ls: _resolve_search_root confines to the workspace (root + paths)
- bash/python/bg cwd: workspace or _AGENT_WORKDIR (keep the #2586 data-dir
default when no workspace is set)
Tests cover edit_file + grep/ls confinement (inside ok, outside rejected).
* Workspace picker: editable path bar + modal style cohesion + cross-platform hardening
- Make the current-folder strip an editable address bar: type/paste a full
path and press Enter to navigate (also reaches other Windows drives and
hidden dirs the up-only browser cannot).
- Reuse shared modal CSS: drop bespoke .workspace-modal-content/.workspace-btn*
in favour of base .modal-content/.modal-body and the .confirm-btn button
family; separators/hover use var(--border). Net -31 CSS lines.
- Fix the path field overflowing the modal right edge (flex stretch + margin
vs an overflow:auto scrollbar-feedback loop): full-bleed, no h-margin.
- Cross-platform confinement: normcase the workspace commonpath check so
containment holds on case-insensitive filesystems (Windows/macOS).
- Make tests OS-portable: sibling temp dirs instead of /etc, python os.getcwd()
instead of pwd. 5 pass.
read_file/grep/glob/ls are in ALWAYS_AVAILABLE but the on-disk write tools
(write_file, edit_file) were only surfaced via per-query tool-RAG retrieval.
On a bare 'edit X' request the retriever could miss them, so the model was
never offered edit_file/write_file and wrongly fell back to edit_document
(editor panel) or improvised with bash sed. Add both to ALWAYS_AVAILABLE
next to read_file; they stay admin-gated by tool_security so non-admin
exposure is unchanged.
Fixes#2683
Restores src.webhook_manager after a review-regression test imports it against a fake src.database. Fixes one focused #2580 CI-baseline pollution bucket.
* feat: round-limit handling — Continue affordance at the cap + configurable cap
When the agent loop runs out of rounds (per-message step cap, default 20)
while still actively using tools, it stopped silently mid-task. Now:
1. The loop emits a `rounds_exhausted` SSE event at the cap, and the UI shows
a "Continue" pill at the bottom of the chat that resumes the task from where
it left off. Repeated cap-hits each get a fresh Continue (multiple continues
in a row).
2. The cap is configurable in Settings → Agent ("Max steps per message"),
validated on the client, at the save endpoint, and at the read site.
- src/agent_loop.py: track `_exhausted_rounds` (set only when a full
tool-executing round completes on the last allowed round — i.e. the agent
wanted to keep going); emit `{"type":"rounds_exhausted","rounds":N}` (logged).
- routes/chat_routes.py: read `agent_max_rounds` (clamped 1..200), pass as
`max_rounds`; forward the new event through the SSE relay.
- routes/auth_routes.py: validate numeric settings on save (int + clamp;
agent_max_rounds 1..200, agent_max_tool_calls 0..1000; 400 on non-int).
- src/settings.py: default `agent_max_rounds = 20`.
- static/: Settings input + client-side clamp; the Continue pill (reuses the
existing .stopped-indicator / .continue-btn classes and theme vars
--border/--fg/--bg/--accent); appended to the chat container so it survives
the message re-render at stream finalize. chat.js cache version bumped.
* test: cover rounds_exhausted emission (cap-hit vs normal finish)
Drives the real stream_agent_loop with mocked LLM stream / tool exec / settings:
a tool block every round exhausts the cap and must emit rounds_exhausted; a
plain answer hits the done-break and must not. Guards the for/else logic.
Restores both sys.modules and parent src.database package state after the webhook SSRF tests import src.webhook_manager against the real database module. Fixes one focused #2580 CI-baseline pollution bucket.
Keeps src.request_models real and restores both sys.modules and parent routes.session_routes package attributes after temporary test stubs. Restores one focused part of the Python CI baseline tracked in #2580.
Slash-command replies and the echoed /setup command are persisted to session
history so they render in the transcript, but they are UI chatter the user
never meant as conversation. They were sent to the model on the next turn,
which then commented on '/setup ...' and exposed transient values (e.g. the
Copilot device user_code) to the LLM.
- get_context_messages() (the LLM-API view) now skips messages tagged
metadata.source == 'slash'. Display/history-load paths use raw history and
are unaffected.
- slashCommands.js tags the echoed user command with source:'slash' too (the
assistant replies already carried it); the user line was the one untagged
path that still reached context.
Fixes#2634.
truncate_messages deletes db_messages[keep_count:] (a no-op when
keep_count >= the real message total) then unconditionally wrote
db_session.message_count = keep_count. When keep_count exceeds the
number of messages that actually exist — e.g. the manage_session AI
tool defaults keep_count to 10, and the HTTP truncate endpoint passes
any client value — the persisted count is set too high (10 on a
3-message session), diverging from the real row count. That column
gates lazy DB-hydration in get_session (message_count > 0) and is
surfaced to the history UI, so it is correctness-relevant. Clamp to
min(keep_count, len(db_messages)); the in-memory slice already caps
naturally.
Default image installs requirements.txt only. Set INSTALL_OPTIONAL=true
at build time to add requirements-optional.txt (PyMuPDF, markitdown, etc.)
without baking AGPL into the standard distributed image.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(provider): add GitHub Copilot provider with device-flow auth
Adds GitHub Copilot as a model provider, so Copilot models (gpt-4o/4.1/5,
Claude, Gemini, …) work through the normal chat + agent loop, incl. native
tool calling and vision.
Auth is one-click via the GitHub OAuth device flow; the access token is stored
as the endpoint's (encrypted) api_key and sent directly as `Authorization:
Bearer` (no Copilot-token exchange, no refresh — matching how editors talk to
the Copilot API). Copilot is a normal ModelEndpoint detected by host; the only
provider-specific behaviour is a small set of required request headers,
injected centrally.
Sign-in is available from Settings → model endpoints ("Connect GitHub
Copilot") and from chat via `/setup copilot`.
- src/copilot.py (new), routes/copilot_routes.py (new): constants, header
builders, device-flow start/poll, model discovery, owner-scoped endpoint
provisioning.
- src/llm_core.py, src/endpoint_resolver.py: detect `copilot`, inject headers,
per-request x-initiator/vision.
- src/agent_loop.py: allowlist api.githubcopilot.com for native tool schemas.
- src/model_context.py: known context windows for Copilot (no unauthenticated
/models probe).
- static/, README, tests/test_copilot*.py.
* Tidy copilot_routes: clarify supports_tools, note _PENDING is per-process
* fix: renaming a user leaves their API tokens resolving to the old owner
* Drive rename token-cache test through the real auth resolver instead of patching a closure
* fix(llm): auto-detect <think> in content stream for unregistered thinking models
_THINKING_MODEL_PATTERNS only covers known model families by name. Qwen3-derived
models with non-standard names (e.g. Qwopus, custom QwQ forks) are not matched,
so their <think>...</think> content streams through as visible chat text instead
of being routed to the thinking display.
When the first content delta opens with <think> and the model was not already
identified as a thinking model, dynamically flag the stream as a thinking model
for the remainder of the response. This enables the existing </think> repair path
(line below) and ensures the frontend receives the full <think>...</think> wrapper
it needs to split thinking from the final answer.
The check is restricted to the very first content delta (_first_content_sent is
False) to avoid misidentifying models that happen to write "<think>" mid-answer.
Fixes#2225
Related: #2420 (covered by separate PR from @AmmarS-Analyst), #2224 (@RaresKeY)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(llm): replace inert _thinking_model flag with _in_think_tag state machine
The original auto-detect set _thinking_model=True on the first <think> chunk
but still emitted it as a regular delta and set _first_content_sent=True
immediately, so no subsequent chunk could enter the repair path.
Replace with _in_think_tag bool: enter thinking mode when first content starts
with <think>, route all chunks to the thinking channel until </think> is found,
then the tail becomes the first regular delta. Adds three regression tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(llm): replace _first_content_sent guard with _think_open_stripped
Opening-tag stripping used `not _first_content_sent` as the guard, but
_first_content_sent stays False throughout the entire think block (it only
flips when regular content is emitted). So `find(">")` ran on every
reasoning chunk — not just the first — and silently truncated everything
before the first ">" in any reasoning text containing comparisons, arrows,
or code.
Fix: add `_think_open_stripped = False` alongside `_in_think_tag`. Use it
as the strip guard in both the "still inside <think>" path and the
"</think> found in same chunk" split path. Set it True once the opening
tag is consumed so all subsequent chunks reach the thinking channel
unmolested.
Add regression test: 3-chunk stream where the middle chunk contains
"c > d" — confirms "more c " is not dropped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes module-level core.database stubbing from the compare endpoint owner-scope regression test and patches ModelEndpoint per test with monkeypatch. Restores one focused part of the Python CI baseline tracked in #2580.
Agent subprocesses (bash, python) previously inherited the container's default
working directory (/app), so files created with relative paths landed in the
ephemeral container layer and were silently destroyed on any docker compose up
--build or container recreation.
Set cwd=_AGENT_WORKDIR (resolved to <repo_root>/data at import time) and
HOME=_AGENT_WORKDIR on both subprocess launchers so that:
- pwd inside a bash tool returns the persistent data directory
- relative paths and ~ resolve to a location that survives rebuilds
- the agent can still cd to any absolute path it needs
The resolution uses pathlib.Path(__file__).parent.parent / "data", which
works for both Docker (/app/src → /app/data) and manual installs
(<repo>/src → <repo>/data) without requiring a new env var or compose change.
Fixes#2512
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Odysseus only supports llama.cpp on Windows (vLLM/SGLang are
explicitly blocked). llama.cpp requires GGUF, so AWQ/GPTQ/FP8
safetensors models without a GGUF alternate should not be
recommended in the Cookbook on Windows hosts.
Changes:
- hardware.py: add 'platform': 'windows' to _detect_windows()
so downstream logic can identify Windows hosts.
- fit.py: include is_windows in the existing GGUF-only filter
alongside apple_silicon and consumer_amd.
- tests: add test_hwfit_windows.py with regression tests.
Fixes#122, #614 (root cause: unservable models recommended).
Python's bool('false') returns True because the string is non-empty.
A JS client serialising a boolean as the string 'false' would have
supports_tools or is_enabled silently flipped to True — so 'disable
tool support' would actually enable it.
Use an explicit lookup dict for supports_tools and a case-insensitive
string check for is_enabled so both string and native bool inputs are
handled correctly.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Updates the stale gallery owner-filter null-user test to match current single-user/auth-disabled behavior. Restores one focused part of the Python CI baseline tracked in #2580.
A system message that arrives without a 'content' key — possible via
malformed tool results — raised a KeyError in the hot path of llm_call,
llm_call_async, and stream_llm. Replace m["content"] with
m.get("content") or "" in all three functions so a missing key degrades
to an empty string instead of crashing.
Also removes a redundant .rstrip() after .strip() in _model_activity_key.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The retry loop raised immediately for any non-success HTTP response
regardless of attempt count. For transient upstream errors (rate limit,
bad gateway, gateway timeout) the function should back off and retry
within the existing attempt budget.
Also lets ConnectError / ConnectTimeout retry when the host has not been
cooled and attempts remain, instead of always raising on the first
connect failure.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Updates the PDF marker regression test to check corrupted markers at line level instead of using a broad substring assertion. Restores one focused part of the Python CI baseline tracked in #2580.
Two bugs prevented pinned models from appearing in the chat model picker:
1. _fetch_models() only used _cached_model_ids(), ignoring pinned_models.
Since Fireworks AI doesn't list kimi-k2p6-turbo in /v1/models, the
cached list was empty, so the endpoint showed as offline with no models.
2. _curate_models() filtered unknown pinned IDs into models_extra, but the
chat UI only reads models (primary list). Pinned models stayed invisible.
Fix: use _visible_models() to merge cached + pinned, then promote pinned
IDs from models_extra to models so they appear in the dropdown.
Closes#1521 follow-up
Updates the split_chunks containment regression test to use deterministic non-repeating records instead of a repeating fixture that could produce accidental substring matches. Restores one focused part of the Python CI baseline tracked in #2580.
Updates endpoint/model-route test HTTP mocks to accept the verify keyword argument passed by endpoint probing code. Restores one focused part of the Python CI baseline tracked in #2580.
