Cookbook scheduler: calendar events drive model serve windows (experimental, feature-flagged)

Add a calendar-driven scheduler so a user can pick a model in Cookbook, click "Schedule…" instead of "Launch", choose time windows + days of the week + (optional) end date, and have Odysseus auto-launch the serve when the window starts and hard-kill it when the window ends. The calendar IS the source of truth — events on a designated calendar are interpreted as serve schedules, so editing the event in the calendar UI immediately changes the schedule.

Whole feature is gated by setting `cookbook_scheduler_enabled` (default False). Disabling the setting silences the reconciler and the API refuses requests; setting + three new files = entire surface, easy to revert.

New files:
  - src/cookbook_scheduler.py — background reconciler: ticks every 60s, reads next ±90s of calendar events on the designated calendar, launches/kills serves to match. Honors "refuse if GPUs busy" (skips with reason, no retry). Adopts pre-existing manual serves matching the event's model so window-end cleanup still applies. Tags scheduler-owned tasks with `_scheduledBy: <event_uid>` so it never kills serves it doesn't own.
  - routes/cookbook_schedule_routes.py — POST /api/cookbook/schedule/from-cookbook builds RRULE+ICS events from the modal's input (model, slots[], days[], until). GET /upcoming returns the next 24h with per-event status (scheduled / running / adopted / skipped / failed / ended) for the UI. POST /reconcile-now manually kicks the reconciler.
  - static/js/cookbookSchedule.js — Schedule button click handler + modal. Daily/hourly time slot picker, multi-slot ("+ add another time slot"), weekday chips with Weekdays/Weekend/Every-day quicksets, optional Until date. Calls /from-cookbook on save. Whole module is a single IIFE; deleting the file plus its <script> tag removes the UI surface.

Existing files touched (minimal):
  - app.py: register the new router + add the reconcile loop as a startup task (~10 lines, all in one block). Reconcile loop checks the feature flag on every tick, so leaving it running with the flag off costs ~one settings lookup per minute.
  - static/index.html: one new <script> tag for cookbookSchedule.js.
  - static/js/cookbookServe.js: add a "Schedule…" button next to the existing Launch button. Hidden by default; cookbookSchedule.js reveals it after confirming the feature flag is on.
  - static/style.css: ~80 lines for the modal styles (mobile-aware via @media).

User choices baked in:
  - Calendar events are the source of truth.
  - Refuse to launch if GPUs busy (skip + log reason in scheduler.events[uid].reason).
  - Hard kill at event end.
  - No retry on a skipped event within the window.
  - Multi-slot per day supported (one calendar event per slot, shared RRULE).
  - Pre-existing manual serves get adopted at window start so they're killed at end.

Known follow-ups (not in this commit):
  - Settings UI to pick the schedule calendar + toggle the feature flag.
  - Calendar event color/badge for status (running/skipped/failed).
  - "Lazy launch on first request" — currently launches at event start. Replacing _launch_serve with a proxy that defers vllm until the first chat request is a contained future change.

app.py

@@ -640,6 +640,14 @@ app.include_router(setup_shell_routes())
 from routes.cookbook_routes import setup_cookbook_routes
 app.include_router(setup_cookbook_routes())
 
+# Cookbook scheduler — calendar-driven serve windows.
+# Feature-flagged on the `cookbook_scheduler_enabled` setting (default
+# off); disabling the setting silences the reconciler and the API
+# refuses requests. Delete this block + src/cookbook_scheduler.py +
+# routes/cookbook_schedule_routes.py to remove the feature entirely.
+from routes.cookbook_schedule_routes import setup_cookbook_schedule_routes
+app.include_router(setup_cookbook_schedule_routes())
+
 # Hardware model fitting (cookbook "What Fits?" tab)
 from routes.hwfit_routes import setup_hwfit_routes
 app.include_router(setup_hwfit_routes())
@@ -1061,6 +1069,14 @@ async def _startup_event():
                 logger.warning(f"Nightly skill audit failed: {e}")
 
     _startup_tasks.append(asyncio.create_task(_skill_audit_nightly_loop()))
+
+    # Cookbook scheduler reconcile loop. Internally checks the
+    # cookbook_scheduler_enabled setting on every tick, so leaving this
+    # task running with the feature disabled costs ~one settings lookup
+    # per minute. Remove this line to dispose of the feature.
+    from src.cookbook_scheduler import reconcile_loop as _cookbook_reconcile_loop
+    _startup_tasks.append(asyncio.create_task(_cookbook_reconcile_loop()))
+
     logger.info("Application startup complete")
 
 async def _shutdown_event():

routes/cookbook_schedule_routes.py

@@ -0,0 +1,208 @@
+"""Cookbook schedule routes — turns the Cookbook \"Schedule\" modal into
+calendar events on the designated schedule calendar, and exposes a
+diagnostic /upcoming endpoint for the UI.
+
+All routes live under /api/cookbook/schedule/* so the whole file can be
+removed by deleting one router-registration line in app.py. The setup
+function is a no-op when `cookbook_scheduler_enabled` is False.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+from datetime import datetime, timedelta, timezone
+from typing import Any, Dict, List, Optional
+
+from fastapi import APIRouter, Body, HTTPException, Request
+
+from core.middleware import require_admin
+
+logger = logging.getLogger(__name__)
+
+
+_DAYS = {"MO", "TU", "WE", "TH", "FR", "SA", "SU"}
+_HHMM_RE = re.compile(r"^([01]?\d|2[0-3]):([0-5]\d)$")
+
+
+def setup_cookbook_schedule_routes() -> APIRouter:
+    router = APIRouter(prefix="/api/cookbook/schedule", tags=["cookbook-schedule"])
+
+    @router.get("/upcoming")
+    async def upcoming(request: Request, hours: int = 24):
+        """Next N hours of scheduled events with reconciler status.
+
+        Drives the "what's running, what's queued" badges in the
+        Cookbook UI. Cheap read — no SSH, just calendar + state file.
+        """
+        require_admin(request)
+        from src.settings import get_setting
+        if not get_setting("cookbook_scheduler_enabled", False):
+            return {"enabled": False, "events": []}
+        calendar_href = get_setting("cookbook_schedule_calendar_href", "") or ""
+        if not calendar_href:
+            return {"enabled": True, "calendar_href": "", "events": []}
+
+        hours = max(1, min(int(hours or 24), 24 * 14))
+        now = datetime.now(timezone.utc)
+        end = now + timedelta(hours=hours)
+        from src.cookbook_scheduler import _fetch_calendar_events, _read_state
+        events = await _fetch_calendar_events(calendar_href, now - timedelta(minutes=5), end)
+        state = _read_state()
+        tracked = (state.get("scheduler") or {}).get("events") or {}
+        out: List[Dict[str, Any]] = []
+        for ev in events:
+            uid = ev.get("uid") or ev.get("id") or ""
+            if not uid:
+                continue
+            t = tracked.get(uid) or {}
+            out.append({
+                "uid": uid,
+                "title": ev.get("summary") or "",
+                "start": ev.get("dtstart") or ev.get("start"),
+                "end": ev.get("dtend") or ev.get("end"),
+                "status": t.get("status") or "scheduled",
+                "reason": t.get("reason") or "",
+                "session_id": t.get("session_id") or "",
+            })
+        return {"enabled": True, "calendar_href": calendar_href, "events": out}
+
+    @router.post("/from-cookbook")
+    async def schedule_from_cookbook(request: Request, body: Dict[str, Any] = Body(default_factory=dict)):
+        """Create one or more calendar events from the Cookbook Schedule modal.
+
+        Body shape:
+          {
+            "model": "Qwen3.5-397B-A17B-AWQ",     # display title
+            "preset": "Qwen3.5-397B-A17B-AWQ",    # optional, matched to saved preset
+            "repo_id": "...",                     # optional, for non-preset launches
+            "cmd": "vllm serve ...",              # optional
+            "host": "pewds@192.168.1.12",         # optional
+            "port": 8003,                         # optional
+            "slots": [
+              {"start": "09:00", "end": "17:00"}, # one or more time windows per day
+              {"start": "21:00", "end": "23:30"}
+            ],
+            "days": ["MO","TU","WE","TH","FR"],   # weekdays this repeats
+            "until": "2026-12-31",                # optional end date, else forever
+            "start_date": "2026-06-05"            # optional first day, else today
+          }
+
+        Creates one calendar event per slot (so split-shift schedules
+        are visible as separate blocks). All events share the same
+        RRULE so they can be edited together by changing one.
+        """
+        require_admin(request)
+        from src.settings import get_setting
+        if not get_setting("cookbook_scheduler_enabled", False):
+            raise HTTPException(400, "Cookbook scheduler is not enabled in Settings.")
+        calendar_href = get_setting("cookbook_schedule_calendar_href", "") or ""
+        if not calendar_href:
+            raise HTTPException(400, "No Cookbook schedule calendar is configured in Settings.")
+
+        title = (body.get("model") or body.get("title") or "").strip()
+        if not title:
+            raise HTTPException(400, "model (title) is required")
+        slots = body.get("slots") or []
+        if not isinstance(slots, list) or not slots:
+            raise HTTPException(400, "at least one time slot is required")
+        for s in slots:
+            if not isinstance(s, dict):
+                raise HTTPException(400, "slot must be an object")
+            if not _HHMM_RE.match(str(s.get("start") or "")):
+                raise HTTPException(400, f"slot.start must be HH:MM, got {s.get('start')!r}")
+            if not _HHMM_RE.match(str(s.get("end") or "")):
+                raise HTTPException(400, f"slot.end must be HH:MM, got {s.get('end')!r}")
+        days = [d for d in (body.get("days") or []) if d in _DAYS]
+        if not days:
+            # Default to every day if the user didn't pick.
+            days = list(_DAYS)
+
+        # Compose the cookbook: YAML block dropped into event DESCRIPTION
+        # so the reconciler knows how to launch.
+        yaml_lines = ["cookbook:"]
+        for k in ("preset", "repo_id", "cmd", "host", "port"):
+            v = body.get(k)
+            if v:
+                yaml_lines.append(f"  {k}: {v}")
+        if len(yaml_lines) == 1:
+            # Fall back: the title alone is the preset name. Reconciler
+            # will preset-match against saved presets at launch time.
+            yaml_lines.append(f"  preset: {title}")
+        description = "\n".join(yaml_lines)
+
+        # First-occurrence date defaults to today (UTC) so the schedule
+        # applies starting now. RRULE-BYDAY handles day filtering.
+        start_date = body.get("start_date")
+        if start_date:
+            try:
+                d0 = datetime.strptime(start_date, "%Y-%m-%d").replace(tzinfo=timezone.utc)
+            except ValueError:
+                raise HTTPException(400, "start_date must be YYYY-MM-DD")
+        else:
+            d0 = datetime.now(timezone.utc).replace(hour=0, minute=0, second=0, microsecond=0)
+        until = (body.get("until") or "").strip()
+        until_clause = ""
+        if until:
+            try:
+                u = datetime.strptime(until, "%Y-%m-%d")
+                until_clause = f";UNTIL={u.strftime('%Y%m%dT235959Z')}"
+            except ValueError:
+                raise HTTPException(400, "until must be YYYY-MM-DD")
+
+        rrule = f"FREQ=WEEKLY;BYDAY={','.join(days)}{until_clause}"
+
+        # Create one event per slot. Call /api/calendar/events directly
+        # so we don't reinvent CalDAV plumbing.
+        import httpx
+        from core.middleware import INTERNAL_TOOL_HEADER, INTERNAL_TOOL_TOKEN
+        headers = {INTERNAL_TOOL_HEADER: INTERNAL_TOOL_TOKEN}
+        created: List[str] = []
+        for slot in slots:
+            sh, sm = [int(x) for x in str(slot["start"]).split(":")]
+            eh, em = [int(x) for x in str(slot["end"]).split(":")]
+            dtstart = d0.replace(hour=sh, minute=sm)
+            dtend = d0.replace(hour=eh, minute=em)
+            if dtend <= dtstart:
+                # Overnight: schedule the end on the next day.
+                dtend = dtend + timedelta(days=1)
+            ev_body = {
+                "summary": title,
+                "dtstart": dtstart.isoformat(),
+                "dtend": dtend.isoformat(),
+                "all_day": False,
+                "description": description,
+                "calendar_href": calendar_href,
+                "rrule": rrule,
+                "color": "#3b82f6",
+            }
+            try:
+                async with httpx.AsyncClient(timeout=15) as client:
+                    r = await client.post(
+                        "http://localhost:7000/api/calendar/events",
+                        json=ev_body, headers=headers,
+                    )
+                    if r.status_code >= 400:
+                        logger.warning(f"schedule: calendar event create failed: {r.status_code} {r.text[:200]}")
+                        continue
+                    data = r.json()
+            except Exception as e:
+                logger.warning(f"schedule: calendar event create errored: {e}")
+                continue
+            uid = data.get("uid") or data.get("id") or ""
+            if uid:
+                created.append(uid)
+        if not created:
+            raise HTTPException(500, "Failed to create any calendar events for this schedule")
+        return {"ok": True, "created": created, "slots": len(slots), "rrule": rrule}
+
+    @router.post("/reconcile-now")
+    async def reconcile_now(request: Request):
+        """Manual kick of the reconciler. Useful for testing + the
+        \"Run now\" button in the Cookbook UI."""
+        require_admin(request)
+        from src.cookbook_scheduler import _reconcile_once
+        return await _reconcile_once()
+
+    return router

src/cookbook_scheduler.py

@@ -0,0 +1,456 @@
+"""Cookbook scheduler — calendar-driven model launches.
+
+Calendar events on a designated calendar (configurable via setting
+`cookbook_schedule_calendar_href`) are interpreted as serve schedules.
+The reconciler ticks every ~60s, reads events whose window contains
+"now", and reconciles the running serves against them:
+
+  - Event starts in window AND no matching serve running → launch via
+    existing /api/model/serve. If GPU is busy, mark event "skipped"
+    with reason. No retry.
+  - Event ends in window AND a scheduled serve is running → hard-kill.
+  - Pre-existing manual serve matching the event's model → adopt it
+    (mark as owned by the event so it gets stopped at window end).
+
+Everything in this module is gated by setting `cookbook_scheduler_enabled`.
+Setting that to False fully disables the feature without touching code.
+
+Event description format (YAML-ish, single nested key):
+  cookbook:
+    preset: Qwen3.5-397B-A17B-AWQ            # or repo_id + cmd + host
+    repo_id: deepseek-ai/DeepSeek-V4-Flash
+    cmd: vllm serve /mnt/HADES/models/...
+    host: pewds@192.168.1.12
+    port: 8003
+
+If only the title is given, the title is matched against saved preset
+names (case-insensitive substring match).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import re
+import time
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+import httpx
+
+logger = logging.getLogger(__name__)
+
+
+# Schedule-owned tasks are tagged with this so we can tell them apart
+# from manual launches when deciding whether to hard-kill at window end.
+SCHEDULE_OWNER_KEY = "_scheduledBy"
+COOKBOOK_BASE_URL = "http://localhost:7000"
+
+
+def _internal_headers() -> Dict[str, str]:
+    """Match the in-process loopback auth path used by chat-agent tools."""
+    from core.middleware import INTERNAL_TOOL_HEADER, INTERNAL_TOOL_TOKEN
+    return {INTERNAL_TOOL_HEADER: INTERNAL_TOOL_TOKEN}
+
+
+def _parse_event_yaml(description: str) -> Dict[str, Any]:
+    """Pull the `cookbook:` block out of an event description.
+
+    Deliberately tolerant: we don't want a calendar-edit typo (a stray
+    `>`, a tab, etc.) to silently drop the event. Returns {} on any
+    error so the caller falls back to title-match against presets.
+    """
+    if not isinstance(description, str) or "cookbook:" not in description:
+        return {}
+    try:
+        block_start = description.index("cookbook:")
+        block = description[block_start:].split("\n")
+        out: Dict[str, Any] = {}
+        for line in block[1:]:
+            if not line.startswith(("  ", "\t")):
+                # First non-indented line ends the block.
+                if line.strip() == "" and not out:
+                    continue
+                break
+            k, _, v = line.strip().partition(":")
+            v = v.strip().strip("'").strip('"')
+            if k and v:
+                out[k] = v
+        return out
+    except Exception as e:
+        logger.debug(f"event yaml parse failed (ignored): {e}")
+        return {}
+
+
+def _now_utc() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def _parse_iso(s: str) -> Optional[datetime]:
+    if not s:
+        return None
+    try:
+        # Accept both ISO with and without timezone; assume UTC if naive.
+        s2 = s.replace("Z", "+00:00")
+        dt = datetime.fromisoformat(s2)
+        if dt.tzinfo is None:
+            dt = dt.replace(tzinfo=timezone.utc)
+        return dt
+    except Exception:
+        return None
+
+
+async def _fetch_calendar_events(calendar_href: str, start: datetime, end: datetime) -> List[Dict[str, Any]]:
+    """List events on a single calendar in [start, end].
+
+    Reuses /api/calendar/events. RRULE expansion happens server-side so
+    we get concrete occurrences, not the master recurring event.
+    """
+    headers = _internal_headers()
+    params = {
+        "start": start.isoformat(),
+        "end": end.isoformat(),
+        "calendar": calendar_href,
+    }
+    try:
+        async with httpx.AsyncClient(timeout=15) as client:
+            r = await client.get(
+                f"{COOKBOOK_BASE_URL}/api/calendar/events",
+                params=params, headers=headers,
+            )
+            if r.status_code >= 400:
+                logger.debug(f"calendar/events returned {r.status_code}: {r.text[:200]}")
+                return []
+            data = r.json()
+        return data.get("events", []) if isinstance(data, dict) else []
+    except Exception as e:
+        logger.warning(f"reconciler: failed to fetch calendar events: {e}")
+        return []
+
+
+async def _gpus_busy(host: str) -> bool:
+    """Best-effort: are any GPUs on `host` already under non-trivial load?
+
+    Used to honor "refuse to launch if GPUs busy" semantics. We don't
+    block on a vllm process that's currently loading our OWN target —
+    that's handled separately (idempotent registration). The check is
+    "is there a foreign process holding GPU memory".
+    """
+    headers = _internal_headers()
+    try:
+        async with httpx.AsyncClient(timeout=10) as client:
+            params = {"host": host} if host else {}
+            r = await client.get(
+                f"{COOKBOOK_BASE_URL}/api/cookbook/gpus",
+                params=params, headers=headers,
+            )
+            if r.status_code >= 400:
+                return False
+            data = r.json() or {}
+    except Exception:
+        return False
+    for gpu in data.get("gpus") or []:
+        used_mb = int(gpu.get("used_mb") or 0)
+        # 500 MB threshold: enough to exclude an idle display driver
+        # (usually <300 MB) but catch any real allocation.
+        if used_mb > 500:
+            return True
+    return False
+
+
+def _resolve_event_payload(event: Dict[str, Any], presets: List[Dict[str, Any]]) -> Optional[Dict[str, Any]]:
+    """Turn a calendar event into a serve payload (or None if unschedulable).
+
+    Tries event description's `cookbook:` block first; falls back to a
+    case-insensitive preset-name match against the event title.
+    """
+    parsed = _parse_event_yaml(event.get("description") or "")
+    if parsed.get("repo_id") or parsed.get("cmd"):
+        return {
+            "repo_id": parsed.get("repo_id") or parsed.get("model") or (event.get("summary") or ""),
+            "cmd": parsed.get("cmd") or "",
+            "remote_host": parsed.get("host") or parsed.get("remote_host") or "",
+            "port": parsed.get("port"),
+        }
+    # Title-based preset lookup.
+    title = (event.get("summary") or "").strip()
+    if not title:
+        return None
+    preset_name = parsed.get("preset") or title
+    lname = preset_name.lower()
+    chosen = next(
+        (p for p in presets if isinstance(p, dict) and (p.get("name") or "").lower() == lname),
+        None,
+    )
+    if chosen is None:
+        chosen = next(
+            (p for p in presets if isinstance(p, dict) and lname in (p.get("name") or "").lower()),
+            None,
+        )
+    if chosen is None:
+        return None
+    cmd = (chosen.get("cmd") or "").strip()
+    # Adopted presets have no usable cmd — they can't be relaunched
+    # from the scheduler.
+    if not cmd or cmd.startswith("(adopted"):
+        logger.info(f"scheduler: preset {preset_name!r} has no cmd; cannot schedule")
+        return None
+    return {
+        "repo_id": chosen.get("model") or chosen.get("modelId") or "",
+        "cmd": cmd,
+        "remote_host": chosen.get("host") or chosen.get("remoteHost") or "",
+        "port": chosen.get("port"),
+    }
+
+
+def _state_path() -> Path:
+    return Path("/app/data/cookbook_state.json")
+
+
+def _read_state() -> Dict[str, Any]:
+    p = _state_path()
+    if not p.exists():
+        return {}
+    try:
+        return json.loads(p.read_text(encoding="utf-8"))
+    except Exception:
+        return {}
+
+
+def _write_state(state: Dict[str, Any]) -> None:
+    try:
+        from core.atomic_io import atomic_write_json
+        atomic_write_json(_state_path(), state)
+    except Exception as e:
+        logger.warning(f"scheduler: state write failed: {e}")
+
+
+async def _launch_serve(payload: Dict[str, Any], event_uid: str) -> Optional[str]:
+    """Hit /api/model/serve. Returns session_id on success, None on failure."""
+    headers = _internal_headers()
+    body = {"repo_id": payload["repo_id"], "cmd": payload["cmd"]}
+    if payload.get("remote_host"):
+        body["remote_host"] = payload["remote_host"]
+    # Pull env/gpu/hf_token from the host's saved server entry, same as
+    # the chat agent's serve_model does. Without this, vllm can't find
+    # its venv binaries.
+    try:
+        async with httpx.AsyncClient(timeout=10) as c:
+            r = await c.get(f"{COOKBOOK_BASE_URL}/api/cookbook/state", headers=headers)
+            st = r.json() if r.headers.get("content-type", "").startswith("application/json") else {}
+    except Exception:
+        st = {}
+    env = (st.get("env") or {}) if isinstance(st, dict) else {}
+    servers = env.get("servers") or []
+    target_host = payload.get("remote_host") or ""
+    srv = next(
+        (s for s in servers if isinstance(s, dict)
+         and (s.get("host") == target_host or s.get("name") == target_host)),
+        {},
+    )
+    if srv.get("env") in ("venv", "conda") and srv.get("envPath"):
+        body["env_prefix"] = f"source {srv['envPath']}/bin/activate" if srv["env"] == "venv" else f"conda activate {srv['envPath']}"
+    if srv.get("hfToken"):
+        body["hf_token"] = srv["hfToken"]
+    if srv.get("port"):
+        body["ssh_port"] = str(srv["port"])
+    if srv.get("platform"):
+        body["platform"] = srv["platform"]
+    try:
+        async with httpx.AsyncClient(timeout=30) as client:
+            r = await client.post(f"{COOKBOOK_BASE_URL}/api/model/serve", json=body, headers=headers)
+            data = r.json() if r.content else {}
+    except Exception as e:
+        logger.warning(f"scheduler: launch failed for event {event_uid}: {e}")
+        return None
+    if not data.get("ok"):
+        err = data.get("error") or data.get("detail") or "unknown"
+        logger.warning(f"scheduler: launch rejected for event {event_uid}: {err}")
+        return None
+    return data.get("session_id")
+
+
+async def _stop_serve(session_id: str, host: str) -> None:
+    headers = _internal_headers()
+    try:
+        async with httpx.AsyncClient(timeout=15) as client:
+            await client.post(f"{COOKBOOK_BASE_URL}/api/model/stop",
+                              json={"session_id": session_id, "remote_host": host},
+                              headers=headers)
+    except Exception as e:
+        logger.warning(f"scheduler: stop failed for {session_id}: {e}")
+
+
+def _mark_event_status(state: Dict[str, Any], event_uid: str, status: str,
+                       reason: str = "", session_id: str = "") -> None:
+    """Track per-event reconciliation status in cookbook_state.scheduler.
+
+    Schema:
+      state.scheduler.events = {
+        "<event_uid>": {
+          "status": "running" | "skipped" | "ended" | "failed",
+          "reason": "<short string>",
+          "session_id": "...",
+          "ts": <ms epoch>,
+        },
+        ...
+      }
+    """
+    sched = state.setdefault("scheduler", {})
+    events = sched.setdefault("events", {})
+    events[event_uid] = {
+        "status": status,
+        "reason": reason,
+        "session_id": session_id,
+        "ts": int(time.time() * 1000),
+    }
+
+
+async def _reconcile_once() -> Dict[str, Any]:
+    """One reconciliation pass. Returns a dict for diagnostics + UI.
+
+    Idempotent: running this twice in a row with no event changes
+    should produce the same state without double-launching or
+    double-killing.
+    """
+    from src.settings import get_setting
+    if not get_setting("cookbook_scheduler_enabled", False):
+        return {"skipped": "disabled"}
+    calendar_href = get_setting("cookbook_schedule_calendar_href", "") or ""
+    if not calendar_href:
+        return {"skipped": "no_calendar_configured"}
+
+    now = _now_utc()
+    # Look ±90s around now so a 60s tick still picks up events that
+    # started 30s ago but haven't been reconciled.
+    window_start = now - timedelta(seconds=90)
+    window_end = now + timedelta(seconds=90)
+    events = await _fetch_calendar_events(calendar_href, window_start, window_end)
+    state = _read_state()
+    presets = state.get("presets") or []
+    sched = state.get("scheduler") or {}
+    tracked = sched.get("events") or {}
+
+    out: Dict[str, Any] = {"events": []}
+    state_dirty = False
+
+    # Classify each event by where `now` falls relative to its window.
+    for ev in events:
+        uid = ev.get("uid") or ev.get("id") or ""
+        if not uid:
+            continue
+        ev_start = _parse_iso(ev.get("dtstart") or ev.get("start") or "")
+        ev_end = _parse_iso(ev.get("dtend") or ev.get("end") or "")
+        if ev_start is None or ev_end is None:
+            continue
+        in_window = ev_start <= now < ev_end
+        just_ended = (ev_end <= now) and (now - ev_end) < timedelta(seconds=90)
+        ev_status = (tracked.get(uid) or {}).get("status")
+        ev_session = (tracked.get(uid) or {}).get("session_id")
+
+        if just_ended and ev_session and ev_status in {"running", "adopted"}:
+            # Window closed → hard-kill (per user choice).
+            payload = _resolve_event_payload(ev, presets) or {}
+            host = payload.get("remote_host") or ""
+            await _stop_serve(ev_session, host)
+            _mark_event_status(state, uid, "ended", session_id=ev_session)
+            state_dirty = True
+            out["events"].append({"uid": uid, "status": "ended", "session_id": ev_session})
+            continue
+
+        if not in_window:
+            continue
+
+        # In window. Determine whether a serve already exists for this event.
+        if ev_status == "running" and ev_session:
+            out["events"].append({"uid": uid, "status": "running", "session_id": ev_session})
+            continue
+        if ev_status == "skipped":
+            # User chose: no retry within the window.
+            out["events"].append({"uid": uid, "status": "skipped",
+                                  "reason": (tracked.get(uid) or {}).get("reason", "")})
+            continue
+
+        payload = _resolve_event_payload(ev, presets)
+        if payload is None:
+            _mark_event_status(state, uid, "failed",
+                               reason="no preset or cmd resolvable from event")
+            state_dirty = True
+            out["events"].append({"uid": uid, "status": "failed", "reason": "no preset"})
+            continue
+
+        # Adoption pass: is a non-scheduled serve already running this model?
+        target_host = payload.get("remote_host") or ""
+        for t in state.get("tasks") or []:
+            if not isinstance(t, dict):
+                continue
+            if t.get("type") != "serve":
+                continue
+            if (t.get("status") or "").lower() not in {"running", "ready", "loading", "warming"}:
+                continue
+            if t.get("remoteHost") != target_host:
+                continue
+            t_model = (t.get("payload") or {}).get("repo_id") or t.get("name") or ""
+            if t_model.split("/")[-1] == (payload["repo_id"] or "").split("/")[-1]:
+                t[SCHEDULE_OWNER_KEY] = uid
+                _mark_event_status(state, uid, "adopted",
+                                   reason="pre-existing serve adopted",
+                                   session_id=t.get("sessionId") or t.get("id") or "")
+                state_dirty = True
+                out["events"].append({"uid": uid, "status": "adopted",
+                                      "session_id": t.get("sessionId")})
+                break
+        else:
+            # No matching pre-existing serve → fresh launch path.
+            if await _gpus_busy(target_host):
+                _mark_event_status(state, uid, "skipped",
+                                   reason="GPUs busy at launch time")
+                state_dirty = True
+                out["events"].append({"uid": uid, "status": "skipped",
+                                      "reason": "GPUs busy"})
+                continue
+            sid = await _launch_serve(payload, uid)
+            if sid:
+                _mark_event_status(state, uid, "running",
+                                   reason="launched by scheduler",
+                                   session_id=sid)
+                state_dirty = True
+                # Tag the new task with the schedule owner so window-end
+                # cleanup knows this is ours, not a manual launch.
+                fresh_state = _read_state()
+                for t in fresh_state.get("tasks") or []:
+                    if isinstance(t, dict) and t.get("sessionId") == sid:
+                        t[SCHEDULE_OWNER_KEY] = uid
+                        break
+                _write_state(fresh_state)
+                state_dirty = False  # we just wrote
+                out["events"].append({"uid": uid, "status": "running",
+                                      "session_id": sid})
+            else:
+                _mark_event_status(state, uid, "skipped",
+                                   reason="serve_model rejected launch")
+                state_dirty = True
+                out["events"].append({"uid": uid, "status": "skipped",
+                                      "reason": "launch rejected"})
+
+    if state_dirty:
+        _write_state(state)
+    out["tick_at"] = now.isoformat()
+    return out
+
+
+async def reconcile_loop() -> None:
+    """Forever-loop reconciler. Registered as a startup task in app.py."""
+    # Stagger the first tick so we don't fight the rest of startup for
+    # CPU + I/O.
+    await asyncio.sleep(15)
+    while True:
+        try:
+            result = await _reconcile_once()
+            if result.get("events"):
+                logger.info(f"scheduler tick: {result}")
+        except Exception as e:
+            logger.warning(f"scheduler tick failed: {e}")
+        await asyncio.sleep(60)

static/index.html

@@ -2266,6 +2266,7 @@
 <script type="module" src="/static/js/chatStream.js"></script>
 <script type="module" src="/static/js/chat.js?v=20260520m"></script>
 <script type="module" src="/static/js/cookbook.js"></script>
+<script src="/static/js/cookbookSchedule.js"></script>
 <script type="module" src="/static/js/search-chat.js"></script>
 <script type="module" src="/static/js/compare/index.js"></script>
 <script type="module" src="/static/js/theme.js"></script>

static/js/cookbookSchedule.js

@@ -0,0 +1,244 @@
+// Cookbook Schedule modal. Click the "Schedule…" button in a serve
+// panel → opens this modal → user picks days + time slots → POST to
+// /api/cookbook/schedule/from-cookbook which writes the calendar event.
+//
+// Whole feature is gated on /api/cookbook/schedule/upcoming returning
+// `enabled: true`. If the server says it's disabled, this module hides
+// all Schedule buttons and never opens the modal.
+//
+// To remove the feature entirely: delete this file + the `<script>`
+// tag that loads it + the `.hwfit-serve-schedule` button in
+// cookbookServe.js. No other code depends on it.
+
+(function () {
+  const DAYS = [
+    { key: "MO", label: "Mon" },
+    { key: "TU", label: "Tue" },
+    { key: "WE", label: "Wed" },
+    { key: "TH", label: "Thu" },
+    { key: "FR", label: "Fri" },
+    { key: "SA", label: "Sat" },
+    { key: "SU", label: "Sun" },
+  ];
+  const WEEKDAYS = ["MO", "TU", "WE", "TH", "FR"];
+
+  let _enabledCache = null;
+
+  async function isEnabled() {
+    if (_enabledCache !== null) return _enabledCache;
+    try {
+      const r = await fetch("/api/cookbook/schedule/upcoming?hours=1", { credentials: "same-origin" });
+      if (!r.ok) { _enabledCache = false; return false; }
+      const data = await r.json();
+      _enabledCache = !!data.enabled;
+      return _enabledCache;
+    } catch (_) {
+      _enabledCache = false;
+      return false;
+    }
+  }
+
+  function esc(s) {
+    return String(s == null ? "" : s)
+      .replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;")
+      .replace(/"/g, "&quot;").replace(/'/g, "&#39;");
+  }
+
+  function slotRowHtml(start, end) {
+    return `
+      <div class="cookbook-schedule-slot">
+        <input type="time" class="cookbook-schedule-start" value="${esc(start || "09:00")}" />
+        <span class="cookbook-schedule-dash">–</span>
+        <input type="time" class="cookbook-schedule-end" value="${esc(end || "17:00")}" />
+        <button type="button" class="cookbook-schedule-slot-remove" title="Remove slot">×</button>
+      </div>`;
+  }
+
+  function dayChipsHtml(selected) {
+    const sel = new Set(selected || WEEKDAYS);
+    return DAYS.map(d =>
+      `<label class="cookbook-schedule-day${sel.has(d.key) ? " active" : ""}">
+        <input type="checkbox" value="${d.key}" ${sel.has(d.key) ? "checked" : ""} />
+        ${esc(d.label)}
+      </label>`).join("");
+  }
+
+  function openModal(config) {
+    // config = {title, preset, repo_id, cmd, host, port}
+    const wrap = document.createElement("div");
+    wrap.className = "cookbook-schedule-modal-backdrop";
+    wrap.innerHTML = `
+      <div class="cookbook-schedule-modal">
+        <div class="cookbook-schedule-modal-header">
+          <strong>Schedule: ${esc(config.title || config.preset || "model")}</strong>
+          <button type="button" class="cookbook-schedule-close" title="Close">×</button>
+        </div>
+        <div class="cookbook-schedule-modal-body">
+          <div class="cookbook-schedule-section">
+            <label class="cookbook-schedule-section-label">When</label>
+            <div class="cookbook-schedule-slots">
+              ${slotRowHtml("09:00", "17:00")}
+            </div>
+            <button type="button" class="cookbook-schedule-add-slot">+ add another time slot</button>
+          </div>
+          <div class="cookbook-schedule-section">
+            <label class="cookbook-schedule-section-label">Repeat on</label>
+            <div class="cookbook-schedule-days">${dayChipsHtml(WEEKDAYS)}</div>
+            <div class="cookbook-schedule-day-quickset">
+              <button type="button" data-set="weekdays">Weekdays</button>
+              <button type="button" data-set="weekend">Weekend</button>
+              <button type="button" data-set="all">Every day</button>
+            </div>
+          </div>
+          <div class="cookbook-schedule-section">
+            <label class="cookbook-schedule-section-label">Until</label>
+            <div class="cookbook-schedule-until">
+              <label><input type="radio" name="until-mode" value="forever" checked /> Forever</label>
+              <label><input type="radio" name="until-mode" value="date" /> Until
+                <input type="date" class="cookbook-schedule-until-date" disabled />
+              </label>
+            </div>
+          </div>
+          <div class="cookbook-schedule-error" style="display:none;"></div>
+        </div>
+        <div class="cookbook-schedule-modal-footer">
+          <button type="button" class="cookbook-btn cookbook-schedule-cancel">Cancel</button>
+          <button type="button" class="cookbook-btn cookbook-schedule-save">Save schedule</button>
+        </div>
+      </div>`;
+    document.body.appendChild(wrap);
+
+    const $ = (sel) => wrap.querySelector(sel);
+    const $$ = (sel) => Array.from(wrap.querySelectorAll(sel));
+
+    const close = () => wrap.remove();
+    $(".cookbook-schedule-close").onclick = close;
+    $(".cookbook-schedule-cancel").onclick = close;
+    wrap.addEventListener("click", (e) => { if (e.target === wrap) close(); });
+
+    // Add / remove slot rows.
+    $(".cookbook-schedule-add-slot").onclick = () => {
+      const slots = $(".cookbook-schedule-slots");
+      const tmp = document.createElement("div");
+      tmp.innerHTML = slotRowHtml("18:00", "23:00");
+      slots.appendChild(tmp.firstElementChild);
+    };
+    wrap.addEventListener("click", (e) => {
+      if (e.target.classList && e.target.classList.contains("cookbook-schedule-slot-remove")) {
+        const slots = $$(".cookbook-schedule-slot");
+        if (slots.length > 1) e.target.closest(".cookbook-schedule-slot").remove();
+      }
+    });
+
+    // Day quickset chips.
+    $$(".cookbook-schedule-day-quickset button").forEach(btn => {
+      btn.onclick = () => {
+        const sel = btn.dataset.set;
+        const want = sel === "weekdays" ? new Set(WEEKDAYS)
+                   : sel === "weekend" ? new Set(["SA", "SU"])
+                   : new Set(DAYS.map(d => d.key));
+        $$(".cookbook-schedule-day input").forEach(inp => {
+          inp.checked = want.has(inp.value);
+          inp.closest(".cookbook-schedule-day").classList.toggle("active", inp.checked);
+        });
+      };
+    });
+    $$(".cookbook-schedule-day input").forEach(inp => {
+      inp.onchange = () => inp.closest(".cookbook-schedule-day").classList.toggle("active", inp.checked);
+    });
+
+    // Until-date radio enables / disables the date picker.
+    $$('input[name="until-mode"]').forEach(r => {
+      r.onchange = () => {
+        const datePicker = $(".cookbook-schedule-until-date");
+        datePicker.disabled = $('input[name="until-mode"]:checked').value !== "date";
+      };
+    });
+
+    $(".cookbook-schedule-save").onclick = async () => {
+      const slots = $$(".cookbook-schedule-slot").map(row => ({
+        start: row.querySelector(".cookbook-schedule-start").value,
+        end: row.querySelector(".cookbook-schedule-end").value,
+      }));
+      const days = $$(".cookbook-schedule-day input:checked").map(i => i.value);
+      const untilMode = $('input[name="until-mode"]:checked').value;
+      const untilDate = untilMode === "date" ? $(".cookbook-schedule-until-date").value : "";
+
+      const errEl = $(".cookbook-schedule-error");
+      errEl.style.display = "none";
+
+      const body = {
+        model: config.title || config.preset || "",
+        preset: config.preset,
+        repo_id: config.repo_id,
+        cmd: config.cmd,
+        host: config.host,
+        port: config.port,
+        slots, days,
+      };
+      if (untilDate) body.until = untilDate;
+
+      try {
+        const r = await fetch("/api/cookbook/schedule/from-cookbook", {
+          method: "POST",
+          credentials: "same-origin",
+          headers: { "Content-Type": "application/json" },
+          body: JSON.stringify(body),
+        });
+        const data = await r.json();
+        if (!r.ok) {
+          errEl.textContent = data.detail || data.error || `HTTP ${r.status}`;
+          errEl.style.display = "block";
+          return;
+        }
+        close();
+        if (window.toast) window.toast(`Scheduled ${slots.length} window(s) on ${days.length} day(s).`, "success");
+      } catch (e) {
+        errEl.textContent = String(e);
+        errEl.style.display = "block";
+      }
+    };
+  }
+
+  // Click-binding: any .hwfit-serve-schedule button inside a serve
+  // panel opens the modal with the panel's current config.
+  document.addEventListener("click", (e) => {
+    const btn = e.target.closest && e.target.closest(".hwfit-serve-schedule");
+    if (!btn) return;
+    e.preventDefault();
+    e.stopPropagation();
+
+    // Reach into the serve panel to read current config. cookbookServe.js
+    // stores the active config on the panel root via data attributes
+    // that we read here without coupling further.
+    const panel = btn.closest("[data-cookbook-serve-panel]") || btn.closest(".doclib-card-expanded") || btn.closest(".doclib-card");
+    const ds = panel ? panel.dataset || {} : {};
+    const config = {
+      title: ds.modelName || ds.preset || panel?.querySelector(".doclib-card-title")?.textContent?.trim() || "model",
+      preset: ds.preset || "",
+      repo_id: ds.repoId || "",
+      cmd: ds.cmd || "",
+      host: ds.host || "",
+      port: ds.port ? Number(ds.port) : undefined,
+    };
+    openModal(config);
+  });
+
+  // Reveal Schedule buttons once we confirm the feature is enabled.
+  async function refreshScheduleButtonVisibility() {
+    const enabled = await isEnabled();
+    document.querySelectorAll(".hwfit-serve-schedule").forEach(btn => {
+      btn.style.display = enabled ? "" : "none";
+    });
+  }
+
+  // Periodically re-check (cheap) so toggling the feature in Settings
+  // takes effect without a full reload.
+  document.addEventListener("DOMContentLoaded", () => {
+    refreshScheduleButtonVisibility();
+    setInterval(refreshScheduleButtonVisibility, 30000);
+  });
+  // Also re-check whenever a serve panel expands.
+  const obs = new MutationObserver(() => refreshScheduleButtonVisibility());
+  obs.observe(document.body, { childList: true, subtree: true });
+})();

static/js/cookbookServe.js

@@ -744,6 +744,11 @@ function _rerenderCachedModels() {
       // pushes Cancel + Launch to the right.
       panelHtml += `<span class="hwfit-serve-actions-spacer"></span>`;
       panelHtml += `<button class="cookbook-btn hwfit-serve-cancel" type="button" title="Close this configuration panel">Cancel</button>`;
+      // Schedule button — opens a modal that converts the current serve
+      // config into a calendar event. Hidden unless the scheduler feature
+      // flag is on (cookbookSchedule.js reads /api/cookbook/schedule/upcoming
+      // on init and toggles visibility).
+      panelHtml += `<button class="cookbook-btn hwfit-serve-schedule" type="button" title="Schedule this model to run on a recurring window" style="display:none;"><svg width="11" height="11" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="vertical-align:-1px;margin-right:4px;flex-shrink:0;"><rect x="3" y="4" width="18" height="18" rx="2" ry="2"/><line x1="16" y1="2" x2="16" y2="6"/><line x1="8" y1="2" x2="8" y2="6"/><line x1="3" y1="10" x2="21" y2="10"/></svg>Schedule…</button>`;
       panelHtml += `<button class="cookbook-btn hwfit-serve-launch"><svg width="11" height="11" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="vertical-align:-1px;margin-right:4px;flex-shrink:0;"><polygon points="13 2 3 14 12 14 11 22 21 10 12 10 13 2"/></svg>Launch</button>`;
       panelHtml += `</div>`;
       panelHtml += `</div>`;

static/style.css

@@ -35736,3 +35736,96 @@ body.theme-frosted .modal {
      is already ≥16px and never zoomed — leave it so we don't shrink it. */
   .doc-email-richbody.doc-font-m { font-size: 16px !important; }
 }
+
+/* ── Cookbook Schedule modal (feature-flagged) ─────────────────────── */
+.cookbook-schedule-modal-backdrop {
+  position: fixed; inset: 0; background: rgba(0,0,0,.5);
+  display: flex; align-items: center; justify-content: center;
+  z-index: 10000;
+}
+.cookbook-schedule-modal {
+  background: var(--bg-secondary, #1e1e22);
+  color: var(--text-primary, #e6e6e6);
+  border: 1px solid var(--border, #2d2d33);
+  border-radius: 12px; width: min(560px, 92vw); max-height: 90vh;
+  display: flex; flex-direction: column; overflow: hidden;
+  box-shadow: 0 12px 40px rgba(0,0,0,.5);
+}
+.cookbook-schedule-modal-header {
+  padding: 14px 18px; border-bottom: 1px solid var(--border, #2d2d33);
+  display: flex; align-items: center; justify-content: space-between;
+}
+.cookbook-schedule-close {
+  background: transparent; border: none; color: inherit;
+  font-size: 22px; line-height: 1; cursor: pointer; padding: 0 6px;
+}
+.cookbook-schedule-modal-body { padding: 16px 18px; overflow-y: auto; }
+.cookbook-schedule-section { margin-bottom: 16px; }
+.cookbook-schedule-section-label {
+  display: block; font-size: 12px; opacity: .7;
+  margin-bottom: 6px; text-transform: uppercase; letter-spacing: .04em;
+}
+.cookbook-schedule-slot {
+  display: flex; align-items: center; gap: 6px; margin-bottom: 6px;
+}
+.cookbook-schedule-slot input[type="time"] {
+  background: var(--bg-primary, #131316); color: inherit;
+  border: 1px solid var(--border, #2d2d33); border-radius: 6px;
+  padding: 6px 8px; font: inherit;
+}
+.cookbook-schedule-slot-remove {
+  background: transparent; border: 1px solid var(--border, #2d2d33);
+  color: inherit; border-radius: 6px; width: 28px; height: 28px;
+  cursor: pointer; font-size: 18px; line-height: 1;
+}
+.cookbook-schedule-add-slot {
+  background: transparent; color: var(--accent, #6aa8ff);
+  border: 1px dashed var(--border, #2d2d33); border-radius: 6px;
+  padding: 6px 10px; cursor: pointer; font: inherit; margin-top: 4px;
+}
+.cookbook-schedule-days {
+  display: flex; flex-wrap: wrap; gap: 6px; margin-bottom: 6px;
+}
+.cookbook-schedule-day {
+  display: inline-flex; align-items: center; gap: 4px;
+  padding: 6px 10px; border: 1px solid var(--border, #2d2d33);
+  border-radius: 999px; cursor: pointer; font-size: 13px;
+  user-select: none;
+}
+.cookbook-schedule-day input { display: none; }
+.cookbook-schedule-day.active {
+  background: var(--accent, #6aa8ff); color: #fff;
+  border-color: var(--accent, #6aa8ff);
+}
+.cookbook-schedule-day-quickset {
+  display: flex; gap: 6px; flex-wrap: wrap;
+}
+.cookbook-schedule-day-quickset button {
+  background: transparent; color: var(--text-secondary, #aaa);
+  border: 1px solid var(--border, #2d2d33); border-radius: 6px;
+  padding: 4px 8px; cursor: pointer; font-size: 12px;
+}
+.cookbook-schedule-until { display: flex; gap: 16px; flex-wrap: wrap; }
+.cookbook-schedule-until label {
+  display: inline-flex; align-items: center; gap: 6px;
+}
+.cookbook-schedule-until-date {
+  background: var(--bg-primary, #131316); color: inherit;
+  border: 1px solid var(--border, #2d2d33); border-radius: 6px;
+  padding: 4px 8px;
+}
+.cookbook-schedule-error {
+  color: #ff6b6b; background: rgba(255,107,107,.08);
+  border: 1px solid rgba(255,107,107,.3); border-radius: 6px;
+  padding: 8px 12px; font-size: 13px;
+}
+.cookbook-schedule-modal-footer {
+  padding: 12px 18px; border-top: 1px solid var(--border, #2d2d33);
+  display: flex; justify-content: flex-end; gap: 8px;
+}
+
+/* Mobile (Firefox + others): single-column slot rows + larger touch targets. */
+@media (max-width: 600px) {
+  .cookbook-schedule-modal { width: 96vw; }
+  .cookbook-schedule-day { padding: 8px 14px; font-size: 14px; }
+}