Code-Inc/intelligence-terminal

Fork 0

Files

MrSphay 60ec835cc3

Codex Template Compliance / template-compliance (pull_request) Successful in 7s

Details

Build / test-and-image (pull_request) Successful in 26s

Details

docs: record tagged tool parser fix

2026-07-05 23:17:47 +02:00

25 KiB

Raw Blame History

Agent Handoff

Last updated: 2026-07-05

Dynamic DAVE Presence

Gitea issue #70 was closed by merged PR #71. Production merge commit: ca68541adf9931e311f1a79a09cb87b3cae85928.
Implementation commit: 08b6016 feat: add dynamic autonomous DAVE presence; handoff commit: f2d8e89.
The implementation deliberately uses no fixed daily schedule. It evaluates at randomized intervals, is pulled forward by material/critical sweep deltas, delays itself after operator chat activity, and lengthens quiet periods.
Hard controls: profile quiet hours, daily cap, minimum message gap, bounded evaluation interval, persisted state in runs/dave-presence-state.json, and agent-core rejection of mutating tools outside operator-initiated chat.
Presence can produce a grounded warning, update, all-clear, practical suggestion, or one useful question; it stays silent when another message would add noise. It never claims consciousness or continuous observation.
New env keys start with DAVE_PRESENCE_; the repository default remains opt-in.
Local lightweight verification passed: Node syntax checks, package.json parse, Compose config, and git diff --check. Heavy tests/build were not run locally per kit policy.
PR runs 915-916 and production runs 917-919 passed, including unit tests, Compose validation, Docker build/publish, release dry-run, and template compliance.
Dockge was updated from the Gitea Registry and explicitly configured with DAVE_PRESENCE_ENABLED=true, max 4 messages/day, 75-minute minimum gap, randomized 45-180 minute evaluation bounds, 60-minute post-interaction idle delay, and a 5-minute timer resolution.
Live /api/health reported davePresence.enabled=true, running=true, dynamic=true, sentToday=0, a variable nextEvaluationAt, the existing encrypted profile preserved, and no sweep error. The container reported running/healthy.
Live verification then exposed issue #73: an invalid optional profile timezone caused lastReason=invalid_timezone. PR #74 fixed runtime resolution to fall back to DAVE_PRESENCE_TIMEZONE without logging or modifying encrypted profile data. Merge commit: f0a8162202bfe0a0a8ed2a00dc4f0f7092677eea.
PR runs 925-926 and production runs 927-929 passed. After redeployment and the startup timer, live health reported lastReason=not_due instead of invalid_timezone, with dynamic presence running, the profile preserved, the container healthy, and no sweep error.

Latest Completed Work

Issue #76 / PR #77 fixed a second local-model protocol leak: ChatML-style output such as <|tool_call>call:get_evidence{query:"...",limit:5}<tool_call|> was previously treated as user-facing text. Merge commit: 7f140021f037cbc351c9a3b39a5eb3610bfc4939.
The controlled agent now parses bounded tagged calls into the normal allowlisted tool path without eval, recognizes tagged JSON envelopes, and treats malformed protocol-like output as a repairable protocol error. Tool tags are never accepted as final Telegram text.
The exact observed Russia-query format and a malformed-tag variant are regression fixtures. PR runs 935-936 and production runs 937-939 passed.
The registry image was redeployed to Dockge. Live health reported running/healthy, Telegram agent enabled with 14 tools, dynamic DAVE presence enabled, encrypted profile preserved, and no sweep error.
Canonical repository: https://git.wilkensxl.de/Code-Inc/intelligence-terminal
LiteLLM implementation merge: 5c4bf80eb0c19bd59080f5432a2a344798d7a3ce
Merged PR: #48 feat: add LiteLLM provider and publish Code-Inc image
Completed issue: #47 Add first-class LiteLLM provider and publish updated image
LiteLLM is implemented through the OpenAI-compatible /chat/completions API and requires LLM_BASE_URL, LLM_API_KEY, and LLM_MODEL.
The build workflow now targets git.wilkensxl.de/code-inc/intelligence-terminal and publishes only from the production branch, not from pull requests.
Gitea Actions runs 231-235 passed for the PR and production merge, including unit tests, Compose validation, Docker build, release dry-run, and template compliance.
The first code-inc registry publication was verified through the Gitea Package API on 2026-07-03.
PR #52 / issue #51 removed the hard-coded 90-second/4096-token idea-generation override. LLM ideas now respect LLM_TIMEOUT_MS and LLM_MAX_TOKENS.
PR #54 / issue #53 fixed prediction persistence after successful LLM generation and added a SQLite-backed regression test.
Live Dockge verification on 2026-07-04 used LLM_TIMEOUT_MS=300000 and LLM_MAX_TOKENS=4096 with the heim-llm LiteLLM alias. The completed sweep produced six parsed ideas, reported ideasSource=llm, persisted memory, and had no lastSweepError.
Production implementation commit: 14d9276c30e06cafcaee8177ba7377fdf5f26277.
Issues #47, #51, and #53 are complete. Issue #21 tracks the failing security scan and #45 tracks the dependency workflow.
PR #57 / issue #56 added conversational Telegram AI chat for normal text plus /ask and /reset, bounded in-memory history, current intelligence grounding, typing activity, strict chat allowlisting, and plain-text replies.
Private Telegram chat messages are excluded from OSINT ingestion, and polling is serialized while a model response is running.
Gitea Actions runs 263-267 passed for the feature and production merge.
Live Dockge verification on 2026-07-05 reported telegramAiChat.enabled=true. A real heim-llm question using current dashboard context completed in 34 seconds and the answer was delivered successfully through the configured Telegram bot.
Current Telegram-chat implementation commit: c86407d4f8bfb8a445bb7f4685ff545b479244a1.
PR #60 / issue #59 added the controlled Telegram terminal agent with 13 allowlisted tools, bounded multi-step decisions, chat-bound confirmation for mutations, /tools, /trace, and proactive sweep analysis.
Tool-agent production commit: d13652a70b77263f357b487d50bab3af5585a309.
Issue #61 was completed and closed by merged PR #62.
Security Manager implementation commit: 0c7ddc5a6cb85487274a8bdf754d48a967ba2c84.
The Security Manager adds language-first Telegram onboarding, explicit consent/minimal setup, an AES-256-GCM encrypted profile under runs/security-profile.enc, profile commands, a read-only get_security_profile agent tool, location/personal relevance guidance, and deterministic alert-level/quiet-hours enforcement.
PR validation runs 883-886 passed. Production merge commit c0afc6d2e88b7602148d786dd249351323885ac2 passed build/publish run 887, release dry-run 888, and template compliance 889.
Registry tags latest, 20260705, and c0afc6d2e88b7602148d786dd249351323885ac2 were verified through the Gitea Package API on 2026-07-05.
The Dockge stack at C:\docker\intelligence-terminal was updated from the registry image. A cryptographically random SECURITY_PROFILE_ENCRYPTION_KEY was added without printing it. Live health reported the container healthy, LiteLLM configured, Telegram AI and the 14-tool agent enabled, and the encrypted profile store configured/available with no profile yet.
Issue #64 fixed a local-model protocol leak where an exhausted tool loop could expose a raw tool_call JSON object as the Telegram answer. PR #65 merged as e47c23e685d833d5f7433dc27b0834854c8c6152.
Exhausted loops now switch to a separate tool-free finalization prompt, allow one bounded repair attempt, and fail closed with a localized user-facing message if the model still requests tools. Internal protocol JSON is never returned as the answer.
PR runs 895-896 and production runs 897-899 passed. The registry latest image was deployed to Dockge; live health reported running/healthy, 14 tools enabled, no sweep error, and the encrypted Security Manager profile preserved (profileExists=true).
Issue #67 / PR #68 added DAVE as the consistent synthetic Security Manager identity across the tool agent, tool-free finalizer, provider-only chat, and first-start language prompt. Production merge: c07a65231ffc6b4d2f0c823b91aaf2eb13900e05.
DAVE adapts language, formality, directness, verbosity, sentence length, formatting, and technical depth from the newest message plus bounded chat history. It does not imitate errors, hostility, panic, discriminatory language, or unsupported certainty, and it never claims human consciousness, emotions, a body, or access beyond allowlisted tools.
PR runs 905-906 and production runs 907-909 passed. The new registry image was deployed to Dockge; live health remained running/healthy, the 14-tool agent was enabled, no sweep error was present, and the encrypted profile persisted.

Repository State

Project: Crucix fork / Intelligence Terminal

Local workspace:

C:\Users\MrSphay\Documents\Codex\Crucix\intelligence-terminal

Remotes:

origin   https://git.wilkensxl.de/Code-Inc/intelligence-terminal.git
upstream https://github.com/calesthio/Crucix.git

Current branch tip:

Run `git rev-parse HEAD` after clone/pull. The production branch contains Security Manager merge commit `c0afc6d2e88b7602148d786dd249351323885ac2` plus this release handoff update.

Production baseline before the current LiteLLM work:

c159c83a0768486c8c6f445b458b760dba4ba385

The default production branch points to this commit before the current PR:

origin/codex/production-intelligence-terminal

Gitea repository:

https://git.wilkensxl.de/Code-Inc/intelligence-terminal

Default branch observed through the Gitea API:

codex/production-intelligence-terminal

Agent Kit Requirements Applied

The mandatory kit was cloned and reviewed first:

C:\Users\MrSphay\Documents\Codex\Crucix\agent-kit

Rules applied from the kit:

Keep agent context in source control: AGENTS.md, .codex/project.md, and this handoff file.
Use Gitea Ubuntu runners for heavy verification and package publishing.
Keep Docker/Dockge operation first-class.
Do not commit secrets, .env, private logs, tokens, or generated runs/ data.
Add report-only maintenance workflows for security, dependency checks, repo cleanup, release dry runs, and template compliance.
Poll pushed Gitea Actions until terminal state when a token is available.
Use only lightweight local checks (node --check, JSON parsing, Compose config, git diff --check). Run unit tests, builds, audits, and image publication on the Gitea Ubuntu runners.

Security Manager

First startup asks for Deutsch or English before any profile question.
Consent offers full, minimal, or cancelled setup; /skip is available for profile inputs.
Allowed profile scope: preferred name, country/region/city level, timezone, household counts, mobility, general travel pattern, risk priorities, critical dependencies, alert preference, and quiet hours.
Exact addresses, identity documents, passwords/API keys, accounts, detailed diagnoses, booking data, and private contacts are explicitly excluded.
SECURITY_PROFILE_ENCRYPTION_KEY must be a unique secret of at least 32 characters. It must be preserved across deployments or the encrypted profile cannot be read.
Commands: /profile, /onboarding, /language, /profile_delete.
Health and metrics expose only profile availability/configuration timestamps and errors, not profile contents.
The configured LLM can obtain the approved profile only through the read-only get_security_profile allowlisted tool.
Non-FLASH proactive messages respect the profile's alert preference and quiet hours. FLASH can override quiet hours.

What Was Implemented

Docker And Runtime

Docker image is Docker-first and Dockge/Pangolin suitable.
Browser auto-open is disabled by default through AUTO_OPEN_BROWSER=false.
Runtime health checks now work in the container without wget or host browser tools.
runs is persisted through a volume.
A later fix added docker-entrypoint.sh to prepare /app/runs before dropping privileges, so mounted volumes work with the non-root Node runtime.
docker-compose.yml uses the Gitea Registry image by default:

git.wilkensxl.de/code-inc/intelligence-terminal:latest

API And Health

Added or hardened:

GET /api/health
GET /api/data
GET /api/metrics
POST /api/sweep
POST /api/action

Health now reports:

starting
healthy
degraded
stale
error

It also reports:

last sweep timestamps
stale/bootstrap state
data age
source health
source errors
LLM configuration state
Telegram/Discord enabled state
memory store state

Live Data And Source Degradation

Existing runs/latest.json is only treated as bootstrap/stale data until a real sweep completes.
Sweeps update sourceHealth, SSE/API data, and memory state.
RSS/news feed failures no longer silently look like fresh valid data.
safeFetch now tracks request counts, failures, bytes, source labels, hosts, and recent fetch events.
safeFetch has better timeout/retry/backoff/error behavior and reports HTML-as-API-error cases.
Yahoo Finance fetches are more explicit about source errors and HTML/API failures.
ACLED missing credentials now degrade transparently.
Telegram polling has quieter network-error backoff logs.

LLM Integration

Added unified OpenAI-compatible provider layer:

lib/llm/openai-compatible.mjs

Supported provider paths include:

openrouter
openai
openai-compatible
local-openai
lmstudio
lm-studio
ollama

Relevant environment keys:

LLM_PROVIDER
LLM_BASE_URL
LLM_API_KEY
LLM_MODEL
LLM_TEMPERATURE
LLM_MAX_TOKENS
LLM_TIMEOUT_MS
OPENROUTER_SITE_URL
OPENROUTER_APP_NAME

OpenRouter Free and local OpenAI-compatible endpoints are documented in README.md and .env.example.

Memory

Added Phase-1 SQLite memory:

lib/intelligence-store.mjs
runs/intelligence.db

It uses node:sqlite when available and gracefully falls back when unavailable.

Dashboard

Implemented:

interactive Sensor Grid layer modes
focus/hide/normal states persisted in localStorage
Space Watch icon/orbit toggle
map/globe filtering consistency
flat map label redraw handling
live server-mode data loading from /api/data even when jarvis.html still contains an offline inline snapshot
Terminal Actions panel with Status, Sweep, and Brief buttons

Important UI markers in the final code:

layerModes
spaceDisplayMode
toggleSpaceDisplay()
shouldShowType()
runTerminalAction()

Briefings

Brief output now includes:

Source Integrity
evidence links
event IDs
configurable verbosity through BRIEF_VERBOSITY

Documentation

Updated:

README.md
.env.example
docs/sources/README.md
docs/sources/opensky.md
docs/sources/acled.md
docs/sources/telegram.md
docs/sources/firms.md
docs/sources/maritime.md
docs/security-review.md
docs/release-checklist.md

README includes:

Gitea Registry pull example
Dockge-compatible compose example
full .env examples
OpenRouter Free setup
LM Studio setup
Ollama setup
local OpenAI-compatible setup
Pangolin/reverse proxy notes

Registry And Images

Production registry image:

git.wilkensxl.de/code-inc/intelligence-terminal

The legacy mrsphay package remains available. Verified code-inc tags from the LiteLLM production merge:

latest
20260703
5c4bf80eb0c19bd59080f5432a2a344798d7a3ce

Operator pull command:

docker pull git.wilkensxl.de/code-inc/intelligence-terminal:latest

Published manifest digest:

sha256:5e29483ebfd9baae368673adc790789f02aed2d5d5d3a550fe55a4b71b5b62dd

Relevant successful runner executions:

PR build:             https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/231
PR template check:    https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/232
Production publish:   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/233
Release dry-run:      https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/234
Template compliance: https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/235

Security Manager release runs:

PR build:             https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/885
PR template check:    https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/886
Production publish:   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/887
Release dry-run:      https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/888
Template compliance: https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/889

Gitea Actions

Workflows present:

.gitea/workflows/build.yml
.gitea/workflows/security-scan.yml
.gitea/workflows/repo-cleanup.yml
.gitea/workflows/dependency-check.yml
.gitea/workflows/release-dry-run.yml
.gitea/workflows/template-compliance.yml

Final runs for commit 53470cc701ec322080a89d220aef449b25850590 were polled through the Gitea API and succeeded:

build.yml on main: success
build.yml on codex/production-intelligence-terminal: success
release-dry-run.yml on main: success
release-dry-run.yml on codex/production-intelligence-terminal: success
template-compliance.yml on main: success
template-compliance.yml on codex/production-intelligence-terminal: success

Relevant run URLs:

https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/23
https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/24
https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/25
https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/26
https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/27
https://git.wilkensxl.de/Code-Inc/intelligence-terminal/actions/runs/28

Repository secret expected by the registry publish workflow:

REGISTRY_TOKEN

Local token note:

GITEA_TOKEN was visible in the final Codex process.
It was used only for Gitea API checks and not printed.

Issue Sync

Open upstream GitHub issues were reviewed on 2026-05-17 from:

https://github.com/calesthio/Crucix/issues

The upstream list contained 24 open issues. Issues already handled by this fork were not copied as open work, including the Docker stale-dashboard incident (#105), map label redraw (#70), Sensor Grid controls (#72), space display toggle (#51), source docs (#52), Dockge/CasaOS docs (#78), LLM timeout (#87), inject/static helper confusion (#100), network metrics (#101), Telegram polling backoff (#104), and briefing/evidence context (#75).

Issues not relevant to this fork were also not copied, including the Wallpaper Engine redesign (#41), the fork-inflation discussion (#107), empty/unclear placeholders (#79/#80), and the general use-case discussion (#93).

The following Gitea issues were created for real remaining work:

#1 Reddit source must stop unauthenticated .json scraping
   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/issues/1

#2 Send operator alerts when dashboard data remains stale
   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/issues/2

#3 ACLED credentialed integration needs regression test and diagnostics
   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/issues/3

#4 Complete memory and prediction loop beyond Phase-1 SQLite
   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/issues/4

#5 Remove old inline dashboard snapshot from production builds
   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/issues/5

#6 Harden Terminal Actions for public reverse-proxy deployments
   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/issues/6

#7 Replace ADS-B stub with real disabled/degraded source handling
   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/issues/7

#8 Clean inherited public-demo and upstream marketing references
   https://git.wilkensxl.de/Code-Inc/intelligence-terminal/issues/8

Verification Already Performed

Local lightweight checks:

npm run test:unit
npm audit --omit=dev --audit-level=high
docker compose --env-file .env.example config
node --check server.mjs
node --check dashboard/inject.mjs
node --check lib/llm/openai-compatible.mjs
git diff --check

Unit test result:

21 tests passing
0 failing

Audit result:

0 high vulnerabilities

Docker build and smoke test were performed locally earlier against the now-legacy mrsphay image namespace. Current deployments must use the code-inc image documented above:

docker build -t git.wilkensxl.de/mrsphay/intelligence-terminal:latest .
docker run --rm -d --name intelligence-terminal-smoke -p 127.0.0.1::3117 -e AUTO_OPEN_BROWSER=false git.wilkensxl.de/mrsphay/intelligence-terminal:latest

Smoke test observations:

Server booted.
No xdg-open error.
Initial sweep completed.
/api/health moved from starting to degraded with transparent source errors.
Degraded state was expected without all optional API keys.

Additional checks after fixing the dashboard live-data bug and Terminal Actions:

node --check server.mjs
npm run test:unit
docker compose --env-file .env.example config
git diff --check

The dashboard script was also syntax-checked after extracting script blocks from dashboard/public/jarvis.html.

Important Commits

7e85a54 chore: apply agent kit project structure
85f97bb feat: harden intelligence runtime and llm providers
42b7fc2 docs: add registry dockge and dashboard operations
d072390 ci: align gitea workflows with agent kit
0559481 ci: fix gitea registry publish login
f3c9331 ci: fix agent kit compliance checks
c2d572e fix: prepare runs volume before dropping privileges
8e096b2 ci: harden gitea workflow reruns
e933586 merge: reconcile main with production branch
4262c7e docs: expand agent handoff
53470cc fix: load live dashboard data and add terminal actions
d13652a merge: controlled Telegram terminal agent (PR #60)
0c7ddc5 feat: add privacy-aware security manager onboarding
b42b393 fix: prevent agent tool protocol leakage
e47c23e merge: prevent Telegram agent protocol leakage (PR #65)
994c806 feat: add DAVE adaptive synthetic persona
c07a652 merge: add DAVE adaptive synthetic persona (PR #68)

The large implementation commit 85f97bb and the dashboard/action fix 53470cc are contained in both:

origin/codex/production-intelligence-terminal
origin/main

How To Continue In A Fresh Codex Environment

Clone the Gitea repository:

git clone https://git.wilkensxl.de/Code-Inc/intelligence-terminal.git
cd intelligence-terminal
git checkout codex/production-intelligence-terminal

Confirm the expected commit:

git rev-parse HEAD

Expected after PR #62 merges:

The production branch should contain tool-agent commit d13652a and Security Manager commit 0c7ddc5 plus the handoff update/merge commit.

Read these files first:

AGENTS.md
.codex/project.md
docs/agent-handoff.md
README.md
.env.example

If checking Actions, use GITEA_TOKEN from the environment. Do not print it.

PowerShell check:

if ($env:GITEA_TOKEN) { "GITEA_TOKEN=set" } else { "GITEA_TOKEN=missing" }

Useful commands:

npm run test:unit
docker compose --env-file .env.example config
docker pull git.wilkensxl.de/code-inc/intelligence-terminal:latest

Start with Dockge/Pangolin using the README compose example and a .env based on .env.example.

Remaining Risks And Follow-Ups

Some sources will report degraded until optional keys are set, especially ACLED, FRED, EIA, and Cloudflare Radar.
OpenSky can rate-limit with HTTP 429; this is now visible in health instead of hidden.
GDELT/OFAC can time out under runner/network conditions; health reports this explicitly.
Browser-level visual verification of the full dashboard should be repeated after any future UI change.
The project still inherits the original Crucix broad source surface. Future work should prefer focused source-by-source tests over broad refactors.
If a new Codex environment sees non-fast-forward branch pushes, fetch first and preserve remote commits. Do not force-push without explicit approval.
A production deployment must add SECURITY_PROFILE_ENCRYPTION_KEY before expecting first-start onboarding. A changed or lost key intentionally fails closed and does not overwrite existing ciphertext.
Security Manager first-start onboarding remains intentionally incomplete until the operator answers the language and consent prompts in Telegram; no personal profile is created automatically.
The operator completed Security Manager onboarding before the #64 live deployment; the encrypted profile survived the container recreation. Never log or copy its contents into issues or handoff files.

Operator Pull Command

For deployment:

docker pull git.wilkensxl.de/code-inc/intelligence-terminal:latest

For a pinned deployment:

docker pull git.wilkensxl.de/code-inc/intelligence-terminal:20260703

25 KiB Raw Blame History