Merge remote-tracking branch 'origin/codex/production-intelligence-terminal' into codex/issue-17-sse-heartbeat
# Conflicts: # README.md # test/fetch-utils.test.mjs
This commit is contained in:
MrSphay
122
README.md
122
README.md
@@ -1,13 +1,11 @@
|
||||
<div align="center">
|
||||
|
||||
# Crucix
|
||||
# Intelligence Terminal
|
||||
|
||||
**Your own intelligence terminal. 27 sources. One command. Zero cloud.**
|
||||
**Self-hosted intelligence dashboard. 27 open sources. Docker-first. No telemetry.**
|
||||
|
||||
## [Visit The Live Site: crucix.live](https://www.crucix.live/)
|
||||
|
||||
[](https://www.crucix.live/)
|
||||
[](https://www.crucix.live/)
|
||||
[](https://git.wilkensxl.de/MrSphay/intelligence-terminal)
|
||||
[](https://git.wilkensxl.de/MrSphay/-/packages/container/intelligence-terminal/latest)
|
||||
|
||||
[](#quick-start)
|
||||
[](LICENSE)
|
||||
@@ -15,12 +13,7 @@
|
||||
[](#data-sources-27)
|
||||
[](#docker)
|
||||
|
||||
**Enter The Signal Network**
|
||||
|
||||
[](https://x.com/crucixmonitor)
|
||||
[](https://discord.gg/ChVy7SF4)
|
||||
|
||||
|
||||
|
||||
|
||||
<details>
|
||||
<summary>More screenshots</summary>
|
||||
@@ -37,22 +30,24 @@
|
||||
|
||||
</div>
|
||||
|
||||
> **Live website:** [https://www.crucix.live/](https://www.crucix.live/)
|
||||
> Explore the public demo first, then clone the repo to run Crucix locally.
|
||||
> **Supported deployment:** private home-server or lab deployment through Docker, Dockge, Pangolin, or local Node.js.
|
||||
> Runtime data stays in your configured `runs/` volume and API keys are operator-owned.
|
||||
> **Source:** [git.wilkensxl.de/MrSphay/intelligence-terminal](https://git.wilkensxl.de/MrSphay/intelligence-terminal)
|
||||
> Pull the image or clone the repository to run Intelligence Terminal on your own infrastructure.
|
||||
|
||||
Crucix pulls satellite fire detection, flight tracking, radiation monitoring, satellite constellation tracking, economic indicators, live market prices, conflict data, sanctions lists, and social sentiment from 27 open-source intelligence feeds — in parallel, every 15 minutes — and renders everything on a single self-contained Jarvis-style dashboard.
|
||||
Intelligence Terminal pulls satellite fire detection, flight tracking, radiation monitoring, satellite constellation tracking, economic indicators, live market prices, conflict data, sanctions lists, and social sentiment from 27 open-source intelligence feeds in parallel, every 15 minutes, and renders everything on a single self-contained dashboard.
|
||||
|
||||
Hook it up to an LLM and it becomes a **two-way intelligence assistant** — pushing multi-tier alerts to Telegram and Discord when something meaningful changes, responding to commands like `/brief` and `/sweep` from your phone, and generating actionable trade ideas grounded in real cross-domain data. Your own analyst that watches the world while you sleep.
|
||||
Hook it up to an LLM and it becomes a **two-way intelligence assistant**: pushing multi-tier alerts to Telegram and Discord when something meaningful changes, responding to commands like `/brief` and `/sweep` from your phone, and generating trade ideas grounded in real cross-domain data.
|
||||
|
||||
Try the live demo first at [https://www.crucix.live/](https://www.crucix.live/), then clone the repo when you want the full local stack.
|
||||
Run it locally with Node.js or pull the published Docker image for a home-server deployment.
|
||||
|
||||
No cloud. No telemetry. No subscriptions. Just `node server.mjs` and you're running.
|
||||
|
||||
## Token / Asset Warning
|
||||
|
||||
> [!WARNING]
|
||||
> **Crucix has not launched any official token, coin, NFT, airdrop, presale, or other blockchain-based asset.**
|
||||
> Any token or digital asset using the Crucix name, logo, or branding is not affiliated with or endorsed by Crucix.
|
||||
> **Intelligence Terminal has not launched any official token, coin, NFT, airdrop, presale, or other blockchain-based asset.**
|
||||
> Any token or digital asset using the Intelligence Terminal or Crucix name, logo, or branding is not affiliated with or endorsed by this project.
|
||||
> Do not buy it, promote it, connect a wallet to claim it, sign transactions, or send funds based on third-party posts, DMs, or websites.
|
||||
|
||||
---
|
||||
@@ -61,7 +56,7 @@ No cloud. No telemetry. No subscriptions. Just `node server.mjs` and you're runn
|
||||
|
||||
Most of the world's real-time intelligence — satellite imagery, radiation levels, conflict events, economic indicators, flight tracking, maritime activity — is publicly available. It's just scattered across dozens of government APIs, research institutions, and open data feeds that nobody has time to check individually.
|
||||
|
||||
Crucix brings it all into one place. Not behind a paywall, not locked in an enterprise platform, not requiring a security clearance. Just open data, aggregated and cross-correlated on your own machine, updated every 15 minutes.
|
||||
Intelligence Terminal brings it all into one place. Not behind a paywall, not locked in an enterprise platform, not requiring a security clearance. Just open data, aggregated and cross-correlated on your own machine, updated every 15 minutes.
|
||||
|
||||
It was built for anyone who wants to understand what's actually happening in the world right now — researchers, journalists, traders, OSINT analysts, or just curious people who believe access to information shouldn't depend on your budget.
|
||||
|
||||
@@ -71,8 +66,8 @@ It was built for anyone who wants to understand what's actually happening in the
|
||||
|
||||
```bash
|
||||
# 1. Clone the repo
|
||||
git clone https://github.com/calesthio/Crucix.git
|
||||
cd Crucix
|
||||
git clone https://git.wilkensxl.de/MrSphay/intelligence-terminal.git
|
||||
cd intelligence-terminal
|
||||
|
||||
# 2. Install dependencies (just Express)
|
||||
npm install
|
||||
@@ -135,6 +130,8 @@ PORT=3117
|
||||
REFRESH_INTERVAL_MINUTES=15
|
||||
AUTO_OPEN_BROWSER=false
|
||||
STALE_DATA_MAX_AGE_MINUTES=60
|
||||
STALE_ALERT_COOLDOWN_MINUTES=60
|
||||
DASHBOARD_URL=https://intelligence.example.internal
|
||||
TERMINAL_ACTIONS_ENABLED=true
|
||||
SWEEP_TOKEN=
|
||||
SSE_HEARTBEAT_INTERVAL_MS=25000
|
||||
@@ -189,6 +186,8 @@ LLM_MODEL=your-model
|
||||
|
||||
For Pangolin or another reverse proxy, forward HTTP traffic to `intelligence-terminal:3117` (or the `PORT` you set). Missing API keys do not crash sweeps; affected sources are reported as degraded in `/api/health`.
|
||||
|
||||
When data remains stale past `STALE_DATA_MAX_AGE_MINUTES`, the server sends an operator alert through configured Telegram/Discord channels after failed or degraded sweep attempts. `STALE_ALERT_COOLDOWN_MINUTES` prevents repeated stale alerts from spamming every refresh interval. Set `DASHBOARD_URL` to the Pangolin/public URL you want included in those alerts.
|
||||
|
||||
The dashboard Terminal Actions panel can trigger `status`, `sweep`, and `brief` through `/api/action`. Leave `TERMINAL_ACTIONS_ENABLED=true` for a private home-server deployment. For an internet-exposed deployment, set `SWEEP_TOKEN` and pass it through trusted automation, or set `TERMINAL_ACTIONS_ENABLED=false` to disable browser-triggered actions. If you protect actions with `SWEEP_TOKEN`, the browser can send it from `localStorage.crucix_sweep_token`.
|
||||
|
||||
#### Reverse Proxy SSE
|
||||
@@ -205,6 +204,39 @@ Recommended proxy settings:
|
||||
|
||||
If you raise the heartbeat interval, keep it shorter than the lowest idle timeout in the proxy chain.
|
||||
|
||||
#### Scenario Watchlist
|
||||
|
||||
Intelligence Terminal can track operator hypotheses across sweeps with a runtime scenario file at `runs/scenarios.json`. On first run, the server creates three disabled starter examples:
|
||||
|
||||
- Middle East energy shock
|
||||
- Macro stress spillover
|
||||
- Regional escalation risk
|
||||
|
||||
Enable or add scenarios by editing `runs/scenarios.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"scenarios": [
|
||||
{
|
||||
"id": "middle-east-energy-shock",
|
||||
"enabled": true,
|
||||
"name": "Middle East energy shock",
|
||||
"description": "Energy supply risk building from regional conflict.",
|
||||
"regions": ["Middle East", "Iran", "Strait of Hormuz"],
|
||||
"categories": ["osint", "energy", "maritime"],
|
||||
"keywords": ["missile", "strike", "hormuz", "oil"],
|
||||
"thresholds": { "watching": 2, "building": 4, "confirmed": 7 },
|
||||
"invalidation": "WTI normalizes and urgent regional signals fade."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Malformed scenario config degrades safely: sweeps continue and the dashboard shows the watchlist as a config issue. Scenario state is persisted in `runs/scenario-state.json`; delete that file to reset state transitions without deleting definitions.
|
||||
|
||||
Scenario states are `dormant`, `watching`, `building`, and `confirmed`. The dashboard shows active scenario state, confidence, score, and recent trigger time. Briefings include a `Scenario Watchlist` section when one or more scenarios change state.
|
||||
|
||||
#### Build And Publish Your Gitea Image
|
||||
|
||||
```bash
|
||||
@@ -263,7 +295,7 @@ The server runs a sweep cycle every 15 minutes (configurable). Each cycle:
|
||||
6. Pushes update to all connected browsers via SSE
|
||||
|
||||
### Telegram Bot (Two-Way)
|
||||
Crucix doubles as an interactive Telegram bot. Beyond sending alerts, it responds to commands directly from your chat:
|
||||
Intelligence Terminal doubles as an interactive Telegram bot. Beyond sending alerts, it responds to commands directly from your chat:
|
||||
|
||||
| Command | What It Does |
|
||||
|---------|-------------|
|
||||
@@ -280,7 +312,7 @@ This requires `TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHAT_ID` in `.env`. The bot pol
|
||||
|
||||
### Discord Bot (Two-Way)
|
||||
|
||||
Crucix also supports Discord as a full-featured bot with slash commands and rich embed alerts. It mirrors the Telegram bot's capabilities with Discord-native formatting.
|
||||
Intelligence Terminal also supports Discord as a full-featured bot with slash commands and rich embed alerts. It mirrors the Telegram bot's capabilities with Discord-native formatting.
|
||||
|
||||
| Command | What It Does |
|
||||
|---------|-------------|
|
||||
@@ -295,7 +327,7 @@ Alerts are delivered as rich embeds with color-coded sidebars: red for FLASH, ye
|
||||
|
||||
**Webhook fallback:** If you don't want to run a full bot, set `DISCORD_WEBHOOK_URL` instead. This enables one-way alerts (no slash commands) with zero dependencies — no `discord.js` needed.
|
||||
|
||||
**Optional dependency:** The full bot requires `discord.js`. Install it with `npm install discord.js`. If it's not installed, Crucix automatically falls back to webhook-only mode.
|
||||
**Optional dependency:** The full bot requires `discord.js`. Install it with `npm install discord.js`. If it's not installed, Intelligence Terminal automatically falls back to webhook-only mode.
|
||||
|
||||
### Optional LLM Layer
|
||||
Connect cloud or local OpenAI-compatible LLM providers for enhanced analysis:
|
||||
@@ -342,9 +374,12 @@ These three unlock the most valuable economic and satellite data. Each takes abo
|
||||
|
||||
| Key | Source | How to Get |
|
||||
|-----|--------|------------|
|
||||
| `ACLED_EMAIL` + `ACLED_PASSWORD` | Armed conflict event data | [acleddata.com/register](https://acleddata.com/register/) — free, OAuth2 |
|
||||
| `ACLED_EMAIL` + `ACLED_PASSWORD` | Armed conflict event data | [acleddata.com/register](https://acleddata.com/register/) — free, OAuth2. `ACLED_USER` / `ACLED_USERNAME` are accepted as email aliases |
|
||||
| `AISSTREAM_API_KEY` | Maritime AIS vessel tracking | [aisstream.io](https://aisstream.io/) — free |
|
||||
| `ADSB_API_KEY` | Unfiltered flight tracking | [RapidAPI](https://rapidapi.com/adsbexchange/api/adsbexchange-com1) — ~$10/mo |
|
||||
| `REDDIT_CLIENT_ID` + `REDDIT_CLIENT_SECRET` | Reddit social sentiment | [reddit.com/prefs/apps](https://www.reddit.com/prefs/apps/) — create a script app |
|
||||
|
||||
Reddit is OAuth-only in this fork. If the Reddit credentials are missing or rejected, the Reddit source is reported as degraded and no unauthenticated `reddit.com/.../hot.json` fallback is used.
|
||||
|
||||
### LLM Provider (optional, for AI-enhanced ideas)
|
||||
|
||||
@@ -393,14 +428,14 @@ Alerts work with or without an LLM on both Telegram and Discord. With an LLM con
|
||||
|
||||
### Without Any Keys
|
||||
|
||||
Crucix still works with zero API keys. 18+ sources require no authentication at all. Sources that need keys return structured errors and the rest of the sweep continues normally.
|
||||
Intelligence Terminal still works with zero API keys. 18+ sources require no authentication at all. Sources that need keys return structured errors and the rest of the sweep continues normally.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
crucix/
|
||||
intelligence-terminal/
|
||||
├── server.mjs # Express dev server (SSE, auto-refresh, LLM, bot commands)
|
||||
├── crucix.config.mjs # Configuration with env var overrides + delta thresholds
|
||||
├── diag.mjs # Diagnostic script — run if server fails to start
|
||||
@@ -540,6 +575,9 @@ All settings are in `.env` with sensible defaults:
|
||||
|----------|---------|-------------|
|
||||
| `PORT` | `3117` | Dashboard server port |
|
||||
| `REFRESH_INTERVAL_MINUTES` | `15` | Auto-refresh interval |
|
||||
| `STALE_DATA_MAX_AGE_MINUTES` | `60` | Data age threshold for stale health state |
|
||||
| `STALE_ALERT_COOLDOWN_MINUTES` | `60` | Minimum time between repeated operator stale-data alerts |
|
||||
| `DASHBOARD_URL` | local URL | Dashboard URL included in operator alerts |
|
||||
| `LLM_PROVIDER` | disabled | `anthropic`, `openai`, `gemini`, `codex`, `openrouter`, `minimax`, `mistral`, or `grok` |
|
||||
| `LLM_API_KEY` | — | API key (not needed for codex) |
|
||||
| `LLM_MODEL` | per-provider default | Override model selection |
|
||||
@@ -589,7 +627,7 @@ This tests every import one by one, checks your Node.js version, and verifies po
|
||||
|
||||
**3. Check if port 3117 is already in use:**
|
||||
|
||||
A previous Crucix instance may still be running in the background.
|
||||
A previous Intelligence Terminal instance may still be running in the background.
|
||||
|
||||
```powershell
|
||||
# Windows PowerShell
|
||||
@@ -611,7 +649,7 @@ Then try starting again. You can also change the port by setting `PORT=3118` in
|
||||
```bash
|
||||
node --version
|
||||
```
|
||||
Crucix requires Node.js 22 or later. If you have an older version, download the latest LTS from [nodejs.org](https://nodejs.org/).
|
||||
Intelligence Terminal requires Node.js 22 or later. If you have an older version, download the latest LTS from [nodejs.org](https://nodejs.org/).
|
||||
|
||||
### Dashboard shows empty panels after first start
|
||||
|
||||
@@ -621,11 +659,11 @@ This is normal — the first sweep takes 30–60 seconds to query all 27 sources
|
||||
|
||||
Expected behavior. Sources that require API keys will return structured errors if the key isn't set. The rest of the sweep continues normally. Check the Source Integrity section in the dashboard (or the server logs) to see which sources failed and why. The 3 most impactful free keys to add are `FRED_API_KEY`, `FIRMS_MAP_KEY`, and `EIA_API_KEY`.
|
||||
|
||||
OpenSky can also return `HTTP 429` when its public hotspots are queried too aggressively. Crucix does not try to evade that limit. Instead, it surfaces the throttle/error in source health and preserves the most recent non-empty air traffic snapshot from `runs/` so the dashboard flight layer does not suddenly go blank on a throttled sweep.
|
||||
OpenSky can also return `HTTP 429` when its public hotspots are queried too aggressively. Intelligence Terminal does not try to evade that limit. Instead, it surfaces the throttle/error in source health and preserves the most recent non-empty air traffic snapshot from `runs/` so the dashboard flight layer does not suddenly go blank on a throttled sweep.
|
||||
|
||||
### Telegram bot not responding to commands
|
||||
|
||||
Make sure both `TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHAT_ID` are set in `.env`. The bot only responds to messages from the configured chat ID (security measure). You should see `[Crucix] Telegram alerts enabled` and `[Crucix] Bot command polling started` in the server logs on startup. If not, double-check your token with `curl https://api.telegram.org/bot<YOUR_TOKEN>/getMe`.
|
||||
Make sure both `TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHAT_ID` are set in `.env`. The bot only responds to messages from the configured chat ID (security measure). You should see Telegram alert and bot polling startup lines in the server logs. If not, double-check your token with `curl https://api.telegram.org/bot<YOUR_TOKEN>/getMe`.
|
||||
|
||||
### Discord bot not responding to slash commands
|
||||
|
||||
@@ -656,29 +694,21 @@ To update them: run the dashboard, wait for a sweep to complete, then use your b
|
||||
|
||||
## Contributing
|
||||
|
||||
Found a bug? Want to add a 28th source? PRs welcome. Each source is a standalone module in `apis/sources/` — just export a `briefing()` function that returns structured data and add it to the orchestrator in `apis/briefing.mjs`.
|
||||
Found a bug? Want to add a 28th source? PRs welcome. Each source is a standalone module in `apis/sources/` - just export a `briefing()` function that returns structured data and add it to the orchestrator in `apis/briefing.mjs`.
|
||||
|
||||
If you find this useful, a star helps others find it too.
|
||||
If you find this useful, a star on the Gitea repository helps other operators find it too.
|
||||
|
||||
For contribution guidelines, review expectations, and source-add rules, see `CONTRIBUTING.md`. For security reports, see `SECURITY.md`.
|
||||
|
||||
## Contact
|
||||
|
||||
For partnerships, integrations, or other non-issue inquiries, you can reach me at `celesthioailabs@gmail.com`.
|
||||
For bugs, feature requests, and integration ideas, use the Gitea issue tracker so discussion stays visible and actionable:
|
||||
|
||||
For bugs and feature requests, please use GitHub Issues so discussion stays visible and actionable.
|
||||
https://git.wilkensxl.de/MrSphay/intelligence-terminal/issues
|
||||
|
||||
---
|
||||
## Upstream And License
|
||||
|
||||
## Star History
|
||||
|
||||
<a href="https://www.star-history.com/?repos=calesthio%2FCrucix&type=date&legend=top-left">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/image?repos=calesthio/Crucix&type=date&theme=dark&legend=top-left" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/image?repos=calesthio/Crucix&type=date&legend=top-left" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/image?repos=calesthio/Crucix&type=date&legend=top-left" />
|
||||
</picture>
|
||||
</a>
|
||||
Intelligence Terminal is an AGPL-3.0-only Crucix fork focused on Docker-first home-server operation, source health transparency, Gitea Registry delivery, and operator-owned deployments. Upstream project credit remains with the original Crucix project.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user