> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hoox.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# monitor-trading

# 📈 Monitoring Operations

Algorithmic trading demands high-integrity, real-time observability. Because Hoox microservices are distributed across Cloudflare's global edge network, tracking health, logs, and message queues requires a consolidated management plane.

This guide outlines how to use the Hoox CLI to audit system status, monitor queue backlogs, stream live edge logs, and manage the emergency kill switch.

***

## 🟢 1. Probing Worker Health Status

Every Hoox worker is configured with a secure `/health` endpoint that validates local CPU capacity, binding access, and D1 database connection integrity.

To probe and audit all active endpoints simultaneously:

```bash theme={null}
hoox check health
```

> `hoox monitor status` was removed in v0.8.0. The single source of truth for health checks is now `hoox check health`.

### Expected Output

The CLI runs asynchronous concurrent health probes, displaying green indicators and public route locations:

```text theme={null}
┌─────────────────────────────────────────────────────────────┐
│               Hoox Microservice Health Probes                │
├─────────────────────────────────────────────────────────────┤
│  hoox (Gateway)     ... ✅ OK  https://hoox.alpha.workers.dev│
│  trade-worker       ... ✅ OK  (Internal Binding Verified)   │
│  d1-worker          ... ✅ OK  (Internal Binding Verified)   │
│  telegram-worker    ... ✅ OK  (Internal Binding Verified)   │
│  agent-worker       ... ✅ OK  (Internal Binding Verified)   │
│  web3-wallet-worker ... ✅ OK  (Internal Binding Verified)   │
│  email-worker       ... ✅ OK  (Internal Binding Verified)   │
│  report-worker      ... ✅ OK  (Internal Binding Verified)   │
│  analytics-worker   ... ✅ OK  (Internal Binding Verified)   │
│                                                             │
│  System Status: HEALTHY (0 warnings, 0 errors)              │
└─────────────────────────────────────────────────────────────┘
```

### Interpreting Health States

| Indicator   | Meaning                               | Action Required                            |
| ----------- | ------------------------------------- | ------------------------------------------ |
| ✅ OK        | All bindings active, D1 responsive    | None — operating normally                  |
| ⚠️ DEGRADED | Some bindings unavailable             | Check `hoox logs tail <worker>` for errors |
| ❌ FAILED    | Worker unreachable or D1 disconnected | Run `hoox repair check` immediately        |
| 🔄 STARTING | Worker is still booting after deploy  | Wait 30–60 seconds and retry               |

***

## 🚨 2. Governing the Global Kill Switch

The **Global Kill Switch** is your emergency brake. Stored inside the sub-millisecond `CONFIG_KV` namespace, flipping this parameter instantly blocks all incoming trade signals globally in under 10 seconds, without requiring code updates or worker redeployments.

```bash theme={null}
hoox monitor kill-switch show

# B. Emergency HALT - disable all edge trade execution immediately
hoox monitor kill-switch on

# C. Resume normal operations
hoox monitor kill-switch off
```

### Kill Switch Behavior

When activated, the following sequence occurs automatically:

<ol>
  <li>
    <code>hoox</code> gateway rejects all incoming <code>/webhook</code> POST
    requests with <code>503 Service Unavailable</code>.
  </li>

  <li>
    A <code>KILL\_SWITCH\_ACTIVE</code> event is logged to D1 for audit trail.
  </li>

  <li>
    <code>agent-worker</code> halts its 5-minute cron execution loop.
  </li>

  <li>
    <code>trade-worker</code> queues any in-flight payloads for retry (not
    abandoned).
  </li>

  <li>Telegram notification is sent to alert you.</li>
</ol>

<Warning>
  When the Kill Switch is turned `ON`, the `hoox` gateway router immediately
  rejects all incoming TradingView alerts and API webhook requests with a `503
      Service Unavailable` error and logs a `KILL_SWITCH_ACTIVE` warning. Active
  positions must be flattened manually or via `hoox monitor trades close-all`.
</Warning>

***

## 🗃️ 3. Auditing Recent Transactions

To review execution history directly from your terminal:

```bash theme={null}
# Print the 10 most recent trades executed across all exchanges
hoox monitor trades

# Print the 50 most recent trades
hoox monitor trades 50

# Filter by specific exchange
hoox monitor trades --exchange bybit

# Filter by date range (ISO format)
hoox monitor trades --from 2026-05-15 --to 2026-05-20
```

This aggregates entries from your production D1 database and prints a formatted terminal table outlining: `TIMESTAMP | REQUEST_ID | EXCHANGE | SYMBOL | ACTION | QUANTITY | PRICE | STATUS`

### Trade Status Codes

| Status     | Meaning                                        |
| ---------- | ---------------------------------------------- |
| `Filled`   | Order fully executed on the exchange           |
| `Partial`  | Only portion filled; remainder queued          |
| `Rejected` | Exchange returned a business logic error       |
| `Failed`   | Max queue retries exhausted; logged for review |
| `Enqueued` | Submitted to Cloudflare Queues for async retry |

***

## 📨 4. Inspecting Queue Depth (Guaranteed Delivery Backlog)

If network volatility causes exchange API dropouts, Hoox offloads trade execution payloads to Cloudflare Queues. To check if there is an active execution backlog:

```bash theme={null}
hoox monitor queue-depth
```

This displays the number of pending messages in the queue:

* **`0`**: System is running normally (all trades executing on the fast-path direct service binding).
* **`> 0`**: Exchange API is experiencing rate-limits or downtime. Trade-worker is retrying transactions asynchronously in the background.

### Queue Health Indicators

| Queue Depth |  Risk Level | Recommended Action                                                                                 |
| :---------: | :---------: | -------------------------------------------------------------------------------------------------- |
|      0      |  🟢 Normal  | No action needed                                                                                   |
|     1–5     | 🟡 Watchful | Monitor exchange health pages                                                                      |
|     6–20    | 🟠 Elevated | Check exchange maintenance schedule; consider `hoox config kv set webhooks:queue_mode direct_only` |
|     20+     | 🔴 Critical | Verify exchange API keys are valid; check rate limit headers; scale manually if needed             |

***

## 📝 5. Real-Time Log Streaming & Telemetry

When debugging custom strategies, you can stream verbose console logs directly from Cloudflare's global edge nodes to your local terminal using Wrangler bindings:

```bash theme={null}
# A. Stream live console logs for a specific worker
hoox logs tail trade-worker

# B. Stream gateway traffic in real-time
hoox logs tail hoox

# C. Stream all workers simultaneously (diagnostic mode)
hoox logs tail --all

# D. Filter logs by severity level
hoox logs tail trade-worker --level error
```

### Downloading Logs for Off-Line Analysis

To download historical logs offloaded to your R2 storage bucket for compliance audits or tax reports:

```bash theme={null}
hoox logs download hoox --output logs/gateway-backup.log

# Download trade-worker logs for the last 24 hours
hoox logs download trade-worker --output logs/trade-execution.log --since 24h
```

***

## 📊 6. System Metrics Overview

Hoox tracks key performance indicators via Cloudflare Analytics Engine. Access summary metrics via:

```bash theme={null}
# View current system metrics snapshot
hoox monitor metrics

# Or use the Dashboard Command Center for visual charts
hoox dashboard
```

### Key Metrics Tracked

| Metric           | Source            | Description                      |
| ---------------- | ----------------- | -------------------------------- |
| Orders/Min       | analytics-worker  | Trade execution throughput       |
| Avg Latency (ms) | analytics-worker  | Signal-to-fill round-trip time   |
| Error Rate (%)   | analytics-worker  | Failed orders / total attempts   |
| Queue Depth      | Cloudflare Queues | Pending execution backlog        |
| Daily P\&L       | D1 Database       | Realized profit/loss calculation |
| Drawdown %       | agent-worker      | Current daily asset drawdown     |

***

<Tip>
  If the system status check returns a red error indicator (e.g. `d1-worker ...
      ❌ FAILED`), immediately proceed to the [**Self-Healing & Repair
  Guide**](repair) to run automated diagnostics and restore service bindings!
</Tip>

### 🔗 Next Steps

* [**Self-Healing & Repair**](repair) — Diagnose connection drops, recreate broken bindings, and rebuild environments.
* [**Terminal UI Operations**](tui) — Monitor health status, tail logs, and edit configurations interactively in a full-screen GUI cockpit.
* [**Infrastructure Management**](manage-infra) — Manage KV config namespaces, Queue parameters, and R2 storage buckets.