# Staging deploy (PM2 + full catalog pipeline)

End-to-end guide for **everything implemented in `business-mcp.lucos.com`** on a staging VM:

| Track | Tickets | What it does |
| ----- | ------- | -------------- |
| Schema harvest | TW-297 | `INFORMATION_SCHEMA` → `catalogs/schemas/` |
| Sensitivity review | TW-298 | Labels columns; gate before Qdrant |
| Catalog index | TW-299 | Embed reviewed schema → Qdrant `lucos_business_catalog` |
| `catalog_search` tool | TW-300 | MCP semantic search over that collection |
| MNGT RO views + tools | TW-309–313 | MySQL views + six entity read tools |
| cake RO views + tools | TW-314–317 | MySQL views on three instances + four reporting tools |
| MCP runtime | TW-323–326 | PM2 service, gateway authz, SSE/Streamable HTTP |

MNGT **database** steps (views, RO grants) are detailed in [`MNGT-DEPLOYMENT.md`](./MNGT-DEPLOYMENT.md).
cake **database** steps are in [`CAKE-RO-VIEWS.md`](./CAKE-RO-VIEWS.md) — note cake spans **three**
MySQL instances (Admin, whale, Shorty), so it needs three DSNs and three accounts, plus a nightly
job for its day-sharded views.

---

## Architecture

```
┌─────────────────────────────────────────────────────────────────────────┐
│ Staging VM (or operator workstation with VPN)                           │
│                                                                         │
│  CLI jobs (on demand / cron):                                           │
│    harvest:schemas → sensitivity:apply/gate → catalog:index             │
│                                                                         │
│  PM2 (always on):                                                       │
│    business-mcp-staging → catalog_search, MNGT tools, health_stub       │
│                                                                         │
│  nginx → 127.0.0.1:3333 (/sse, /mcp, /healthz)                          │
└─────────────────────────────────────────────────────────────────────────┘
         │                    │                    │
         ▼                    ▼                    ▼
   MySQL (staging)     Qdrant (staging)    api.lucos.com (staging gateway)
   - harvest RO DSN    lucos_business_     JWT authz + audit (TW-325)
   - MNGT lucos_ro_*   catalog             │
   views + RO DSN                          OpenAI API (embeddings)
```

**Catalog data flow:**

```
TW-297 harvest → TW-298 sensitivity → TW-299 index → TW-300 catalog_search
     │                  │                  │                  │
 catalogs/schemas/   overrides +      Qdrant points      MCP tool (PM2)
 staging/admin.json  gate pass        (vectors)          embeds query → search
```

**MNGT data flow** (parallel track):

```
TW-309 views (DBA) → TW-310 connector → TW-311–313 MCP tools (PM2)
```

Use **one PM2 instance** (`instances: 1`). MCP SSE is long-lived.

---

## Deployment order (first time)

Do these in order on staging:

| Step | Work | Doc section |
| ---- | ---- | ----------- |
| 1 | Install Node, PM2, clone repo | [§1 Install](#1-install) |
| 2 | Host env file (vault secrets) | [§2 Environment](#2-environment) |
| 3 | Qdrant reachable (local or shared staging) | [§3 Qdrant](#3-qdrant) |
| 4 | Schema harvest (staging Admin, then other targets) | [§4 Schema harvest](#4-schema-harvest-tw-297) |
| 5 | Sensitivity review + gate | [§5 Sensitivity review](#5-sensitivity-review-tw-298) |
| 6 | Catalog index into Qdrant | [§6 Catalog index](#6-catalog-index-tw-299) |
| 7 | MNGT views + RO user (DBA) | [`MNGT-DEPLOYMENT.md`](./MNGT-DEPLOYMENT.md) |
| 8 | PM2 start MCP server | [§7 PM2 MCP server](#7-pm2-mcp-server) |
| 9 | nginx TLS | [§8 nginx](#8-nginx) |
| 10 | Smoke all enabled tools | [§9 Smoke tests](#9-smoke-tests) |

You can run steps 4–6 before or in parallel with MNGT DB work. The MCP server can start earlier with only `health_stub`; enable other tools when their backends are ready.

---

## 1. Install

```bash
cd /opt/lucos/business-mcp.lucos.com
nvm install          # Node 24.19.0 — .nvmrc
npm ci
npm test             # recommended before first deploy
```

Install PM2 globally or via nvm: `npm install -g pm2`.

---

## 2. Environment (`.env` in repo root)

Use a **repo-root `.env`** for PM2 and CLI jobs. Node loads it via `--env-file-if-exists=.env` (npm scripts + `ecosystem.config.cjs`). **Never commit `.env`.**

```bash
cd /opt/lucos/business-mcp.lucos.com
cp deploy/business-mcp.staging.env.example .env
chmod 600 .env
# edit all CHANGE_ME values
```

No `source` / `/etc/lucos` step — just run `npm` / `pm2` from the repo root after `.env` exists.

Templates:

| File | Use |
| ---- | --- |
| [`deploy/business-mcp.staging.env.example`](../deploy/business-mcp.staging.env.example) | Staging values (gateway off mock, tools on) → copy to `.env` |
| [`.env.example`](../.env.example) | Local/dev defaults (mock broker, tools commented) |

After editing `.env`, reload PM2 so the process picks up changes:

```bash
pm2 reload business-mcp-staging --update-env
```

### Environment variables by track

| Track | Required vars | Used by |
| ----- | ------------- | ------- |
| MCP runtime | `MCP_HOST`, `MCP_PORT`, `MCP_ALLOWED_HOSTS`, `LUCOS_BROKER_MOCK_ALLOW`, `LUCOS_GATEWAY_*`, `LUCOS_POLICY_ENABLED`, `LUCOS_TOOL_ENABLE_*` | PM2 |
| Schema harvest | `SCHEMA_HARVEST_*_DSN_STAGING` per target | `npm run harvest:schemas` |
| Sensitivity | (none — reads catalogs in git) | `npm run sensitivity:*` |
| Catalog index | `OPENAI_API_KEY`, `QDRANT_URL`, `QDRANT_BUSINESS_COLLECTION` | `npm run catalog:index` |
| `catalog_search` | Same as index + `LUCOS_TOOL_ENABLE_CATALOG_SEARCH=true` | PM2 tool |
| MNGT tools | `MNGT_RO_DSN_STAGING` | PM2 tools |
| cake tools | `CAKE_{ADMIN,WHALE,SHORTY}_RO_DSN_STAGING` | PM2 tools |
| AdCenter (optional) | `ADCENTER_LEVEL1_KEYS_STAGING` | `adcenter_get_campaign` |

**Never on hosted staging:** `MNGT_RO_FIXTURE`, `CATALOG_SEARCH_FIXTURE`.

---

## 3. Qdrant

`catalog_search` and `catalog:index` need a Qdrant instance with collection **`lucos_business_catalog`** (separate from code RAG `adpilot_embeddings`).

**Option A — Qdrant on same VM (simple staging):**

```bash
docker run -d --name qdrant-staging -p 6333:6333 \
  -v qdrant_staging_data:/qdrant/storage \
  qdrant/qdrant:latest
```

Set `QDRANT_URL=http://127.0.0.1:6333` in the env file.

**Option B — Shared staging Qdrant:** set `QDRANT_URL` and `QDRANT_API_KEY` to your team host.

The index job creates the collection if missing (1536-dim cosine, `text-embedding-3-small`).

---

## 4. Schema harvest (TW-297)

Pulls **INFORMATION_SCHEMA only** into `catalogs/schemas/staging/{target}.json`. No business rows.

```bash
cd /opt/lucos/business-mcp.lucos.com
# Admin first (MNGT-aligned)
npm run harvest:schemas -- --env staging --target admin
# → catalogs/schemas/staging/admin.json
```

Other allowlisted targets (same pattern, set matching `SCHEMA_HARVEST_*_DSN_STAGING`):

```bash
npm run harvest:schemas -- --env staging --target shorty
npm run harvest:schemas -- --env staging --target adcenter
npm run harvest:schemas -- --env staging --target whale
npm run harvest:schemas -- --env staging --target keywords
```

**Fixture (no DB — CI/local only):**

```bash
npm run harvest:schemas -- --env staging --target admin \
  --fixture tests/fixtures/schema-harvest-admin.json
```

**After harvest:** commit reviewed catalogs to git (or keep on VM if your process keeps catalogs deploy-only). Re-harvest when schema changes; then re-run sensitivity + index.

Details: [`jobs/schema_harvest/README.md`](../jobs/schema_harvest/README.md).

---

## 5. Sensitivity review (TW-298)

Gate between harvest and Qdrant. Blocks `pii`, `secret`, `deny_index`, and unresolved `unknown` columns from indexing.

```bash
# Apply name rules + existing overrides
npm run sensitivity:apply -- \
  --catalog catalogs/schemas/staging/admin.json \
  --overrides catalogs/sensitivity/staging/admin.overrides.json

# Must pass before catalog:index
npm run sensitivity:gate -- --catalog catalogs/schemas/staging/admin.json
```

If gate fails: edit `catalogs/sensitivity/staging/admin.overrides.json` (human labels), re-run `sensitivity:apply` and `sensitivity:gate`.

Repeat per indexed target (`admin`, `shorty`, …).

Details: [`jobs/sensitivity_review/README.md`](../jobs/sensitivity_review/README.md).

---

## 6. Catalog index (TW-299)

Embeds **indexable** columns (public/internal only) into Qdrant. Requires gate pass + `OPENAI_API_KEY`.

```bash
# Dry-run (chunk counts, no network)
npm run catalog:index -- --catalog catalogs/schemas/staging/admin.json --dry-run

# Live index (OpenAI + Qdrant)
npm run catalog:index -- --catalog catalogs/schemas/staging/admin.json
```

Index additional targets after each passes sensitivity:

```bash
npm run catalog:index -- --catalog catalogs/schemas/staging/adcenter.json
# etc.
```

**Refresh cadence:** after schema harvest or sensitivity changes, re-run `catalog:index` for affected catalogs. The job deletes prior points for the same `environment` + `targetId` before upserting.

Details: [`jobs/catalog_index/README.md`](../jobs/catalog_index/README.md).

---

## 7. PM2 MCP server

After Qdrant is indexed (for `catalog_search`) and MNGT DSN is set (for entity tools):

```bash
cd /opt/lucos/business-mcp.lucos.com
# requires .env in this directory (see §2)
pm2 start ecosystem.config.cjs --only business-mcp-staging --env staging
pm2 save
pm2 startup          # run printed sudo command
```

Always use `--only business-mcp-staging` so production is not started. Do not `pm2 reload ecosystem.config.cjs` without `--only`.

Config: [`ecosystem.config.cjs`](../ecosystem.config.cjs).

```bash
pm2 status
pm2 logs business-mcp-staging
pm2 reload business-mcp-staging --update-env
```

**Staged tool rollout** — enable in `.env` or `policies/tool_enablement.json`, then `pm2 reload --update-env`:

1. `health_stub`
2. `catalog_search` (needs index + `OPENAI_API_KEY`)
3. MNGT tools (needs `MNGT_RO_DSN_STAGING`)
4. `adcenter_get_campaign` (needs AdCenter keys)
5. cake tools (needs all three `CAKE_*_RO_DSN_STAGING` **and** the nightly shard-view job on Shorty —
   without it, `get_postback_report` / `get_ivt_report` go stale the next day)

---

## 8. nginx

[`deploy/nginx-staging.conf.example`](../deploy/nginx-staging.conf.example) — TLS + SSE-friendly proxy.

```bash
sudo cp deploy/nginx-staging.conf.example /etc/nginx/sites-available/business-mcp-staging
sudo ln -sf /etc/nginx/sites-available/business-mcp-staging /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
```

| Path | Client |
| ---- | ------ |
| `https://<host>/mcp` | ChatGPT (streamable HTTP, **primary**) |
| `https://<host>/sse` | Cursor / legacy SSE |
| `https://<host>/healthz` | Liveness |

DEV Cursor: `https://dev-mcp.lucos.com/sse` (legacy) or `https://dev-mcp.lucos.com/mcp` (preferred). Reload the MCP server in Cursor after token/server changes if discovery times out.

Set `MCP_ALLOWED_HOSTS=<public-hostname>` in `.env` (e.g. `dev-mcp.lucos.com`). Binding stays `127.0.0.1`; nginx forwards `Host: dev-mcp.lucos.com`, and the MCP SDK rejects unknown Host headers otherwise (`Invalid Host`).

---

## 9. Smoke tests

### Service health

```bash
curl -fsS http://127.0.0.1:3333/healthz
curl -fsS https://business-mcp-staging.lucos.com/healthz
```

### `catalog_search` (TW-300)

From Cursor MCP or any MCP client (tool must be enabled):

```json
{
  "query": "publisher agreement columns",
  "limit": 5,
  "environment": "staging"
}
```

**Pass criteria:**

- Returns hits from `lucos_business_catalog` with schema/table metadata
- No row data, no blocked column names in payloads
- Fails clearly if `OPENAI_API_KEY` or Qdrant is misconfigured

**CLI sanity (on VM, same env):**

```bash
npm run catalog:index -- --catalog catalogs/schemas/staging/admin.json --dry-run
```

### MNGT tools

See [`MNGT-DEPLOYMENT.md`](./MNGT-DEPLOYMENT.md) §3.3. Example:

```json
{ "adv_id": 19880, "environment": "staging" }
```

### `health_stub`

```json
{ "ping": "staging" }
```

---

## Usage (ongoing operations)

### Refresh schema catalog (schema changed in staging DB)

```bash
npm run harvest:schemas -- --env staging --target admin
npm run sensitivity:apply -- \
  --catalog catalogs/schemas/staging/admin.json \
  --overrides catalogs/sensitivity/staging/admin.overrides.json
npm run sensitivity:gate -- --catalog catalogs/schemas/staging/admin.json
npm run catalog:index -- --catalog catalogs/schemas/staging/admin.json
# commit catalogs/overrides to git if that is your source of truth
```

No PM2 reload needed for catalog refresh — `catalog_search` reads Qdrant live.

### Deploy code updates

```bash
cd /opt/lucos/business-mcp.lucos.com
git pull
npm ci
npm test              # optional
# keep existing .env (gitignored); edit if new vars were added
pm2 reload business-mcp-staging --update-env
```

### Cursor MCP config

```json
{
  "mcpServers": {
    "lucos-business-staging": {
      "url": "https://business-mcp-staging.lucos.com/mcp"
    }
  }
}
```

Send `Authorization: Bearer <Lucos JWT>` on tool calls when gateway is live (TW-325).

### What each MCP tool needs at runtime

| Tool | Backend |
| ---- | ------- |
| `health_stub` | None |
| `catalog_search` | OpenAI embed + Qdrant `lucos_business_catalog` |
| `get_advertiser` … `get_publisher_spaf_status` | `MNGT_RO_DSN_STAGING` + views |
| `adcenter_get_campaign` | `ADCENTER_LEVEL1_KEYS_STAGING` |

---

## Rollback

```bash
# Kill switch — all tools
export LUCOS_POLICY_ENABLED=false
pm2 reload business-mcp-staging --update-env

# Revert app
git checkout <previous-sha>
npm ci
pm2 reload business-mcp-staging --update-env
```

Qdrant points persist until re-indexed; disabling `catalog_search` stops queries without deleting vectors.

---

## Production (later)

- Use a production `.env` on the prod host (same keys; never reuse staging secrets)
- `SCHEMA_HARVEST_ALLOW_PROD=true` for prod harvest jobs
- `MNGT_RO_DSN_PROD` + `MNGT_RO_ALLOW_PROD=true`
- `pm2 start ecosystem.config.cjs --only business-mcp-production --env production`
- See [`MNGT-DEPLOYMENT.md`](./MNGT-DEPLOYMENT.md) for prod MNGT DB steps

---

## References

| Topic | Path |
| ----- | ---- |
| PM2 config | [`ecosystem.config.cjs`](../ecosystem.config.cjs) |
| Staging `.env` template | [`deploy/business-mcp.staging.env.example`](../deploy/business-mcp.staging.env.example) → `.env` |
| nginx example | [`deploy/nginx-staging.conf.example`](../deploy/nginx-staging.conf.example) |
| Schema harvest | [`jobs/schema_harvest/README.md`](../jobs/schema_harvest/README.md) |
| Sensitivity | [`jobs/sensitivity_review/README.md`](../jobs/sensitivity_review/README.md) |
| Catalog index | [`jobs/catalog_index/README.md`](../jobs/catalog_index/README.md) |
| MNGT DB + tools | [`MNGT-DEPLOYMENT.md`](./MNGT-DEPLOYMENT.md) |
| All env keys | [`.env.example`](../.env.example) |

Tickets: TW-297 · TW-298 · TW-299 · TW-300 · TW-309–313 · TW-323–326