# MongoDB — metadata store (TW-19)

## Decision

**MongoDB** is the persistent metadata store for Repo Sync (TW-19). It holds repository records, snapshots, and sync job history — not git file content and not Redis stream events.

| Store | Technology | Purpose |
| --- | --- | --- |
| Git clones | Filesystem (`GIT_WORKSPACE_PATH`) | Source tree on disk |
| Event bus | Redis Streams | Downstream indexing events |
| **Metadata** | **MongoDB** | Repositories, snapshots, sync jobs (TW-19) |

## Environments

| Environment | MongoDB | Notes |
| --- | --- | --- |
| **Local dev** | Docker `mongo:7` in this repo's `docker-compose.yml` | Port **27018** on host (avoids clashing with other projects on 27017) |
| **Staging / prod** | Dedicated MongoDB Atlas cluster or database | Separate from unrelated Atlas projects |

Do **not** reuse another project's Atlas database or local Mongo instance — use a dedicated database name (`adpilot_repo_sync`) even if sharing a cluster later.

## Local Docker setup (this project)

From `adpilot-indexing-repo-sync.com/`:

```sh
# MongoDB only (e.g. app run via PM2 or make dev)
docker compose up mongodb -d

# Full stack: Mongo + Redis + app
docker compose up --build
```

Default connection strings:

| Where app runs | `MONGO_URI` |
| --- | --- |
| Host (`make dev`, PM2) | `mongodb://localhost:27018/adpilot_repo_sync` |
| Inside compose (`repo-sync-service`) | `mongodb://mongodb:27017/adpilot_repo_sync` |

Verify Mongo is up:

```sh
docker compose ps mongodb
docker compose exec mongodb mongosh --eval 'db.runCommand({ ping: 1 })'
```

Optional UI (dev profile):

```sh
docker compose --profile dev up mongo-express -d
# http://localhost:8082  (user/pass: dev/dev — local only)
```

## Database layout

Database: `adpilot_repo_sync`

| Collection | Documents |
| --- | --- |
| `repositories` | `repo_id`, clone URL, default ref, `last_commit_sha`, `last_sync_at`, status |
| `snapshots` | `snapshot_id`, `repo_id`, `commit_sha`, `ref`, `file_count`, status, timestamps |
| `sync_jobs` | `job_id`, `repo_id`, `ref`, status, error, started/completed |
| `indexing_runs` | `indexing_run_id`, `job_id`, `repo_id`, `snapshot_id`, mode (`fresh`/`reindex`), expected/processed counts, per-stage status (TW-92) |
| `snapshot_files` | per-file inventory for a snapshot: `file_path`, `file_kind`, `change_type`, `status` (TW-92) |

### `indexing_runs` (TW-92)

Key fields:

| Field | Purpose |
| --- | --- |
| `indexing_run_id` | Unique run ID (`run_` + 16 hex) |
| `expected_code_files` / `expected_docs_files` | Fan-in gate for Code Parser finalization |
| `expected_commits` | Commit delta queue size |
| `processed_*` / `failed_files` | Updated by downstream workers (TW-91, TW-93) |
| `stages` | `repo_sync`, `docs_parse`, `code_parse`, `graph_finalize`, `commit_intel` |

Indexes: unique `indexing_run_id`; compound `repo_id` + `started_at`; compound `repo_id` + `snapshot_id`.

### `snapshot_files` (TW-92)

One document per emitted `files.changed` record. Initial `status` is `pending`. Used to verify snapshot file inventory before graph finalization.

Indexes: compound `repo_id` + `snapshot_id`; unique `repo_id` + `snapshot_id` + `file_path`.

## Environment variables

| Variable | Local default | Description |
| --- | --- | --- |
| `MONGO_URI` | `mongodb://localhost:27018/adpilot_repo_sync` | MongoDB connection string |
| `MONGODB_DATABASE` | `adpilot_repo_sync` | Database name (optional if URI includes path) |

Copy from `env.example`. Required when `ENVIRONMENT` is not `local`.

## Implementation status

- **TW-19:** Mongo client, domain models, and `MetadataStore` for repositories, snapshots, sync jobs
- **TW-92:** `indexing_runs` and `snapshot_files` collections with CRUD in `internal/store/mongo/`
- **SyncService** persists metadata, indexing run, and snapshot file inventory before publishing Redis events
- **`/readyz`:** includes Mongo ping (with Redis)

## Atlas (staging / production)

1. Create a **new** Atlas project or a **new database** under an indexing-specific cluster (not the unrelated app's DB).
2. Database name: `adpilot_repo_sync`
3. Network access: allow deployment IPs / VPC peering
4. Database user: least-privilege user scoped to `adpilot_repo_sync`
5. Connection string example:

```text
mongodb+srv://repo-sync-user:<password>@cluster.example.mongodb.net/adpilot_repo_sync?retryWrites=true&w=majority
```

Store the URI in secrets (env / secret manager), never commit it.

## Separation from other Mongo projects

If another repo already runs Mongo on `localhost:27017` (Atlas Local, `mongod`, or compose):

- This repo uses host port **27018** → no conflict
- Volume name `adpilot_repo_sync_mongo_data` → isolated data
- Database name `adpilot_repo_sync` → isolated namespace
- Do not point `MONGO_URI` at the other project's database

## Verify indexing run (local)

After a sync:

```sh
mongosh mongodb://localhost:27018/adpilot_repo_sync --eval '
  db.indexing_runs.find().sort({started_at:-1}).limit(1).pretty()
  db.snapshot_files.countDocuments({snapshot_id:"snap_..."})
'
```