# Minabot production

Live: https://mina.teppelinlabs.com

SSH: `ssh syntaxbullet@46.62.156.93` (key authentication; passwordless sudo).
Deployment root: `/opt/minabot`. Compose manages `app`, `caddy`, and `monitor`.
Docker starts at boot; containers use `restart: unless-stopped`.

## Release and configuration

Current image: `minabot:20260907-fonts-1`, built on the Linux amd64 server from
commit `1721ee4`, including development write protection and bundled share fonts.
Its immutable image ID is recorded by `docker image inspect`; source is retained
under `/opt/minabot/releases/20260907-fonts-1`, with `source.tar.gz` and build
logs in `/opt/minabot`. Bun 1.3.14 and Caddy images are pinned by digest.

`/opt/minabot/.env` selects release images. `/opt/minabot/.env.production` contains
runtime secrets (0600). Discord client and bot credentials were copied from the
effective development environment; production has a separate generated cookie
signing secret. Runtime uses `APP_ORIGIN=https://mina.teppelinlabs.com`,
`TRUST_PROXY=true`, `HOST=0.0.0.0`, and `DATABASE_PATH=/data/minabot.sqlite`.

`TRUST_PROXY` only accepts HTTPS from `X-Forwarded-Proto`, retains the actual
request Host, and must only be enabled behind the private Caddy proxy. Port 3000
is not published. Caddy exposes TCP 80/443 and persists certificate state in
`minabot_caddy_data` and `minabot_caddy_config`. SSH remains on 22. Fail2ban uses
the iptables-compatible SSH action alongside Docker/UFW.

The registered Discord callback is
`https://mina.teppelinlabs.com/api/auth/discord/callback`; localhost remains registered.

## Data and backups

`/srv/minabot/data` is bind-mounted at `/data`, owned by UID 1000. A verified
`VACUUM INTO` snapshot copied the development database, including its one account,
four habits, history, reminders, and live Discord state. The transfer hash and
counts are recorded in `/opt/minabot/migration-manifest.json`. Local development
was stopped before the snapshot so only production manages the existing live cards.
Development now blocks channel writes at both snapshot-send and live-card entry
points. It also skips copied live-card recovery and change scheduling. Preview and
download remain available. Stubbed tests inject their own transport/configuration;
real writes through the environment loader are enabled only in production.

Daily verified snapshots and pre-migration snapshots are stored under
`/srv/minabot/data/backups`, retaining seven snapshots. The restore rehearsal
passed using a separate file; production data was not replaced by the rehearsal.
Restore revokes sessions and disables reminder opt-ins; see [OPERATIONS.md](OPERATIONS.md).

Off-server backup replication and delivered uptime alerts are **not configured**:
the operator has no destinations set up yet. The monitor checks public HTTPS every
minute in a separate container on this server and writes failure/recovery events
to its logs. It cannot report a complete server outage to an external destination.

## Common commands

Run after SSH login:

```sh
cd /opt/minabot
sudo docker compose ps
sudo docker compose logs --tail=100 app caddy monitor
sudo docker compose exec -T app bun scripts/backup.ts
sudo docker compose exec -T monitor bun scripts/monitor.ts --once
```

When sending several shell commands over SSH stdin, redirect noninteractive
`docker compose exec` calls from `/dev/null` so they do not consume later commands.

For updates, build a new release tag, run the container smoke test, take a backup,
update `APP_IMAGE` in `.env`, stop the old app, then run
`sudo docker compose up -d --wait --wait-timeout 120`. Never run overlapping app
workers or `docker compose down -v`. Keep the last known-good image and a matching
pre-migration snapshot. The initial `-1` image predates the proxy fix and is not
a suitable rollback target for working production sign-in.

## Verified September 7, 2026

- Typecheck, build, 197 tests, and production HTTP smoke tests passed.
- The Linux runtime passed migrations, native 1920×1080 PNG rendering, and
  persistence across a process restart using a disposable database.
- Public HTTPS health returns `{"status":"ok"}`; HTTP redirects to HTTPS.
- Browser Discord sign-in completed and displayed the migrated account and habits.
- Read-only authenticated production API/PNG checks passed; unauthenticated access
  and cross-origin writes were rejected. Temporary verification sessions were removed.
- Snapshot integrity, restore rehearsal, and transferred database hash passed.
- Application port 3000 is unpublished and an external HTTP probe timed out.
- Docker and Fail2ban are active; runtime secrets and the SQLite file are mode 0600.

Live Discord test posts/DMs were not sent. Existing configured workers continue
their normal production behavior.

## Share fonts

Instrument Serif and Noto Sans (400/500/600) are bundled for both browser and native
canvas share renderers. Mac/Linux fixture text metrics match exactly; pixel-channel
differences are at most 2/255 from native rasterization. Missing native font files
fail startup. Renderer version 3 refreshes existing live cards with the new fonts.