wirabasalamah/Qris-Soundbox
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Production readiness hardening and ops tooling

Browse Source
This commit is contained in:
wirabasalamah
2026-05-29 10:10:12 +07:00
parent e0b8f9af9a
commit 648e77cee9
68 changed files with 12222 additions and 848 deletions
Download Patch File Download Diff File Expand all files Collapse all files

214
README.md
View File

@ -14,6 +14,7 @@ Paket ini berisi blueprint final v1 untuk platform merchant aggregator QRIS + so
- 09-screen-inventory.md
- 10-design-blueprint.md
- 11-low-fi-wireframes.md
- 18-mqtt-broker-mosquitto-debian13.md
## Tujuan
Dokumen ini dibuat supaya tim bisa langsung mulai:
@ -33,7 +34,9 @@ Dokumen ini dibuat supaya tim bisa langsung mulai:
- middleware idempotency untuk endpoint sensitif
- endpoint awal:
- `GET /health`
- `GET /health/deep`
- `GET /admin/health` (dengan `Authorization: Bearer <token>`)
- `GET /admin/health/deep`
- `POST /admin/login`
- `POST /admin/sample-idempotent`
- `POST /admin/merchants`
@ -51,6 +54,7 @@ Dokumen ini dibuat supaya tim bisa langsung mulai:
- `POST /admin/devices`
- `GET /admin/devices`
- `GET /admin/devices/{id}`
- `POST /admin/devices/{id}/credentials/rotate`
- `POST /admin/devices/{id}/bind`
- `POST /admin/devices/{id}/unbind`
- `POST /admin/devices/{id}/commands`
@ -64,6 +68,40 @@ Dokumen ini dibuat supaya tim bisa langsung mulai:
- `GET /admin/devices/{id}/mqtt-messages`
- `GET /admin/audit-logs`
- `GET /admin/ledger-entries`
- `GET /admin/dashboard/summary`
- Memuat settlement finance summary termasuk `settlement_adjustment_amount` dan `settlement_adjusted_paid_amount`.
- `POST /admin/settlement-batches`
- `GET /admin/settlement-batches`
- `GET /admin/settlement-batches/{batchId}`
- `GET /admin/settlement-batches/{batchId}/export.csv`
- Query `format=bank_generic` tersedia untuk CSV upload bank generik; default tetap settlement report standard.
- `POST /admin/settlement-batches/{batchId}/mark-paid`
- `PATCH /admin/settlement-batches/{batchId}/reference`
- `POST /admin/settlement-batches/{batchId}/adjustments`
- Adjustment disimpan di tabel formal `settlement_batch_adjustments` dan ikut tampil di detail batch admin/merchant.
- Metadata adjustment lama otomatis dibackfill ke tabel formal saat schema initialization berjalan.
- `GET /admin/settlement-adjustments`
- Report adjustment lintas settlement batch dengan filter `merchant_id`, `adjustment_type`, `from`, `to`, dan `limit`.
- `GET /admin/settlement-adjustments/export.csv`
- Export CSV untuk report adjustment settlement dengan filter yang sama.
- `POST /admin/exports/settlement-adjustments`
- Membuat job export CSV adjustment settlement secara async, dengan permission `settlement:export`.
- `GET /admin/exports/{jobId}`
- Melihat status job export.
- `GET /admin/exports/{jobId}/download`
- Download hasil CSV saat job sudah `completed`.
- `POST /admin/settlement-batches/{batchId}/mark-failed`
- `POST /admin/settlement-batches/{batchId}/cancel`
- `POST /admin/settlement-batches/{batchId}/reprocess`
- `GET /admin/reconciliation/settlement-batches`
- `GET /admin/observability/summary`
- `POST /merchant/login`
- `GET /merchant/profile`
- `GET /merchant/settlement-summary`
- `GET /merchant/settlement-batches`
- `GET /merchant/settlement-batches/{batchId}`
- `GET /merchant/settlement-batches/{batchId}/export.csv`
- Query `format=bank_generic` tersedia untuk CSV upload bank generik; default tetap settlement report standard.
- `GET /admin/transactions`
- `GET /admin/transactions/{transactionId}`
- `POST /admin/transactions`
@ -81,10 +119,24 @@ Dokumen ini dibuat supaya tim bisa langsung mulai:
```bash
npm install
cp .env.example .env
npm run db:migrate
npm run dev
npm run build && npm start
```
Schema migration memakai tabel `schema_migrations`. Migration awal `001_current_schema_bootstrap` membaseline schema saat ini; migration berikutnya bisa ditambahkan ke folder `migrations/` dengan format `NNN_description.sql` atau `NNN_description.mts`.
Untuk mode broker MQTT sungguhan, set `MQTT_PUBLISH_MODE=broker` di `.env` bersama `MQTT_BROKER_URL`, `MQTT_USERNAME`, `MQTT_PASSWORD`, dan `MQTT_CLIENT_ID`. Smoke test otomatis tetap override ke `MQTT_PUBLISH_MODE=simulator` agar tidak bergantung pada broker eksternal.
Dynamic QR expiry sweep berjalan otomatis bila `DYNAMIC_QR_EXPIRY_SCHEDULER_ENABLED=true`; interval dan batch size diatur lewat `DYNAMIC_QR_EXPIRY_SWEEP_INTERVAL_MS` dan `DYNAMIC_QR_EXPIRY_SWEEP_LIMIT`.
Finance light membuat ledger `gross_income`, `platform_fee`, dan `merchant_payable` saat transaksi menjadi `paid`. Rate fee default diatur lewat `FINANCE_PLATFORM_FEE_BPS`.
Settlement light membuat batch dari ledger `merchant_payable` yang belum dibatch, dikelompokkan per merchant, lalu admin dapat menandai batch sebagai `paid`.
UI `settlement-batch-management` sudah membaca batch real, menampilkan KPI settlement, membuka drawer detail batch, event history payout, generate batch, confirm paid dengan reference payout/note, mark failed/cancel dengan reason, reprocess failed/cancelled batch, dan download CSV payout report.
Admin dashboard overview juga menampilkan summary settlement/finance dari agregasi DB: pending payout, paid payout, total platform fee, jumlah batch, batch terbaru, dan download CSV payout report per batch.
UI `merchant-settlement-history` sudah membaca settlement batch milik merchant yang login, menampilkan pending/paid payout termasuk adjustment dan adjusted paid amount, membuka detail batch dengan payout event history, dan download CSV payout report merchant-scoped.
### Cleanup data smoke test
```bash
@ -97,7 +149,7 @@ Cleanup hanya menarget entitas smoke (`Smoke Merchant`, `PR-`, `DEV-`) agar data
PORT=3100 ADMIN_TOKEN=admin-dev-token DEVICE_TOKEN=device-dev-token INTEGRATION_WEBHOOK_SECRET=dev-callback-secret PGHOST=127.0.0.1 PGPORT=5432 PGUSER=postgres PGPASSWORD=postgres PGDATABASE=qris_soundbox_platform npm run smoke:flow
```
Smoke flow akan melakukan create merchant/device/transaction + heartbeat + callback paid + verifikasi event/heartbeat/notification, duplicate callback, invalid signature, audit log, ledger placeholder, skenario terminal tanpa binding, dynamic QR API-direct, expiry sweep dynamic QR, dynamic QR MQTT, device config push/retry/status/ack, dan trace MQTT config ack.
Smoke flow akan melakukan create merchant/device/transaction + heartbeat + callback paid + verifikasi event/heartbeat/notification, duplicate callback, invalid signature, audit log, ledger finance light, settlement batch light, skenario terminal tanpa binding, dynamic QR API-direct, expiry sweep dynamic QR, dynamic QR MQTT, device config push/retry/status/ack, dan trace MQTT config ack.
### Smoke test end-to-end (bootstrap + flow + cleanup)
@ -113,6 +165,164 @@ Perintah ini menjalankan:
- jalankan flow smoke lengkap
- hentikan server setelah selesai
### Load test awal
Jalankan backend lokal lalu eksekusi:
```bash
PORT=3120 LOAD_CALLBACKS=30 LOAD_HEARTBEATS=60 LOAD_DYNAMIC_QR=30 LOAD_READS=30 LOAD_CONCURRENCY=10 npm run load:test
```
Script ini membuat data load test sementara, menjalankan burst transaction create, callback QRIS paid, heartbeat device, dynamic QR API, dan observability read, lalu membersihkan merchant/transaksi load test.
Baseline lokal level 2 terakhir: 1610 request, 0 error, durasi 4955.45 ms, throughput perkiraan 324.9 req/s. p95 lokal: transaction create 183.56 ms, callback paid 129.03 ms, heartbeat 90.4 ms, dynamic QR 71.57 ms, observability summary 230.96 ms.
### Smoke test broker MQTT sungguhan
```bash
npm run smoke:mqtt-real
```
Perintah ini membaca `.env`, menjalankan app lokal dengan `MQTT_PUBLISH_MODE=broker`, subscribe ke broker `devices/+/downlink/#`, lalu memverifikasi publish real untuk payment success, config push, dan dynamic QR response. Test akan membersihkan data merchant/transaksi/MQTT trace yang dibuat khusus smoke.
Smoke broker real terakhir lulus terhadap `mqtts://mqtt.iptek.co:8883`: broker connect ok, subscribe `devices/+/downlink/#`, menerima payment success, config push, dynamic QR response, dan cleanup data smoke.
### Async export worker
Export adjustment settlement memakai worker internal yang aktif default lewat `EXPORT_WORKER_ENABLED=true`. Endpoint create job mengembalikan status `pending`, worker mengambil job secara berkala, lalu client dapat polling `GET /admin/exports/{jobId}` sampai `completed` dan download dari `download_url`.
UI `admin-reconciliation-management` sudah memakai flow async ini untuk tombol download adjustment report.
Konfigurasi utama:
- `EXPORT_WORKER_INTERVAL_MS`
- `EXPORT_WORKER_BATCH_SIZE`
- `EXPORT_JOB_STALE_RUNNING_MS`
- `EXPORT_SETTLEMENT_ADJUSTMENT_MAX_ROWS`
- `EXPORT_STORAGE_DIR`
- `EXPORT_RETENTION_DAYS`
Status worker dan count job tampil di `/admin/observability/summary`.
Hasil export baru disimpan sebagai file di `EXPORT_STORAGE_DIR`; metadata job tetap di database. Worker juga membersihkan hasil export yang melewati `EXPORT_RETENTION_DAYS`.
### Provisioning credential MQTT device
```bash
npm run mqtt:provision-device -- --base-url http://127.0.0.1:3000 --device-id <device-id>
```
Perintah ini rotate credential device lewat API admin dan menampilkan command `mosquitto_passwd` untuk broker. Username MQTT device sama dengan `device_id` agar cocok dengan ACL Mosquitto `devices/%u/...`.
Validasi/template ACL Mosquitto:
```bash
npm run mqtt:check-acl -- --print-template
npm run mqtt:check-acl -- --file /etc/mosquitto/acl
```
Smoke ACL memakai dua username device test:
```bash
MQTT_TEST_DEVICE_A_USERNAME=<device-a-id> \
MQTT_TEST_DEVICE_A_PASSWORD=<secret-a> \
MQTT_TEST_DEVICE_B_USERNAME=<device-b-id> \
npm run smoke:mqtt-acl
```
UI device technical detail juga menyediakan tombol `Rotate Credential` dan modal one-time secret untuk operator.
### Backup dan restore readiness
```bash
npm run backup:production -- --out ./backups --include-mosquitto
npm run restore:plan -- --backup ./backups/<file>.dump
npm run restore:validate
```
`backup:production` membuat dump Postgres format custom dan opsional menyalin file Mosquitto passwd/ACL. `restore:plan` default hanya menampilkan command restore; tambahkan `--execute` hanya pada database restore yang memang sudah disiapkan. `restore:validate` menjalankan migration idempotent, health check, admin deep health, dan cek tabel kunci setelah restore.
### Load test staging profile
```bash
npm run load:test:staging
```
Profile staging menjalankan transaction create, QRIS callback, heartbeat, dynamic QR, observability read, dan async export job. Set `BASE_URL`, token, dan env database staging sebelum menjalankan terhadap environment target.
Hasil staging disimpan otomatis ke folder `reports/` melalui `LOAD_REPORT_FILE`.
### UI QA ringan
```bash
npm run ui:qa
```
Checker ini memvalidasi inline script halaman operasional, memastikan export reconciliation memakai async job, dan memastikan navigasi placeholder di halaman operasional utama sudah dibersihkan.
### Rate limiting dan security polish
Rate limit in-memory aktif default untuk login admin/merchant, admin write routes, device routes, dan integration callback routes.
Login admin dan merchant juga dicatat ke audit log dengan action `admin.login.success`, `admin.login.failed`, `merchant.login.success`, dan `merchant.login.failed`.
Halaman `/ui/admin-system-audit-logs` sudah membaca audit log real, menyediakan preset login events, filter action/entity/date, pencarian client-side, KPI login success/failed, dan drawer JSON payload.
Konfigurasi utama:
- `RATE_LIMIT_ENABLED`
- `RATE_LIMIT_LOGIN_WINDOW_MS`
- `RATE_LIMIT_LOGIN_MAX`
- `RATE_LIMIT_DEVICE_WINDOW_MS`
- `RATE_LIMIT_DEVICE_MAX`
- `RATE_LIMIT_ADMIN_WRITE_WINDOW_MS`
- `RATE_LIMIT_ADMIN_WRITE_MAX`
- `TRUST_PROXY`
- `JSON_BODY_LIMIT`
Runbook operasional utama: `OPERATIONAL_RUNBOOK.md`. Checklist pilot: `PILOT_EXECUTION_CHECKLIST.md`. Catatan export storage: `EXPORT_STORAGE_READINESS.md`.
### Membuat admin user production
```bash
npm run admin:create-user -- \
--email finance@example.com \
--name "Finance Ops" \
--role finance \
--password "ganti-dengan-password-kuat"
```
Role yang tersedia: `admin`, `finance`, `ops`, `support`, dan `viewer`. Untuk update password user yang sudah ada, tambahkan `--rotate-password`.
### Membuat merchant user production
```bash
npm run merchant:create-user -- \
--merchant <merchant-id-atau-code> \
--email owner@merchant.com \
--name "Merchant Owner" \
--role owner \
--password "ganti-dengan-password-kuat"
```
Role merchant yang tersedia: `owner`, `finance`, `ops`, dan `viewer`. Untuk update password user yang sudah ada, tambahkan `--rotate-password`.
### Device API auth
Endpoint `/device/*` mendukung credential per-device:
```bash
curl -X POST http://127.0.0.1:3000/device/heartbeat \
-H "Content-Type: application/json" \
-H "X-Device-Id: <device-id>" \
-H "X-Device-Secret: <one-time-secret-yang-dipasang-di-device>" \
-d '{"device_id":"<device-id>","timestamp":"2026-05-28T00:00:00.000Z","network_strength":88,"battery_level":77}'
```
Credential hanya boleh dipakai untuk `device_id` yang sama. `Authorization: Bearer DEVICE_TOKEN` masih tersedia sebagai fallback/dev compatibility.
Untuk production, fallback token global bisa dimatikan:
```env
DEVICE_AUTH_ALLOW_LEGACY_TOKEN=false
```
### Endpoint device lain
- `POST /device/commands/ack`
@ -124,3 +334,5 @@ Perintah ini menjalankan:
Status lanjutan: Fase 1 core flow sudah tercakup smoke e2e. Fase 2 sudah aktif untuk capability resolver, dynamic QR API-direct, dynamic QR MQTT via outbox, dan device config push/status/retry/ack.
Catatan Fase 2 ops: endpoint daftar/detail device admin juga mengirim `health_summary` (`status`, `score`, `age_seconds`, `reasons`) untuk membantu triage device. UI device registry dan device technical detail sudah menampilkan health summary, config drift, dan retry config push.
Catatan credential device: `POST /admin/devices/{id}/credentials/rotate` menerbitkan MQTT username/password satu kali, menyimpan fingerprint secret di database, dan payload device biasa hanya menampilkan ringkasan credential tanpa secret.