Production readiness hardening and ops tooling

This commit is contained in:
2026-05-29 10:10:12 +07:00
parent e0b8f9af9a
commit 648e77cee9
68 changed files with 12222 additions and 848 deletions

View File

@ -1,103 +1,251 @@
# CODEx Handoff QRIS Soundbox Platform
# Codex Handoff - QRIS Soundbox Platform
## Current status
- Fokus terakhir: penyelesaian gap Fase 1 backend + smoke e2e skenario wajib.
- Implementasi backend dan UI sudah mulai dikerjakan di repository (tidak lagi hanya dokumentasi).
- Tambahan terbaru:
- audit log untuk aksi admin/webhook penting
- ledger placeholder `gross_income` saat transaksi menjadi `paid`
- endpoint admin `GET /admin/audit-logs` dan `GET /admin/ledger-entries`
- awal Fase 2: capability resolver + `POST /device/transactions/dynamic-qr` API-direct
- lanjutan Fase 2: MQTT dynamic QR simulator/outbox + device config push/ack
- lanjutan Fase 2 berikutnya: config drift status + retry push config + MQTT trace untuk config ACK
- health summary device Fase 2 untuk admin list/detail: status, score, age_seconds, reasons
- UI ops Fase 2 di device registry/detail: health score/reasons, config drift, retry config push
- dynamic QR expiry sweep via `POST /admin/transactions/expire-due`
- smoke e2e mencakup duplicate callback, invalid signature, ledger, audit, terminal tanpa binding, dynamic QR API-direct, dynamic QR expiry sweep, dynamic QR MQTT, dan device config push/status/retry/ack
- fix UI lokal:
- CSP Helmet dilonggarkan untuk Tailwind CDN, Google Fonts/Material Symbols, dan image Googleusercontent agar desain render normal
- panel kanan login admin dibuat dark glass supaya teks putih terbaca
- input login admin diubah dari email ke username text agar credential dev `admin/admin` bisa dipakai
- Smoke test Fase1 jalannya:
- `smoke:cleanup`
- `smoke:flow` jalan jika server aktif di `localhost:3100`
- `smoke:e2e` ✅ setelah server auto-start di port 3100 (cleanup + full flow berhasil).
Tanggal update: 2026-05-29, Asia/Jakarta.
## Files baru/terbaru yang sudah dibuat
- [UI: admin-system-dashboard](/home/wira/work/codex/qris-soundbox-platform/ui/admin-system-dashboard/index.html)
- Dashboard API wiring dipastikan terhubung ke backend untuk token admin, endpoint summary, dan retry/realtime UI.
- [UI: merchant-onboarding-flow](/home/wira/work/codex/qris-soundbox-platform/ui/merchant-onboarding-flow/index.html)
- Form onboarding disinkron ke API (create merchant, outlet, terminal, device, binding) + status badge flow.
- [UI: merchant-detail-view](/home/wira/work/codex/qris-soundbox-platform/ui/merchant-detail-view/index.html)
- Detail merchant kini ambil data API untuk merchant/outlet/transactions list.
- [UI: device-technical-detail](/home/wira/work/codex/qris-soundbox-platform/ui/device-technical-detail/index.html)
- Device detail sinkronisasi data API: detail device, binding terbaru, heartbeats, events, metrics device, dan stream log.
- [UI: transaction-history-monitoring](/home/wira/work/codex/qris-soundbox-platform/ui/transaction-history-monitoring/index.html)
- Search/filter outlet-terminal dan path transaksi sudah memakai endpoint API admin.
- [README](/home/wira/work/codex/qris-soundbox-platform/README.md)
- Sudah ada script dan langkah smoke test (`smoke:cleanup`, `smoke:flow`, `smoke:e2e`) siap dipakai dan mencakup skenario Fase 1 tambahan.
- [DECISIONS_LOG.md](/home/wira/work/codex/qris-soundbox-platform/DECISIONS_LOG.md)
- Sudah memuat keputusan merchant bank account dan keputusan audit log + ledger placeholder Fase 1.
- [Backend: auditLogStore](/home/wira/work/codex/qris-soundbox-platform/src/shared/store/auditLogStore.ts)
- Store audit log untuk aksi admin/webhook penting.
- [Backend: ledgerStore](/home/wira/work/codex/qris-soundbox-platform/src/shared/store/ledgerStore.ts)
- Store ledger placeholder untuk transaksi paid Fase 1.
- [Backend: deviceCapabilityResolver](/home/wira/work/codex/qris-soundbox-platform/src/shared/services/deviceCapabilityResolver.ts)
- Resolver capability untuk flow dynamic QR API/MQTT.
- [Backend: dynamicQrOrchestrator](/home/wira/work/codex/qris-soundbox-platform/src/shared/services/dynamicQrOrchestrator.ts)
- Membuat transaksi dynamic `awaiting_payment` dan mock QR payload.
- [Backend: dynamicQrExpiry](/home/wira/work/codex/qris-soundbox-platform/src/shared/services/dynamicQrExpiry.ts)
- Sweep transaksi dynamic QR `awaiting_payment` yang sudah melewati `expired_at`.
- [Backend: mqttMessageStore](/home/wira/work/codex/qris-soundbox-platform/src/shared/store/mqttMessageStore.ts)
- Outbox/trace MQTT uplink dan downlink.
- [Backend: deviceConfigStore](/home/wira/work/codex/qris-soundbox-platform/src/shared/store/deviceConfigStore.ts)
- Config versioned dan ACK device.
- [Backend: deviceConfigStatus](/home/wira/work/codex/qris-soundbox-platform/src/shared/services/deviceConfigStatus.ts)
- Derivasi status drift config: `applied`, `pending_ack`, `failed_ack`, `stale_ack`, `never_pushed`.
- [UI: device-registry-monitoring](/home/wira/work/codex/qris-soundbox-platform/ui/device-registry-monitoring/index.html)
- Drawer ops menampilkan health score/reasons, config drift, latest push/ACK, dan retry config push.
- [UI: device-technical-detail](/home/wira/work/codex/qris-soundbox-platform/ui/device-technical-detail/index.html)
- Detail device menampilkan health summary dan config delivery panel dengan retry push.
- [App CSP](/home/wira/work/codex/qris-soundbox-platform/src/app.ts)
- Helmet CSP disesuaikan agar asset desain eksternal dapat dimuat di lokal.
- [UI: admin-login](/home/wira/work/codex/qris-soundbox-platform/ui/admin-login/index.html)
- Login admin API-wired, input username dev, dan kontras panel kanan diperbaiki.
- [UI: admin-login-portal](/home/wira/work/codex/qris-soundbox-platform/ui/admin-login-portal/index.html)
- Baseline portal login ikut diselaraskan untuk username dan kontras.
Dokumen ini adalah snapshot kerja terakhir untuk melanjutkan project tanpa perlu membaca ulang seluruh chat.
## Keputusan penting yang harus diikuti saat lanjut
1. Fase 1 Step 14 harus tetap jalan berurutan sebelum pengembangan Fase 2.
2. Backend target Postgres di local (`qris_soundbox_platform`) sudah dipakai di smoke test.
3. Jalankan smoke dari kondisi bersih (`smoke:cleanup`) untuk hasil yang konsisten.
4. Untuk sementara, pencairan dana mengikuti pola rekening merchant sendiri (sesuai permintaan terakhir), bukan rekening terpusat.
5. Pertahankan format error API yang konsisten: `code`, `message`, `details`, `request_id`, `timestamp`.
6. Ledger Fase 1 masih placeholder `gross_income`; jangan perluas fee/payable sebelum Fase 3 kecuali diminta eksplisit.
7. Dynamic QR Fase 2 saat ini memakai mock QRIS payload lokal; integrasi partner sungguhan belum dipasang.
8. MQTT Fase 2 saat ini memakai simulator HTTP + `mqtt_messages` outbox; broker sungguhan belum dipasang.
9. Config retry Fase 2 mengirim ulang config version yang sama; jangan naikkan versi kecuali settings berubah.
10. Dynamic QR expiry sweep saat ini endpoint admin/manual; bisa dinaikkan menjadi scheduler/background worker.
11. Untuk cek UI lokal, gunakan `http://127.0.0.1:3100/ui/admin-login`; credential dev adalah username `admin`, password `admin`.
## Status Terakhir
## Urutan kerja selanjutnya (disarankan)
1. UI/manual sanity lanjut dari titik terakhir:
- Merchant detail page
- Merchant list/filter
- Device technical detail
- Device list + heartbeat view
- Transaction history + outlet/terminal filter
2. Jalankan lagi `npm run smoke:e2e` sebelum lanjut Fase 2 atau sebelum commit besar.
3. Jika ada regresi, cek log server di `/tmp/qris-smoke-e2e-server.log`.
4. Lanjut Fase 2 berikutnya:
- adapter broker MQTT sungguhan dari `mqtt_messages` outbox
- scheduler otomatis untuk dynamic QR expiry sweep
- filter/sorting UI berbasis `health_summary.score` dan `health_summary.reasons`
- manual visual QA device registry/detail untuk layout mobile dan drawer
5. Sebelum wiring UI baru, pastikan halaman tetap mengikuti desain `design/*/code.html` dan cek kontras teks pada panel transparan/overlay.
- Estimasi MVP / early pilot: 92-94%.
- Estimasi production-ready penuh: 82-85%.
- Platform sudah bukan prototype docs-only. Backend, UI operasional, migration, smoke test, rate limiting, audit logging, async export, runbook, dan script deployment sudah tersedia.
- Fokus terakhir yang selesai: rate limiting + security polish, login audit, admin audit UI real-data, placeholder nav cleanup, dan runbook/checklist produksi.
- Worktree kemungkinan masih dirty karena banyak perubahan aktif. Jangan revert perubahan yang tidak eksplisit diminta.
## Note kalau meneruskan sesi berikutnya
- Kode dan screen yang sudah dimodifikasi tidak perlu diulang dari nol; lanjut dari state saat ini.
- Prioritas saat lanjut: verifikasi “jalur UI sinkron API” lalu lanjutkan smoke flow end-to-end berkala.
- Gunakan [DECISIONS_LOG.md] sebagai rujukan wajib untuk keputusan yang sudah disepakati.
## Verifikasi Terakhir
## Selesai untuk off
- Sudah ada gabungan perubahan di repo: doc + UI + API integration + smoke validation.
- `npm run typecheck`: pass.
- `npm run db:migrate`: pass dan idempotent sampai migration `003_export_job_storage.sql`.
- `npm audit --json`: pass, 0 vulnerability.
- `npm run ui:qa`: pass setelah cleanup placeholder navigation.
- `npm run smoke:e2e`: pass setelah rate limiting dan login audit.
- Quick rate limit test: pass. Login admin salah pertama menghasilkan `401` dengan `RateLimit-Remaining: 0`, request berikutnya menghasilkan `429 RATE_LIMITED`.
- Quick login audit test: pass. Event `admin.login.success`, `admin.login.failed`, `merchant.login.success`, dan `merchant.login.failed` tercatat.
- Quick audit UI API test: pass. `GET /admin/audit-logs?action_contains=.login.&limit=10` mengembalikan event login.
- Production-like env check dummy: pass via `npm run deploy:check-env`, hanya warning opsional untuk `MQTT_SUBSCRIBE`.
- Staging/load/MQTT real sebelumnya sudah pernah diverifikasi: load level 2 1610 requests 0 errors, MQTT broker `mqtts://mqtt.iptek.co:8883` publish/subscribe OK.
## Implementasi Selesai
### 1. Auth, RBAC, dan Security
- Admin session login tersedia lewat `/admin/login`, `/admin/logout`, `/admin/me`.
- Merchant session login tersedia lewat `/merchant/login`, `/merchant/logout`, `/merchant/me`.
- Legacy dev auth bisa dimatikan via env dan production check memblokir konfigurasi yang tidak aman.
- Admin dan merchant bootstrap script tersedia:
- `scripts/create-admin-user.mjs`
- `scripts/create-merchant-user.mjs`
- Password policy bootstrap diperketat:
- minimal 14 karakter;
- wajib lowercase, uppercase, angka, dan simbol;
- menolak kata mudah ditebak seperti product/default/password/admin/merchant/qris/soundbox.
- Rate limiting middleware baru:
- [src/shared/middleware/rateLimit.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/middleware/rateLimit.ts)
- dipasang ke `/admin/login`, `/merchant/login`, admin write routes, `/device`, dan `/integrations`.
- Env security baru:
- `TRUST_PROXY`
- `JSON_BODY_LIMIT`
- `RATE_LIMIT_ENABLED`
- `RATE_LIMIT_AUTH_WINDOW_MS`
- `RATE_LIMIT_AUTH_MAX`
- `RATE_LIMIT_ADMIN_WRITE_WINDOW_MS`
- `RATE_LIMIT_ADMIN_WRITE_MAX`
- `RATE_LIMIT_WRITE_WINDOW_MS`
- `RATE_LIMIT_WRITE_MAX`
- Error code baru `RATE_LIMITED` di [src/shared/errors/index.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/errors/index.ts).
### 2. Audit, Monitoring, dan Logging
- Audit logging login admin:
- `admin.login.success`
- `admin.login.failed`
- Audit logging login merchant:
- `merchant.login.success`
- `merchant.login.failed`
- `auditLogStore` mendukung `actor_type: merchant`.
- Filter audit baru `action_contains` tersedia di:
- [src/shared/store/auditLogStore.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/store/auditLogStore.ts)
- [src/routes/admin.ts](/home/wira/work/codex/qris-soundbox-platform/src/routes/admin.ts)
- Admin audit UI sudah memakai real API, bukan mock:
- [ui/admin-system-audit-logs/index.html](/home/wira/work/codex/qris-soundbox-platform/ui/admin-system-audit-logs/index.html)
- Audit UI memiliki filter action/entity/date/search, preset login events, KPI count, dan drawer JSON detail.
- Observability/health sebelumnya sudah tersedia:
- `/health`
- `/health/ready`
- `/admin/observability/summary`
- `/admin/observability/dead-letter-replays`
- `/admin/observability/mqtt-status`
### 3. MQTT dan Device Operations
- MQTT worker dan policy production sudah diperketat.
- Wildcard subscribe default production dicegah oleh env check.
- MQTT ACL tooling tersedia:
- `scripts/check-mqtt-acl.mjs`
- `scripts/smoke-mqtt-acl.mjs`
- `scripts/provision-mqtt-device.mjs`
- Package scripts:
- `npm run mqtt:provision-device`
- `npm run mqtt:check-acl`
- `npm run smoke:mqtt-acl`
- `npm run smoke:mqtt-real`
- Real MQTT smoke pernah pass dengan broker `mqtts://mqtt.iptek.co:8883`.
### 4. Settlement, Reconciliation, dan Finance Ops
- Settlement batch, merchant settlement history, reconciliation management, adjustment approval, dan device technical detail UI sudah tersedia.
- Admin reconciliation UI sudah menggunakan async export flow dan export history.
- Key UI pages:
- [ui/admin-reconciliation-management/index.html](/home/wira/work/codex/qris-soundbox-platform/ui/admin-reconciliation-management/index.html)
- [ui/settlement-batch-management/index.html](/home/wira/work/codex/qris-soundbox-platform/ui/settlement-batch-management/index.html)
- [ui/merchant-settlement-history/index.html](/home/wira/work/codex/qris-soundbox-platform/ui/merchant-settlement-history/index.html)
- [ui/device-technical-detail/index.html](/home/wira/work/codex/qris-soundbox-platform/ui/device-technical-detail/index.html)
- Placeholder `href="#"` sudah dibersihkan dari UI yang masuk QA.
### 5. Async Export dan Storage
- Async export job sudah tersedia untuk settlement adjustment export.
- Export job worker:
- [src/shared/services/exportJobWorker.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/services/exportJobWorker.ts)
- Export job store:
- [src/shared/store/exportJobStore.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/store/exportJobStore.ts)
- Migration:
- [migrations/002_export_jobs.sql](/home/wira/work/codex/qris-soundbox-platform/migrations/002_export_jobs.sql)
- [migrations/003_export_job_storage.sql](/home/wira/work/codex/qris-soundbox-platform/migrations/003_export_job_storage.sql)
- Export file storage memakai `EXPORT_STORAGE_DIR`, dengan metadata path/size/expiry.
- Export retention cleanup tersedia via worker.
- Admin endpoints:
- `POST /admin/exports/settlement-adjustments`
- `GET /admin/exports`
- `GET /admin/exports/:jobId`
- `GET /admin/exports/:jobId/download`
### 6. Deployment, Backup, Restore, dan Runbook
- Production env checker diperketat:
- [scripts/check-production-env.mjs](/home/wira/work/codex/qris-soundbox-platform/scripts/check-production-env.mjs)
- Backup/restore tooling:
- [scripts/backup-production.mjs](/home/wira/work/codex/qris-soundbox-platform/scripts/backup-production.mjs)
- [scripts/restore-plan.mjs](/home/wira/work/codex/qris-soundbox-platform/scripts/restore-plan.mjs)
- [scripts/restore-drill-validate.mjs](/home/wira/work/codex/qris-soundbox-platform/scripts/restore-drill-validate.mjs)
- Load testing/report tooling:
- [scripts/load-test.mjs](/home/wira/work/codex/qris-soundbox-platform/scripts/load-test.mjs)
- [scripts/run-staging-load-report.mjs](/home/wira/work/codex/qris-soundbox-platform/scripts/run-staging-load-report.mjs)
- Operational docs baru:
- [OPERATIONAL_RUNBOOK.md](/home/wira/work/codex/qris-soundbox-platform/OPERATIONAL_RUNBOOK.md)
- [PILOT_EXECUTION_CHECKLIST.md](/home/wira/work/codex/qris-soundbox-platform/PILOT_EXECUTION_CHECKLIST.md)
- [EXPORT_STORAGE_READINESS.md](/home/wira/work/codex/qris-soundbox-platform/EXPORT_STORAGE_READINESS.md)
- README dan deployment readiness docs sudah direferensikan ke runbook/checklist tersebut.
## Endpoint Penting
- Health:
- `GET /health`
- `GET /health/ready`
- Admin auth/session:
- `POST /admin/login`
- `POST /admin/logout`
- `GET /admin/me`
- Admin audit/observability:
- `GET /admin/audit-logs`
- `GET /admin/observability/summary`
- `GET /admin/observability/dead-letter-replays`
- `GET /admin/observability/mqtt-status`
- Admin export:
- `POST /admin/exports/settlement-adjustments`
- `GET /admin/exports`
- `GET /admin/exports/:jobId`
- `GET /admin/exports/:jobId/download`
- Merchant auth/session:
- `POST /merchant/login`
- `POST /merchant/logout`
- `GET /merchant/me`
- Device and integration routes remain rate-limited for write-heavy paths:
- `/device`
- `/integrations`
## Package Scripts Penting
- `npm run typecheck`
- `npm run db:migrate`
- `npm run smoke:e2e`
- `npm run ui:qa`
- `npm run deploy:check-env`
- `npm run load:test`
- `npm run load:test:staging`
- `npm run backup:production`
- `npm run restore:plan`
- `npm run restore:validate`
- `npm run admin:create-user`
- `npm run merchant:create-user`
- `npm run mqtt:provision-device`
- `npm run mqtt:check-acl`
- `npm run smoke:mqtt-acl`
- `npm run smoke:mqtt-real`
## File Kunci yang Sering Disentuh
- App bootstrap: [src/app.ts](/home/wira/work/codex/qris-soundbox-platform/src/app.ts)
- Env config: [src/config/env.ts](/home/wira/work/codex/qris-soundbox-platform/src/config/env.ts)
- Admin routes: [src/routes/admin.ts](/home/wira/work/codex/qris-soundbox-platform/src/routes/admin.ts)
- Merchant routes: [src/routes/merchant.ts](/home/wira/work/codex/qris-soundbox-platform/src/routes/merchant.ts)
- Audit store: [src/shared/store/auditLogStore.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/store/auditLogStore.ts)
- Export worker: [src/shared/services/exportJobWorker.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/services/exportJobWorker.ts)
- Export store: [src/shared/store/exportJobStore.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/store/exportJobStore.ts)
- Rate limit middleware: [src/shared/middleware/rateLimit.ts](/home/wira/work/codex/qris-soundbox-platform/src/shared/middleware/rateLimit.ts)
- UI QA script: [scripts/ui-qa-check.mjs](/home/wira/work/codex/qris-soundbox-platform/scripts/ui-qa-check.mjs)
- Admin audit UI: [ui/admin-system-audit-logs/index.html](/home/wira/work/codex/qris-soundbox-platform/ui/admin-system-audit-logs/index.html)
- Env sample: [.env.example](/home/wira/work/codex/qris-soundbox-platform/.env.example)
## Decision Log Ringkas
- D-026 sampai D-049: dasar auth, merchant/admin flows, migration, UI awal, dan smoke testing.
- D-050 sampai D-059: production hardening awal, MQTT policy, finance/reconciliation UI, settlement flows.
- D-060 sampai D-069: merchant auth productionization, DB migration idempotent, monitoring/logging, load test, async export.
- D-070 sampai D-074: export storage/history, MQTT ACL, backup/restore, staging load report.
- D-075 sampai D-080: rate limiting/security polish, login audit, audit UI real data, UI QA cleanup, runbook/checklist produksi.
Rujukan utama: [DECISIONS_LOG.md](/home/wira/work/codex/qris-soundbox-platform/DECISIONS_LOG.md).
## Sisa Gap Utama
1. Eksekusi staging nyata dari checklist:
- deploy dengan env final;
- jalankan `deploy:check-env`, migration, smoke, UI QA, load report;
- simpan artefak hasil staging.
2. Pilot real device:
- provisioning device real;
- validasi MQTT ACL per device;
- transaksi QRIS test end-to-end;
- validasi soundbox delivery dan dead-letter handling.
3. Restore drill nyata:
- backup production/staging;
- restore ke database disposable;
- jalankan `restore:validate`;
- dokumentasikan RTO/RPO aktual.
4. Export storage production topology:
- pastikan `EXPORT_STORAGE_DIR` durable, absolute, writable, dan di-backup;
- jika multi-node, perlu shared filesystem/object storage strategy.
5. Manual visual QA:
- buka halaman admin utama di browser;
- cek layout mobile/desktop;
- cek login/session expiry state;
- cek empty/error/loading state.
6. Operational readiness:
- isi PIC, escalation contact, broker credential, backup location, dan pilot merchant list di runbook/checklist.
## Prioritas Lanjutan Disarankan
1. Jalankan full staging rehearsal dari `PILOT_EXECUTION_CHECKLIST.md`.
2. Lakukan manual visual QA admin UI dengan browser.
3. Jalankan restore drill sungguhan pada database disposable.
4. Finalisasi export storage production strategy.
5. Siapkan pilot real merchant/device dan rekam hasilnya di runbook.
## Catatan Penting
- Jangan hidupkan legacy auth di production.
- Jangan gunakan wildcard MQTT subscribe di production kecuali sedang maintenance terkontrol.
- `EXPORT_STORAGE_DIR` harus absolute path dan durable untuk production.
- Rate limiting sekarang aktif secara default jika `RATE_LIMIT_ENABLED=true`; hati-hati saat smoke test berulang pada login endpoint.
- `CODEX_HANDOFF.md` ini adalah snapshot operasional terbaru; untuk detail historis keputusan, baca `DECISIONS_LOG.md`.