add readme file
This commit is contained in:
288
README.md
Normal file
288
README.md
Normal file
@ -0,0 +1,288 @@
|
|||||||
|
# UTMS Agent
|
||||||
|
|
||||||
|
UTMS Agent is a Java 8 background service that connects to an MQTT broker, consumes terminal messages, stores incoming data in PostgreSQL, and publishes download/profile messages back to terminals. It is designed to run as a standard Linux `systemd` service without Tanuki or YAJWS wrappers.
|
||||||
|
|
||||||
|
## Runtime Overview
|
||||||
|
|
||||||
|
The application entrypoint is `id.iptek.utms.agent.Main`.
|
||||||
|
|
||||||
|
Startup flow:
|
||||||
|
|
||||||
|
1. Load `conf/mqtt-agents.cfg`.
|
||||||
|
2. Initialize Log4j from `conf/mqtt-agents-log4j.cfg`.
|
||||||
|
3. Initialize PostgreSQL access through `DBConn`.
|
||||||
|
4. Initialize a global MQTT client for scheduled task publishing.
|
||||||
|
5. Initialize a MinIO client for presigned download URLs.
|
||||||
|
6. Load terminal ID cache asynchronously.
|
||||||
|
7. Start queue handlers for heartbeat, diagnostic, and device init messages.
|
||||||
|
8. Start configured MQTT consumer workers.
|
||||||
|
9. Start Quartz scheduler for pending download tasks.
|
||||||
|
|
||||||
|
Shutdown flow:
|
||||||
|
|
||||||
|
1. Stop Quartz scheduler so no new publish jobs start.
|
||||||
|
2. Drain queue handlers gracefully for up to 30 seconds.
|
||||||
|
3. Force interrupt queue work if the graceful timeout is exceeded.
|
||||||
|
4. Stop MQTT workers.
|
||||||
|
5. Disconnect the global MQTT client.
|
||||||
|
6. Stop the executor service.
|
||||||
|
|
||||||
|
## Project Layout
|
||||||
|
|
||||||
|
```text
|
||||||
|
src/id/iptek/utms/agent/
|
||||||
|
Main.java Application lifecycle and service entrypoint
|
||||||
|
AppConfig.java Properties config loader
|
||||||
|
db/ Database connection and utility classes
|
||||||
|
dao/ PostgreSQL data access objects
|
||||||
|
model/ Message, task, app, profile, and terminal models
|
||||||
|
queue/ In-memory delay queues and queue handlers
|
||||||
|
scheduler/job/ Quartz jobs for download task scheduling/publishing
|
||||||
|
worker/ MQTT consumer workers
|
||||||
|
util/ Shared utility classes
|
||||||
|
|
||||||
|
conf/
|
||||||
|
mqtt-agents.cfg Main runtime configuration
|
||||||
|
mqtt-agents-log4j.cfg File-only rolling log configuration
|
||||||
|
|
||||||
|
lib/ Runtime dependency jars
|
||||||
|
utms-agent.service Example systemd unit
|
||||||
|
build.xml NetBeans/Ant project file
|
||||||
|
```
|
||||||
|
|
||||||
|
Ignored generated/runtime folders:
|
||||||
|
|
||||||
|
```text
|
||||||
|
build/
|
||||||
|
dist/
|
||||||
|
logs/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Main Components
|
||||||
|
|
||||||
|
### Main
|
||||||
|
|
||||||
|
`Main.java` owns the service lifecycle. It accepts one required argument:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
java -jar utms-agent.jar /opt/utms-agent/conf/mqtt-agents.cfg
|
||||||
|
```
|
||||||
|
|
||||||
|
It also accepts the old wrapper-style argument form for compatibility:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
java -cp ... id.iptek.utms.agent.Main id.iptek.utms.agent.Main /opt/utms-agent/conf/mqtt-agents.cfg
|
||||||
|
```
|
||||||
|
|
||||||
|
### MQTT Workers
|
||||||
|
|
||||||
|
MQTT workers are created dynamically by `WorkerFactory` from the `mqtt.consumers` property.
|
||||||
|
|
||||||
|
Configured consumers:
|
||||||
|
|
||||||
|
```properties
|
||||||
|
mqtt.consumers = heartbeat_c, diagnostic_c, device_init_c, download_ack_c
|
||||||
|
```
|
||||||
|
|
||||||
|
Each consumer has its own:
|
||||||
|
|
||||||
|
- topic
|
||||||
|
- thread count
|
||||||
|
- client ID prefix
|
||||||
|
- MQTT credentials
|
||||||
|
- keepalive
|
||||||
|
- clean session setting
|
||||||
|
- reconnect setting
|
||||||
|
- worker class
|
||||||
|
- queue mode where applicable
|
||||||
|
|
||||||
|
Worker classes:
|
||||||
|
|
||||||
|
| Worker | Purpose |
|
||||||
|
| --- | --- |
|
||||||
|
| `HeartbeatWorker` | Consumes `HEARTBEAT` messages and queues them for persistence. |
|
||||||
|
| `DiagnosticWorker` | Consumes `DIAGNOSTIC` messages and queues diagnostic details. |
|
||||||
|
| `DeviceInitWorker` | Consumes device init requests and queues profile response work. |
|
||||||
|
| `DownloadTaskAckWorker` | Consumes download ACK messages and sends confirm ACKs when needed. |
|
||||||
|
|
||||||
|
The base `Worker` class uses Eclipse Paho MQTT. It supports automatic reconnect and has a reconnect delay when the broker is unavailable.
|
||||||
|
|
||||||
|
### Queues
|
||||||
|
|
||||||
|
`DelayMessageQueue<T>` is the internal queue abstraction.
|
||||||
|
|
||||||
|
Supported modes:
|
||||||
|
|
||||||
|
| Mode | Behavior |
|
||||||
|
| --- | --- |
|
||||||
|
| `SINGLE` | Each message is handled separately. |
|
||||||
|
| `BATCH` | Messages are collected until capacity is reached or the flush interval expires. |
|
||||||
|
|
||||||
|
The queue shutdown is graceful. During service stop, queues are asked to drain before workers are disconnected. If queue work does not finish within 30 seconds, remaining work is forced to stop.
|
||||||
|
|
||||||
|
### Scheduler
|
||||||
|
|
||||||
|
Quartz runs `DownloadTaskSchedulerJob` every `scheduler.periode.minutes`.
|
||||||
|
|
||||||
|
The scheduler:
|
||||||
|
|
||||||
|
1. Finds pending download tasks.
|
||||||
|
2. Schedules `DownloadTaskPublisherJob`.
|
||||||
|
3. Publishes task messages over MQTT.
|
||||||
|
4. Regenerates expired MinIO presigned URLs before publishing.
|
||||||
|
5. Updates last broadcast timestamps in the database.
|
||||||
|
|
||||||
|
### MinIO URLs
|
||||||
|
|
||||||
|
`DownloadTaskPublisherJob` uses MinIO presigned URLs for app and icon downloads.
|
||||||
|
|
||||||
|
Relevant config:
|
||||||
|
|
||||||
|
```properties
|
||||||
|
fileserver.endpoint = https://...
|
||||||
|
fileserver.bucket = application-bucket
|
||||||
|
fileserver.icon.path = icons
|
||||||
|
fileserver.icon.expiry = 7
|
||||||
|
fileserver.icon.timeunit = DAYS
|
||||||
|
fileserver.app.path = apps
|
||||||
|
fileserver.app.expiry = 7
|
||||||
|
fileserver.app.timeunit = DAYS
|
||||||
|
```
|
||||||
|
|
||||||
|
If a stored URL expiry is null or in the past, the publisher generates a fresh URL before sending the MQTT message.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Main config file:
|
||||||
|
|
||||||
|
```text
|
||||||
|
conf/mqtt-agents.cfg
|
||||||
|
```
|
||||||
|
|
||||||
|
Important sections:
|
||||||
|
|
||||||
|
| Section | Keys |
|
||||||
|
| --- | --- |
|
||||||
|
| Logging | `log4j.file` |
|
||||||
|
| Database | `db.driver`, `db.url`, `db.user`, `db.password` |
|
||||||
|
| MQTT global publisher | `mqtt.broker.url`, `mqtt.clientid_prefix`, `mqtt.user`, `mqtt.password`, `mqtt.keepalive`, `mqtt.maxinflight`, `mqtt.cleansession`, `mqtt.autoreconnect` |
|
||||||
|
| Consumers | `mqtt.consumers`, `mqtt.consumer.<name>.*` |
|
||||||
|
| Pending task query | `pendingtask.query.limit`, `pendingtask.periode.minutes` |
|
||||||
|
| MinIO | `fileserver.*` |
|
||||||
|
| Scheduler | `scheduler.periode.minutes` |
|
||||||
|
|
||||||
|
`mqtt.keepalive` and `mqtt.consumer.<name>.keepalive` are in seconds because Eclipse Paho expects seconds.
|
||||||
|
|
||||||
|
Do not commit production credentials. Replace database, MQTT, and MinIO secrets before deploying.
|
||||||
|
|
||||||
|
## Logging
|
||||||
|
|
||||||
|
Logging is file-only. Console logging is disabled.
|
||||||
|
|
||||||
|
Current Log4j output:
|
||||||
|
|
||||||
|
```text
|
||||||
|
logs/utms-agent.log
|
||||||
|
```
|
||||||
|
|
||||||
|
Rotation:
|
||||||
|
|
||||||
|
```properties
|
||||||
|
log4j.appender.FILE.MaxFileSize=100MB
|
||||||
|
log4j.appender.FILE.MaxBackupIndex=10
|
||||||
|
```
|
||||||
|
|
||||||
|
The Linux service user must be able to write to the `logs/` directory under the service working directory.
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
This is a NetBeans/Ant Java project. If Ant is installed:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ant clean jar
|
||||||
|
```
|
||||||
|
|
||||||
|
The expected runtime artifact is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
dist/utms-agent.jar
|
||||||
|
```
|
||||||
|
|
||||||
|
The jar is run with the dependency jars in `dist/lib`.
|
||||||
|
|
||||||
|
## Manual Build Fallback
|
||||||
|
|
||||||
|
If Ant is not available, compile with Java 8 and package a jar with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
javac -cp "lib/*;lib/Mqtt/*;lib/Quartz/*;lib/Minio-Client/*" -d build/classes <all source files>
|
||||||
|
jar cfm dist/utms-agent.jar build/MANIFEST.MF -C build/classes .
|
||||||
|
```
|
||||||
|
|
||||||
|
On Linux, use `:` instead of `;` in the classpath.
|
||||||
|
|
||||||
|
## Run Locally
|
||||||
|
|
||||||
|
```bash
|
||||||
|
java -jar dist/utms-agent.jar conf/mqtt-agents.cfg
|
||||||
|
```
|
||||||
|
|
||||||
|
The process writes logs to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
logs/utms-agent.log
|
||||||
|
```
|
||||||
|
|
||||||
|
## Linux systemd
|
||||||
|
|
||||||
|
Example unit file:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=UTMS Agent
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=tomcat
|
||||||
|
Group=tomcat
|
||||||
|
WorkingDirectory=/opt/utms-agent
|
||||||
|
ExecStart=/usr/bin/java -jar /opt/utms-agent/utms-agent.jar /opt/utms-agent/conf/mqtt-agents.cfg
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=10
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
```
|
||||||
|
|
||||||
|
Install example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo mkdir -p /opt/utms-agent
|
||||||
|
sudo cp dist/utms-agent.jar /opt/utms-agent/
|
||||||
|
sudo cp -r dist/lib /opt/utms-agent/
|
||||||
|
sudo cp -r conf /opt/utms-agent/
|
||||||
|
sudo mkdir -p /opt/utms-agent/logs
|
||||||
|
sudo chown -R tomcat:tomcat /opt/utms-agent
|
||||||
|
sudo cp utms-agent.service /etc/systemd/system/
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable utms-agent
|
||||||
|
sudo systemctl start utms-agent
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful commands:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl status utms-agent
|
||||||
|
sudo systemctl stop utms-agent
|
||||||
|
tail -f /opt/utms-agent/logs/utms-agent.log
|
||||||
|
```
|
||||||
|
|
||||||
|
## Operational Notes
|
||||||
|
|
||||||
|
- Tanuki and YAJWS wrappers are no longer used.
|
||||||
|
- The app does not log to console.
|
||||||
|
- MQTT worker keepalive values are seconds, not milliseconds.
|
||||||
|
- Queue shutdown is graceful for up to 30 seconds, then forced.
|
||||||
|
- MinIO presigned URLs are regenerated before publish if expired.
|
||||||
|
- `build/`, `dist/`, and `logs/` are ignored by Git.
|
||||||
Reference in New Issue
Block a user