utms-agent/README.md

UTMS Agent

UTMS Agent is a Java 8 background service that connects to an MQTT broker, consumes terminal messages, stores incoming data in PostgreSQL, and publishes download/profile messages back to terminals. It is designed to run as a standard Linux systemd service without Tanuki or YAJWS wrappers.

Runtime Overview

The application entrypoint is id.iptek.utms.agent.Main.

Startup flow:

1. Load conf/mqtt-agents.cfg.
2. Initialize Log4j from conf/mqtt-agents-log4j.cfg.
3. Initialize PostgreSQL access through DBConn.
4. Initialize a global MQTT client for scheduled task publishing.
5. Initialize a MinIO client for presigned download URLs.
6. Load terminal ID cache asynchronously.
7. Start queue handlers for heartbeat, diagnostic, and device init messages.
8. Start configured MQTT consumer workers.
9. Start Quartz scheduler for pending download tasks.

Shutdown flow:

1. Stop Quartz scheduler so no new publish jobs start.
2. Drain queue handlers gracefully for up to 30 seconds.
3. Force interrupt queue work if the graceful timeout is exceeded.
4. Stop MQTT workers.
5. Disconnect the global MQTT client.
6. Stop the executor service.

Project Layout

src/id/iptek/utms/agent/
  Main.java                         Application lifecycle and service entrypoint
  AppConfig.java                    Properties config loader
  db/                               Database connection and utility classes
  dao/                              PostgreSQL data access objects
  model/                            Message, task, app, profile, and terminal models
  queue/                            In-memory delay queues and queue handlers
  scheduler/job/                    Quartz jobs for download task scheduling/publishing
  worker/                           MQTT consumer workers
  util/                             Shared utility classes

conf/
  mqtt-agents.cfg                   Main runtime configuration
  mqtt-agents-log4j.cfg             File-only rolling log configuration

lib/                                Runtime dependency jars
utms-agent.service                  Example systemd unit
build.xml                           NetBeans/Ant project file

Ignored generated/runtime folders:

build/
dist/
logs/

Main Components

Main

Main.java owns the service lifecycle. It accepts one required argument:

java -jar utms-agent.jar /opt/utms-agent/conf/mqtt-agents.cfg

It also accepts the old wrapper-style argument form for compatibility:

java -cp ... id.iptek.utms.agent.Main id.iptek.utms.agent.Main /opt/utms-agent/conf/mqtt-agents.cfg

MQTT Workers

MQTT workers are created dynamically by WorkerFactory from the mqtt.consumers property.

Configured consumers:

mqtt.consumers = heartbeat_c, diagnostic_c, device_init_c, download_ack_c

Each consumer has its own:

• topic
• thread count
• client ID prefix
• MQTT credentials
• keepalive
• clean session setting
• reconnect setting
• worker class
• queue mode where applicable

Worker classes:

Worker	Purpose
HeartbeatWorker	Consumes HEARTBEAT messages and queues them for persistence.
DiagnosticWorker	Consumes DIAGNOSTIC messages and queues diagnostic details.
DeviceInitWorker	Consumes device init requests and queues profile response work.
DownloadTaskAckWorker	Consumes download ACK messages and sends confirm ACKs when needed.

The base Worker class uses Eclipse Paho MQTT. It supports automatic reconnect and has a reconnect delay when the broker is unavailable.

Queues

DelayMessageQueue<T> is the internal queue abstraction.

Supported modes:

Mode	Behavior
SINGLE	Each message is handled separately.
BATCH	Messages are collected until capacity is reached or the flush interval expires.

The queue shutdown is graceful. During service stop, queues are asked to drain before workers are disconnected. If queue work does not finish within 30 seconds, remaining work is forced to stop.

Scheduler

Quartz runs DownloadTaskSchedulerJob every scheduler.periode.minutes.

The scheduler:

1. Finds pending download tasks.
2. Schedules DownloadTaskPublisherJob.
3. Publishes task messages over MQTT.
4. Regenerates expired MinIO presigned URLs before publishing.
5. Updates last broadcast timestamps in the database.

MinIO URLs

DownloadTaskPublisherJob uses MinIO presigned URLs for app and icon downloads.

Relevant config:

fileserver.endpoint = https://...
fileserver.bucket = application-bucket
fileserver.icon.path = icons
fileserver.icon.expiry = 7
fileserver.icon.timeunit = DAYS
fileserver.app.path = apps
fileserver.app.expiry = 7
fileserver.app.timeunit = DAYS

If a stored URL expiry is null or in the past, the publisher generates a fresh URL before sending the MQTT message.

Configuration

Main config file:

conf/mqtt-agents.cfg

Important sections:

Section	Keys
Logging	log4j.file
Database	db.driver, db.url, db.user, db.password
MQTT global publisher	mqtt.broker.url, mqtt.clientid_prefix, mqtt.user, mqtt.password, mqtt.keepalive, mqtt.maxinflight, mqtt.cleansession, mqtt.autoreconnect
Consumers	mqtt.consumers, mqtt.consumer.<name>.*
Pending task query	pendingtask.query.limit, pendingtask.periode.minutes
MinIO	fileserver.*
Scheduler	scheduler.periode.minutes

mqtt.keepalive and mqtt.consumer.<name>.keepalive are in seconds because Eclipse Paho expects seconds.

Do not commit production credentials. Replace database, MQTT, and MinIO secrets before deploying.

Logging

Logging is file-only. Console logging is disabled.

Current Log4j output:

logs/utms-agent.log

Rotation:

log4j.appender.FILE.MaxFileSize=100MB
log4j.appender.FILE.MaxBackupIndex=10

The Linux service user must be able to write to the logs/ directory under the service working directory.

Build

This is a NetBeans/Ant Java project. If Ant is installed:

ant clean jar

The expected runtime artifact is:

dist/utms-agent.jar

The jar is run with the dependency jars in dist/lib.

Manual Build Fallback

If Ant is not available, compile with Java 8 and package a jar with:

javac -cp "lib/*;lib/Mqtt/*;lib/Quartz/*;lib/Minio-Client/*" -d build/classes <all source files>
jar cfm dist/utms-agent.jar build/MANIFEST.MF -C build/classes .

On Linux, use : instead of ; in the classpath.

Run Locally

java -jar dist/utms-agent.jar conf/mqtt-agents.cfg

The process writes logs to:

logs/utms-agent.log

Linux systemd

Example unit file:

[Unit]
Description=UTMS Agent
After=network.target

[Service]
Type=simple
User=tomcat
Group=tomcat
WorkingDirectory=/opt/utms-agent
ExecStart=/usr/bin/java -jar /opt/utms-agent/utms-agent.jar /opt/utms-agent/conf/mqtt-agents.cfg
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

Install example:

sudo mkdir -p /opt/utms-agent
sudo cp dist/utms-agent.jar /opt/utms-agent/
sudo cp -r dist/lib /opt/utms-agent/
sudo cp -r conf /opt/utms-agent/
sudo mkdir -p /opt/utms-agent/logs
sudo chown -R tomcat:tomcat /opt/utms-agent
sudo cp utms-agent.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable utms-agent
sudo systemctl start utms-agent

Useful commands:

sudo systemctl status utms-agent
sudo systemctl stop utms-agent
tail -f /opt/utms-agent/logs/utms-agent.log

Operational Notes

• Tanuki and YAJWS wrappers are no longer used.
• The app does not log to console.
• MQTT worker keepalive values are seconds, not milliseconds.
• Queue shutdown is graceful for up to 30 seconds, then forced.
• MinIO presigned URLs are regenerated before publish if expired.
• build/, dist/, and logs/ are ignored by Git.