Integrating Python ML into a Java Control System Without Rewriting the Stack

Keep acquisition, inference, decisions and actuation as separate responsibilities

The prototype began with three independent assets: an existing Java control environment, a Python computer-vision service, and an RC car whose front lights could be controlled over MQTT. Replacing the Java runtime with a Python application would have discarded mature camera, event, dashboard and device-integration capabilities. Embedding CPython into the JVM would have coupled dependency lifecycles and made failures harder to isolate.

The selected design treats Python as a local processing module with a narrow message contract. JPEG frames travel from Java to Python over ZeroMQ. Python returns compact semantic events such as MOTION and NO_MOTION. Banalytics receives those events, evaluates an explicit rule, and publishes the requested state through MQTT. The device-side Agent then converts the MQTT state into a GPIO or controller action.

Integrate around the existing control system rather than replacing it

The same pattern works when the Python side contains YOLO, segmentation, OCR, anomaly detection or a custom scientific model. The model is not allowed to become the whole application; it remains one replaceable stage in a larger operational pipeline.

Two compute nodes, three protocol boundaries and one physical outcome

The industrial workstation runs camera acquisition, JPEG conversion, ZeroMQ exchange, Event Manager and the MQTT publisher. A Python process on the same workstation subscribes to frames and publishes motion-state events. The RC car carries another Banalytics Agent, subscribes to the selected MQTT topic and applies the resulting ON/OFF state to its lighting hardware.

The demonstrator used a RabbitMQ MQTT endpoint between the two Agents. That broker placement is not a platform requirement: a DIY build can use the embedded MQTT Server or another broker on the local network. What matters is the topic and payload contract, not where the broker runs.

Follow the frame from camera acquisition to the headlights

Expose every boundary on the integration dashboard

The demonstrator dashboard places the frame sender, Python process, event receiver, Event Manager rules, MQTT publisher, remote subscriber and hardware actions in one operational view. The numbered annotations correspond to the architecture diagram. This makes it possible to ask a precise question when the lights do not react: was a frame sent, did Python emit an event, did the rule match, was MQTT published, and did the device-side action run?

Send binary frames in one direction and small state events in the other

The frame channel carries JPEG bytes rather than Java or Python objects. This removes language-specific serialization and lets the receiver use numpy.frombuffer() followed by cv2.imdecode(). The result channel deliberately carries a much smaller vocabulary: the prototype emits MOTION on transition into the active state and NO_MOTION after a quiet interval.

A compact OpenCV motion detector with transition events

The original service subscribes to JPEG frames on port 5555 and publishes state transitions on port 5556. It uses frame differencing rather than a trained model, which keeps the example easy to reproduce while exercising the same process and protocol boundaries that a larger model would use.

import zmq
import numpy as np
import cv2
from datetime import datetime
import time

INPUT_ENDPOINT = "tcp://localhost:5555"  # input with JPEG
OUTPUT_ENDPOINT = "tcp://*:5556"         # sending events

context = zmq.Context()

# Receiver (JPEG frames)
receiver = context.socket(zmq.SUB)
receiver.connect(INPUT_ENDPOINT)
receiver.setsockopt_string(zmq.SUBSCRIBE, "")

# Sender (events)
sender = context.socket(zmq.PUB)
sender.bind(OUTPUT_ENDPOINT)

print(f"Listening on {INPUT_ENDPOINT}, sending events to {OUTPUT_ENDPOINT}")

prev_frame = None

CONTOUR_THRESHOLD = 3000
BLUR_SIZE = (5, 5)
THRESHOLD = 25

MOTION_COOLDOWN = 1.0
NO_MOTION_INTERVAL = 3.0

last_motion_time = 0
motion_active = False

while True:
    data = receiver.recv()

    np_arr = np.frombuffer(data, np.uint8)
    frame = cv2.imdecode(np_arr, cv2.IMREAD_COLOR)
    if frame is None:
        continue

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    gray = cv2.GaussianBlur(gray, BLUR_SIZE, 0)

    if prev_frame is None:
        prev_frame = gray
        continue

    delta = cv2.absdiff(prev_frame, gray)
    thresh = cv2.threshold(delta, THRESHOLD, 255, cv2.THRESH_BINARY)[1]
    thresh = cv2.dilate(thresh, None, iterations=2)

    contours, _ = cv2.findContours(
        thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
    )

    motion = False
    max_area = 0

    for contour in contours:
        area = cv2.contourArea(contour)
        if area > CONTOUR_THRESHOLD:
            motion = True
            max_area = max(max_area, area)

    now_time = time.time()

    if motion:
        if not motion_active and now_time - last_motion_time > MOTION_COOLDOWN:
            timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
            message = "MOTION"
            sender.send_string(message)
            print(
                f"[{timestamp}] Motion Detected - "
                f"Zone size: {int(max_area)} px | Sent: {message}"
            )
            motion_active = True

        last_motion_time = now_time
    elif motion_active and now_time - last_motion_time > NO_MOTION_INTERVAL:
        timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
        message = "NO_MOTION"
        sender.send_string(message)
        print(f"[{timestamp}] No motion detected | Sent: {message}")
        motion_active = False

    prev_frame = gray

The cooldown prevents rapid repeated activation events. The quiet interval provides hysteresis before the inactive state is announced. For a detector that operates per camera or per zone, keep this state separately for every source key rather than sharing one prev_frame and one timer.

The device side remains a normal MQTT-controlled embedded system

The RC platform contains its own embedded computer, Banalytics Agent, MQTT subscriber and lighting actions. That separation is important: Python requests a semantic state, but the car-side node decides how that state maps to Arduino, Modbus or GPIO hardware. A local rule can also reject a command when the system is in manual mode, the connection is stale or another interlock is active.

Harden the boundaries before the model is allowed to affect equipment

ZeroMQ PUB/SUB is intentionally simple and does not by itself provide durable delivery or a complete control audit. MQTT can provide QoS and retained state, but QoS is not a substitute for command expiry, acknowledgement and hardware feedback. Choose semantics according to the physical consequence of a missed, duplicated or delayed message.

Reproduce the pattern in small, testable increments

1. Choose a harmless output. Begin with an LED, dashboard indicator or notification. Do not begin with motion, heat, access control or another consequential actuator.
2. Prove acquisition independently. Configure a USB, RTSP or ONVIF camera and confirm that live video remains usable without the Python process.
3. Create the frame channel. Use a ZeroMQ Socket and an explicit frame sender. Limit the resolution and rate before measuring Python throughput.
4. Run the reference service. Install Python, PyZMQ, NumPy and OpenCV in an isolated environment. Verify MOTION/NO_MOTION output with no actuator connected.
5. Ingest results. Use a ZeroMQ Socket Listener to convert Python output into Banalytics events.
6. Add an explainable rule. Configure Event Manager with separate ON and OFF branches, cooldowns and a visible disabled state.
7. Add MQTT last. Connect an MQTT v3 Connector or local broker and publish to a dedicated test topic before attaching hardware.
8. Commission the device side. Map the topic to a low-risk output through Pi4J, Arduino, Modbus or another controlled interface. Verify manual override and stale-command behaviour.
9. Introduce failure tests. Stop Python, disconnect MQTT, delay frames, restart either Agent and send duplicate events. The resulting state should be observable and safe.

A practical starter bill of materials is deliberately modest: one PC or mini PC for Banalytics and Python, a USB or network camera, a local MQTT broker, and a low-voltage LED output on a Raspberry Pi or microcontroller. The RC platform is useful for demonstration, but the integration pattern can be proven on a desk before it is mobile.

Replace the model without changing the control contract

Motion detection is only one implementation of the Python stage. A later service can run YOLO, segmentation, pose estimation, OCR, acoustic analysis or a domain-specific model while continuing to publish the same versioned event schema. The Java side remains responsible for camera lifecycle, orchestration, operator visibility and device integration. The physical node remains responsible for its own hardware mapping and interlocks.

This division also helps a small DIY project grow into a multi-device lab. Frame processors can move to a GPU workstation, multiple Agents can publish through MQTT, and dashboards can aggregate events without requiring model code to learn the details of every camera and actuator. The boundaries are the reusable asset.