White paper AI integration showcase 12 min read

Integrating an LLM with an Edge Agent

A governed tool architecture that turns natural-language intent into typed reads, approved actions, and operator-facing business workflows.

An effective Agent assistant is not a prompt wrapped around an unrestricted API. It is a layered contract: the LLM discovers only selected capabilities, reads their specifications on demand, submits structured calls, receives real results, pauses before side effects, and hands asynchronous work back to the local business UI. This showcase follows that design through event analysis and network-device discovery on a running Banalytics Agent.

Banalytics Gemini assistant describing event types while Event History remains open beside the chat — A real operator session: the assistant reasons over registered event contracts while the Agent's own Event History interface remains visible and authoritative.

Planning an LLM interface for operational systems?

Tell us which components, data, and actions operators need to reach. We will help define a typed capability boundary, permission model, and representative validation workflow.

Discuss an Agent assistant

// The integration problem

Natural language is useful; unconstrained execution is not

Operators naturally ask for outcomes: show today's counter statistics, find cameras, explain a fault, restart a bounded service, or open the right panel. The underlying Agent, however, works with component identities, typed method parameters, time boundaries, permission scopes, and asynchronous operations. An LLM can translate between those worlds, but only if the operating layer remains responsible for what is discoverable and executable.

Sending the complete Agent API in every prompt does not scale and gives the model too much irrelevant detail. Allowing arbitrary method names, SQL, HQL, expressions, or shell commands turns a language error into an operational error. Treating a model's prose as evidence of completion is equally dangerous: a sentence that says a scan is running is not the same thing as a local component actually reporting progress.

The architecture described here keeps language interpretation probabilistic and execution deterministic. Gemini decides which declared capability may answer the request. The Banalytics Agent decides whether that capability exists, whether this assistant may use it, whether the parameters conform, whether human approval is required, and what result is returned.

// Reference architecture

Separate provider conversation, Agent governance, and component business logic

Layer	Responsibility	What it must not assume
Operator chat	Capture intent, attachments, approvals, visible progress, tool activity, final answers, and navigation buttons.	That any model sentence proves local work has completed.
Gemini adapter	Translate conversation messages and native function declarations to the Google Generative Language API; return text and structured function calls.	That a provider-generated function call is authorised or valid.
AI toolbox	Build the permitted catalogue, load AI specifications, validate references and arguments, enforce result limits, classify access, and create exact confirmation proposals.	That a method is safe because its name sounds read-only.
Component contract	Expose a small business API with parameter schemas, return meaning, pagination, side-effect description, and domain-specific cautions.	That the LLM understands an internal entity model or serialized payload layout.
Business UI	Show authoritative state, live progress, device results, configuration, and domain operations after the chat has selected the right component.	That chat should duplicate every specialist interface.

Provider-specific interfaces currently live in the Gemini module while that integration is being validated. The critical separation for the running system is already present: the provider adapter speaks model protocol, the core toolbox governs Agent capabilities, and each component owns its business method.

// Capability discovery

Use progressive disclosure instead of a giant prompt

The Agent starts with a compact catalogue. Each line has a short reference, component kind, class, title, state, class summary, and the instance description supplied by the operator. This is enough for Gemini to resolve a phrase such as "event history of this server" or "find devices on the production network" without loading every method in the installation.

Find the purpose. Search or inspect catalogue summaries and choose an instance by its title and site description, not merely by Java class name.
Resolve real ambiguity. If exactly one instance is relevant, use it. If several instances would produce materially different data or effects, ask one short question that names them.
Load the contract. Call describe_components to obtain the versioned AI specification only for the chosen components.
Submit a typed call. Use the declared method and JSON parameter schema. Unknown methods, unknown fields, invalid bounds, and ungranted scopes fail at the Agent boundary.
Continue from evidence. Feed the result back to Gemini so it can page, refine, call the next component, open a UI, or write the final answer.

This progressive disclosure keeps the recurring system prompt small while letting the model understand complex requests. It also makes a capability reviewable: an operator can see which component and method the assistant actually selected.

// AI specification

Describe business capabilities, not implementation internals

A component AI specification contains a concise class summary plus a method list. Every method declares an access class, purpose, JSON Schema parameters, return meaning, and notes that affect safe use. The instance's editable AI description remains short: it identifies what that particular configured component represents on the site.

Contract element	Why the LLM needs it	Example
Purpose summary	Select the correct component without loading its full API.	"Stored history of every event raised on this agent."
Access class	Let the Agent decide whether execution is immediate or must become a confirmation proposal.	`read` for statistics; `action` for subnet scanning.
Parameter schema	Reject guessed fields, invalid dates, oversized pages, traversal strings, and other malformed inputs before business code runs.	A half-open time range or an address and mask returned by the component.
Return semantics	Prevent plausible but wrong interpretation of values.	`eventCount` counts rows; `valueTotal` sums counter deltas.
Operational note	Guide multi-step behavior that a signature alone cannot express.	After an asynchronous scan starts, call `show_in_ui` in the same turn.

Short defaults should be supplied for every Thing, Task, and Action description, then supplemented by the owner when AI access is enabled. The default explains the class; the owner's text explains the instance. Neither should contain credentials or replace the formal method schema.

// Showcase 1: analytical read

Calculate CountingEvent statistics without exposing arbitrary queries

The operator first asks which system events exist and then requests the CountingEvent specification. Gemini uses describe_event_types to obtain the registered event contracts. It learns that key identifies what was counted, value is the delta in one event, and the correct historical workflow is EventHistoryThing.readCountingStatistics.

Gemini explains the CountingEvent contract and asks for the date boundary needed to calculate statistics — The model asks for the missing dates because choosing a two-day period changes the result; it does not ask the operator to reconfirm read access already granted to the component.

After the operator specifies 8 to 9 August 2026, Gemini resolves the Event History instance, loads its contract, and calls the typed aggregate. The method filters stored rows by the real EventStore fields needed for the time window and EVT_COUNTING, deserializes the typed payload, groups by key, and returns both counts and value totals.

Gemini returns CountingEvent totals and groups for the selected two-day period — The assistant reports 69 event rows, a total counting value of 77, and the per-key groups returned by the Agent for the requested period.

This method boundary matters. EventStore has row fields such as id, nodeUuid, dateTime, and messageType; the event itself is a serialized payload. CountingEvent.key and CountingEvent.value are not queryable EventStore columns. Giving an LLM arbitrary HQL would not simplify that semantic mismatch: it would expand authority while preserving the possibility of a wrong query. A typed aggregate makes the intended calculation portable, testable, bounded, and understandable to both the model and the operator.

// Showcase 2: approved action

Find devices, approve the scan, and hand off to the business UI

The next request is conversationally simple: "help me find network devices." The assistant resolves the enabled Network Device Discovery instance and reads its specification. Because a machine can have several interfaces, it first calls readAvailableSubnets. Two candidates are returned, so Gemini asks which one the operator intends to scan.

Gemini lists two local network interfaces and asks the operator which subnet to scan — The chosen address and mask are copied from the component result. The model does not fabricate a subnet from the operator's wording.

scanSubnet creates network traffic and starts work in the background, so its AI specification classifies it as an action. The Agent does not execute the model proposal immediately. It displays the component, method, human-readable effect, and exact arguments. Approval is tied to that pending call and can be used once.

Operator confirmation for scanSubnet followed by a button that opens Network Device Discovery — Once approved, the stored proposal executes exactly once. Gemini then calls `show_in_ui` and returns a button rather than producing repeated chat messages or claiming that the asynchronous scan is already complete.

The operator opens the component panel to watch discovery progress and inspect the resulting hosts, cameras, web services, RTSP endpoints, and MQTT candidates. The specialist UI remains responsible for live state; chat remains responsible for understanding the request and providing the governed transition into that workflow.

Network Device Discovery business UI listing discovered hosts and service candidates — Business UI hand-off avoids forcing the LLM to poll a long-running scan or flatten a rich operational result into chat.

// Governance

Apply least privilege at the assistant and component levels

The Gemini assistant defines global ceilings for six scopes: read, write, action, lifecycle, configuration, and UI. The operator then grants a subset to each component instance. A read-only Event History can therefore coexist with action and UI access to one Device Discovery instance without exposing every component on the Agent.

Fail closed on description and access metadata. A missing or invalid AI specification must not silently create a callable read method.
Validate before presenting approval. The target, method, action type, parameter schema, and current grant should be valid before an operator sees a proposal.
Bind approval to exact arguments. Confirmation for one subnet scan must not authorise a scan of another subnet or a different action.
Consume confirmation once. A retry after success, crash, or duplicate UI event must not repeat a side effect without a new operator decision.
Keep read disclosure in scope. Read calls do not need a confirmation card, but their results may be sent to an external model provider and still require access control and data minimisation.
Separate navigation from execution. UI scope permits a trusted button; it does not grant the model the user's business-panel permissions.

// Conversation lifecycle

Make progress visible without exposing private model reasoning

Tool-based turns can take several provider requests. The panel therefore writes safe progress states to durable chat history before model and tool steps. Operators see that the assistant is preparing context, contacting Gemini, executing a named tool, waiting for approval, resuming, completing, failing, or being interrupted. Tool rows can be expanded for evidence; private chain-of-thought is neither requested nor displayed.

The assistant uses one conversation worker. A new send is rejected while work is active, preventing overlapping turns and duplicate UI results. Approval or rejection resumes the stored conversation once. Stopping the component interrupts active work and leaves a terminal state; merely reopening a stopped panel can still read status without generating a recurring "component not initialized" failure.

A strict turn rule is essential: there is no background model execution after Gemini returns text without another function call. The assistant must never say "wait, I will continue" and then stop. It either emits the next structured call in the current loop or gives the best complete answer available, including the exact coverage if pagination or a configured iteration limit was reached.

// Failure boundaries

Design for model, provider, data, and component failure separately

Failure	Required behavior	Operator evidence
Model chooses the wrong component	Use title and instance description; ask one concise question when several choices materially differ.	Visible component reference and specification/tool row.
Model invents a method or argument	Reject it against the loaded AI specification and JSON Schema.	Error names the invalid call without executing business code.
Provider stops after promising more work	Treat text without a function call as terminal; prohibit future-work promises in the system instructions.	One terminal request status, not a silent spinner.
Tool result is too large	Use typed filtering, aggregation, pagination, and result caps; report incomplete coverage.	Call arguments, totals, page state, and truncation status.
Action waits for approval	Persist the exact proposal and pause the tool loop without retrying or choosing an alternative route.	Confirmation card with target, effect, and arguments.
Local operation is asynchronous	Start it once and offer the component business UI for live progress.	Open component button and authoritative local panel.
GeminiThing stops or restarts	Interrupt active conversation work, keep durable history, and never silently replay an unresolved side effect.	Stopped/interrupted status and retained confirmation outcome.

// Validation plan

Test representative operator journeys, not only tool endpoints

Catalogue selection. Create two similarly named components and verify the assistant chooses from instance context or asks once when the difference matters.
Typed analytical read. Ask for a period, grouping, and totals; compare the answer to the component API and verify paging or aggregate coverage.
Permission denial. Remove a per-component grant and confirm the assistant reports the component and missing scope without attempting another route.
Exact approval. Inspect the proposed component, method, description, and arguments; approve once, then verify duplicate clicks or transport retries do not repeat it.
UI hand-off. Run an asynchronous operation and verify one usable navigation button appears in the same completed turn.
Lifecycle interruption. Stop the assistant during provider work and while waiting for approval; confirm history and status remain understandable after restart.
External data review. Inspect every catalogue field, description, specification, tool result, and attachment type that can reach the provider.
Cost and latency envelope. Measure requests per operator turn, context size, output budget, reasoning level, tool-result size, and worst-case iteration limit.

// Takeaway

Make the LLM an interpreter inside an operational contract

The useful part of LLM integration is its ability to understand intent across domain language: "today", "counter keys", "the production network", or "show me where this is running". The trustworthy part comes from everything around the model: compact discovery, explicit AI specifications, typed component methods, fail-closed permissions, exact confirmation, durable progress, and authoritative business UI.

This pattern avoids two unhelpful extremes. The assistant is more capable than a fixed command menu because it can plan multi-step reads and choose components from natural language. It is safer and easier to test than an unrestricted agent because it can do only what the local, versioned contracts and current grants allow. That is the architecture that lets chat become a practical operator interface rather than a second, opaque control plane.

For configuration details, see the Gemini AI Assistant. For deployment roles and external data boundaries, see AI Modules and Deployment Responsibility.