Module Deep Dive

Chat & Sessions

What happens between a message arriving and an answer appearing — history, caching, streaming, and the two different things called a session.

Overview

Read this first — especially the part about the word "session", which means two unrelated things.

Chat is where people actually meet an agent. It is the testing ground during development and the interface in production, and the same path serves both — a message sent from the studio and a message arriving from a channel run through the identical sequence.

Underneath it are three moving parts that are easy to confuse. Conversation memory stores the turns and their summaries so the agent has context. The semantic cache tries to answer a repeated question without running the agent at all. And the real-time layer pushes tokens, tool activity and approval requests to the browser while the run is still in progress.

"Session" means two different things here, in two different collections. An auth session records that a person or an API key is signed in — one row per user, per session type, per client. A conversation is a chat thread: a session id that groups messages, carries summaries, and expires on its own idle clock. They share a word and nothing else.

Two kinds of session

Different collections, different lifetimes, different purposes.

Signing in and holding a conversation are tracked entirely separately

The order of a turn

Six stages, in a fixed sequence. The order is load-bearing — the safety check runs before anything touches the database.

Global safety baseline — pattern-based injection check, no database and no model call.
Resolve the agent type — a configured workflow, or a direct model chat.
Fetch conversation history for context.
Try the semantic cache — suppressed for workflow requests, because those carry their own guardrail nodes that a cached answer would bypass.
Run the workflow or the direct model.
Persist and update the cache, including the turn summary.

Modules

Conversation Memory

Messages are stored in a chat-history collection, grouped by session id. Each write also stamps the session's last-updated time and can tag it with the agent that handled the turn, so a conversation list can be filtered per agent without a separate index of its own.

Lifecycle

Idle expiry

A TTL index on the last-updated field, with the window set in App Configuration. Because every message write refreshes that timestamp, the clock measures inactivity rather than age — an active conversation never expires out from under someone.

End vs clear

Ending a conversation marks it completed but keeps the history. Clearing deletes the messages. Two separate actions, because "I'm done with this thread" and "remove this" are different intentions.

TTL setup

Reconciled once per database connection. A reconnect produces a new connection, so a genuinely transient failure is retried rather than being permanently skipped — and a failure never blocks the turn.

Turn summaries

Every turn gets a short summary written alongside it. Those summaries — not the full transcript — are what gets replayed as context on later turns, capped to the most recent few and de-duplicated, and presented to the model as background it should treat as secondary to the current request.

Summaries are generated by a model at low temperature for determinism, with a naive truncation fallback if no model is configured or the call fails — so a summarisation outage degrades quality rather than breaking the turn.

Image turns get a special case, and the reason is instructive. An image agent runs on an image model that does not support chat, so asking it to summarise throws. Worse, the naive fallback would splice the entire base64 blob into the stored summary — the data has no whitespace, so a token-based truncation treats it as a single token and keeps all of it. Image turns are therefore detected and given a short static summary instead. Even the image dimensions are read from outside the encoded data, so a digit run that happens to appear inside the blob is never reported as a size.

The Semantic Cache

Before running an agent, the platform checks whether a semantically similar question has already been answered in this conversation. A match above a cosine similarity of 0.9 returns the stored answer without a model call.

Lookups are scoped to the user and the session, and workflow requests additionally key on the agent and its guardrail — so changing an agent's guardrail invalidates its cached answers rather than silently serving results produced under the old policy.

Why similarity alone is dangerous

Two different people's CVs embed to nearly identical vectors. The natural-language shape of "here is my CV, extract the details" is the same regardless of whose CV it is — so on cosine similarity alone, the second person receives the first person's parsed result. Similarity is the right tool for the instruction and completely the wrong tool for the identity.

The fix is a deterministic fingerprint of everything in a turn that must match exactly rather than approximately. The fingerprint becomes part of the lookup key, while the natural-language instruction is still left to drive semantic matching as normal.

URLs

Normalised to host, path and query so trivial differences do not fragment the cache.

Emails

Lowercased and de-duplicated — a strong per-person identity anchor, and the one thing a plain-prose document reliably carries.

Attachment ids

The specific file the turn operates on.

Embedded payloads

Fenced code blocks and JSON. JSON is canonicalised first — keys sorted, whitespace stripped — so the same data formatted differently still shares one entry, while genuinely different data does not.

Person name

Best-effort extraction, as extra discrimination layered on top of the email.

Order and record ids

A hash-prefixed token, or a bare token mixing letters and digits. Two turns quoting different order numbers must not share an entry even when their prose is identical.

The gates are what make this safe in both directions. Name extraction only fires when the turn already carries an email or a URL — a CV always has contact details, an ordinary question like "what is the weather?" does not, so forcing exact-match on conversational prose would destroy the hit rate. The payload hash is likewise gated on a structured block actually being present, not on the query being long. And over-extraction is deliberately safe: a wrong or empty name only lowers the hit rate — it can never cause a false hit, because the email already pins identity.

Streaming & Stop

While a run is in progress the browser receives typed content-part events over a socket — the same content-block model used by modern assistant APIs. The chat interface keys its renderer on the event type, so text appends, an image swaps a placeholder, and a tool call shows as a chip.

The terminal HTTP response remains the source of truth; these events are the incremental layer on top of it. Clients join a room named after the conversation, so every open tab on the same thread sees the same stream.

Stopping a generation

Each in-flight request registers an abort controller against its session. A stop — whether it arrives over the socket or as an HTTP call — resolves that controller and signals the live provider call. The registry is cleaned up when the request finishes, whether it succeeded, errored or was aborted.

Stopping is not a discard. The partial text is frozen in the bubble with a stopped indicator, the partial output is recorded, and the run is written to analytics with a stopped status rather than vanishing.

Approval gates

A team's human-in-the-loop block emits a review request into the conversation room and waits for a response. Each request carries its own id, the block's name, the prompt, the surrounding context and whether it fires before or after the step.

Unavailability auto-approves rather than hanging. If the real-time layer is not initialised, the gate resolves as approved with that reason recorded — a run is not left blocked forever because nobody could have been asked.

Origins

Allowed socket origins come from configuration. When none is set the wildcard remains, so existing clients keep working — but in production that absence is logged loudly rather than passing silently. Handshake tokens are verified when supplied, and a missing one does not reject the connection, which keeps older clients functional.

Feedback

A thumbs up or down does three things, in a deliberate order.

Tag the analytics record for that turn, so the reaction is visible in dashboards.
Append to a feedback audit collection — symmetric for both ratings, never purged, a full history.
Only then, for a dislike, purge the cached copy so the same answer is not served again.

The durable records are written first, on purpose. A dislike has a destructive side effect — it deletes a cache entry. Recording the reaction in two independent places before any deletion means a dislike is never a silent, unrecoverable delete: the signal survives even though the cached answer does not.

Attachments

Files are uploaded to a dedicated chat directory and recorded with their original name, MIME type, size, path, the uploading user and the conversation they belong to. Stored filenames combine a timestamp with a generated identifier, so two people uploading the same filename never collide.

10 MB per-file limit
Upload directory created on demand
Original extension preserved
Scoped to user and conversation
Deletable individually
Ids feed the cache fingerprint

Attachment text is extracted and folded into the turn's context, which is why an attachment id is one of the exact-match components of the cache key — the same question about a different file is a different question.

Auth Sessions

The other kind of session: a record that an identity is active. Three types — sso, general and api — with a status of active, expired or revoked.

One row per identity

A unique index on email, session type and client. A user signing in repeatedly refreshes one record rather than accumulating rows.

API sessions are upserted

An API invocation either creates the identity's row or refreshes its last-login time and the agent or team it most recently invoked — so API usage is attributable without a sign-in step.

Expiry is checked, then written

Validating a token that has passed its expiry marks the record expired before rejecting it, so a lapsed session does not stay marked active.

Clients resolve two ways

A client reference is accepted either as an identifier or as a registered domain string, so callers are not forced to know the internal id.

Revocation carries a reason

Revoked sessions record why, rather than just disappearing from the active list.

How It Works — Flow Examples

Four diagrams: the turn, the cache decision, stopping, and what a reaction triggers.

Input / output Data handling Decision Gate / side effect Result

Example 1 — One chat turn Turn

Safety first, before any database work. The cache is skipped entirely for workflow runs, because a cached answer would bypass their guardrail nodes.

Six stages in a fixed order — the cheapest checks run before the expensive ones

Example 2 — Deciding a cache hit Cache

The fingerprint must match exactly; only then does similarity get a vote. That split is what keeps two people's documents from sharing an entry.

Exact for identity, similar for instruction — the instruction is the only part left to the embedding

Example 3 — Stopping a generation Stop

One registry serves both the socket event and the HTTP call, so a stop behaves the same however it arrives.

A stop is recorded as an outcome, not treated as a failure

Example 4 — What a reaction triggers Feedback

Two durable records are written before the one destructive step, so the signal outlives the cache entry it removes.

The destructive step happens last, and only for a dislike

Reference Tables

Stream events, session types, and what goes into the cache key.

Stream event types

Event	What the interface does with it
text_delta	Appends tokens to the streaming bubble
image	Swaps a placeholder for the generated image
tool_call	Shows a chip naming the tool being called
status	Displays a progress label with optional detail
response_replace	Replaces the streamed content wholesale
done	Finalises the bubble

Session records

Field	Meaning
Auth session
email	Unified identity, lowercased
sessionType	sso, general or api
clientId	Which client the identity belongs to
status	active, expired or revoked
initialLogin / lastLogin	First and most recent activity
revokedReason	Why a session was revoked
Conversation
sessionId	Groups every message in one thread
agentId	Tags the thread with the agent that handled it
summary	Per-turn summary replayed as later context
status	Set to completed when a thread is ended
updatedAt	Drives the idle expiry clock

Cache key components

Component	Match	Gate
Instruction text	Semantic, above 0.9	Always
User and session	Exact	Always — privacy scope
Agent and guardrail	Exact	Workflow requests only
URLs	Exact, normalised	When present
Emails	Exact, de-duplicated	When present
Attachment ids	Exact	When attached
Embedded payloads	Hash of canonicalised JSON	Only when a structured block exists
Person name	Exact, normalised	Only when an email or URL is present
Record ids	Exact, lowercased	Hash-prefixed, or letter-and-digit mix