Models & Providers

A Provider is one vendor connection. It is the only place a credential is stored, and the only record that knows where a self-hosted endpoint lives.

Testing takes one of two paths depending on the connection method.

Hosted vendors are verified by calling the vendor's own model-catalogue endpoint with the supplied credential. Each vendor in the catalogue declares its URL and how authentication is carried — a bearer header, a custom header, or a URL template — plus any extra headers the vendor requires, such as an API version. A real catalogue response is proof the key works; anything else is normalised into a specific error with a code and status rather than a raw exception.

Self-hosted endpoints are probed instead. Three candidate paths are tried in order — the Ollama tag listing, the OpenAI-compatible model listing used by tools such as LM Studio and LocalAI, and finally the bare root URL — each with a five-second timeout. The first one that responds successfully proves the server is reachable; if none do, the error names the URL that was tried.

Beyond testing, a provider's live model list can be fetched using its stored credential, returning each model's name, type and — where the vendor publishes them — its token limits and parameter ranges. That is how the model form can offer real choices instead of a free-text field.

A Model record is one usable configuration. The same vendor model can be registered several times with different parameters, different pricing, or scoped to a different environment — production, development or staging — and each is independently selectable.

The modelType field decides what a model is for and which of its fields mean anything. Two families matter most: generate for chat and reasoning, and embeddings for vectors.

You configure one set of parameters. What actually reaches the vendor differs, because the vendors themselves differ — and the mapping layer handles that rather than leaving you to discover it through failed calls.

Unset means unset. A parameter left null is not sent at all, so the vendor applies its own default. Only explicitly configured values are forwarded, which is why a fresh model behaves exactly like the vendor's own defaults until you change something.

There is an optional platform-wide ceiling on output tokens, applied identically to every vendor. It defaults to zero, meaning no code-level cap — the model governs its own limit and whatever it returns is passed through verbatim.

Truncation is detected rather than guessed. Each vendor signals it differently — OpenAI and Ollama report a length stop, Anthropic reports hitting max tokens, Gemini reports its own equivalent — and all of them are normalised to a single truncated flag on the response. Every response also carries the finish reason, the ceiling that was actually requested, and the provider and model it came from, so a truncated answer is diagnosable from the trace instead of appearing as a mysteriously short reply.

Rates live on the model record, expressed per one million tokens in USD. Three figures are held: input price, output price, and a separate rate for cached tokens.

Token usage is tracked per phase, not just as a single total — classification, generation, safety checks, routing and embeddings are counted separately. Each phase is then priced individually, so the summary shows not only what a run cost but where the money went.

A pricing-configured flag travels with the summary. That distinguishes a genuine zero from a model that simply has no rates entered — without it, an unpriced model would look free rather than unmeasured. A model counts as priced when any one of its three rates is non-zero.

• Gated behind its own pricing permission
• Capped at 500 rows per request
• Only the four pricing fields are touched
• Matched by model name, case-insensitively
• Optional title narrows ambiguous matches
• Every matching record is updated, not just the first
• Rows with nothing to change are reported as skipped

Some parts of the platform need a model without anyone choosing one — the classifier, template matching, semantic search and every embedding job. That is what SYSTEM_DEFAULT is for, and it is enforced rather than merely conventional.