01/Overview
Core Concepts
The building blocks that recur throughout the platform.
The following concepts apply across the gateway. Getting comfortable with them makes flow routing much easier to grasp.
- Node
- A physical LLM server/provider (Ollama, vLLM, OpenAI-compatible, Bedrock, Cursor…). Its health, load and metrics are tracked.
- Model mapping
- Binds a display model name to the real model on a node. The catalog is made of these mappings.
- Model group
- Groups multiple members under one virtual model name and picks among them with a strategy.
- Member
- A member of a group: a model + preferred nodes + capability tags (vision, code…) + cost.
- Strategy
- Determines selection within a group: priority, weighted, round_robin, content, auto or flow.
- Flow
- The superset strategy: routing via an admin-drawn entry → router → member graph, plus failure resilience.
#Strategy at a glance
| Strategy | Selection logic | Typical use |
|---|---|---|
| priority | Lowest priority number | Simple failover |
| weighted | Weighted random | Load spreading |
| round_robin | Cycle in order | Even distribution |
| content | By LLM classifier category | Topic-based routing |
| auto | Input size / vision rules | Automatic model pick |
| flow | Router graph (superset) | Advanced routing |
The flow strategy can contain every other strategy as router nodes — which is why most of this documentation is devoted to it.