Rise Router: Unified Token Entry for Enterprise LLMs

Rise Router is the unified entry and governance plane for all enterprise LLM Token traffic; Rise ModelX is one of Router's inference backends (internal compute · training-inference integrated). Applications integrate only against Router's OpenAI-compatible endpoint; internally, Router dispatches requests to local ModelX or to external managed channels by policy. The split exists because ModelX manages compute (GPU pooling, scheduling, training, inference engines) and serves AI application teams and compute operations; Router manages traffic (unified entry, routing policies, key custody, dual-rail Token billing, compliance audit) and serves business integrators, finance, and compliance. Different concerns, different evolution cadences — shipped as two parallel standalone products but delivered together in customer deployments.

Through tenant / API-key / business-tag access control. Each tenant or API key can be bound to an allowlisted channel set: sensitive workloads such as core trading and risk are restricted to local Rise ModelX (domestic GPUs), while non-sensitive workloads get controlled egress. Every outbound request is forced through the Router audit log, combined with PII detection, sensitive-word filtering, schedules, and rate limits. Router also supports content-aware routing (customizable per customer) as an extra defense layer for sensitive data. Router's role is to govern traffic that must go out; sensitive workloads should stay on the local compute stack.

LiteLLM / OneAPI / Portkey are protocol proxy layers solving multi-upstream adaptation. Rise Router layers enterprise-grade Token governance on top: per tenant / project / business-line Token billing and budget hard-limits, multi-channel cost comparison and smart routing, centralized API key custody with rotation and revocation, full egress audit, PII / sensitive-word interception. The real differentiator isn't the gateway core (open-source implementations are similar) — it's that Rise Router governs internal enterprise GPU compute (via Rise ModelX) and external channels together, on a single governance plane with GPU-hour + Token dual-rail billing, scenario-based internal/external split, and unified audit. Xinference / vLLM / SGLang are inference engines — local backends managed by ModelX, not peer products to Router.

Six channel types: local inference (vLLM / SGLang / MindIE instances managed by Rise ModelX), domestic LLMs (XC-LLM / Zhipu / iFlytek / Kimi), cloud MaaS (Bailian / Qianfan / TI / MA), aggregators (SiliconFlow / Volcano Ark), foreign models (OpenAI / Anthropic / Google — on-demand for workloads that genuinely need them), and enterprise BYOC (customer-specified upstream). Broad channel coverage, all accessed through the same OpenAI-compatible interface — switching paths requires no application code changes.

Upstream vendor API keys and ModelX internal credentials are vaulted on the Router governance plane — application code no longer holds any LLM credential, eliminating key sprawl and leak risk. Supports key rotation (periodic upstream credential refresh transparent to applications), key revocation (individual key instant invalidation), per-tenant/project authorization (teams see only their authorized channels and budgets), and access policies (schedules / IP allowlists / rate limits). Full egress audit logs meet finance and state-enterprise compliance requirements.

Router uses the actual upstream tokenizer to precisely meter prompt / completion tokens per request — not gateway estimates. Multi-dimensional attribution (tenant / project / business-line / API key), time-series aggregation, top-N usage, automatic monthly statements. The same FinOps dashboard renders local GPU-hours + external Tokens in a combined view (a core capability of Router as the unified entry), feeding enterprise finance systems directly by department.

Budget behavior is configurable: per-tenant or business-line hard-limits with an action choice on threshold — auto-downgrade (reroute to cheaper channels), alert without blocking (warn but let traffic through), or hard-block (reject new requests). Beyond budgets: cost anomaly alerts (usage spikes), schedules (off-peak / night dispatch), and rate limits — layered protection against a single tenant or runaway Agent draining the budget.

Full OpenAI-compatible Chat / Completions / Embeddings / Function Call / JSON Mode / streaming SSE support — Agent frameworks can swap upstream models with zero code changes. Multimodal models are covered too: text, image, audio, and video models all flow through Router's unified entry, with Agents directly invoking image understanding, speech recognition, video generation, and more — all subject to the same Token metering, budget control, and compliance audit. Virtual ModelName lets Router auto-route across channels by policy (cost / latency / availability); combined with rate limits and budget hard-limits, autonomous Agents can't run away in external channels — Agent Token consumption rates far exceed human-triggered traffic, making budget control essential for external channel governance.