Portus · API & AI Gateway

Portus is a Rust-based Kubernetes gateway that handles HTTP, LLM, and MCP traffic through one data plane with Gateway API support and local policy enforcement.

Visit Website
Portus · API & AI Gateway

Introduction

Overview

Portus is an API and AI gateway built in Rust for Kubernetes and standalone deployments. It places conventional HTTP routing, LLM providers, and MCP tool servers behind one data plane, allowing them to share routing and policy controls.

The gateway implements Kubernetes Gateway API resources and provisions a separate Deployment, Service, and PodDisruptionBudget for each Gateway. Configuration is compiled from custom resources into protobuf and streamed to the relevant data plane.

Key Features

  • Supports HTTPRoute, GRPCRoute, TLSRoute, TCPRoute, UDPRoute, ListenerSet, and BackendTLSPolicy.
  • Applies policies for timeouts, retries, rate limits, circuit breakers, health checks, CORS, access lists, body limits, and authentication.
  • Routes LLM requests by model while keeping provider credentials inside the cluster.
  • Tracks token budgets by key, user, tenant, or route, reserving usage before a call and settling it afterward.
  • Routes MCP JSON-RPC calls by method and tool name, with federation across multiple tool servers.
  • Reloads configuration atomically over mutually authenticated gRPC without dropping active connections.

Architecture and Operation

Portus reconcilers watch Gateway API and Portus resources, publish changes into a shared store, and compile a configuration slice for each gateway. The data plane builds route maps by port and hostname, applies Gateway API precedence, passively ejects failing endpoints, and drains in-flight requests during shutdown.

Authentication and budget checks use locally available state so requests do not wait on the optional ledger service. The ledger issues hashed gateway keys, synchronizes shared budget counters, records usage, and obtains signing keys for OAuth issuers.

Deployment Options

In Kubernetes, Portus supports version 1.32 or later, Helm 3.8 or later, and both amd64 and arm64 architectures. A standalone mode can instead load one YAML configuration file and operate as a reverse proxy outside a cluster.

Standalone deployments retain routing, policies, and TLS capabilities. They also support automatic certificate management, hot configuration reloads, and periodic resolution of backend hostnames for services running in environments such as Docker Compose or on virtual machines.