Red Hat AI Inference 3.5 MaaS governance on AKS

Red Hat AI Inference 3.5 adds the Model-as-a-Service (MaaS) governance layer to Azure Kubernetes Service (AKS), letting platform engineers enforce API key authentication, subscription tiers, and rate limits on models served through vLLM and KServe. The inference stack and the governance layer now run on non-OpenShift Kubernetes distributions, and the MaaS components install alongside the existing platform without modifying it.

What MaaS adds to the platform

The inference platform includes cert-manager, Istio, KServe, llm-d, LeaderWorkerSet, the Red Hat AI Inference operator, and the cloud manager. MaaS adds two layers on top: Red Hat Connectivity Link (RHCL), the productized Kuadrant, which handles authentication and rate limiting, and MaaS itself, which manages subscriptions, API keys, and model access control.

The request flow runs through the MaaS Gateway, then Authorino validates the API key or JWT, Limitador checks the token rate limit, and the request reaches the model backend, vLLM via KServe. Failure cases map to specific responses: no API key or an invalid key returns 401 Unauthorized, a valid key with the wrong subscription tier returns 403 Forbidden, a valid key with exceeded quota returns 429 Too Many Requests, and a valid request returns 200 with model output.

On a vanilla Kubernetes distribution, MaaS requires external tools that OpenShift provides built-in. The deployment uses Helm charts for installation, cert-manager for TLS certificates, and an OIDC-compliant identity provider such as Azure Active Directory on AKS.

Deploying a model before governance

The starting point is a running model with no authentication, no rate limiting, and no API keys. Applying a single LLMInferenceService Custom Resource Definition deploys a Qwen3-0.6B model. The llmisvc-controller-manager creates a Deployment for the vLLM server, a router-scheduler Deployment for request routing, an InferencePool for load balancing, an HTTPRoute connecting the gateway to the model, a Service for internal networking, and a self-signed TLS certificate.

On the test cluster, an AKS deployment in the East US region with Kubernetes 1.34.8, 3 CPU nodes and 1 GPU node with an NVIDIA T4, the model showed Ready: True with an externally routable URL. Two pods ran: the vLLM server with Qwen3-0.6B, which started with an init container downloading model weights from Hugging Face and then launched vLLM v0.24.0 with the CUDA runtime image, and the router-scheduler distributing inference traffic. The GPU pod scheduled onto the T4 node because it requested nvidia.com/gpu: 1. MaaS runs on CPU, only model serving requires GPU.

A direct call to the model through the inference gateway returned 200 with no API key, no authentication, and no rate limiting. The endpoint is OpenAI-compatible and supports all vLLM serving arguments, including disabling Qwen3's chain-of-thought reasoning. The same request through the MaaS gateway also returned 200 because no MaaSSubscription or MaaSAuthPolicy resources existed yet. Anyone with the IP address could call any model, consume unlimited GPU resources, and usage was untracked.

Inside the MaaS deployment

The governance layer installs across several namespaces. The kuadrant-operators namespace holds four operator controllers: kuadrant-operator-controller-manager, authorino-operator, limitador-operator-controller-manager, and dns-operator-controller-manager. The kuadrant-system namespace runs authorino, which validates API keys and JWTs, and limitador-limitador, which enforces rate limits. The redhat-ods-applications namespace contains ai-gateway-operator, maas-controller, maas-default-gateway-istio, payload-processing, and payload-pre-processing. The redhat-ai-gateway-infra namespace runs maas-api, the REST API for subscriptions and API keys, and maas-postgres, a PostgreSQL instance storing API key hashes. MaaS subscription and auth policy resources go in the models-as-a-service namespace.

Red Hat Connectivity Link provides the policy enforcement. Authorino validates API keys against the MaaS API and validates JWT tokens against the configured OIDC provider, on AKS that is Azure Active Directory. Limitador checks token counters against rate limits defined in each subscription. RHCL installs 16 CRDs, but only authpolicies.kuadrant.io and tokenratelimitpolicies.kuadrant.io matter for MaaS. Users do not create these directly. When a MaaSSubscription is created, the maas-controller translates it into a Kuadrant TokenRateLimitPolicy, and Limitador enforces that policy at the gateway on every request.

The AI Gateway Operator coordinates the deployment chain. The Helm chart installs the underlying platform manager, opendatahub-operator, and creates a Platform CR with MaaS enabled. The opendatahub-operator detects the platform type, AKS, and deploys ai-gateway-operator. The ai-gateway-operator reads the AIGateway CR, selects xKS-specific manifests, and deploys maas-controller. The maas-controller sets up the MaaS API, gateway, auth policies, and rate limit policies. Helm values enable this chain with components.aigateway.managementState set to Managed and components.modelsAsAService.managementState set to Managed.