Inference Gateway UI

The Inference Gateway UI is a Next.js application that provides a user-friendly interface to interact with AI models through the Inference Gateway service. It enables easy access to various language models through a consistent interface, streamlining the process of sending requests and receiving responses.

Table of Contents

• Table of Contents
• Key Features
• Development
• Configuration
  • General Settings
  • Storage Settings
  • Authentication Settings
  • Keycloak Auth Provider
• Docker
• Kubernetes Deployment
  • Quick Installation
  • Deployment Options
    • 1. Combined Deployment with Inference Gateway Backend
    • 2. UI-Only Deployment (Connecting to External Gateway)
    • 3. Deployment with Ingress for External Access
  • Key Configuration Parameters
  • Complete Example
• Deployment
• Related Projects
• License
• Contributing

Key Features

• 🎨 Modern Interface: Clean, responsive design with Next.js 15 and Tailwind
• 🔌 Seamless API Integration: Connects directly to the Inference Gateway API
• 🛠️ Model Selection: Support for multiple language models
• 💬 Chat Interface: Intuitive chat UI for interacting with AI models
• 🧰 Tool Support: Enables AI models to use tools, including web search
• 🔍 Web Search: Integrated web search for models to retrieve current info
• 🔒 Authentication: Optional authentication using NextAuth.js
• 📱 Responsive Design: Works on desktop and mobile devices
• 🐳 Docker Support: Easy containerization and deployment
• ☸️ Kubernetes Ready: Includes configurations for Kubernetes deployment
• 🧩 Component Library: Built with shadcn/ui components and Radix UI
• 🔄 State Management: Efficient state management with React hooks

Development

# Install dependencies
npm install

# Run development server
npm run dev

The development server will be available at http://localhost:3000.

Configuration

The UI can be configured using the following environment variables:

General Settings

Variable	Default	Description
NODE_ENV	development	Node environment
PORT	3000	Port to run the application on
HOSTNAME	0.0.0.0	Hostname to bind to
INFERENCE_GATEWAY_URL	http://localhost:8080	Inference Gateway URL
LOG_LEVEL	debug	Logging level (debug in dev, info in prod)

Storage Settings

Variable	Default	Description
STORAGE_TYPE	local	Storage type for chat history
DB_CONNECTION_URL	-	Connection URL (required for postgres)

Examples:

• Local storage (default): No connection URL needed
• PostgreSQL: postgresql://username:password@host:port/database

Authentication Settings

Variable	Default	Description
AUTH_ENABLE	false	Enable authentication
AUTH_SECURE_COOKIES	false	Use secure cookies (true for HTTPS)
NEXTAUTH_URL	http://localhost:3000	URL of this application
NEXTAUTH_SECRET	-	Secret to encrypt session cookies
NEXTAUTH_TRUST_HOST	true	Trust the host header from proxy
NEXTAUTH_REFRESH_TOKEN_ENABLED	true	Enable refresh token rotation

Keycloak Auth Provider

Variable	Default	Description
AUTH_OIDC_KEYCLOAK_CLIENT_ID	app-client	OIDC client ID
AUTH_OIDC_KEYCLOAK_CLIENT_SECRET	-	OIDC client secret
AUTH_OIDC_KEYCLOAK_ISSUER	(see note)	OIDC issuer URL
AUTH_OIDC_KEYCLOAK_SCOPES	openid profile email	OIDC scopes

Default issuer: http://localhost:8080/realms/app-realm

Github Auth Provider

Variable	Default	Description
AUTH_OIDC_GITHUB_CLIENT_ID	app-client	OIDC client ID
AUTH_OIDC_GITHUB_CLIENT_SECRET	-	OIDC client secret
AUTH_OIDC_GITHUB_ISSUER	(see note above)	OIDC issuer URL
AUTH_OIDC_GITHUB_SCOPES	read:user user:email	OIDC scopes

Google Auth Provider

Variable	Default	Description
AUTH_OIDC_GOOGLE_CLIENT_ID	app-client	OIDC client ID
AUTH_OIDC_GOOGLE_CLIENT_SECRET	-	OIDC client secret
AUTH_OIDC_GOOGLE_ISSUER	(see note above)	OIDC issuer URL
AUTH_OIDC_GOOGLE_SCOPES	(see below)	OIDC scopes

Default Google scopes: userinfo.email and userinfo.profile

Docker

Pre-built container images are available on the GitHub Container Registry. You can use these images directly:

# Pull the pre-built image
docker pull ghcr.io/inference-gateway/ui:latest

# Run the container with the pre-built image
docker run -p 3000:3000 \
  -e INFERENCE_GATEWAY_URL=http://localhost:8080 \
  ghcr.io/inference-gateway/ui:latest

Alternatively, you can build the image locally:

# Build the Docker image locally
docker build -t inference-gateway-ui --target dev .

# Run the container with the locally built image
docker run -p 3000:3000 \
  -v $(pwd):/app \
  -e INFERENCE_GATEWAY_URL=http://localhost:8080 \
  inference-gateway-ui

Kubernetes Deployment

The Inference Gateway UI provides a Helm chart for easy deployment to Kubernetes environments. The chart is available as an OCI artifact, which can be deployed directly without adding a Helm repository.

Quick Installation

helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --version 0.7.1 \
  --create-namespace \
  --namespace inference-gateway \
  --set gateway.enabled=true \
  --set gateway.envFrom.secretRef=inference-gateway

Deployment Options

The Helm chart supports multiple deployment scenarios:

1. Combined Deployment with Inference Gateway Backend

Deploy the UI with the Inference Gateway backend as a dependency (recommended):

helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --create-namespace \
  --namespace inference-gateway \
  --set gateway.enabled=true \
  --set gateway.envFrom.secretRef=inference-gateway

2. UI-Only Deployment (Connecting to External Gateway)

Deploy the UI separately and connect it to an existing Inference Gateway instance:

helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --create-namespace \
  --namespace inference-gateway \
  --set gateway.enabled=false \
  --set-string "env[0].name=INFERENCE_GATEWAY_URL" \
  --set-string "env[0].value=http://your-gateway-service:8080"

3. Deployment with Ingress for External Access

Enable ingress for accessing the UI from outside the cluster:

helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --create-namespace \
  --namespace inference-gateway \
  --set gateway.enabled=true \
  --set gateway.envFrom.secretRef=inference-gateway \
  --set ingress.enabled=true \
  --set ingress.className=nginx \
  --set "ingress.hosts[0].host=ui.inference-gateway.local" \
  --set "ingress.hosts[0].paths[0].path=/" \
  --set "ingress.hosts[0].paths[0].pathType=Prefix" \
  --set "ingress.tls[0].secretName=inference-gateway-ui-tls" \
  --set "ingress.tls[0].hosts[0]=ui.inference-gateway.local"

Key Configuration Parameters

Parameter	Description	Default
replicaCount	Number of UI replicas to deploy	1
image.repository	UI image repository	ghcr.io/inference-gateway/ui
image.tag	UI image tag (defaults to appVersion)	""
gateway.enabled	Deploy the Gateway backend with UI	true
ingress.enabled	Enable ingress for external access	false
ingress.className	Ingress controller class to use	nginx
resources	CPU/Memory limits and requests	See values.yaml
config.*	Environment variables for UI	See values.yaml

Complete Example

For a complete example including a local Kubernetes setup with k3d, ingress configuration, and more, refer to the Kubernetes example directory.

# Example with comprehensive configuration
helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --version 0.7.1 \
  --create-namespace \
  --namespace inference-gateway \
  --set replicaCount=1 \
  --set gateway.enabled=true \
  --set gateway.envFrom.secretRef=inference-gateway \
  --set resources.limits.cpu=500m \
  --set resources.limits.memory=512Mi \
  --set resources.requests.cpu=100m \
  --set resources.requests.memory=128Mi \
  --set-string "env[0].name=NODE_ENV" \
  --set-string "env[0].value=production" \
  --set-string "env[1].name=NEXT_TELEMETRY_DISABLED" \
  --set-string "env[1].value=1"

Deployment

The application is automatically packaged as a Docker image and published to GitHub Container Registry (ghcr.io) when a new release is created.

To pull the latest release:

docker pull ghcr.io/inference-gateway/ui:latest

Related Projects

• Inference Gateway - The main gateway service
• Documentation - Project documentation

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please see our Contributing Guide for details on how to get started, coding standards, development workflow, and more.