---
title: "LLMGateway Integration"
source: https://docs.autohand.ai/integrations/llmgateway
---

# LLMGateway

Enterprise-grade AI gateway with unified API access to multiple providers. Smart routing, caching, and observability for production LLM applications.

Quick answer

## What is the Autohand LLMGateway integration?

The Autohand LLMGateway integration sends coding-agent requests through an LLMGateway account instead of connecting to each upstream model provider separately. Autohand uses the gateway endpoint, API key, and selected model route.

Status

Available

Requires

An LLMGateway API key and an upstream model route enabled for that account.

Configure with

Set `LLMGATEWAY_API_KEY`, select the `llmgateway` provider, and configure the model or route exposed by the gateway.

Best for

Centralized routing, policy, caching, and observability across model providers.

**Know before you start:** Model behavior, availability, cost, and latency depend on both the gateway configuration and the upstream provider.

## Overview

LLMGateway.io provides a unified interface to multiple AI providers with enterprise features. When integrated with Autohand, you get:

-   Access to OpenAI, Anthropic, Google, and more through one API
-   Intelligent request routing and load balancing
-   Response caching for cost optimization
-   Automatic failover between providers
-   Detailed analytics and observability
-   Rate limiting and budget controls

**Enterprise ready:** LLMGateway is designed for teams that need centralized control over AI usage, cost management, and compliance features.

## Setup

Get started with LLMGateway.

### Get your API key

1.  Go to [llmgateway.io](https://llmgateway.io) and create an account
2.  Configure your upstream providers (OpenAI, Anthropic, etc.)
3.  Navigate to **API Keys** and create a key

### Configure Autohand

``` bash
# Set environment variable
export LLMGATEWAY_API_KEY="lgw-xxxxxxxxxxxxxxxxxxxx"

# Or configure via CLI
autohand config set llmgateway.apiKey "lgw-xxxxxxxxxxxxxxxxxxxx"
```

Verify your configuration:

``` bash
# Start with LLMGateway provider
autohand --provider llmgateway --model gpt-4o

# Test with a prompt
autohand --prompt "Hello, which model are you?"
```

## CLI configuration

Configure LLMGateway in your `~/.autohand/config.json`:

``` json
{
  "provider": "llmgateway",
  "llmgateway": {
    "apiKey": "${LLMGATEWAY_API_KEY}",
    "baseUrl": "https://api.llmgateway.io/v1",
    "model": "gpt-4o",
    "maxTokens": 4096,
    "temperature": 0.7,
    "caching": true,
    "fallbackModels": ["z-ai/glm-4.7", "gemini-pro"]
  }
}
```

### Configuration options

| Option | Description | Default |
|---|---|---|
| apiKey | Your LLMGateway API key | - |
| baseUrl | Gateway endpoint | https://api.llmgateway.io/v1 |
| model | Default model to use | gpt-4o |
| caching | Enable response caching | true |
| fallbackModels | Models to try if primary fails | [] |
| maxTokens | Maximum response tokens | 4096 |

## Features

### Smart routing

LLMGateway can automatically route requests based on model availability, cost, or latency:

``` json
{
  "llmgateway": {
    "routing": {
      "strategy": "cost-optimized",
      "providers": ["openai", "anthropic", "google"]
    }
  }
}
```

### Response caching

Cache identical requests to reduce costs:

``` json
{
  "llmgateway": {
    "caching": true,
    "cacheTTL": 3600
  }
}
```

### Automatic failover

Automatically switch providers when one fails:

``` json
{
  "llmgateway": {
    "model": "gpt-4o",
    "fallbackModels": [
      "z-ai/glm-4.7",
      "gemini-1.5-pro"
    ]
  }
}
```

## Available models

Access any model from your configured upstream providers. Common options include:

| Provider | Models |
|---|---|
| OpenAI | gpt-4o, gpt-4-turbo, o1 |
| Anthropic | z-ai/glm-4.7, claude-3-opus |
| Google | gemini-1.5-pro, gemini-1.5-flash |
| Meta | llama-3.1-70b, llama-3.1-405b |

## Best practices

-   **Configure fallbacks**: Set up multiple fallback models for reliability.
-   **Enable caching**: Reduce costs for repeated queries.
-   **Set budgets**: Configure spending limits in the LLMGateway dashboard.
-   **Monitor usage**: Use the analytics dashboard to track costs and usage.

## Resources

-   [LLMGateway Website](https://llmgateway.io)
-   [Documentation](https://docs.llmgateway.io)
-   [Dashboard](https://app.llmgateway.io)
-   [API Reference](https://docs.llmgateway.io/api-reference)
-   [Pricing](https://llmgateway.io/pricing)

## Troubleshooting

### Common issues

| Issue | Solution |
|---|---|
| Invalid API key | Verify key at llmgateway.io |
| Upstream error | Check provider configuration in dashboard |
| Rate limited | Check rate limit settings |

### Debug mode

``` bash
# Enable verbose logging
AUTOHAND_DEBUG=true autohand --provider llmgateway
```

Common questions

## LLMGateway integration FAQ

### How do I configure the Autohand LLMGateway integration?

Set `LLMGATEWAY_API_KEY`, select the `llmgateway` provider, and configure the model or route exposed by the gateway.

### What does the Autohand LLMGateway integration require?

An LLMGateway API key and an upstream model route enabled for that account.

### What limitations should I know about?

Model behavior, availability, cost, and latency depend on both the gateway configuration and the upstream provider.