LLMGateway Integration - Autohand Docs

Q: How do I configure the Autohand LLMGateway integration?

Set LLMGATEWAY_API_KEY, select the llmgateway provider, and configure the model or route exposed by the gateway.

Quick answer

What is the Autohand LLMGateway integration?

The Autohand LLMGateway integration sends coding-agent requests through an LLMGateway account instead of connecting to each upstream model provider separately. Autohand uses the gateway endpoint, API key, and selected model route.

Status: Available
Requires: An LLMGateway API key and an upstream model route enabled for that account.
Configure with: Set LLMGATEWAY_API_KEY, select the llmgateway provider, and configure the model or route exposed by the gateway.
Best for: Centralized routing, policy, caching, and observability across model providers.

Know before you start: Model behavior, availability, cost, and latency depend on both the gateway configuration and the upstream provider.

Overview

LLMGateway.io provides a unified interface to multiple AI providers with enterprise features. When integrated with Autohand, you get:

Access to OpenAI, Anthropic, Google, and more through one API
Intelligent request routing and load balancing
Response caching for cost optimization
Automatic failover between providers
Detailed analytics and observability
Rate limiting and budget controls

Enterprise ready: LLMGateway is designed for teams that need centralized control over AI usage, cost management, and compliance features.

Setup

Get started with LLMGateway.

Get your API key

Go to llmgateway.io and create an account
Configure your upstream providers (OpenAI, Anthropic, etc.)
Navigate to API Keys and create a key

Configure Autohand

# Set environment variable
export LLMGATEWAY_API_KEY="lgw-xxxxxxxxxxxxxxxxxxxx"

# Or configure via CLI
autohand config set llmgateway.apiKey "lgw-xxxxxxxxxxxxxxxxxxxx"

Verify your configuration:

# Start with LLMGateway provider
autohand --provider llmgateway --model gpt-4o

# Test with a prompt
autohand --prompt "Hello, which model are you?"

CLI configuration

Configure LLMGateway in your ~/.autohand/config.json:

{
  "provider": "llmgateway",
  "llmgateway": {
    "apiKey": "${LLMGATEWAY_API_KEY}",
    "baseUrl": "https://api.llmgateway.io/v1",
    "model": "gpt-4o",
    "maxTokens": 4096,
    "temperature": 0.7,
    "caching": true,
    "fallbackModels": ["z-ai/glm-4.7", "gemini-pro"]
  }
}

Configuration options

Option	Description	Default
`apiKey`	Your LLMGateway API key	-
`baseUrl`	Gateway endpoint	`https://api.llmgateway.io/v1`
`model`	Default model to use	`gpt-4o`
`caching`	Enable response caching	`true`
`fallbackModels`	Models to try if primary fails	`[]`
`maxTokens`	Maximum response tokens	`4096`

Features

Smart routing

LLMGateway can automatically route requests based on model availability, cost, or latency:

{
  "llmgateway": {
    "routing": {
      "strategy": "cost-optimized",
      "providers": ["openai", "anthropic", "google"]
    }
  }
}

Response caching

Cache identical requests to reduce costs:

{
  "llmgateway": {
    "caching": true,
    "cacheTTL": 3600
  }
}

Automatic failover

Automatically switch providers when one fails:

{
  "llmgateway": {
    "model": "gpt-4o",
    "fallbackModels": [
      "z-ai/glm-4.7",
      "gemini-1.5-pro"
    ]
  }
}

Available models

Access any model from your configured upstream providers. Common options include:

Provider	Models
OpenAI	gpt-4o, gpt-4-turbo, o1
Anthropic	z-ai/glm-4.7, claude-3-opus
Google	gemini-1.5-pro, gemini-1.5-flash
Meta	llama-3.1-70b, llama-3.1-405b

Best practices

Configure fallbacks: Set up multiple fallback models for reliability.
Enable caching: Reduce costs for repeated queries.
Set budgets: Configure spending limits in the LLMGateway dashboard.
Monitor usage: Use the analytics dashboard to track costs and usage.

Resources

Troubleshooting

Common issues

Issue	Solution
Invalid API key	Verify key at llmgateway.io
Upstream error	Check provider configuration in dashboard
Rate limited	Check rate limit settings

Debug mode

# Enable verbose logging
AUTOHAND_DEBUG=true autohand --provider llmgateway

Common questions

LLMGateway integration FAQ

How do I configure the Autohand LLMGateway integration?

Set LLMGATEWAY_API_KEY, select the llmgateway provider, and configure the model or route exposed by the gateway.

What does the Autohand LLMGateway integration require?

An LLMGateway API key and an upstream model route enabled for that account.

What limitations should I know about?

Model behavior, availability, cost, and latency depend on both the gateway configuration and the upstream provider.