GCP Vertex AI

Overview

Google Cloud Vertex AI provides enterprise access to AI models with Google Cloud infrastructure. When integrated with Autohand, you get:

• Access to Gemini 1.5 Pro, Gemini 1.5 Flash, and Gemini 2.0
• Open-source models like Llama 3.1 and Mistral through Model Garden
• Up to 2 million token context window with Gemini
• Enterprise security with Google Cloud IAM
• Regional endpoints for data residency
• Integration with existing GCP infrastructure

Setup

Get started with GCP Vertex AI.

Prerequisites

• Google Cloud project with Vertex AI API enabled
• Service account with Vertex AI User role
• gcloud CLI installed and configured

Enable Vertex AI

# Enable the Vertex AI API
gcloud services enable aiplatform.googleapis.com

# Set your project
gcloud config set project YOUR_PROJECT_ID

Configure authentication

# Option 1: Application Default Credentials (recommended for development)
gcloud auth application-default login

# Option 2: Service account key
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"

Configure Autohand

# Set environment variables
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_REGION="us-central1"

# Or configure via CLI
autohand config set vertex.project "your-project-id"
autohand config set vertex.region "us-central1"

Verify your configuration:

# Start with Vertex AI provider
autohand --provider vertex --model gemini-1.5-pro

# Test with a prompt
autohand --prompt "Hello, which model are you?"

CLI configuration

Configure Vertex AI in your ~/.autohand/config.json:

{
  "provider": "vertex",
  "vertex": {
    "project": "${GOOGLE_CLOUD_PROJECT}",
    "region": "us-central1",
    "model": "gemini-1.5-pro",
    "maxTokens": 8192,
    "temperature": 0.7
  }
}

Configuration options

Available models

Vertex AI offers Google's Gemini models and open-source alternatives.

Gemini models

Model Garden (open-source)

Switch models

# Set default model
autohand config set model "gemini-1.5-pro"

# Use during a session
/model vertex/gemini-2.0-flash

Best practices

• Use ADC in development: Application Default Credentials are easier to manage locally.
• Use service accounts in production: Create dedicated service accounts with minimal permissions.
• Choose the right region: Deploy in regions close to your users for lower latency.
• Monitor costs: Use Google Cloud Billing to track AI spending.
• Use Gemini 1.5 Pro for large context: Ideal for analyzing entire codebases.

Recommended configuration

{
  "vertex": {
    "model": "gemini-1.5-pro",
    "temperature": 0.3,
    "maxTokens": 8192
  }
}

Resources

• Vertex AI Overview
• Vertex AI Documentation
• Generative AI on Vertex AI
• Vertex AI Console
• Pricing
• Google Cloud Status

Troubleshooting

Common issues

Debug mode

# Enable verbose logging
AUTOHAND_DEBUG=true autohand --provider vertex

# Test authentication
gcloud auth application-default print-access-token