API Gateway Proxy
Production Setup

Prerequisites Checklist

Before starting production setup, ensure all prerequisites are met:

• Cloud provider account (AWS, GCP, or Azure) with appropriate IAM permissions
• Kubernetes cluster (1.25+) with kubectl configured
• Container registry access (Docker Hub, ECR, GCR)
• Domain name with DNS management access
• SSL certificate for HTTPS (Let's Encrypt or commercial)
• API keys from all LLM providers you'll use
• Monitoring stack (Prometheus, Grafana) or managed alternative
• Secrets management solution (Vault, AWS Secrets Manager)
• CI/CD pipeline configured (GitHub Actions, GitLab CI, etc.)
• Runbook documentation for incident response

Infrastructure Provisioning

Step 1: Create Kubernetes Namespace

kubectl create namespace api-gateway-production

# Add labels for network policies
kubectl label namespace api-gateway-production \
  environment=production \
  app=api-gateway

Step 2: Create Secrets

kubectl create secret generic api-provider-keys \
  --from-literal=OPENAI_API_KEY=sk-your-key \
  --from-literal=ANTHROPIC_API_KEY=sk-ant-your-key \
  --namespace=api-gateway-production

# Create TLS secret for HTTPS
kubectl create secret tls gateway-tls \
  --cert=path/to/cert.pem \
  --key=path/to/key.pem \
  --namespace=api-gateway-production

Step 3: Deploy Redis Cache

# Using Helm for Redis deployment
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install redis bitnami/redis \
  --namespace api-gateway-production \
  --set auth.enabled=true \
  --set auth.password=your-redis-password \
  --set replica.replicaCount=2

Production Configuration

Gateway Configuration File

# config/production.yaml
environment: production
log_level: info

server:
  http_port: 8080
  https_port: 8443
  admin_port: 9090
  graceful_shutdown_timeout: 30s

authentication:
  enabled: true
  providers:
    - type: api_key
      header: X-API-Key
      rate_limit_per_key: 1000/minute
  
rate_limiting:
  enabled: true
  global_limit: 50000/minute
  per_endpoint_limits:
    - path: /v1/chat/completions
      limit: 500/minute

cache:
  enabled: true
  backend: redis
  url: redis://redis-master:6379
  ttl: 3600
  max_memory: 2gb

providers:
  openai:
    api_key: ${OPENAI_API_KEY}
    timeout: 30s
    retry_attempts: 3
    
  anthropic:
    api_key: ${ANTHROPIC_API_KEY}
    timeout: 30s

monitoring:
  metrics_enabled: true
  metrics_port: 9090
  tracing_enabled: true
  tracing_endpoint: jaeger-collector:14268

Deployment Automation

Terraform Infrastructure

# main.tf
provider "kubernetes" {
  config_path = "~/.kube/config"
}

resource "kubernetes_deployment" "api_gateway" {
  metadata {
    name      = "api-gateway"
    namespace = "api-gateway-production"
  }
  
  spec {
    replicas = 3
    
    selector {
      match_labels = {
        app = "api-gateway"
      }
    }
    
    template {
      metadata {
        labels = {
          app = "api-gateway"
        }
      }
      
      spec {
        container {
          name  = "gateway"
          image = "api-gateway:v2.1.0"
          
          port {
            container_port = 8080
          }
          
          env_from {
            secret_ref {
              name = "api-provider-keys"
            }
          }
          
          resources {
            limits = {
              cpu    = "2000m"
              memory = "2Gi"
            }
            requests = {
              cpu    = "500m"
              memory = "512Mi"
            }
          }
        }
      }
    }
  }
}

CI/CD Pipeline (GitHub Actions)

name: Deploy to Production

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Configure kubectl
        uses: azure/setup-kubectl@v3
        
      - name: Deploy to Kubernetes
        run: |
          kubectl apply -f k8s/production/
          kubectl rollout status deployment/api-gateway \
            -n api-gateway-production

Verification Steps

Post-Deployment Checks