API Gateway OpenAI

Why Use an OpenAI API Gateway?

OpenAI's API is powerful, but production deployments require more than simple API calls. An API gateway layer provides critical infrastructure: secure key management, intelligent caching, comprehensive logging, and seamless failover.

Whether you're building chatbots, content generation systems, or AI-powered applications, an OpenAI gateway ensures reliability, cost control, and scalability. Centralize your AI infrastructure and let the gateway handle the complexities.

Core Capabilities

Implementation Example

Here's a production-ready Node.js gateway for OpenAI's Chat Completions API:

const express = require('express');
const axios = require('axios');
const Redis = require('ioredis');

const app = express();
const redis = new Redis(process.env.REDIS_URL);

app.use(express.json());

// Cache key generator (simplified)
function getCacheKey(model, messages) {
    return `openai:${model}:${JSON.stringify(messages)}`;
}

// OpenAI proxy endpoint
app.post('/v1/chat/completions', async (req, res) => {
    const { model, messages, temperature = 0.7 } = req.body;
    
    try {
        // Check cache first
        const cacheKey = getCacheKey(model, messages);
        const cached = await redis.get(cacheKey);
        
        if (cached) {
            console.log('Cache hit');
            return res.json(JSON.parse(cached));
        }
        
        // Forward to OpenAI
        const response = await axios.post(
            'https://api.openai.com/v1/chat/completions',
            { model, messages, temperature },
            {
                headers: {
                    'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
                    'Content-Type': 'application/json'
                },
                timeout: 30000
            }
        );
        
        // Log usage
        const usage = response.data.usage;
        console.log(`Tokens: ${usage.total_tokens}, Cost: $${calculateCost(model, usage)}`);
        
        // Cache response (5 minute TTL)
        await redis.setex(cacheKey, 300, JSON.stringify(response.data));
        
        res.json(response.data);
        
    } catch (error) {
        // Handle errors gracefully
        if (error.response?.status === 429) {
            return res.status(429).json({ error: 'Rate limited, please retry' });
        }
        res.status(500).json({ error: 'Internal server error' });
    }
});

function calculateCost(model, usage) {
    // GPT-4 pricing (example)
    const inputPrice = 0.03 / 1000;
    const outputPrice = 0.06 / 1000;
    return (usage.prompt_tokens * inputPrice) + (usage.completion_tokens * outputPrice);
}

app.listen(3000, () => {
    console.log('OpenAI Gateway running on port 3000');
});

Best Practices

1. Use Streaming for Chat Applications

Implement Server-Sent Events (SSE) streaming for real-time chat. Users see responses as they're generated, improving perceived latency and engagement.

2. Implement Request Validation

Validate all incoming requests before forwarding to OpenAI. Check message length, token count, and content safety to prevent abuse and unexpected costs.

3. Set Up Alerts and Monitoring

Monitor token usage, error rates, and response times in real-time. Set up alerts for unusual patterns or approaching budget limits.

4. Use Multiple API Keys

Distribute requests across multiple OpenAI API keys to maximize throughput. Implement key rotation and load balancing for optimal performance.