NLP Framework Integration

OpenAI API Gateway
for Haystack

Integrate OpenAI API Gateway with Haystack NLP framework. Build production-ready search and QA systems with document retrieval, question answering, and semantic search.

Haystack Pipeline with Gateway

End-to-end NLP pipeline powered by OpenAI through gateway

1
Document Store
Store and index documents in Elasticsearch, Pinecone, or other vector databases
2
Retriever
Retrieve relevant documents using BM25, embedding, or hybrid search
3
Gateway Connector
Route all OpenAI API calls through gateway for caching and monitoring
4
Reader / Generator
Generate answers or summaries using GPT models via gateway

Gateway Benefits for Haystack

Enterprise features for your NLP pipelines

⚡
Caching
Cache embeddings and GPT responses to reduce API costs and latency.
🔄
Auto Retries
Automatic retry with backoff for failed API calls in long pipelines.
📊
Usage Tracking
Track token usage per pipeline, document, and query.
🔒
Rate Limiting
Prevent quota exhaustion during bulk document processing.
🎯
Model Fallback
Fall back to smaller models when rate limits are hit.
📈
Observability
Full tracing of pipeline execution with Haystack integration.

Code Example

Build a Haystack pipeline with Gateway

🌾 Haystack with Gateway Python 
# Import Haystack and Gateway components
from haystack import Pipeline
from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes import EmbeddingRetriever
from haystack.nodes import GatewayPromptNode

# Create document store
document_store = InMemoryDocumentStore(embedding_dim=1536)

# Create retriever with gateway embeddings
retriever = EmbeddingRetriever(
    document_store=document_store,
    embedding_model="text-embedding-3-small",
    gateway_endpoint="https://gateway.example.com",
    gateway_api_key="your-key"
)

# Create prompt node with gateway
prompt_node = GatewayPromptNode(
    model_name_or_path="gpt-4",
    gateway_endpoint="https://gateway.example.com",
    gateway_api_key="your-key",
    max_length=500,
    cache=True
)

# Build pipeline
pipeline = Pipeline()
pipeline.add_node(retriever, name="Retriever", inputs=["Query"])
pipeline.add_node(prompt_node, name="Generator", inputs=["Retriever"])

# Run query
result = pipeline.run(query="What is the main topic?")
print(result["answers"][0].answer)

Haystack Components

Gateway-ready components for your pipelines

📚
Document Stores
Elasticsearch, Pinecone, Weaviate
🔍
Retrievers
BM25, Embedding, Hybrid
📝
Readers
Extractive QA models
💬
Generators
GPT-based answer generation
🎯
Rankers
Re-rank retrieved documents
📄
Converters
PDF, DOCX, TXT parsing