Build AI Agents with LLM Proxy
Create intelligent agents that use tools, maintain memory, and execute complex workflows. Connect to multiple AI providers through a single proxy for maximum flexibility and control.
# Create an agent with tool calling agent = Agent( llm=get_proxy_llm(), tools=[search, calculator, email], memory=ConversationBuffer() ) # Agent autonomously uses tools result = agent.run("Find AI papers and email summary")
Persistent Memory
Conversation & task context
Autonomous Execution
Self-directed task completion
Multi-Provider
Switch LLMs seamlessly
Tool Calling
Execute real-world actions
Agents can call functions, APIs, and external services. The proxy handles tool definitions and responses across all providers.
Memory Systems
Context that persists
Agents maintain conversation history, user preferences, and task context across interactions.
# Conversation memory conv_memory = ConversationBufferMemory() # Entity memory entity_memory = EntityMemory() # Vector memory for RAG vector_memory = VectorStoreMemory( embedding=proxy_embeddings() )
Planning & Reasoning
Break down complex tasks
Agents use chain-of-thought reasoning to plan multi-step actions and adapt to changing conditions.
Multi-Model Routing
Right model for each task
Route different agent tasks to optimal models. Use fast models for simple tools, powerful models for reasoning.
# Route based on task complexity router = ModelRouter( simple="gpt-3.5-turbo", # Fast tool calls complex="gpt-4", # Deep reasoning creative="claude-3-opus" # Content generation )
Error Handling
Graceful failure recovery
Agents handle tool failures, retry logic, and fallback strategies automatically through proxy configuration.
Agent Architecture
Build Your Agent
Configure LLM Connection
Point your agent framework to the proxy endpoint. Use standard OpenAI-compatible configuration for seamless integration.
llm = ChatOpenAI( model="gpt-4", api_key="proxy-key", base_url="https://proxy.example.com/v1" )
Define Agent Tools
Create tools your agent can use. Tools can call APIs, query databases, or perform any programmatic action.
@tool def search_web(query: str) -> str: """Search the web for information""" return search_api.query(query) tools = [search_web, calculator, email]
Add Memory Layer
Configure memory to maintain context. Choose from conversation, entity, or vector-based memory systems.
memory = ConversationBufferMemory( memory_key="chat_history", return_messages=True ) agent = AgentExecutor( agent=agent, tools=tools, memory=memory )
Deploy & Monitor
Deploy your agent and monitor all interactions through the proxy dashboard. Track tool usage, costs, and performance.
# Execute agent with monitoring result = agent.invoke({ "input": "Research AI trends and create report" }) # All calls logged via proxy print(result["output"])
Built-in Agent Tools
Start Building Agents
Create intelligent agents with tool calling, memory, and autonomous execution. Connect to all AI providers through a single unified proxy.
Related Guides
Tool Calling Guide
Function calling with proxy
Implement function and tool calling with automatic schema translation across all providers.
Streaming Responses
Real-time token delivery
Stream agent thoughts and tool results in real-time for responsive user experiences.
Multi-Provider Setup
Access multiple AI models
Configure your proxy to access OpenAI, Anthropic, Google, and other providers.