> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Langchain (Python)

> Add Prisma AIRS AI Gateway's observability, reliability, caching, and cost control to any Langchain app.

> [!NOTE]
>
> Screenshots on this page show the previous product branding and navigation. The workflow they
> illustrate is current; the interface around it has since been rebranded to Prisma AIRS.

> [!NOTE]
>
> This guide covers Langchain **Python**. For JS, see [Langchain JS](https://docs.portkey.ai/docs/aigw/integrations/libraries/langchain-js.md).

Langchain provides a unified interface for building LLM applications. Add the AI Gateway to get production-grade features: full observability, automatic fallbacks, semantic caching, and cost controls—all without changing your Langchain code.

## Quick Start

Add the AI Gateway to any Langchain app with 3 parameters:

```python theme={"system"}
from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    model="@openai-prod/gpt-4o",       # Provider slug from Model Catalog
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY"          # Your AI Gateway API key
)

response = model.invoke("Tell me a joke")
print(response.content)
```

![](https://mintcdn.com/portkey-docs/T0lFtdapIPX8YtCI/images/libraries/langchain-logs.gif?s=83f80a28ac2103950e5683f90faa16b2)

All requests now appear in the AI Gateway logs

That's it! You now get:

* ✅ Full observability (costs, latency, logs)
* ✅ Dynamic model selection per request
* ✅ Automatic fallbacks and retries (via configs)
* ✅ Budget controls per team/project

## Why Add the AI Gateway to Langchain?

Langchain handles application orchestration. The AI Gateway adds production features:

> **Observability**
>
> Every request logged with costs, latency, tokens. Team-level analytics and debugging.

> **Dynamic Model Selection**
>
> Switch models per request. Route simple queries to cheap models, complex to advanced—automatically tracked.

> **Production Reliability**
>
> Automatic fallbacks, smart retries, load balancing—configured once, works everywhere.

> **Cost & Access Control**
>
> Budget limits per team/project. Rate limiting. Centralized credential management.

## Setup

### 1. Install Packages

```bash theme={"system"}
pip install langchain-openai
```

### 2. Add Provider in Model Catalog

1. Go to [**Model Catalog → Add Provider**](https://stratacloudmanager.paloaltonetworks.com/)
2. Select your provider (OpenAI, Anthropic, Google, etc.)
3. Choose existing credentials or create new by entering your API keys
4. Name your provider (e.g., `openai-prod`)

Your provider slug will be **`@openai-prod`** (or whatever you named it).

> **[Complete Model Catalog Guide →](https://docs.portkey.ai/docs/aigw/product/model-catalog.md)**
>
> Set up budgets, rate limits, and manage credentials

### 3. Get AI Gateway API Key

Create your AI Gateway API key at [stratacloudmanager.paloaltonetworks.com](https://stratacloudmanager.paloaltonetworks.com/)

### 4. Use in Your Code

Replace your existing `ChatOpenAI` initialization:

```python theme={"system"}
# Before (direct to OpenAI)
model = ChatOpenAI(
    model="gpt-4o",
    api_key="OPENAI_API_KEY"
)

# After (via the AI Gateway)
model = ChatOpenAI(
    model="@openai-prod/gpt-4o",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY"
)
```

**That's the only change needed!** All your existing Langchain code (agents, chains, LCEL, etc.) works exactly the same.

## Switching Between Providers

Just change the model string—everything else stays the same:

```python theme={"system"}
# OpenAI
model = ChatOpenAI(
    model="@openai-prod/gpt-4o",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY"
)

# Anthropic
model = ChatOpenAI(
    model="@anthropic-prod/claude-sonnet-4",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY"
)

# Google Gemini
model = ChatOpenAI(
    model="@google-prod/gemini-2.0-flash",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY"
)
```

> [!NOTE]
>
> The AI Gateway implements OpenAI-compatible APIs for all providers, so you always use `ChatOpenAI` regardless of which model you're calling.

## Using with Langchain Agents

Langchain agents are the primary use case. The AI Gateway works seamlessly with `create_agent`:

```python theme={"system"}
from langchain.agents import create_agent
from langchain.tools import tool
from langchain_openai import ChatOpenAI

@tool
def search(query: str) -> str:
    """Search for information."""
    return f"Results for: {query}"

@tool
def get_weather(location: str) -> str:
    """Get weather for a location."""
    return f"Weather in {location}: Sunny, 72°F"

model = ChatOpenAI(
    model="@openai-prod/gpt-4o",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY"
)

agent = create_agent(model, tools=[search, get_weather])

result = agent.invoke({
    "messages": [{"role": "user", "content": "What's the weather in NYC and search for AI news"}]
})
```

Every agent step is logged in the AI Gateway:

* Model calls with prompts and responses
* Tool executions with inputs and outputs
* Full trace of the agent's reasoning
* Costs and latency for each step

![](https://mintcdn.com/portkey-docs/T0lFtdapIPX8YtCI/images/libraries/langchain-logs.gif?s=83f80a28ac2103950e5683f90faa16b2)

Agent traces in Portkey

## Works With All Langchain Features

✅ **Agents** - Full compatibility with `create_agent`
✅ **LCEL** - LangChain Expression Language
✅ **Chains** - All chain types supported
✅ **Streaming** - Token-by-token streaming
✅ **Tool Calling** - Function/tool calling
✅ **LangGraph** - Complex workflows

### Streaming

```python theme={"system"}
model = ChatOpenAI(
    model="@openai-prod/gpt-4o",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY",
    streaming=True
)

for chunk in model.stream("Write a short story"):
    print(chunk.content, end="", flush=True)
```

### Tool Calling

```python theme={"system"}
from pydantic import BaseModel, Field

class GetWeather(BaseModel):
    '''Get current weather in a location'''
    location: str = Field(..., description="City and state, e.g. San Francisco, CA")

model = ChatOpenAI(
    model="@openai-prod/gpt-4o",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY"
)

model_with_tools = model.bind_tools([GetWeather])
response = model_with_tools.invoke("What's the weather in NYC?")
print(response.tool_calls)
```

## Dynamic Model Selection

For dynamic model routing based on query complexity or task type, use **AI Gateway Configs** with conditional routing:

```python theme={"system"}
import json
from langchain_openai import ChatOpenAI

# Define routing config (created in Strata Cloud Manager)
config = {
  "strategy": {
    "mode": "conditional",
    "conditions": [
      {
        "query": {"metadata.complexity": {"$eq": "simple"}},
        "then": "cheap-model"
      },
      {
        "query": {"metadata.complexity": {"$eq": "complex"}},
        "then": "advanced-model"
      }
    ],
    "default": "cheap-model"
  },
  "targets": [
    {
      "name": "cheap-model",
      "override_params": {"model": "@openai-prod/gpt-4o-mini"}
    },
    {
      "name": "advanced-model",
      "override_params": {"model": "@openai-prod/o1"}
    }
  ]
}

model = ChatOpenAI(
    model="gpt-4o",  # Default model
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY",
    default_headers={"x-portkey-config": json.dumps(config)}
)

# Route to cheap model
response1 = model.invoke(
    "What is 2+2?",
    config={"metadata": {"complexity": "simple"}}
)

# Route to advanced model
response2 = model.invoke(
    "Solve this differential equation...",
    config={"metadata": {"complexity": "complex"}}
)
```

### Use Cases

**1. Cost Optimization**

Route by query complexity automatically:

```python theme={"system"}
def smart_invoke(prompt, complexity="simple"):
    return model.invoke(
        prompt,
        config={"metadata": {"complexity": complexity}}
    )

# Automatic routing
answer1 = smart_invoke("What is 2+2?", complexity="simple")
answer2 = smart_invoke("Explain quantum mechanics", complexity="complex")
```

**2. Model Specialization by Task**

Route different task types to specialized models:

```python theme={"system"}
config = {
  "strategy": {
    "mode": "conditional",
    "conditions": [
      {"query": {"metadata.task": {"$eq": "code"}}, "then": "coding-model"},
      {"query": {"metadata.task": {"$eq": "creative"}}, "then": "creative-model"}
    ],
    "default": "coding-model"
  },
  "targets": [
    {
      "name": "coding-model",
      "override_params": {"model": "@openai-prod/gpt-4o"}
    },
    {
      "name": "creative-model",
      "override_params": {"model": "@anthropic-prod/claude-sonnet-4"}
    }
  ]
}

def route_by_task(prompt, task_type):
    return model.invoke(
        prompt,
        config={"metadata": {"task": task_type}}
    )

code = route_by_task("Write a sorting algorithm", task_type="code")
story = route_by_task("Write a sci-fi story", task_type="creative")
```

**3. Dynamic Agent Model Selection**

Use different models for different agent steps:

```python theme={"system"}
import json
from langchain.agents import create_agent
from langchain.tools import tool

@tool
def complex_calculation(query: str) -> str:
    """Perform complex calculations."""
    return "42"

model = ChatOpenAI(
    model="gpt-4o",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY",
    default_headers={"x-portkey-config": json.dumps(config)}
)

agent = create_agent(model, tools=[complex_calculation])

# Agent routes based on task complexity
result = agent.invoke({
    "messages": [{"role": "user", "content": "Calculate quantum probabilities"}],
    "metadata": {"complexity": "complex"}
})
```

All routing decisions are tracked in the AI Gateway with full observability—see which models were used, costs per model, and performance comparisons.

> **[Conditional Routing Guide →](https://docs.portkey.ai/docs/aigw/product/ai-gateway/conditional-routing.md)**
>
> Learn more about conditional routing and advanced patterns

### When to Use Dynamic Routing

**Use conditional routing** when you need:

* ✅ Cost optimization based on query complexity
* ✅ Model specialization by task type
* ✅ Automatic failover and fallbacks
* ✅ A/B testing with traffic distribution

**Use fixed models** when you need:

* ✅ Simple, predictable behaviour
* ✅ Consistent model across all requests
* ✅ Easier debugging

## Advanced Features via Configs

For production features like fallbacks, caching, and load balancing, use AI Gateway Configs:

```python theme={"system"}

model = ChatOpenAI(
    model="@openai-prod/gpt-4o",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY",
    default_headers={
        "x-portkey-config": "pc_your_config_id",  # Created in Strata Cloud Manager
    }
)
```

> **[Learn About Configs →](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md)**
>
> Set up fallbacks, retries, caching, load balancing, and more

## Langchain Embeddings

Create embeddings via the AI Gateway:

```python theme={"system"}
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="@openai-prod/text-embedding-3-small",
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY",
)

vectors = embeddings.embed_documents(["Hello world", "Goodbye world"])
```

> [!NOTE]
>
> The AI Gateway supports OpenAI embeddings via `OpenAIEmbeddings`. For other providers (Cohere, Voyage), call the [embeddings endpoint](https://docs.portkey.ai/docs/aigw/api-reference/embeddings/create-embedding.md) directly.

## Migration from Direct OpenAI

Already using Langchain with OpenAI? Just update 3 parameters:

```python theme={"system"}
# Before
from langchain_openai import ChatOpenAI
import os

model = ChatOpenAI(
    model="gpt-4o",
    api_key=os.getenv("OPENAI_API_KEY"),
    temperature=0.7
)

# After (add 2 parameters, change 1)
model = ChatOpenAI(
    model="@openai-prod/gpt-4o",      # Add provider slug
    base_url="https://aigw.portkey.ai/v1", # Add this
    api_key="PORTKEY_API_KEY",         # Change to AI Gateway API key
    temperature=0.7                     # Keep existing params
)
```

**Benefits:**

* Zero code changes to your existing Langchain logic
* Instant observability for all requests
* Production-grade reliability features
* Cost controls and budgets

## Next Steps

> **[Model Catalog](https://docs.portkey.ai/docs/aigw/product/model-catalog.md)**
>
> Set up providers, budgets, and access control

> **[Configs](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md)**
>
> Configure fallbacks, caching, and routing

> **[Observability](https://docs.portkey.ai/docs/aigw/product/observability.md)**
>
> Track costs, performance, and usage

> **[Guardrails](https://docs.portkey.ai/docs/aigw/product/guardrails.md)**
>
> Add PII detection and content filtering


## Related topics

- [Langchain (JS/TS)](https://docs.portkey.ai/docs/aigw/integrations/libraries/langchain-js.md)
- [Overview](https://docs.portkey.ai/docs/aigw/integrations/libraries.md)
- [Tracing](https://docs.portkey.ai/docs/aigw/product/observability/traces.md)
- [Langchain Agents](https://docs.portkey.ai/docs/aigw/integrations/agents/langchain-agents.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.