Getting Started with AI Gateway

Connect to 1,600+ LLMs through Portkey’s unified API with observability, reliability, and cost controls.

Portkey’s AI Gateway provides a unified interface to 1,600+ LLMs with enterprise features: observability, automatic retries, fallbacks, caching, and cost controls—all through a simple API.

Quick Start

Get started in 3 steps:

Python

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add provider in Model Catalog (e.g., @openai-prod)
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",  # @provider-slug/model-name
    messages=[{"role": "user", "content": "What is a fractal?"}]
)

print(response.choices[0].message.content)

Portkey also supports Anthropic’s native /messages endpoint. See Using with Anthropic SDK below.

Add Provider in Model Catalog

Before making requests, add a provider:

1. Go to Model Catalog → Add Provider
2. Select your provider (OpenAI, Anthropic, etc.)
3. Choose existing credentials or enter your API key
4. Name your provider (e.g., openai-prod)

Your provider slug will be @openai-prod (the name you chose with @ prefix).

Complete Model Catalog Guide →

Set up budgets, rate limits, and manage credentials

Switch Between Providers

Change the model string to use different providers—same code, different models:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

# OpenAI
response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Anthropic
response = portkey.chat.completions.create(
    model="@anthropic-prod/claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "Hello!"}],
    max_tokens=250  # Required for Anthropic
)

# Mistral
response = portkey.chat.completions.create(
    model="@mistral-prod/mistral-large-latest",
    messages=[{"role": "user", "content": "Hello!"}]
)

Examples

Vision

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/800px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"}}
        ]
    }],
    max_tokens=300
)

print(response.choices[0].message.content)

Function Calling

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

tools = [{
        "type": "function",
        "function": {
        "name": "get_weather",
        "description": "Get current weather in a location",
          "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state, e.g. San Francisco, CA"}
            },
            "required": ["location"]
        }
        }
}]

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Boston?"}],
  tools=tools,
  tool_choice="auto"
)

print(response.choices[0].message.tool_calls)

Image Generation

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

image = portkey.images.generate(
    model="@openai-prod/dall-e-3",
    prompt="A serene mountain landscape at sunset",
    size="1024x1024"
)

print(image.data[0].url)

Embeddings

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

embeddings = portkey.embeddings.create(
    model="@openai-prod/text-embedding-3-small",
    input=["Hello world", "Goodbye world"]
)

print(embeddings.data[0].embedding[:5])  # First 5 dimensions

Audio Transcription

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

transcription = portkey.audio.transcriptions.create(
    model="@openai-prod/whisper-1",
    file=open("/path/to/audio.mp3", "rb")
)

print(transcription.text)

Using with OpenAI SDK

Use your existing OpenAI code with Portkey—just change 2 parameters:

Python

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# Change base_url and api_key
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url=PORTKEY_GATEWAY_URL
)

# Use model with @provider-slug prefix
response = client.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

Using with Anthropic SDK

Portkey fully supports Anthropic’s native /messages endpoint. Use the Anthropic SDK directly with Portkey:

Python

import anthropic

client = anthropic.Anthropic(
        api_key="PORTKEY_API_KEY",
    base_url="https://api.portkey.ai"
    )

message = client.messages.create(
    model="@anthropic-prod/claude-sonnet-4-5-20250929",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude!"}]
)

print(message.content[0].text)

All Portkey features (caching, observability, configs) work with the Anthropic SDK—just add the x-portkey-* headers.

Anthropic Integration Guide →

Learn about prompt caching, extended thinking, and more Anthropic features

Gateway Features

Add production features through configs:

from portkey_ai import Portkey

# Attach a config for retries, caching, fallbacks
portkey = Portkey(
        api_key="PORTKEY_API_KEY",
    config="pc-your-config-id"  # Created in Portkey dashboard
)

Automatic Retries

Retry failed requests with exponential backoff

Fallbacks

Automatically switch to backup providers

Caching

Cache responses to reduce costs and latency

Load Balancing

Distribute requests across multiple providers

Gateway Configs Guide →

Learn how to create and use configs

Supported Integrations

Portkey integrates with the entire AI ecosystem:

LLM Providers

1,600+ models from OpenAI, Anthropic, Google, Mistral, Cohere, and 30+ providers

Agent Frameworks

LangChain, CrewAI, AutoGen, OpenAI Agents, Strands, and more

Libraries

LangChain, LlamaIndex, Vercel AI SDK, and popular frameworks

Guardrails

Aporia, Pillar, Patronus, and content safety providers

Vector Databases

Pinecone, Weaviate, and vector store integrations

MCP Servers

Model Context Protocol servers and tools

Next Steps

Observability

Track costs, latency, and usage

Prompt Library

Manage and version prompts

Guardrails

Add PII detection and content filtering

Model Catalog

Manage providers, budgets, and access