> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI Swarm

> The Portkey x Swarm integration brings advanced AI gateway capabilities, full-stack observability, and reliability features to build production-ready AI agents.

Swarm is an experimental framework by OpenAI for building multi-agent systems. It showcases the handoff & routines pattern, making agent coordination and execution lightweight, highly controllable, and easily testable. Portkey integration extends Swarm's capabilities with production-ready features like observability, reliability, and more.

## Getting Started

**Step 1**

### 1. Install the Portkey SDK

```sh theme={"system"}
pip install -U portkey-ai
```

**Step 2**

### 2. Configure the LLM Client used in OpenAI Swarm

To build Swarm Agents with Portkey, you'll need two keys:

* **Portkey API Key**: Sign up on the [Portkey app](https://app.portkey.ai) and copy your API key.

* **Provider**: Add your LLM provider in [Model Catalog](https://app.portkey.ai/model-catalog) to get a provider slug (e.g., `@openai-prod`)

```py theme={"system"}
from swarm import Swarm, Agent
from portkey_ai import Portkey

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY", # defaults to os.environ.get("PORTKEY_API_KEY")
    provider="@YOUR_PROVIDER" 
    )

client = Swarm(client=portkey)
```

**Step 3**

### 3. Create and Run an Agent

In this example we are building a simple Weather Agent using OpenAI Swarm with Portkey.

```py theme={"system"}
def get_weather(location) -> str:
    return "{'temp':67, 'unit':'F'}"


agent = Agent(
    name="Agent",
    instructions="You are a helpful agent.",
    functions=[get_weather],
)

messages = [{"role": "user", "content": "What's the weather in NYC?"}]

response = client.run(agent=agent, messages=messages)
print(response.messages[-1]["content"])
```

## E2E example with Function Calling in OpenAI Swarm

Here's a complete example showing function calling and agent interaction:

```py theme={"system"}
from swarm import Swarm, Agent
from portkey_ai import Portkey

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY", # defaults to os.environ.get("PORTKEY_API_KEY")
    provider="@YOUR_PROVIDER" 
    )

client = Swarm(client=portkey)


def get_weather(location) -> str:
    return "{'temp':67, 'unit':'F'}"


agent = Agent(
    name="Agent",
    instructions="You are a helpful agent.",
    functions=[get_weather],
)

messages = [{"role": "user", "content": "What's the weather in NYC?"}]

response = client.run(agent=agent, messages=messages)
print(response.messages[-1]["content"])
```

> The current temperature in New York City is 67°F.

## Enabling Portkey Features

By routing your OpenAI Swarm requests through Portkey, you get access to the following production-grade features:

> [card] [Interoperability](#1-interoperability-calling-different-llms)
>
> Call various LLMs like Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, and AWS Bedrock with minimal code changes.

> [card] [Caching](#2-caching-speed-up-agent-responses)
>
> Speed up agent responses and save costs by storing past responses in the Portkey cache. Choose between Simple and Semantic cache modes.

> [card] [Reliability](#3-reliability-keep-your-agents-running-smoothly)
>
> Set up fallbacks between different LLMs, load balance requests across multiple instances, set automatic retries, and request timeouts.

> [card] [Metrics](#4-observability-understand-your-agents)
>
> Get comprehensive logs of agent interactions, including cost, tokens used, response time, and function calls. Send custom metadata for better analytics.

> [card] [Logs](#5-logs-and-traces)
>
> Access detailed logs of agent executions, function calls, and interactions. Debug and optimize your agents effectively.

> [card] [Security & Compliance](#6-security-compliance-enterprise-ready-controls)
>
> Implement budget limits, role-based access control, and audit trails for your agent operations.

> [card] [Continuous Improvement](#7-continuous-improvement)
>
> Capture and analyze user feedback to improve agent performance over time.

## 1. Interoperability - Calling Different LLMs

When building with Swarm, you might want to experiment with different LLMs or use specific providers for different agent tasks. Portkey makes this seamless - you can switch between OpenAI, Anthropic, Gemini, Mistral, or cloud providers without changing your agent code.

Instead of managing multiple API keys and provider-specific configurations, Portkey's Model Catalog gives you a single point of control. Here's how you can use different LLMs with your Swarm agents:

> [tab] Anthropic
>
> ```python theme={"system"}
> portkey = Portkey(
>     api_key="YOUR_PORTKEY_API_KEY",
>     provider="@ANTHROPIC_PROVIDER" #Just change the provider slug to your preferred LLM provider
> )
>
> client = Swarm(client=portkey)
> ```

> [tab] Azure OpenAI
>
> ```python theme={"system"}
> portkey = Portkey(
>     api_key="YOUR_PORTKEY_API_KEY",
>     provider="@AZURE_OPENAI_PROVIDER" #Just change the provider slug to your preferred LLM provider
> )
>
> client = Swarm(client=portkey)
> ```

## 2. Caching - Speed Up Agent Responses

Agent operations often involve repetitive queries or similar tasks. Every time your agent makes an LLM call, you're paying for tokens and waiting for responses. Portkey's caching system can significantly reduce both costs and latency.

Portkey offers two powerful caching modes:

**Simple Cache**: Perfect for exact matches - when your agents make identical requests. Ideal for deterministic operations like function calling or FAQ-type queries.

**Semantic Cache**: Uses embedding-based matching to identify similar queries. Great for natural language interactions where users might ask the same thing in different ways.

```python theme={"system"}
config = {
    "cache": {
        "mode": "semantic",  # or "simple" for exact matching
        "max_age": 3600000  # cache duration in milliseconds
    }
}

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY",
    provider="@YOUR_PROVIDER",
    config=config
)
```

## 3. Reliability - Keep Your Agents Running Smoothly

When running agents in production, things can go wrong - API rate limits, network issues, or provider outages. Portkey's reliability features ensure your agents keep running smoothly even when problems occur.

> [card] [Automatic Retries](https://docs.portkey.ai/docs/product/ai-gateway/automatic-retries.md)
>
> Handles temporary failures automatically. If an LLM call fails, Portkey will retry the same request for the specified number of times - perfect for rate limits or network blips.

> [card] [Request Timeouts](https://docs.portkey.ai/docs/product/ai-gateway/request-timeouts.md)
>
> Prevent your agents from hanging. Set timeouts to ensure you get responses (or can fail gracefully) within your required timeframes.

> [card] [Conditional Routing](https://docs.portkey.ai/docs/product/ai-gateway/conditional-routing.md)
>
> Send different requests to different providers. Route complex reasoning to GPT-4, creative tasks to Claude, and quick responses to Gemini based on your needs.

> [card] [Fallbacks](https://docs.portkey.ai/docs/product/ai-gateway/fallbacks.md)
>
> Keep running even if your primary provider fails. Automatically switch to backup providers to maintain availability.

> [card] [Load Balancing](https://docs.portkey.ai/docs/product/ai-gateway/load-balancing.md)
>
> Spread requests across multiple API keys or providers. Great for high-volume agent operations and staying within rate limits.

## 4. [Observability - Understand Your Agents](https://docs.portkey.ai/docs/product/observability.md)

Building agents is the first step - but how do you know they're working effectively? Portkey provides comprehensive visibility into your agent operations through multiple lenses:

**Metrics Dashboard**: Track 40+ key performance indicators like:

* Cost per agent interaction
* Response times and latency
* Token usage and efficiency
* Success/failure rates
* Cache hit rates

![](https://mintlify.s3.us-west-1.amazonaws.com/portkey-docs/images/autogen/autogen-4.gif)

#### Send Custom Metadata with your requests

Add trace IDs to track specific workflows:

```python theme={"system"}
portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY",
    provider="@YOUR_PROVIDER",
    trace_id="weather_workflow_123",
    metadata={
        "agent": "weather_agent",
        "environment": "production"
    }
)
```

## 5. [Logs and Traces](https://docs.portkey.ai/docs/product/observability/logs.md)

Logs are essential for understanding agent behavior, diagnosing issues, and improving performance. They provide a detailed record of agent activities and tool use, which is crucial for debugging and optimizing processes.

Access a dedicated section to view records of agent executions, including parameters, outcomes, function calls, and errors. Filter logs based on multiple parameters such as trace ID, model, tokens used, and metadata.

![](https://mintlify.s3.us-west-1.amazonaws.com/portkey-docs/images/autogen/autogen-4.gif)

## 6. [Security & Compliance - Enterprise-Ready Controls](https://docs.portkey.ai/docs/product/enterprise-offering/security-portkey.md)

When deploying agents in production, security is crucial. Portkey provides enterprise-grade security features:

> [card] Budget Controls
>
> Set and monitor spending limits per provider. Get alerts before costs exceed thresholds.

> [card] Access Management
>
> Control who can access what. Assign roles and permissions for your team members.

> [card] Audit Logging
>
> Track all changes and access. Know who modified agent settings and when.

> [card] Data Privacy
>
> Configure data retention and processing policies to meet your compliance needs.

Configure these settings in the [Portkey Dashboard](https://app.portkey.ai) or programmatically through the API.

## 7. Continuous Improvement

Now that you know how to trace & log your Llamaindex requests to Portkey, you can also start capturing user feedback to improve your app!

You can append qualitative as well as quantitative feedback to any `trace ID` with the `portkey.feedback.create` method:

```py Adding Feedback theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@YOUR_OPENAI_PROVIDER"
)

feedback = portkey.feedback.create(
    trace_id="YOUR_LLAMAINDEX_TRACE_ID",
    value=5,  # Integer between -10 and 10
    weight=1,  # Optional
    metadata={
        # Pass any additional context here like comments, _user and more
    }
)

print(feedback)
```

## [Portkey Config](https://docs.portkey.ai/docs/product/ai-gateway/configs.md)

Many of these features are driven by Portkey's Config architecture. The Portkey app simplifies creating, managing, and versioning your Configs.

For more information on using these features and setting up your Config, please refer to the [Portkey documentation](https://docs.portkey.ai).

![](https://mintcdn.com/portkey-docs/T0lFtdapIPX8YtCI/images/libraries/libraries-3.avif?fit=max\&auto=format\&n=T0lFtdapIPX8YtCI\&q=85\&s=39c20b2f2dff75c78ed061797342e4bc)

***

> [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact)
>
> Contact Us


## Related topics

- [October](https://docs.portkey.ai/docs/changelog/2024/oct.md)
- [Overview](https://docs.portkey.ai/docs/integrations/agents.md)
- [OpenAI](https://docs.portkey.ai/docs/integrations/llms/openai.md)
- [Azure OpenAI](https://docs.portkey.ai/docs/integrations/llms/azure-openai/azure-openai.md)
- [OpenAI Codex](https://docs.portkey.ai/docs/integrations/libraries/codex.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.