> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Anthropic

> Integrate Anthropic's Claude models with Prisma AIRS AI Gateway

The AI Gateway provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including [Anthropic's Claude APIs](https://docs.anthropic.com/claude/reference/getting-started-with-the-api).

With the AI Gateway, take advantage of features like fast AI gateway access, observability, and more, while securely managing API keys through [Model Catalog](https://docs.portkey.ai/docs/aigw/product/model-catalog.md).

> **All Models**
>
> Full support for all Claude models including Sonnet and Haiku 4-5

> **All Endpoints**
>
> `/messages`, `count-tokens` and more fully supported

> **Multi-Provider Support**
>
> Use Claude from Anthropic, Bedrock, and Vertex with native SDK support

## Quick Start

Get Anthropic working in 3 steps:

```sh cURL icon="square-terminal" theme={"system"}
# 1. Add @anthropic provider in model catalog
# 2. Use it:

# /chat/completions endpoint (OpenAI-compatible)
curl https://aigw.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PORTKEY_API_KEY" \
  -d '{
    "model": "@anthropic/claude-sonnet-4-5-20250929",
    "messages": [
      { "role": "user", "content": "What is Portkey's AI Gateway?" }
    ],
    "max_tokens": 250
  }'

# /messages endpoint (Anthropic native) - also supported
curl https://aigw.portkey.ai/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PORTKEY_API_KEY" \
  -d '{
    "model": "@anthropic/claude-sonnet-4-5-20250929",
    "max_tokens": 250,
    "messages": [
      { "role": "user", "content": "What is Portkey's AI Gateway?" }
    ]
  }'
```

```python OpenAI Py icon="python" theme={"system"}
from openai import OpenAI

# 1. Install: pip install openai
# 2. Add @anthropic provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # AI Gateway API key
    base_url="https://aigw.portkey.ai/v1"
)

response = client.chat.completions.create(
    model="@anthropic/claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "What is Portkey's AI Gateway?"}],
    max_tokens=250  # Required for Anthropic
)

print(response.choices[0].message.content)
```

```js OpenAI JS icon="square-js" theme={"system"}
import OpenAI from "openai"

// 1. Install: npm install openai
// 2. Add @anthropic provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // AI Gateway API key
    baseURL: "https://aigw.portkey.ai/v1"
})

const response = await client.chat.completions.create({
    model: "@anthropic/claude-sonnet-4-5-20250929",
    messages: [{ role: "user", content: "What is Portkey's AI Gateway?" }],
    max_tokens: 250  // Required for Anthropic
})

console.log(response.choices[0].message.content)
```

```python Anthropic Py theme={"system"}
import anthropic

# 1. Install: pip install anthropic
# 2. Add @anthropic provider in model catalog
# 3. Use it:

client = anthropic.Anthropic(
    api_key="dummy", # auth happens via the Authorization header
    default_headers={"Authorization": "Bearer YOUR_PORTKEY_API_KEY"},
    base_url="https://aigw.portkey.ai"
)

message = client.messages.create(
    model="@anthropic/claude-sonnet-4-5-20250929",
    max_tokens=250,
    messages=[{"role": "user", "content": "What is Portkey's AI Gateway?"}]
)

print(message.content)
```

```typescript Anthropic TS theme={"system"}
import Anthropic from '@anthropic-ai/sdk'

// 1. Install: npm install @anthropic-ai/sdk
// 2. Add @anthropic provider in model catalog
// 3. Use it:

const anthropic = new Anthropic({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://aigw.portkey.ai"
})

const msg = await anthropic.messages.create({
    model: "@anthropic/claude-sonnet-4-5-20250929",
    max_tokens: 250,
    messages: [{ role: "user", content: "What is Portkey's AI Gateway?" }],
})

console.log(msg)
```

> [!NOTE]
>
> **Tip:** You can also send `x-portkey-provider: @anthropic` as a header and use just `model="claude-sonnet-4-5-20250929"` in the request.

* **`max_tokens` is required** - Always specify this parameter
* **System prompts** - Handled differently (see System Prompts section below)
* **Model naming** - Use full model names like `claude-sonnet-4-5-20250929`

## Add Provider in Model Catalog

1. Go to [**Model Catalog → Add Provider**](https://stratacloudmanager.paloaltonetworks.com/)
2. Select **Anthropic**
3. Choose existing credentials or create new by entering your [Anthropic API key](https://console.anthropic.com/settings/keys)
4. Name your provider (e.g., `anthropic-prod`)

> **[Complete Setup Guide →](https://docs.portkey.ai/docs/aigw/product/model-catalog.md)**
>
> See all setup options, code examples, and detailed instructions

## Basic Usage

### Chat Completions

### System Prompts

Anthropic handles system prompts differently than OpenAI. With the AI Gateway, you can use the OpenAI-compatible format:

The AI Gateway automatically formats this for Anthropic's API.

### Streaming

Streaming works the same as OpenAI:

### Catch Overloaded Error on Stream

Anthropic's API can return an `overloaded_error` inside a streaming response with HTTP status 200. The error appears as an SSE event:

```text theme={"system"}
event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}
```

By default, the gateway treats this as a successful (status 200) response and streams the error directly to the client, which means retry, fallback, and circuit breaker strategies do not activate (they rely on HTTP status codes).

When **Catch Overloaded Error on Stream** is enabled on an Anthropic integration, the gateway intercepts these errors before they reach the client and converts them into HTTP `529` responses, allowing your retry and fallback strategies to trigger automatically.

> [!NOTE]
>
> This feature is only available for the Anthropic provider. Other providers (e.g., Bedrock) handle overload errors at the HTTP level, where existing retry/fallback already applies. It also only applies to **streaming** requests — non-streaming Anthropic requests already return HTTP 529 directly.

#### How it works

When enabled on an integration, the gateway:

1. Reads the first chunk of the Anthropic streaming response before committing it to the client
2. Skips any keepalive ping events
3. If the first meaningful event is an `overloaded_error`, returns an HTTP `529` response instead of the stream
4. If the first event is normal content, continues streaming as usual with no data loss

If no retry strategy is present and an `overloaded_error` is found, the request fails as a normal request with error `529`.

The `529` response integrates with the gateway's existing error handling and supports all existing config combinations:

* **Retry**: Triggers automatically when retry is configured
* **Fallback**: Moves to the next target in a fallback strategy
* **Circuit breaker**: Counts as a failure for circuit breaker thresholds

> [!NOTE]
>
> Performance: There is zero overhead when the setting is disabled. When enabled, only the first event is inspected before the stream is committed.

#### How to enable

**1. Enable the flag on your Anthropic integration**

Go to **Model Catalog → Integrations → Anthropic** and enable the **Catch Overloaded Error on Stream** flag, then create or update the integration.

**2. Add 529 to your retry status codes**

In your [config](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md), add `529` to the retry `on_status_codes` (or fallback `on_status_codes`). This supports all existing config combinations.

**3. Attach the config to your API key**

Attach the updated config to your API key so the new behaviour applies to all routed requests.

Once enabled, all Anthropic streaming requests routed through the gateway are checked for overloaded errors.

#### Example: Fallback on overload

With a fallback config using two Anthropic integrations (both with **Catch Overloaded Error on Stream** enabled), if the primary returns an overloaded error during streaming, the gateway automatically retries with the backup:

```json theme={"system"}
{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "provider": "anthropic", "virtual_key": "anthropic-primary" },
    { "provider": "anthropic", "virtual_key": "anthropic-backup" }
  ]
}
```

#### Error response

When an overloaded error is detected, the client receives:

```http theme={"system"}
HTTP/1.1 529
{
  "error": {
    "message": "Overloaded",
    "type": "overloaded_error",
    "param": null,
    "code": null
  }
}
```

## Advanced Features

### Vision (Multimodal)

The AI Gateway supports Anthropic's vision models including `claude-sonnet-4-5-20250929`, `claude-3-5-sonnet`, `claude-3-haiku`, `claude-3-opus`, and `claude-3.7-sonnet`. Use the same format as OpenAI:

> [!NOTE]
>
> Anthropic **only accepts base64-encoded images** and does not support image URLs. Use the same base64 format to send images to both Anthropic and OpenAI models.

> [!NOTE]
>
> To prompt with PDFs, update the `url` field to: `data:application/pdf;base64,BASE64_PDF_DATA`

### PDF Support

Anthropic Claude processes PDFs to extract text, analyze charts, and understand visual content. PDF support is available on:

* Claude 3.7 Sonnet (`claude-3-7-sonnet-20250219`)
* Claude 3.5 Sonnet (`claude-3-5-sonnet-20241022`, `claude-3-5-sonnet-20240620`)
* Claude Sonnet 4-5 (`claude-sonnet-4-5-20250929`)
* Claude 3.5 Haiku (`claude-3-5-haiku-20241022`)

**Limitations:**

* Maximum request size: 32MB
* Maximum pages per request: 100
* Format: Standard PDF (no passwords/encryption)

### Extended Thinking (Reasoning Models)

Models like `claude-3-7-sonnet-latest` support [extended thinking](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#streaming-extended-thinking). Get the model's reasoning as it processes the request.

> [!NOTE]
>
> The assistant's thinking response is returned in the `response_chunk.choices[0].delta.content_blocks` array, not the `response.choices[0].message.content` string.

Set `strict_open_ai_compliance=False` to use this feature:

### Using /messages Route

The AI Gateway supports Anthropic's `/messages` endpoint, allowing you to use either Anthropic's native SDK or the AI Gateway's SDK with full gateway features.

#### Using Anthropic's Native SDK

```python Anthropic Py theme={"system"}
import anthropic

client = anthropic.Anthropic(
    api_key="dummy", # auth happens via the Authorization header
    default_headers={"Authorization": "Bearer YOUR_PORTKEY_API_KEY"},
    base_url="https://aigw.portkey.ai"
)

message = client.messages.create(
    model="@your-provider-slug/claude-sonnet-4-5-20250929",
    max_tokens=250,
    messages=[{"role": "user", "content": "Hello, Claude"}]
)

print(message.content)
```

```typescript Anthropic TS theme={"system"}
import Anthropic from '@anthropic-ai/sdk'

const anthropic = new Anthropic({
    apiKey: "YOUR_PORTKEY_API_KEY",
    baseURL: "https://aigw.portkey.ai"
})

const msg = await anthropic.messages.create({
    model: "@your-provider-slug/claude-sonnet-4-5-20250929",
    max_tokens: 1024,
    messages: [{ role: "user", content: "Hello, Claude" }]
})

console.log(msg)
```

#### Using the AI Gateway's SDK

```sh cURL icon="square-terminal" theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/messages' \
--header 'x-portkey-provider: anthropic' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_PORTKEY_API_KEY' \
--data-raw '{
    "model": "@your-provider-slug/claude-sonnet-4-5-20250929",
    "max_tokens": 1024,
    "stream": true,
    "messages": [
        {
            "role": "user",
            "content": "What is the weather like in Chennai?"
        }
    ]
}'
```

> [!NOTE]
>
> You can use all AI Gateway features (like caching, observability, configs) with this route. Just add the `x-portkey-config`, `x-portkey-provider`, `x-portkey-...` headers.

### Prompt Caching

The AI Gateway works with Anthropic's prompt caching feature to save time and money. Refer to this guide:

> **[Prompt Caching](https://docs.portkey.ai/docs/aigw/integrations/llms/anthropic/prompt-caching.md)**
>
> Learn how to enable prompt caching for Anthropic requests

### Structured Outputs

Ensure that the model always follows your supplied JSON schema with the AI Gateway's structured outputs support.

> **[Structured Outputs](https://docs.portkey.ai/docs/aigw/integrations/llms/anthropic/structured-outputs.md)**
>
> Learn how to use Pydantic, Zod, or JSON schema for structured data from Anthropic

### Web Search

Anthropic Claude models support web search as a tool, allowing the model to search the web for up-to-date information.

```sh cURL icon="square-terminal" theme={"system"}
curl POST 'https://aigw.portkey.ai/v1/chat/completions' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer $PORTKEY_API_KEY' \
  --header 'x-portkey-strict-open-ai-compliance: false' \
  --data '{
    "model": "@my-anthropic/claude-haiku-4-5",
    "max_tokens": 1024,
    "messages": [
        {
            "role": "user",
            "content": "What is the latest news on Poland?"
        }
    ],
    "tools": [
        {
            "type": "web_search",
            "web_search": {
                "name": "web_search_20250305",
                "max_uses": 2
            }
        }
    ]
}'
```

> [!NOTE]
>
> Set `strict_open_ai_compliance` to `false` (or use the header `x-portkey-strict-open-ai-compliance: false`) to receive citations in the response.

### Files API

The AI Gateway supports Anthropic's [Files API](https://docs.anthropic.com/en/docs/build-with-claude/files) (beta), enabling you to upload, list, retrieve, and delete files through the gateway. Uploaded files can be referenced in chat completions using `file_id` instead of re-uploading content each request.

> **[Files API](https://docs.portkey.ai/docs/aigw/integrations/llms/anthropic/files.md)**
>
> Upload, list, retrieve, and delete files — then use them in chat completions

### Service Tier

When routing Chat Completions requests to Anthropic, the AI Gateway automatically translates OpenAI's `service_tier` parameter to Anthropic's native `speed` parameter:

| `service_tier` | Anthropic `speed` |
| - | - |
| `auto` | `fast` |
| `standard_only` | `standard` |
| `default` | `standard` |
| `fast` | `fast` |
| `standard` | `standard` |
| unknown value | omitted |

```sh cURL icon="square-terminal" theme={"system"}
curl -X POST https://aigw.portkey.ai/v1/chat/completions \
  -H "Authorization: Bearer $PORTKEY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "@anthropic/claude-sonnet-4-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello!"}],
    "service_tier": "auto"
  }'
```

### Beta Features

The AI Gateway supports Anthropic's beta features through headers. Pass the beta feature name as the value:

```sh cURL icon="square-terminal" theme={"system"}
x-portkey-anthropic-beta: "token-efficient-tools-2025-02-19"
```

## Next Steps

> **[Add Metadata](https://docs.portkey.ai/docs/aigw/product/observability/metadata.md)**
>
> Add metadata to your Anthropic requests

> **[Gateway Configs](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md)**
>
> Add gateway configs to your Anthropic requests

> **[Tracing](https://docs.portkey.ai/docs/aigw/product/observability/traces.md)**
>
> Trace your Anthropic requests

> **[Fallbacks](https://docs.portkey.ai/docs/aigw/product/ai-gateway/fallbacks.md)**
>
> Setup fallback from OpenAI to Anthropic


## Related topics

- [Claude Code with Anthropic](https://docs.portkey.ai/docs/aigw/integrations/libraries/claude-code-anthropic.md)
- [Anthropic Computer Use](https://docs.portkey.ai/docs/aigw/integrations/libraries/anthropic-computer-use.md)
- [HoneyHive](https://docs.portkey.ai/docs/aigw/integrations/tracing-providers/honeyhive.md)
- [Amazon Bedrock Mantle](https://docs.portkey.ai/docs/aigw/integrations/llms/bedrock-mantle.md)
- [Claude Platform on AWS](https://docs.portkey.ai/docs/aigw/integrations/llms/claude-platform-aws/claude-platform-aws.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.