> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Azure AI Foundry

> Learn how to integrate Azure AI Foundry with Portkey to access a wide range of AI models with enhanced observability and reliability features.

Azure AI Foundry provides a unified platform for enterprise AI operations, model building, and application development. With Portkey, you can seamlessly integrate with various models available on Azure AI Foundry and take advantage of features like observability, prompt management, fallbacks, and more.

## Quick Start

```python Python icon="python" theme={"system"}
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @azure-foundry provider in Model Catalog
# 3. Use it:

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@azure-foundry"
)

response = portkey.chat.completions.create(
    model="DeepSeek-V3-0324",  # Your deployed model name
    messages=[{"role": "user", "content": "Tell me about cloud computing"}]
)

print(response.choices[0].message.content)
```

```js Javascript icon="square-js" theme={"system"}
import Portkey from "portkey-ai";

// 1. Install: npm install portkey-ai
// 2. Add @azure-foundry provider in Model Catalog
// 3. Use it:

const portkey = new Portkey({
  apiKey: "PORTKEY_API_KEY",
  provider: "@azure-foundry",
});

const response = await portkey.chat.completions.create({
  model: "DeepSeek-V3-0324", // Your deployed model name
  messages: [{ role: "user", content: "Tell me about cloud computing" }],
});

console.log(response.choices[0].message.content);
```

```sh cURL icon="square-terminal" theme={"system"}
# 1. Add @azure-foundry provider in Model Catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "DeepSeek-V3-0324",
    "messages": [{"role": "user", "content": "Tell me about cloud computing"}]
  }'
```

***

## Add Provider in Model Catalog

To integrate Azure AI Foundry with Portkey, you'll create a provider in the Model Catalog. This securely stores your Azure AI Foundry credentials, allowing you to use a simple identifier in your code instead of handling sensitive authentication details directly.

> [card] [OpenAI models on Azure](https://docs.portkey.ai/docs/integrations/llms/azure-openai)
>
> If you're specifically looking to use OpenAI models on Azure, you should use
> [Azure OpenAI](https://docs.portkey.ai/docs/integrations/llms/azure-openai) instead, which is optimized
> for OpenAI models.

***

### Understanding Azure AI Foundry Deployments

Azure AI Foundry offers three different ways to deploy models, each with unique endpoints and configurations:

1. **AI Services**: Azure-managed models accessed through Azure AI Services endpoints
2. **Managed**: User-managed deployments running on dedicated Azure compute resources
3. **Serverless**: Seamless, scalable deployment without managing infrastructure

You can learn more about the Azure AI Foundry deployment [here](https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/deployments-overview).

***

## Creating Your Azure AI Foundry Provider

Integrate Azure AI Foundry with Portkey to centrally manage your AI models and deployments. This guide walks you through setting up the provider using API key authentication.

### Prerequisites

Before creating your provider, you'll need:

* An active Azure AI Foundry account
* Access to your Azure AI Foundry portal
* A deployed model on Azure Foundry

### Step 1: Navigate to Model Catalog

Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers) and select **Azure AI Foundry** as your provider.

![Creating Azure AI Foundry Provider](https://mintcdn.com/portkey-docs/Az_PJRPMq602xEZC/images/product/model-catalog/integrations-page.png?fit=max\&auto=format\&n=Az_PJRPMq602xEZC\&q=85\&s=0750acc88ba68cab3d53d3cd2e27f4fa)

### Step 2: Configure Provider Details

Fill in the basic information for your provider:

* **Name**: A descriptive name for this provider (e.g., "Azure AI Production")
* **Short Description**: Optional context about this provider's purpose
* **Slug**: A unique identifier used in API calls (e.g., "@azure-ai-prod")

  ![](https://mintcdn.com/portkey-docs/wJcXkLPJHZBcCBJt/images/llms/azure/azure-1.3.png?fit=max\&auto=format\&n=wJcXkLPJHZBcCBJt\&q=85\&s=740483bf55b101f1212507ca1b0aba4c)

### Step 3: Set Up Authentication

Portkey supports three authentication methods for Azure AI Foundry. For most use cases, we recommend using the **Default (API Key)** method.

> [tab] Default (API Key)
>
> ### Gather Your Azure Credentials
>
> From your Azure AI Foundry portal, you'll need to collect:
>
> ![](https://mintcdn.com/portkey-docs/wJcXkLPJHZBcCBJt/images/llms/azure/azure-1.2.png?fit=max\&auto=format\&n=wJcXkLPJHZBcCBJt\&q=85\&s=bf814038cfbf4b2a3d5948e999a2bace)
>
> 1. Navigate to your model deployment in Azure AI Foundry
> 2. Click on the deployment to view details
> 3. Copy the **API Key** from the authentication section
> 4. Copy the **Target URI** - this is your endpoint URL
> 5. Note the **API Version** from your deployment URL
> 6. **Azure Deployment Name** (Optional): Only required for Managed Services deployments
>
> #### Enter Credentials in Portkey
>
> ![](https://mintcdn.com/portkey-docs/wJcXkLPJHZBcCBJt/images/llms/azure/azure-1.3.png?fit=max\&auto=format\&n=wJcXkLPJHZBcCBJt\&q=85\&s=740483bf55b101f1212507ca1b0aba4c)

> [tab] Azure Managed Entity
>
> For managed Azure deployments:
>
> Required parameters:
>
> * **Azure Managed ClientID**: Your managed client ID
>
> * **Azure Foundry URL**: The base endpoint URL for your deployment, formatted according to your deployment type:
>   * For AI Services: `https://your-resource-name.services.ai.azure.com/models`
>   * For project-scoped OpenAI v1 deployments: `https://your-resource-name.services.ai.azure.com/api/projects/your-project-name/openai/v1`
>   * For Managed: `https://your-model-name.region.inference.ml.azure.com/score`
>   * For Serverless: `https://your-model-name.region.models.ai.azure.com`
>
> * **Azure API Version**: The API version to use (e.g., "2024-05-01-preview"). This is required if you have api version in your deployment url.
>   **Examples:**
>   * If your URL is `https://mycompany-ai.westus2.services.ai.azure.com/models?api-version=2024-05-01-preview`, the API version is `2024-05-01-preview`
>
> * **Azure Deployment Name**: (Optional) Required only when a single resource contains multiple deployments.
>
> ![Default Authentication Setup](https://mintcdn.com/portkey-docs/PaRorqtknQiAf2YW/images/product/azure-2.png?fit=max\&auto=format\&n=PaRorqtknQiAf2YW\&q=85\&s=7bdc3203e677d4474a1a89450615ae5a)

> [tab] Azure Entra ID
>
> To use this authentication your azure application need to have the role of: `conginitive services user`.
> Enterprise-level authentication with Azure Entra ID:
>
> Required parameters:
>
> * **Azure Entra ClientID**: Your Azure Entra client ID
>
> * **Azure Entra Secret**: Your client secret
>
> * **Azure Entra Tenant ID**: Your tenant ID
>
> * **Azure Foundry URL**: The base endpoint URL for your deployment, formatted according to your deployment type:
>   * For AI Services: `https://your-resource-name.services.ai.azure.com/models`
>   * For project-scoped OpenAI v1 deployments: `https://your-resource-name.services.ai.azure.com/api/projects/your-project-name/openai/v1`
>   * For Managed: `https://your-model-name.region.inference.ml.azure.com/score`
>   * For Serverless: `https://your-model-name.region.models.ai.azure.com`
>
> * **Azure API Version**: The API version to use (e.g., "2024-05-01-preview"). This is required if you have api version in your deployment url.
>   **Examples:**
>   * If your URL is `https://mycompany-ai.westus2.services.ai.azure.com/models?api-version=2024-05-01-preview`, the API version is `2024-05-01-preview`
>
> * **Azure Deployment Name**: (Optional) Required only when a single resource contains multiple deployments. Common in Managed deployments.
>
> You can Learn more about these [Azure Entra Resources here](https://learn.microsoft.com/en-us/azure/ai-services/authentication)
>
> ![Default Authentication Setup](https://mintcdn.com/portkey-docs/PaRorqtknQiAf2YW/images/product/azure-3.png?fit=max\&auto=format\&n=PaRorqtknQiAf2YW\&q=85\&s=f0ebb61941227edc5ea7f3b75fd2f1c2)

For project-scoped OpenAI v1 deployments, enter the full URL ending with `/api/projects/<project-name>/openai/v1`, such as `https://your-resource-name.services.ai.azure.com/api/projects/your-project-name/openai/v1`, exactly as shown in Azure AI Foundry. Portkey appends the request route, such as `/chat/completions`, when sending traffic to the deployment.

## Adding Multiple Models to Your Azure AI Foundry Provider

You can deploy multiple models through a single Azure AI Foundry provider by using Portkey's custom models feature.

### Steps to Add Additional Models

1. Navigate to your Azure AI Foundry provider in Model Catalog
2. Select the **Model Provisioning** step
3. Click **Add Model** in the top-right corner

![](https://mintcdn.com/portkey-docs/pTimO0vUV04jRx9O/images/custom-model.gif?s=937a422620790fb07c154363c3d5ee5c)

#### Configure Your Model

Enter the following details for your Azure deployment:

**Model Slug**: Use your Azure Model Deployment name exactly as it appears in Azure AI Foundry

![Azure Deployment Name](https://mintcdn.com/portkey-docs/wJcXkLPJHZBcCBJt/images/llms/azure/azure-1.2.png?fit=max\&auto=format\&n=wJcXkLPJHZBcCBJt\&q=85\&s=bf814038cfbf4b2a3d5948e999a2bace)

**Short Description**: Optional description for team reference

**Model Type**: Select "Custom model"

**Base Model**: Choose the model that matches your deployment's API structure (e.g., select `gpt-4` for GPT-4 deployments)

> [!NOTE]
>
> This is just for reference. If you can't find the particular model, you can
> just choose a similar model.

**Custom Pricing**: Enable to track costs with your negotiated rates

Once configured, this model will be available alongside others in your provider, allowing you to manage multiple Azure deployments through a single set of credentials.

## Using Anthropic Models on Azure AI Foundry

Azure AI Foundry supports Anthropic models (Claude) through a slightly different configuration process. Follow these steps to integrate Anthropic models with Portkey.

### Step 1: Create an Azure Foundry Provider

When creating the provider for Anthropic models, you'll need to configure the following:

1. **Azure API Key**: Copy this from your Azure Foundry console
2. **Azure Target URI**: From your Foundry console, you'll get a URL like:

   ```
   https://resource-name-swedencentral.services.ai.azure.com/anthropic/v1/messages
   ```

   You need to strip your URL till `/anthropic`:

   ```
   https://resource-name-swedencentral.services.ai.azure.com/anthropic
   ```

> [!NOTE]
>
> For Anthropic models on Azure Foundry, you don't need to provide the **Azure
> API Version** or **Deployment Name** fields.

### Step 2: Configure Workspace Provisioning

After setting up the provider credentials, proceed with the workspace provisioning step as usual.

### Step 3: Add Your Anthropic Model

In the **Model Provisioning** step:

1. Click the **+ Add Model** button at the top
2. Configure the model with these details:
   * **Model Slug**: Enter your deployment name from the Azure Foundry console
   * **Base Model**: Search for and select your Anthropic model (e.g., `claude-opus-4-5-20251101`, `claude-sonnet-4-5-20250929`, `claude-haiku-4-5-20251001`, `claude-opus-4-1-20250805`)
3. Save the configuration

### Making Requests to Anthropic Models

Once configured, you can call your Anthropic model using the Model Slug you saved:

> [tab] NodeJS
>
> ```javascript theme={"system"}
> import Portkey from 'portkey-ai';
>
> const client = new Portkey({
>   apiKey: 'PORTKEY_API_KEY',
>   provider: '@AZURE_FOUNDRY_ANTHROPIC_PROVIDER'
> });
>
> const response = await client.chat.completions.create({
>   messages: [{ role: "user", content: "Hello, Claude!" }],
>   model: "your-azure-deployment-name", // Use the Model Slug you configured
> });
>
> console.log(response.choices[0].message.content);
> ```

> [tab] Python
>
> ```python theme={"system"}
> from portkey_ai import Portkey
>
> client = Portkey(
>   api_key="PORTKEY_API_KEY",
>   provider="@AZURE_FOUNDRY_ANTHROPIC_PROVIDER"
> )
>
> response = client.chat.completions.create(
>   model="your-azure-deployment-name", # Use the Model Slug you configured
>   messages=[
>     {"role": "user", "content": "Hello, Claude!"}
>   ]
> )
>
> print(response.choices[0].message.content)
> ```

> [tab] cURL
>
> ```sh theme={"system"}
> curl https://api.portkey.ai/v1/chat/completions \
>   -H "Content-Type: application/json" \
>   -H "x-portkey-api-key: $PORTKEY_API_KEY" \
>   -H "x-portkey-provider: $AZURE_FOUNDRY_ANTHROPIC_PROVIDER" \
>   -d '{
>     "model": "your-azure-deployment-name",
>     "messages": [
>       { "role": "user", "content": "Hello, Claude!" }
>     ]
>   }'
> ```

### Using the /messages Route with Azure Foundry Anthropic Models

Access Anthropic models on Azure AI Foundry through Anthropic's native `/messages` endpoint using Portkey's SDK or Anthropic's SDK.

> [!NOTE]
>
> The `/messages` route provides access to Anthropic-native features like
> extended thinking, prompt caching, and native streaming formats when using
> Claude models on Azure AI Foundry.

> [tab] cURL
>
> ```sh theme={"system"}
> curl --location 'https://api.portkey.ai/v1/messages' \
> --header 'Content-Type: application/json' \
> --header 'x-portkey-api-key: YOUR_PORTKEY_API_KEY' \
> --data-raw '{
>     "model": "@your-azure-foundry-anthropic-provider/your-azure-deployment-name",
>     "max_tokens": 1024,
>     "messages": [
>       {
>         "role": "user",
>         "content": "Hello, Claude!"
>       }
>     ]
>   }'
> ```

> [tab] Anthropic Python SDK
>
> ```python theme={"system"}
> import anthropic
>
> client = anthropic.Anthropic(
>     api_key="dummy", # we will use portkey's provider slug
>     default_headers={"x-portkey-api-key": "YOUR_PORTKEY_API_KEY"},
>     base_url="https://api.portkey.ai"
> )
>
> message = client.messages.create(
>     model="@your-azure-foundry-anthropic-provider/your-azure-deployment-name",
>     max_tokens=1024,
>     messages=[
>         {"role": "user", "content": "Hello, Claude!"}
>     ],
> )
> print(message.content)
> ```

> [tab] Anthropic TS SDK
>
> ```typescript theme={"system"}
> import Anthropic from '@anthropic-ai/sdk';
>
> const anthropic = new Anthropic({
>   apiKey: 'PORTKEY_API_KEY', // we will use portkey's provider slug
>   baseURL: "https://api.portkey.ai",
> });
>
> const msg = await anthropic.messages.create({
>   model: "@your-azure-foundry-anthropic-provider/your-azure-deployment-name",
>   max_tokens: 1024,
>   messages: [{ role: "user", content: "Hello, Claude!" }],
> });
> console.log(msg);
> ```

## Using the Responses API with Azure AI Foundry

Portkey supports the [Responses API](https://docs.portkey.ai/docs/product/ai-gateway/responses-api.md) on Azure AI Foundry in **two modes**. Pick the one that matches your Azure deployment.

| Mode | Default? | How it works | When to use |
| - | - | - | - |
| **Gateway Transformation** | Yes | Portkey converts Responses → Chat Completions → Responses around your Azure deployment | Any model deployed on Azure AI Foundry (DeepSeek, Cohere, Llama, Mistral, etc.) |
| **Direct (Proxy)** | Opt-in via header | Portkey forwards your Responses payload as-is to Azure's native `/responses` endpoint | Azure deployments that natively support `/responses` (e.g. Azure AI Agents) |

In both modes, Portkey logging, tracing, guardrails, caching, retries, and fallbacks continue to work.

### Mode 1 - Gateway Transformation (Default)

This is the default behaviour and requires no extra configuration. Portkey accepts the request at `/v1/responses`, adapts the payload to Chat Completions, calls Azure AI Foundry's `/chat/completions` endpoint, and translates the response back to the Responses API format before returning it to you.

Because the call is made over Chat Completions on Azure's side, **every model on your Azure AI Foundry provider works** — even ones that don't natively expose a Responses endpoint.

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@AZURE_FOUNDRY_PROVIDER"
)

response = portkey.responses.create(
    model="DeepSeek-V3-0324",  # Any model on your Azure AI Foundry provider
    input="Explain the difference between AKS and ACI in one paragraph."
)

print(response.output_text)
```

```js JavaScript theme={"system"}
import Portkey from "portkey-ai";

const portkey = new Portkey({
  apiKey: "PORTKEY_API_KEY",
  provider: "@AZURE_FOUNDRY_PROVIDER",
});

const response = await portkey.responses.create({
  model: "DeepSeek-V3-0324",
  input: "Explain the difference between AKS and ACI in one paragraph.",
});

console.log(response.output_text);
```

```sh cURL theme={"system"}
curl https://api.portkey.ai/v1/responses \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @AZURE_FOUNDRY_PROVIDER" \
  -d '{
    "model": "DeepSeek-V3-0324",
    "input": "Explain the difference between AKS and ACI in one paragraph."
  }'
```

> [!NOTE]
>
> Features that rely on Azure storing response state — `previous_response_id`, `store`, retrieve/delete on `/v1/responses/:id`, and built-in tools like `web_search` — are **not** available in this mode. Use multi-turn `input` arrays to carry conversation history. See [Open Responses → Native-Only Features](https://docs.portkey.ai/docs/product/ai-gateway/responses-api.md#native-only-features).

### Mode 2 - Direct Responses (Proxy)

When your Azure AI Foundry deployment exposes a native `/responses` endpoint (for example, **Azure AI Agents**), set the `x-portkey-provider-responses-proxy` header to `true`. Portkey will skip the Chat Completions transformation and forward the request body **exactly as-is** to Azure's native Responses endpoint.

This is required to use Azure-specific parameters such as `agent_reference`, which targets a deployed Azure AI Agent.

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@AZURE_FOUNDRY_PROVIDER",
    # Forwards the request directly to Azure's native /responses endpoint
    provider_responses_proxy=True,
)

response = portkey.responses.create(
    model="gpt-4.1",  # Your Azure deployment name
    input="Summarise the latest Q3 sales numbers for the EMEA region.",
    agent_reference="asst_abc123",  # Azure AI Agent reference
)

print(response.output_text)
```

```js JavaScript theme={"system"}
import Portkey from "portkey-ai";

const portkey = new Portkey({
  apiKey: "PORTKEY_API_KEY",
  provider: "@AZURE_FOUNDRY_PROVIDER",
  providerResponsesProxy: true,
});

const response = await portkey.responses.create({
  model: "gpt-4.1",
  input: "Summarise the latest Q3 sales numbers for the EMEA region.",
  agent_reference: "asst_abc123",
});

console.log(response.output_text);
```

```sh cURL theme={"system"}
curl https://api.portkey.ai/v1/responses \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @AZURE_FOUNDRY_PROVIDER" \
  -H "x-portkey-provider-responses-proxy: true" \
  -d '{
    "model": "gpt-4.1",
    "input": "Summarise the latest Q3 sales numbers for the EMEA region.",
    "agent_reference": "asst_abc123"
  }'
```

> [!NOTE]
>
> Direct mode forwards the payload verbatim. Your Azure deployment must accept the Responses API schema — if it doesn't, requests will fail. Use Gateway Transformation (Mode 1) for everything else.

### Choosing the right mode

* Use **Gateway Transformation** for general-purpose Responses API calls against any Azure-deployed model.
* Use **Direct (Proxy)** when calling **Azure AI Agents** or any Azure deployment that exposes the native Responses endpoint and you need Azure-specific parameters (`agent_reference`, server-side response storage, etc.).

For a deep dive into the Responses API itself — streaming, tool calling, reasoning, vision, structured output — see the [Open Responses](https://docs.portkey.ai/docs/product/ai-gateway/responses-api.md) guide.

## Sample Request

Once you've created your provider, you can start making requests to Azure AI Foundry models through Portkey.

> [tab] NodeJS
>
> Install the Portkey SDK with npm
>
> ```sh theme={"system"}
> npm install portkey-ai
> ```
>
> ```js theme={"system"}
> import Portkey from 'portkey-ai';
>
> const client = new Portkey({
>   apiKey: 'PORTKEY_API_KEY',
>   provider:'@AZURE_FOUNDRY_PROVIDER'
> });
>
> async function main() {
>   const response = await client.chat.completions.create({
>     messages: [{ role: "user", content: "Tell me about cloud computing" }],
>     model: "DeepSeek-V3-0324", // Replace with your deployed model name
>   });
>
>   console.log(response.choices[0].message.content);
> }
>
> main();
> ```

> [tab] Python
>
> Install the Portkey SDK with pip
>
> ```sh theme={"system"}
> pip install portkey-ai
> ```
>
> ```python theme={"system"}
> from portkey_ai import Portkey
>
> client = Portkey(
>   api_key = "PORTKEY_API_KEY",
>   provider = "@AZURE_FOUNDRY_PROVIDER"
> )
>
> response = client.chat.completions.create(
>   model="DeepSeek-V3-0324", # Replace with your deployed model name
>   messages=[
>     {"role": "system", "content": "You are a helpful assistant."},
>     {"role": "user", "content": "Tell me about cloud computing"}
>   ]
> )
>
> print(response.choices[0].message.content)
> ```

> [tab] cURL
>
> ```sh theme={"system"}
> curl https://api.portkey.ai/v1/chat/completions \
>   -H "Content-Type: application/json" \
>   -H "x-portkey-api-key: $PORTKEY_API_KEY" \
>   -H "x-portkey-provider: $AZURE_FOUNDRY_PROVIDER" \
>   -d '{
>     "model": "DeepSeek-V3-0324",
>     "messages": [
>       { "role": "user", "content": "Tell me about cloud computing" }
>     ]
>   }'
> ```

## Advanced Features

### Function Calling

Azure AI Foundry supports function calling (tool calling) for compatible models. Here's how to implement it with Portkey:

```python Python icon="python" theme={"system"}
tools = [{
    "type": "function",
    "function": {
        "name": "getWeather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

response = portkey.chat.completions.create(
    model="DeepSeek-V3-0324",  # Use a model that supports function calling
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What's the weather like in Delhi?"}
    ],
    tools=tools,
    tool_choice="auto"
)

print(response.choices[0])
```

```js Javascript icon="square-js" theme={"system"}
const tools = [
  {
    type: "function",
    function: {
      name: "getWeather",
      description: "Get the current weather",
      parameters: {
        type: "object",
        properties: {
          location: { type: "string", description: "City and state" },
          unit: { type: "string", enum: ["celsius", "fahrenheit"] },
        },
        required: ["location"],
      },
    },
  },
];

const response = await portkey.chat.completions.create({
  model: "DeepSeek-V3-0324", // Use a model that supports function calling
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What's the weather like in Delhi?" },
  ],
  tools,
  tool_choice: "auto",
});

console.log(response.choices[0]);
```

### Vision Capabilities

Process images alongside text using Azure AI Foundry's vision capabilities:

```python Python icon="python" theme={"system"}
response = portkey.chat.completions.create(
    model="Llama-4-Scout-17B-16E",  # Use a model that supports vision
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
                }
            ]
        }
    ],
    max_tokens=500
)

print(response.choices[0].message.content)
```

```js Javascript icon="square-js" theme={"system"}
const response = await portkey.chat.completions.create({
  model: "Llama-4-Scout-17B-16E", // Use a model that supports vision
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "What's in this image?" },
        {
          type: "image_url",
          image_url:
            "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
        },
      ],
    },
  ],
  max_tokens: 500,
});

console.log(response.choices[0].message.content);
```

### Structured Outputs

Get consistent, parseable responses in specific formats:

```python Python icon="python" theme={"system"}
import json

response = portkey.chat.completions.create(
    model="cohere-command-a",  # Use a model that supports response formats
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "List the top 3 cloud providers with their main services"}
    ],
    response_format={"type": "json_object"},
    temperature=0
)

print(json.loads(response.choices[0].message.content))
```

```js Javascript icon="square-js" theme={"system"}
const response = await portkey.chat.completions.create({
  model: "cohere-command-a", // Use a model that supports response formats
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    {
      role: "user",
      content: "List the top 3 cloud providers with their main services",
    },
  ],
  response_format: { type: "json_object" },
  temperature: 0,
});

console.log(JSON.parse(response.choices[0].message.content));
```

***

## Portkey Gateway Features

Portkey provides advanced gateway features for Azure AI Foundry deployments:

### Fallbacks

Create fallback configurations to ensure reliability when working with Azure AI Foundry models:

```json theme={"system"}
{
  "strategy": {
    "mode": "fallback"
  },
  "targets": [
    {
      "provider": "@azure-foundry-prod",
      "override_params": {
        "model": "DeepSeek-V3-0324"
      }
    },
    {
      "provider": "@openai-prod",
      "override_params": {
        "model": "gpt-4o"
      }
    }
  ]
}
```

### Load Balancing

Distribute requests across multiple models for optimal performance:

```json theme={"system"}
{
  "strategy": {
    "mode": "loadbalance"
  },
  "targets": [
    {
      "provider": "@azure-foundry-prod-1",
      "override_params": {
        "model": "DeepSeek-V3-0324"
      },
      "weight": 0.7
    },
    {
      "provider": "@azure-foundry-prod-2",
      "override_params": {
        "model": "cohere-command-a"
      },
      "weight": 0.3
    }
  ]
}
```

### Conditional Routing

Route requests based on specific conditions like user type or content requirements:

```json theme={"system"}
{
  "strategy": {
    "mode": "conditional",
    "conditions": [
      {
        "query": { "metadata.user_type": { "$eq": "premium" } },
        "then": "high-performance-model"
      },
      {
        "query": { "metadata.content_type": { "$eq": "code" } },
        "then": "code-specialized-model"
      }
    ],
    "default": "standard-model"
  },
  "targets": [
    {
      "name": "high-performance-model",
      "provider": "@azure-foundry-prod-1",
      "override_params": {
        "model": "Llama-4-Scout-17B-16E"
      }
    },
    {
      "name": "code-specialized-model",
      "provider": "@azure-foundry-prod-2",
      "override_params": {
        "model": "DeepSeek-V3-0324"
      }
    },
    {
      "name": "standard-model",
      "provider": "@azure-foundry-prod-3",
      "override_params": {
        "model": "cohere-command-a"
      }
    }
  ]
}
```

***

## Managing Prompts

You can manage all prompts to Azure AI Foundry in the [Prompt Library](https://docs.portkey.ai/docs/product/prompt-library.md). Once you've created and tested a prompt in the library, use the `portkey.prompts.completions.create` interface to use the prompt in your application.

```python Python icon="python" theme={"system"}
prompt_completion = portkey.prompts.completions.create(
    prompt_id="Your Prompt ID",
    variables={
       # The variables specified in the prompt
    }
)
```

```js Javascript icon="square-js" theme={"system"}
const promptCompletion = await portkey.prompts.completions.create({
  promptID: "Your Prompt ID",
  variables: {
    // The variables specified in the prompt
  },
});
```

***

## Rerank

Azure AI Foundry supports reranking through Cohere models deployed on the platform. Use the Portkey unified `/rerank` endpoint with the `cohere.` model prefix:

```python Python theme={"system"}
from portkey_ai import Portkey

client = Portkey(
api_key="PORTKEY_API_KEY",
provider="@azure-foundry"
)

result = client.post(
"/v1/rerank",
body={
"model": "cohere.Cohere-rerank-v4.0-pro",
"query": "What is deep learning?",
"documents": [
"Deep learning is a subset of machine learning",
"The weather is sunny today",
"Neural networks have multiple layers"
]
}
)

print(result)

```

```sh cURL theme={"system"}
curl https://api.portkey.ai/v1/rerank \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @azure-foundry" \
  -d '{
    "model": "cohere.Cohere-rerank-v4.0-pro",
    "query": "What is deep learning?",
    "documents": [
      "Deep learning is a subset of machine learning",
      "The weather is sunny today",
      "Neural networks have multiple layers"
    ]
  }'
```

The `cohere.` prefix in the model name is automatically stripped before forwarding to the provider.

***

## OCR (Document Processing)

Azure AI Foundry supports Mistral OCR models for extracting text and structured content from PDFs and images. Use the `/v1/ocr` endpoint:

```python Python theme={"system"}
from portkey_ai import Portkey

client = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@azure-foundry"
)

result = client.post(
    "/ocr",
    model="mistral-ocr-4-0",
    document={
        "type": "document_url",
        "document_url": "data:application/pdf;base64,<BASE64_PDF>"
    },
    include_image_base64=True
)

for page in result.pages:
    print(f"Page {page['index']}: {page['markdown'][:100]}")
```

```sh cURL theme={"system"}
curl https://api.portkey.ai/v1/ocr \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @azure-foundry" \
  -d '{
    "model": "mistral-ocr-4-0",
    "document": {
      "type": "document_url",
      "document_url": "data:application/pdf;base64,<BASE64_PDF>"
    },
    "include_image_base64": true
  }'
```

**Supported Models:** `mistral-ocr-4-0`, `mistral-document-ai-2505`, `mistral-document-ai-2512`

> [!NOTE]
>
> Azure AI Foundry does not support external URLs for OCR. Documents must be sent as base64-encoded data URIs (e.g. `data:application/pdf;base64,...`).

> [!NOTE]
>
> The native `/v1/ocr` endpoint requires gateway version **2.18.0** or higher for hybrid (self-hosted) deployments.

***

## Decisions (Microsoft-Decision-1)

`Microsoft-Decision-1` is a decision model available in Microsoft Foundry. It does not generate text. It reads a piece of text or JSON, answers typed questions about it, and returns probabilities that your application can threshold or rank. Use it for classification, routing, ranking, grading, and yes/no checks.

Portkey routes these calls through `/v1/decisions`, a separate endpoint from chat completions. You still get the gateway's full feature set: authentication, routing, retries, circuit breaking, caching, logging, and cost tracking. See the [Decisions API](https://docs.portkey.ai/docs/product/ai-gateway/decisions-api.md) for the full request and response reference.

> [!NOTE]
>
> `/v1/decisions` does not support streaming. Only before-request [guardrails](https://docs.portkey.ai/docs/product/guardrails/decisions-guardrails.md) apply, scanning the content to judge only (`state`).

> [!NOTE]
>
> This section covers how to call Microsoft-Decision-1 through Portkey. For model details, deployment steps in Azure, regional availability, and responsible-use guidance, see Microsoft's official guide: [Deploy and use Microsoft-Decision-1 in Microsoft Foundry](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision).

### Model at a glance

These values come from [Microsoft's model overview](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision#microsoft-decision-1-at-a-glance).

| Model name | Model version | Deployment type | API type |
| :- | :- | :- | :- |
| `Microsoft-Decision-1` | 1 | `DataZoneStandard` in selected regions, `GlobalStandard` | Decision |

### Request model vs response model

This is the most common source of errors with this model, so read it before you send a request.

| Field | Value | Example |
| :- | :- | :- |
| `model` in your **request** | The **deployment name** you chose in Azure AI Foundry | `@azure-foundry/decision-1` |
| `model` in the **response** | The underlying model | `microsoft-decision-1` |

> [!WARNING]
>
> Do not hardcode `microsoft-decision-1` as the request `model`. Azure looks up a **deployment** with that exact name. If your deployment has a different name, the request fails with `The API deployment microsoft-decision-1 does not exist`. Use the deployment name you see in Foundry. Microsoft describes this in the [Call the decision API](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision) section of its guide.

For example, if you deploy the model with the deployment name `decision-1`, send `"model": "@azure-foundry/decision-1"`. If you name the deployment `pi-decision-1`, send `"model": "@azure-foundry/pi-decision-1"`. In both cases the response reports `"model": "microsoft-decision-1"`.

Portkey logs and prices the call using the response `model`, so cost tracking works the same no matter what you name your deployment.

### Setup

1. Deploy `Microsoft-Decision-1` in your Foundry project by following [Microsoft's deployment steps](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision). From the deployment details page, copy the **resource endpoint**, the **deployment name**, and the **API key**.
2. Create your Azure AI Foundry provider by following [Creating Your Azure AI Foundry Provider](#creating-your-azure-ai-foundry-provider) above. Use the URL as shown in Foundry for your resource, for example `https://your-resource-name.services.ai.azure.com/models`.
3. If you manage models in **Model Provisioning**, add your deployment name as the model slug (for example `decision-1`), exactly as it appears in Foundry. See [Adding Multiple Models](#adding-multiple-models-to-your-azure-ai-foundry-provider).

> [!NOTE]
>
> For decisions requests, Portkey sends the call to the resource root at `/providers/microsoft/v1/systemone`. Any path in the saved Foundry URL, including a `/deployments/<name>` suffix, is dropped. No `api-version` is sent. The deployment is selected by the `model` field in the request body.

Decisions requests work with the same authentication methods as the rest of this integration: API key, Azure Entra ID, and Azure Managed Identity. With an API key, Portkey sends it in the `api-key` header, which is what the Microsoft decision endpoint expects.

### Send a decisions request

Microsoft-Decision-1 uses the Jev request shape: `state` for the content to judge, and a `questions` map. Set `model` to `@provider-slug/deployment-name`. The provider holds the Azure credentials, so no provider headers are needed.

```bash cURL theme={"system"}
curl --request POST \
  --url https://api.portkey.ai/v1/decisions \
  --header 'Content-Type: application/json' \
  --header "x-portkey-api-key: $PORTKEY_API_KEY" \
  --data '{
  "model": "@azure-foundry/decision-1",
  "state": {
    "current_ask": "hello there, how is your day going?"
  },
  "questions": {
    "greeting": {
      "type": "choice",
      "instructions": "Is this a greeting?",
      "criteria": {
        "yes": "a greeting",
        "no": "anything else"
      }
    }
  }
}'
```

Response (values are illustrative):

```json theme={"system"}
{
  "model": "microsoft-decision-1",
  "answers": {
    "greeting": {
      "type": "choice",
      "choice": "yes",
      "probabilities": { "no": 0.0067, "yes": 0.9933 },
      "confidence": 0.9866
    }
  },
  "usage": { "input_tokens": 46, "output_tokens": 1 },
  "provider": "azure-ai"
}
```

The OpenAI request shape (`input` and a `questions` array) also works. See [Request Shapes](https://docs.portkey.ai/docs/product/ai-gateway/decisions-api.md#request-shapes).

### Question types

Each entry in `questions` sets a `type`, which determines the shape of its answer. The `state` can be text or JSON.

| Type | Question it answers | Answer shape |
| :- | :- | :- |
| `noul` | Is this true? | `{ noul: <0 to 1 probability> }` |
| `choice` | Which option is it? | `{ choice, probabilities, confidence }` |
| `score` | How much? | `{ score, legend, probabilities, confidence }` |

The response `answers` object is keyed by the question names in your request.

#### `noul`: gates and filters

Use for yes/no conditions, such as whether a reply promises a refund or a document contains a prompt injection. `criteria` is optional. Define the true and false conditions to make borderline questions more precise.

```json theme={"system"}
{
  "model": "@azure-foundry/decision-1",
  "state": "I refunded your $200. You don't need to contact billing.",
  "questions": {
    "promises_refund": {
      "type": "noul",
      "instructions": "Does this reply promise the customer a refund?",
      "criteria": {
        "true": "Commits to returning money",
        "false": "Makes no commitment about money"
      }
    }
  }
}
```

#### `choice`: classification and routing

Use to pick exactly one label from a fixed set, such as routing tickets or classifying intent. Describe each option clearly. The response includes a probability for every option, so a low top probability is a good signal to escalate to a person or another model.

```json theme={"system"}
{
  "model": "@azure-foundry/decision-1",
  "state": "The API returns 500 on every call.",
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this ticket?",
      "criteria": {
        "billing": "Charges, invoices, and refunds",
        "engineering": "Bugs, errors, and outages",
        "support": "How-to and account questions"
      }
    }
  }
}
```

#### `score`: ranking and grading

Use to place an item on an ordered scale, such as incident severity. List the levels from lowest to highest. The returned score is the probability-weighted average of the level indexes, so a four-level scale runs from `0` through `3` and the result can fall between levels. Prefer scores for relative ordering and thresholds, not as absolute ratings.

```json theme={"system"}
{
  "model": "@azure-foundry/decision-1",
  "state": "Checkout is down for all EU customers since 09:00.",
  "questions": {
    "severity": {
      "type": "score",
      "instructions": "How severe is this incident?",
      "criteria": ["Cosmetic", "Minor", "Major", "Critical"]
    }
  }
}
```

#### Combine questions in one request

Questions that share the same `state` can go in one request. This reduces request count and keeps the context consistent.

```json theme={"system"}
{
  "model": "@azure-foundry/decision-1",
  "state": {
    "message": "Checkout is down again. This is my third request.",
    "customer_contact_count": 3
  },
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Charges, invoices, and refunds",
        "engineering": "Bugs, errors, and outages",
        "support": "How-to and account questions"
      }
    },
    "severity": {
      "type": "score",
      "instructions": "How severe is this issue?",
      "criteria": ["Cosmetic", "Minor", "Major", "Critical"]
    },
    "repeat_contact": {
      "type": "noul",
      "instructions": "Has the customer contacted support before?"
    }
  }
}
```

### Pricing and usage

Usage is reported in the response as `usage.input_tokens` and `usage.output_tokens`. The pricing entry for `microsoft-decision-1` charges for input tokens only. Output tokens, which hold the typed judgment itself, are priced at zero.

### Decisions troubleshooting

| Status | Likely cause | What to do |
| :- | :- | :- |
| `400` | Invalid question type or shape | Check `state`, `questions`, `type`, and `criteria` |
| `401` | Missing, invalid, or expired credential | Check the API key or Entra credentials on the provider |
| `403` | The identity has no access to the deployment | Check role assignments and deployment access in Azure |
| `404` | Wrong endpoint, or the deployment name does not exist | Confirm the `model` value matches your deployment name exactly. See [Request model vs response model](#request-model-vs-response-model) |
| `422` | Unsupported deployment name or question definition | Check the `model` value and each question definition |
| `429` | The deployment rate limit was exceeded | Retry with backoff, or request more quota in Azure. You can also add retries or fallbacks in a [gateway config](https://docs.portkey.ai/docs/product/ai-gateway/configs.md) |

Upstream errors use the standard Portkey error shape, with the message prefixed by `azure-ai`:

```json theme={"system"}
{
  "error": {
    "message": "azure-ai error: The API deployment microsoft-decision-1 does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.",
    "type": null,
    "param": null,
    "code": null
  },
  "provider": "azure-ai"
}
```

### Limitations and responsible use

From Microsoft's guidance for this model:

* **No explanations.** The model returns probabilities, not a written rationale.
* **Wording sensitivity.** Scores can change with how you phrase or order questions and options.
* **Bias.** The model might reflect biases from its base model and training data. Do not use its scores as the sole basis for decisions about individuals.
* **Safety filtering.** If you use it as a safety filter, it can miss subtle harmful content or flag benign content.
* **Validate on your data.** Test thresholds against labeled examples from your own workload before you automate a gate.

See [Known limitations and best practices](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision) in Microsoft's guide for the full list.

***

## Next Steps

> [card] [SDK Reference](https://docs.portkey.ai/docs/api-reference/sdk/list.md)
>
> Complete SDK documentation and API reference

> [card] [Azure OpenAI](https://docs.portkey.ai/docs/integrations/llms/azure-openai)
>
> Use Azure OpenAI for OpenAI-specific models

> [card] [Add Metadata](https://docs.portkey.ai/docs/product/observability/metadata.md)
>
> Add custom metadata to your Azure AI Foundry requests

> [card] [Gateway Configs](https://docs.portkey.ai/docs/product/ai-gateway/configs.md)
>
> Configure advanced gateway features

> [card] [Request Tracing](https://docs.portkey.ai/docs/product/observability/traces.md)
>
> Trace your Azure AI Foundry requests

> [card] [Setup Fallbacks](https://docs.portkey.ai/docs/product/ai-gateway/fallbacks.md)
>
> Create fallback configurations between providers

***

> [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact)
>
> Contact Us


## Related topics

- [Azure OpenAI](https://docs.portkey.ai/docs/integrations/llms/azure-openai/azure-openai.md)
- [May](https://docs.portkey.ai/docs/changelog/2026/may.md)
- [Microsoft Azure](https://docs.portkey.ai/docs/integrations/cloud/azure.md)
- [AKS](https://docs.portkey.ai/docs/self-hosting/hybrid-deployments/azure/aks.md)
- [Authentication](https://docs.portkey.ai/docs/integrations/llms/azure-openai/authentication.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.