> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deepinfra

> Integrate Deepinfra models with Portkey's AI Gateway

Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including [Deepinfra's hosted models](https://deepinfra.com/models/text-generation).

With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through [Model Catalog](https://docs.portkey.ai/docs/product/model-catalog.md).

## Quick Start

Get Deepinfra working in 3 steps:

```python Python icon="python" theme={"system"}
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @deepinfra provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@deepinfra/nvidia/Nemotron-4-340B-Instruct",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```js Javascript icon="square-js" theme={"system"}
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @deepinfra provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@deepinfra/nvidia/Nemotron-4-340B-Instruct",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```python OpenAI Py icon="python" theme={"system"}
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @deepinfra provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@deepinfra/nvidia/Nemotron-4-340B-Instruct",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```js OpenAI JS icon="square-js" theme={"system"}
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @deepinfra provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@deepinfra/nvidia/Nemotron-4-340B-Instruct",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```sh cURL icon="square-terminal" theme={"system"}
# 1. Add @deepinfra provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@deepinfra/nvidia/Nemotron-4-340B-Instruct",
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'
```

> [!NOTE]
>
> **Tip:** You can also set `provider="@deepinfra"` in `Portkey()` and use just `model="nvidia/Nemotron-4-340B-Instruct"` in the request.

## Add Provider in Model Catalog

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Deepinfra**
3. Choose existing credentials or create new by entering your [Deepinfra API key](https://deepinfra.com/dash/api_keys)
4. Name your provider (e.g., `deepinfra-prod`)

> [card] [Complete Setup Guide →](https://docs.portkey.ai/docs/product/model-catalog.md)
>
> See all setup options, code examples, and detailed instructions

## Supported Endpoints

| Endpoint | Supported |
| - | - |
| `/chat/completions` | ✅ |
| `/completions` | ✅ |
| `/embeddings` | ✅ |

## Tool Calling

DeepInfra supports tool calling (function calling) for compatible models. Use the standard OpenAI `tools` format:

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"}
                },
                "required": ["location"]
            }
        }
    }
]

response = portkey.chat.completions.create(
    model="@deepinfra/meta-llama/Meta-Llama-3.1-70B-Instruct",
    messages=[{"role": "user", "content": "What's the weather in London?"}],
    tools=tools,
    tool_choice="auto"
)

print(response.choices[0].message.tool_calls)
```

```js Javascript theme={"system"}
import Portkey from 'portkey-ai'

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" })

const tools = [
    {
        type: "function",
        function: {
            name: "get_weather",
            description: "Get the current weather for a location",
            parameters: {
                type: "object",
                properties: {
                    location: { type: "string", description: "City name" }
                },
                required: ["location"]
            }
        }
    }
]

const response = await portkey.chat.completions.create({
    model: "@deepinfra/meta-llama/Meta-Llama-3.1-70B-Instruct",
    messages: [{ role: "user", content: "What's the weather in London?" }],
    tools,
    tool_choice: "auto"
})

console.log(response.choices[0].message.tool_calls)
```

## Supported Models

Deepinfra hosts a wide range of open-source models for text generation. View the complete list:

> [card] [Deepinfra Models](https://deepinfra.com/models/text-generation)
>
> Browse all available models on Deepinfra

Popular models include:

* `nvidia/Nemotron-4-340B-Instruct`
* `meta-llama/Meta-Llama-3.1-405B-Instruct`
* `Qwen/Qwen2.5-72B-Instruct`

## Next Steps

> [card] [Add Metadata](https://docs.portkey.ai/docs/product/observability/metadata.md)
>
> Add metadata to your Deepinfra requests

> [card] [Gateway Configs](https://docs.portkey.ai/docs/product/ai-gateway/configs.md)
>
> Add gateway configs to your Deepinfra requests

> [card] [Tracing](https://docs.portkey.ai/docs/product/observability/traces.md)
>
> Trace your Deepinfra requests

> [card] [Fallbacks](https://docs.portkey.ai/docs/product/ai-gateway/fallbacks.md)
>
> Setup fallback from OpenAI to Deepinfra

For complete SDK documentation:

> [card] [SDK Reference](https://docs.portkey.ai/docs/api-reference/sdk/list.md)
>
> Complete Portkey SDK documentation

***

> [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact)
>
> Contact Us


## Related topics

- [Deepinfra](https://docs.portkey.ai/docs/guides/integrations/deepinfra.md)
- [Enterprise Gateway](https://docs.portkey.ai/docs/changelog/enterprise.md)
- [Frontend](https://docs.portkey.ai/docs/changelog/frontend.md)
- [Overview](https://docs.portkey.ai/docs/guides/integrations.md)
- [March](https://docs.portkey.ai/docs/changelog/2026/march.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.