> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Predibase

> Use Predibase's open-source and fine-tuned LLMs through Portkey.

## Quick Start

Get started with Predibase in under 2 minutes:

```python Python icon="python" theme={"system"}
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @predibase provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@predibase/llama-3-8b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```js Javascript icon="square-js" theme={"system"}
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @predibase provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@predibase/llama-3-8b-instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python OpenAI Py icon="python" theme={"system"}
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @predibase provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@predibase/llama-3-8b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```js OpenAI JS icon="square-js" theme={"system"}
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @predibase provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@predibase/llama-3-8b-instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```sh cURL icon="square-terminal" theme={"system"}
# 1. Add @predibase provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@predibase/llama-3-8b-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

## Add Provider in Model Catalog

Before making requests, add Predibase to your Model Catalog:

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Predibase**
3. Enter your [Predibase API key](https://app.predibase.com/settings)
4. Name your provider (e.g., `predibase`)

> [card] [Complete Setup Guide](https://docs.portkey.ai/docs/product/model-catalog.md)
>
> See all setup options and detailed configuration instructions

***

## Predibase Capabilities

### Serverless Endpoints

Predibase offers LLMs like **Llama 3**, **Mistral**, **Gemma**, etc. on its [serverless infrastructure](https://docs.predibase.com/user-guide/inference/models#serverless-endpoints) that you can query instantly.

> [!NOTE]
>
> **Sending Predibase Tenant ID**
>
> Predibase expects your **account tenant ID** along with the API key in each request. With Portkey, you can send [**your Tenant ID**](https://app.predibase.com/settings) with the `user` param while making your request.

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@predibase")

response = portkey.chat.completions.create(
    model="llama-3-8b-instruct",
    messages=[{"role": "user", "content": "Hello!"}],
    user="PREDIBASE_TENANT_ID"  # Required: Your Predibase tenant ID
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@predibase'
});

const response = await portkey.chat.completions.create({
    model: "llama-3-8b-instruct",
    messages: [{ role: "user", content: "Hello!" }],
    user: "PREDIBASE_TENANT_ID"  // Required: Your Predibase tenant ID
});

console.log(response.choices[0].message.content);
```

### Using Fine-Tuned Models

Predibase allows you to deploy and use fine-tuned models with adapters. Use the special format with your model identifier from your Predibase dashboard:

> [!NOTE]
>
> **Fine-Tuned Model Format:**\
> `model = base_model:adapter-repo-name/adapter-version-number`
>
> For example: `llama-3-8b:sentiment-analysis/1`

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@predibase")

response = portkey.chat.completions.create(
    model="llama-3-8b:sentiment-analysis/1",  # Base model + adapter
    messages=[{"role": "user", "content": "This product is amazing!"}],
    user="PREDIBASE_TENANT_ID"
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@predibase'
});

const response = await portkey.chat.completions.create({
    model: "llama-3-8b:sentiment-analysis/1",  // Base model + adapter
    messages: [{ role: "user", content: "This product is amazing!" }],
    user: "PREDIBASE_TENANT_ID"
});

console.log(response.choices[0].message.content);
```

### Dedicated Deployments

Route requests to your dedicated deployed models by passing the deployment name in the `model` parameter:

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@predibase")

response = portkey.chat.completions.create(
    model="my-dedicated-mistral-deployment",  # Your deployment name
    messages=[{"role": "user", "content": "Hello!"}],
    user="PREDIBASE_TENANT_ID"
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@predibase'
});

const response = await portkey.chat.completions.create({
    model: "my-dedicated-mistral-deployment",  // Your deployment name
    messages: [{ role: "user", content: "Hello!" }],
    user: "PREDIBASE_TENANT_ID"
});

console.log(response.choices[0].message.content);
```

### JSON Schema Mode

Enforce JSON schema for all Predibase models by setting `response_format` to `json_object` with your schema:

```python Python theme={"system"}
from portkey_ai import Portkey
from pydantic import BaseModel, constr

# Define JSON Schema with Pydantic
class Character(BaseModel):
    name: constr(max_length=10)
    age: int
    strength: int

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@predibase")

response = portkey.chat.completions.create(
    model="llama-3-8b",
    messages=[{"role": "user", "content": "Create a character profile"}],
    user="PREDIBASE_TENANT_ID",
    response_format={
        "type": "json_object",
        "schema": Character.schema()
    }
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@predibase'
});

const response = await portkey.chat.completions.create({
    model: "llama-3-8b",
    messages: [{ role: "user", content: "Create a character profile" }],
    user: "PREDIBASE_TENANT_ID",
    response_format: {
        type: "json_object",
        schema: {
            properties: {
                name: { maxLength: 10, title: "Name", type: "string" },
                age: { title: "Age", type: "integer" },
                strength: { title: "Strength", type: "integer" }
            },
            required: ["name", "age", "strength"],
            title: "Character",
            type: "object"
        }
    }
});

console.log(response.choices[0].message.content);
```

***

## Supported Models

Predibase provides access to various open-source and fine-tuned models:

* Llama 3 (various sizes)
* Mistral
* Zephyr
* Your custom fine-tuned models

Check [Predibase's documentation](https://docs.predibase.com/) for the complete model list and fine-tuning options.

***

## Next Steps

> [card] [Gateway Configs](https://docs.portkey.ai/docs/product/ai-gateway.md)
>
> Add fallbacks, load balancing, and more

> [card] [Observability](https://docs.portkey.ai/docs/product/observability.md)
>
> Monitor and trace your Predibase requests

> [card] [Prompt Library](https://docs.portkey.ai/docs/product/prompt-engineering-studio.md)
>
> Manage and version your prompts

> [card] [Fine-Tuning Logs](https://docs.portkey.ai/docs/product/observability.md)
>
> Track performance of fine-tuned models

For complete SDK documentation:

> [card] [SDK Reference](https://docs.portkey.ai/docs/api-reference/sdk/list.md)
>
> Complete Portkey SDK documentation

***

> [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact)
>
> Contact Us


## Related topics

- [Portkey Models](https://docs.portkey.ai/docs/product/model-catalog/portkey-models.md)
- [Overview](https://docs.portkey.ai/docs/integrations/llms.md)
- [Supported Providers](https://docs.portkey.ai/docs/api-reference/inference-api/supported-providers.md)
- [Create Virtual Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/virtual-keys/create-virtual-key.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.