> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Ollama

> Integrate Ollama-hosted models with Portkey for local LLM deployment with full observability.

Portkey provides a robust gateway to facilitate the integration of your **locally hosted models through Ollama**.

> [accordion] Local Setup (npm or Docker)
>
> First, install the Gateway locally:
>
> ```bash npx theme={"system"}
> npx @portkey-ai/gateway
> ```
>
> ```bash Docker theme={"system"}
> docker run -d -p 8787:8787 portkeyai/gateway:latest
> ```
>
> Then, connect to your local Ollama instance:
>
> ```python Python theme={"system"}
> from portkey_ai import Portkey
>
> portkey = Portkey(
>     base_url="http://localhost:8787/v1",  # Your local Gateway
>     provider="ollama",
>     custom_host="http://localhost:11434"  # Your Ollama instance
> )
>
> response = portkey.chat.completions.create(
>     model="llama3",
>     messages=[{"role": "user", "content": "Hello!"}]
> )
> ```

## Integration Steps

**1. Expose your Ollama API**

Expose your Ollama API using a tunneling service like [ngrok](https://ngrok.com/) or make it publicly accessible. Skip this if you're self-hosting the Gateway.

For using Ollama with ngrok, here's a [useful guide](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-use-ollama-with-ngrok):

```sh theme={"system"}
ngrok http 11434 --host-header="localhost:11434"
```

**2. Add to Model Catalog**

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Enable **"Local/Privately hosted provider"** toggle
3. Select **Ollama** as the provider type
4. Enter your Ollama URL in **Custom Host**: `https://your-ollama.ngrok-free.app`
5. Name your provider (e.g., `my-ollama`)

> [card] [Complete Setup Guide](https://docs.portkey.ai/docs/product/model-catalog.md)
>
> See all setup options

**3. Use in Your Application**

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@my-ollama"
)

response = portkey.chat.completions.create(
    model="llama3",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@my-ollama'
});

const response = await portkey.chat.completions.create({
    model: 'llama3',
    messages: [{ role: 'user', content: 'Hello!' }]
});

console.log(response.choices[0].message.content);
```

**Or use custom host directly:**

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="ollama",
    custom_host="https://your-ollama.ngrok-free.app"
)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: 'ollama',
    customHost: 'https://your-ollama.ngrok-free.app'
});
```

> [!NOTE]
>
> **Important:** For Ollama integration, you only need to pass the base URL to `customHost` **without** the version identifier (such as `/v1`) - Portkey handles the rest!

***

## Supported Models

Ollama supports a wide range of models including:

* Llama 3, Llama 3.1, Llama 3.2
* Mistral, Mixtral
* Gemma, Gemma 2
* Phi-3
* Qwen 2
* And many more!

Check [Ollama's model library](https://ollama.com/library) for the complete list.

***

## Next Steps

> [card] [Gateway Configs](https://docs.portkey.ai/docs/product/ai-gateway.md)
>
> Add retries, timeouts, and fallbacks

> [card] [Observability](https://docs.portkey.ai/docs/product/observability.md)
>
> Monitor your Ollama requests

> [card] [Custom Host Guide](https://docs.portkey.ai/docs/product/ai-gateway/universal-api.md#integrating-local-or-private-models)
>
> Learn more about custom host setup

> [card] [BYOLLM Guide](https://docs.portkey.ai/docs/integrations/llms/byollm.md)
>
> Complete guide for private LLMs

For complete SDK documentation:

> [card] [SDK Reference](https://docs.portkey.ai/docs/api-reference/sdk/list.md)
>
> Complete Portkey SDK documentation

***

> [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact)
>
> Contact Us


## Related topics

- [November](https://docs.portkey.ai/docs/changelog/2024/nov.md)
- [Universal API](https://docs.portkey.ai/docs/product/ai-gateway/universal-api.md)
- [Make Your First Request](https://docs.portkey.ai/docs/introduction/make-your-first-request.md)
- [Anthropic Computer Use](https://docs.portkey.ai/docs/integrations/libraries/anthropic-computer-use.md)
- [Custom hosts](https://docs.portkey.ai/docs/product/ai-gateway/custom-hosts.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.