> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Triton Inference Server

> Integrate Triton-hosted custom models with Prisma AIRS AI Gateway for production observability and reliability.

The AI Gateway provides a robust platform to observe, govern, and manage your **locally** or **privately** hosted custom models using Triton Inference Server.

> [!NOTE]
>
> Here's the official [Triton Inference Server documentation](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/getting_started/quickstart.html) for more details.

## Integration Steps

**1. Expose your Triton Server**

Expose your Triton server using a tunneling service like [ngrok](https://ngrok.com/) or make it publicly accessible. Skip this if you're self-hosting the Gateway.

```sh theme={"system"}
ngrok http 8000 --host-header="localhost:8080"
```

**2. Add to Model Catalog**

1. Go to [**Model Catalog → Add Provider**](https://stratacloudmanager.paloaltonetworks.com/)
2. Enable **"Local/Privately hosted provider"** toggle
3. Select **Triton** as the provider type
4. Enter your Triton server URL in **Custom Host**: `http://localhost:8000/v2/models/mymodel`
5. Add authentication headers if needed
6. Name your provider (e.g., `my-triton`)

> **[Complete Setup Guide](https://docs.portkey.ai/docs/aigw/product/model-catalog.md)**
>
> See all setup options

**3. Use in Your Application**

**Or use custom host directly:**

***

## Next Steps

> **[Gateway Configs](https://docs.portkey.ai/docs/aigw/product/ai-gateway.md)**
>
> Add retries, timeouts, and fallbacks

> **[Observability](https://docs.portkey.ai/docs/aigw/product/observability.md)**
>
> Monitor your Triton deployments

> **[Custom Host Guide](https://docs.portkey.ai/docs/aigw/product/ai-gateway/universal-api.md#integrating-local-or-private-models)**
>
> Learn more about custom host setup

> **[BYOLLM Guide](https://docs.portkey.ai/docs/aigw/integrations/llms/byollm.md)**
>
> Complete guide for private LLMs


## Related topics

- [Supported Providers](https://docs.portkey.ai/docs/aigw/api-reference/inference-api/supported-providers.md)
- [Inference.net](https://docs.portkey.ai/docs/aigw/integrations/llms/inference.net.md)
- [Remote MCP](https://docs.portkey.ai/docs/aigw/product/ai-gateway/remote-mcp.md)
- [Overview](https://docs.portkey.ai/docs/aigw/integrations/llms.md)
- [Enterprise Gateway](https://docs.portkey.ai/docs/changelog/enterprise.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.