> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Arize AX

> Extend Prisma AIRS AI Gateway with Arize AX for unified LLM observability, tracing, and evaluation.

[Arize AX](https://arize.com/products/ax/) is the full-featured observability and evaluation platform from [Arize AI](https://arize.com/?utm_source=portkey-docs\&utm_medium=partner\&utm_campaign=partner-docs\&utm_content=tracing-provider-arize) for production teams, AI-native companies, and enterprises. It is available as managed cloud or enterprise self-hosted deployment.

The AI Gateway is a **production-grade AI Gateway and Observability platform** for AI applications. The [OpenInference Portkey instrumentation](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-portkey) lets Arize AX capture gateway-routed calls, including gateway requests and the underlying LLM completions.

> [!NOTE]
>
> Use Arize AX for production AI observability and evaluation. If you want an open-source path for local development, experimentation, or single-container self-hosting, use the separate [Arize Phoenix integration](https://docs.portkey.ai/docs/aigw/integrations/tracing-providers/phoenix.md).

## Why the AI Gateway + Arize AX?

Thanks to OpenInference instrumentation, the AI Gateway can emit structured traces automatically. This gives you visibility into each LLM call routed through the gateway, making it easier to debug behaviour, inspect token usage, and evaluate production traffic in Arize AX. For production evaluation patterns, see Arize's [agent evaluation guide](https://arize.com/guides/ai-agent-handbook/agent-evaluation/) and [LLM evaluation guide](https://arize.com/resources/llm-evaluation/).

> **AI Gateway Features**
>
> * **3,000+ LLMs**: Single API for OpenAI, Anthropic, AWS Bedrock, and more
> * **Advanced Routing**: Fallbacks, load balancing, conditional routing
> * **Cost Optimization**: Semantic caching and request controls
> * **Security**: PII detection, content filtering, compliance controls

> **Built-in Observability**
>
> * **40+ Key Metrics**: Cost, latency, tokens, error rates
> * **Detailed Logs & Traces**: Request/response bodies and custom tracing
> * **Custom Metadata**: Attach custom metadata to your requests
> * **Custom Alerts**: Real-time monitoring and notifications

With this integration, you can route LLM traffic through the AI Gateway and gain deep observability in Arize AX, bringing together gateway orchestration and ML observability.

## Getting Started

### Prerequisites

* Python 3.10+
* An Arize AX account
* An AI Gateway API key
* An OpenAI API key, or another provider key routed through the AI Gateway

### Step 1: Install dependencies

Install the required packages for Arize AX and the AI Gateway:

```bash theme={"system"}
pip install arize-otel openinference-instrumentation-portkey openai
```

### Step 2: Configure credentials

Set the Arize AX and AI Gateway credentials in the same shell that runs your application:

```bash theme={"system"}
export ARIZE_SPACE_ID="your-space-id"
export ARIZE_API_KEY="your-arize-api-key"
export ARIZE_PROJECT_NAME="portkey-gateway"
export PORTKEY_API_KEY="your-portkey-api-key"
export OPENAI_API_KEY="your-openai-api-key"
```

### Step 3: Register Arize AX and instrument the AI Gateway

**1. Configure Arize AX**

First, set up the Arize AX OpenTelemetry configuration:

```python theme={"system"}
import os
from arize.otel import register
from openinference.instrumentation.portkey import PortkeyInstrumentor

tracer_provider = register(
    space_id=os.environ["ARIZE_SPACE_ID"],
    api_key=os.environ["ARIZE_API_KEY"],
    project_name=os.environ["ARIZE_PROJECT_NAME"],
)

PortkeyInstrumentor().instrument(tracer_provider=tracer_provider)
```

**2. Configure AI Gateway**

Set up the AI Gateway in the same process after registering the `PortkeyInstrumentor`:

## Complete Integration Example

Here's a complete working example that connects AI Gateway with Arize AX for centralized monitoring.

## Verify in Arize AX

Open your Arize AX space and select the project configured by `ARIZE_PROJECT_NAME`. You should see a trace for the AI Gateway request, including the LLM span, prompt, response, token usage, model, and provider metadata.

> **[Arize AX AI Gateway Tracing Guide](https://arize.com/docs/ax/integrations/python-agent-frameworks/portkey/portkey-tracing)**
>
> See the Arize-side the AI Gateway tracing setup, verification steps, and troubleshooting guidance.

> **[Cookbook on the AI Gateway x Arize](https://docs.portkey.ai/docs/guides/integrations/arize-portkey.md)**
>
> Learn how to use the AI Gateway's Universal API to orchestrate multiple LLMs in a structured debate while tracking performance and evaluating outputs with Arize.

## AI Gateway Features

While Arize AX provides observability and evaluation, the AI Gateway delivers a complete AI infrastructure platform. Here's everything you get with the AI Gateway:

### 🚀 Core Gateway Capabilities

> **[3,000+ LLMs](https://docs.portkey.ai/docs/integrations.md)**
>
> Access OpenAI, Anthropic, Google, Cohere, Mistral, Llama, and 3,000+ models through a single unified API. No more managing different SDKs or endpoints.

> **[Universal API](https://docs.portkey.ai/docs/aigw/product/ai-gateway/universal-api.md)**
>
> Use the same code to call any LLM provider. Switch between models and providers without changing your application code.

> **[LLM Integrations](https://docs.portkey.ai/docs/product/integrations)**
>
> Secure vault for API keys with budget limits, rate limiting, and access controls. Never expose raw API keys in your code.

> **[Advanced Configs](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md)**
>
> Define routing strategies, model parameters, and reliability settings in reusable configurations. Version control your AI infrastructure.

### 🛡️ Reliability & Performance

> **[Smart Fallbacks](https://docs.portkey.ai/docs/aigw/product/ai-gateway/fallbacks.md)**
>
> Automatically switch to backup providers when primary fails. Define fallback chains across multiple providers.

> **[Load Balancing](https://docs.portkey.ai/docs/aigw/product/ai-gateway/load-balancing.md)**
>
> Distribute requests across multiple API keys or providers based on custom weights and strategies.

> **[Automatic Retries](https://docs.portkey.ai/docs/aigw/product/ai-gateway/automatic-retries.md)**
>
> Configurable retry logic with exponential backoff for transient failures and rate limits.

> **[Request Timeouts](https://docs.portkey.ai/docs/aigw/product/ai-gateway/request-timeouts.md)**
>
> Set custom timeouts to prevent hanging requests and improve application responsiveness.

> **[Conditional Routing](https://docs.portkey.ai/docs/aigw/product/ai-gateway/conditional-routing.md)**
>
> Route requests to different models based on content, metadata, or custom conditions.

> **[Canary Testing](https://docs.portkey.ai/docs/aigw/product/ai-gateway/canary-testing.md)**
>
> Gradually roll out new models or providers with percentage-based traffic splitting.

### 💰 Cost Optimization

> **[Semantic Caching](https://docs.portkey.ai/docs/aigw/product/ai-gateway/cache-simple-and-semantic.md)**
>
> Intelligent caching that understands semantic similarity. Reduce costs by up to 90% on repeated queries.

> **[Budget Limits](https://docs.portkey.ai/docs/aigw/product/policies/budget-limits.md)**
>
> Set spending limits per provider, team, or project. Get alerts before hitting limits.

> **[Rate Limits](https://docs.portkey.ai/docs/aigw/product/policies/rate-limits.md)**
>
> Set rate limits per provider, team, or project.

> **[Cost Analytics](https://docs.portkey.ai/docs/aigw/product/observability/analytics.md)**
>
> Real-time cost tracking across all providers with detailed breakdowns by model, user, and feature.

### 📊 Built-in Observability

> **[Comprehensive Metrics](https://docs.portkey.ai/docs/aigw/product/observability/analytics.md)**
>
> Track 40+ metrics including latency, tokens, costs, cache hits, error rates, and more in real-time.

> **[Detailed Logs](https://docs.portkey.ai/docs/aigw/product/observability/logs.md)**
>
> Full request/response logging with advanced filtering, search, and export capabilities.

> **[Distributed Tracing](https://docs.portkey.ai/docs/aigw/product/observability/traces.md)**
>
> Trace requests across your entire AI pipeline with correlation IDs and custom metadata.

> **[Custom Alerts](https://docs.portkey.ai/docs/aigw/product/observability/analytics.md)**
>
> Set up alerts on any metric with webhook, email, or Slack notifications.

### 🔒 Security & Compliance

> **[PII Detection](https://docs.portkey.ai/docs/aigw/product/guardrails.md)**
>
> Automatically detect and redact sensitive information like SSN, credit cards, and personal data.

> **[Content Filtering](https://docs.portkey.ai/docs/aigw/product/guardrails.md)**
>
> Block harmful, toxic, or inappropriate content in real-time based on custom policies.

> **[Access Controls](https://docs.portkey.ai/docs/aigw/product/enterprise-offering/access-control-management.md)**
>
> Fine-grained RBAC with team management, user permissions, and audit logs.

> **[Audit Logs](https://docs.portkey.ai/docs/aigw/product/enterprise-offering/access-control-management.md#audit-logs)**
>
> Complete audit trail of all API usage, configuration changes, and user actions.

> **[Data Privacy](https://docs.portkey.ai/docs/aigw/product/enterprise-offering/security.md)**
>
> Zero data retention options and deployment in your own VPC for maximum privacy.

### 🏢 Governance Features

> **[SSO Integration](https://docs.portkey.ai/docs/aigw/product/enterprise-offering/org-management/sso.md)**
>
> SAML 2.0 support for Okta, Azure AD, Google Workspace, and custom IdPs.

> **[Organisation Management](https://docs.portkey.ai/docs/aigw/product/enterprise-offering/org-management.md)**
>
> Multi-workspace support with hierarchical teams and department-level controls.

> **[Access Control](https://docs.portkey.ai/docs/aigw/product/enterprise-offering/access-control-management.md)**
>
> Team and individual level RBAC for AI services, LLMs, logs, and more.

> **[Private Deployments](https://docs.portkey.ai/docs/aigw/self-hosting/hybrid-deployments/architecture.md)**
>
> Deploy the AI Gateway in your own AWS, Azure, or GCP environment with full control.

## Next Steps

> **[Explore AI Gateway Features](https://docs.portkey.ai/docs/aigw/product/ai-gateway.md)**
>
> Discover all AI Gateway capabilities beyond observability

> **[LLM Integrations](https://docs.portkey.ai/docs/product/integrations)**
>
> Secure your API keys and set budgets

> **[Advanced Routing](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md)**
>
> Configure fallbacks, load balancing, and more

> **[Built-in Analytics](https://docs.portkey.ai/docs/aigw/product/observability.md)**
>
> Use the AI Gateway's native observability features


## Related topics

- [Arize Phoenix](https://docs.portkey.ai/docs/aigw/integrations/tracing-providers/phoenix.md)
- [Supported OTel Libraries](https://docs.portkey.ai/docs/aigw/product/observability/opentelemetry/list-of-supported-otel-instrumenters.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.