OpenLIT allows you to simplify your AI development workflow, especially for Generative AI and LLMs. It streamlines essential tasks like experimenting with LLMs, organizing and versioning prompts, and securely handling API keys. With just one line of code, you can enable OpenTelemetry-native observability, offering full-stack monitoring that includes LLMs, vector databases, and GPUs.

OpenLIT’s automatic instrumentation combined with Portkey’s intelligent gateway creates a comprehensive observability solution where every trace captures model performance, prompt versioning, and cost optimization data in real-time.

​
Why OpenLIT + Portkey?

One-Line Instrumentation

Enable complete observability with a single line of code for all AI components

Full-Stack AI Monitoring

Monitor LLMs, vector databases, and GPUs in a unified view

Native OpenTelemetry

Built on OpenTelemetry standards for seamless integration

Production-Ready

Smooth transition from experimentation to production deployment

​
Quick Start

​
Prerequisites

  • Python
  • Portkey account with API key
  • OpenAI API key (or use Portkey’s virtual keys)

​
Step 1: Install Dependencies

Install the required packages for OpenLIT and Portkey integration:

pip install openlit openai portkey-ai

​
Step 2: Configure OpenTelemetry Export

Set up the environment variables to send traces to Portkey’s OpenTelemetry endpoint:

import os

# Configure Portkey endpoint and authentication
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://api.portkey.ai/v1/logs/otel"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-portkey-api-key=YOUR_PORTKEY_API_KEY"

​
Step 3: Initialize OpenLIT with Custom Tracer

Set up OpenTelemetry tracer and initialize OpenLIT:

import openlit
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry import trace

# Create and configure the tracer provider
trace_provider = TracerProvider()
trace_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter()))

# Set the global default tracer provider
trace.set_tracer_provider(trace_provider)

# Create a tracer from the global tracer provider
tracer = trace.get_tracer(__name__)

# Initialize OpenLIT with the custom tracer
# disable_batch=True ensures traces are processed immediately
openlit.init(tracer=tracer, disable_batch=True)

​
Step 4: Configure Portkey Gateway

Set up the OpenAI client to use Portkey’s intelligent gateway:

from openai import OpenAI
from portkey_ai import createHeaders

# Create OpenAI client with Portkey's gateway
client = OpenAI(
    api_key="YOUR_OPENAI_API_KEY",  # Or use a dummy value with virtual keys
    base_url="https://api.portkey.ai/v1",
    default_headers=createHeaders(
        api_key="YOUR_PORTKEY_API_KEY",
        virtual_key="YOUR_VIRTUAL_KEY"  # Optional: Use Portkey's secure key management
    )
)

​
Step 5: Make Instrumented LLM Calls

Now your LLM calls are automatically traced by OpenLIT and enhanced by Portkey:

# Make calls through Portkey's gateway
# OpenLIT instruments the call, Portkey adds gateway intelligence
response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {
            "role": "user",
            "content": "Explain the benefits of OpenTelemetry in AI applications"
        }
    ],
    temperature=0.7
)

print(response.choices[0].message.content)

# You now get:
# 1. Automatic tracing from OpenLIT
# 2. Gateway features from Portkey (caching, fallbacks, routing)
# 3. Combined insights in Portkey's dashboard

​
Complete Example

Here’s a full example bringing everything together:

import os
import openlit
from openai import OpenAI
from portkey_ai import createHeaders
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry import trace

# Step 1: Configure Portkey endpoint
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://api.portkey.ai/v1/logs/otel"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-portkey-api-key=YOUR_PORTKEY_API_KEY"

# Step 2: Set up OpenTelemetry
trace_provider = TracerProvider()
trace_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(trace_provider)
tracer = trace.get_tracer(__name__)

# Step 3: Initialize OpenLIT
openlit.init(tracer=tracer, disable_batch=True)

# Step 4: Configure Portkey Gateway
client = OpenAI(
    api_key="YOUR_OPENAI_API_KEY",
    base_url="https://api.portkey.ai/v1",
    default_headers=createHeaders(
        api_key="YOUR_PORTKEY_API_KEY",
        virtual_key="YOUR_VIRTUAL_KEY"
    )
)

# Step 5: Make instrumented calls
response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What are the key benefits of observability in AI?"}
    ]
)

print(response.choices[0].message.content)

​
Next Steps

Configure Gateway

Set up intelligent routing, fallbacks, and caching

Explore Virtual Keys

Secure your API keys with Portkey’s vault

View Analytics

Analyze costs, performance, and usage patterns

Set Up Alerts

Configure alerts for anomalies and performance issues

​
See Your Traces in Action

Once configured, navigate to the Portkey dashboard to see your OpenLIT instrumentation combined with gateway intelligence:

OpenTelemetry traces in Portkey