Tracing

This feature is available for all plans:-

Developer: 10k Logs / Month with 3 day Log Retention
Production: 100k Logs / Month + $9 for additional 100k with 30 Days Log Retention
Enterprise: Unlimited

This is perfect for agentic workflows, chatbots, or multi-step LLM calls, by helping you understand and optimize your AI application’s performance.

How Tracing Works

Portkey implements OpenTelemetry-compliant tracing. When you include a trace ID with your requests, all related LLM calls are grouped together in the Traces View, appearing as “spans” within that trace.

“Span” is another word for subgrouping of LLM calls. Based on how you instrument, it can refer to another group within your trace or to a single LLM call.

Trace Tree Structure

Portkey uses a tree data structure for tracing, similar to OTel. Each node in the tree is a span with a unique spanId and optional spanName. Child spans link to a parent via the parentSpanId. Parentless spans become root nodes.

traceId
├─ parentSpanId
│  ├─ spanId
│  ├─ spanName

Key - Node	Key - Python	Expected Value	Required?
traceId	trace_id	Unique string	YES
spanId	span_id	Unique string	NO
spanName	span_name	string	NO
parentSpanId	parent_span_id	Unique string	NO

W3C Trace Context Support

Portkey supports the W3C Trace Context standard headers, making it easy to integrate with existing OpenTelemetry-instrumented applications. If you’re already using OpenTelemetry in your stack, you can pass the standard traceparent and baggage headers instead of Portkey-specific headers.

traceparent Header

The traceparent header follows the W3C format: version-trace_id-parent_id-trace_flags Example: 00-bad90143930ea019df8f681254fb2393-42df8ac2dde4ac52-00 When you send a request with the traceparent header:

The trace_id (32 hex characters) is extracted and used as x-portkey-trace-id
The parent_id (16 hex characters) is extracted and used as x-portkey-parent-span-id (the caller’s span)
A new span_id is automatically generated for the gateway’s span
The span_name is automatically set to the HTTP method and path (e.g., POST /v1/chat/completions)

If you provide both traceparent and Portkey-specific headers (x-portkey-trace-id, x-portkey-span-id), the Portkey-specific headers take precedence.

baggage Header

The baggage header allows you to pass key-value metadata following the W3C Baggage specification. Example: userId=alice,environment=production,requestType=chat When you send a request with the baggage header:

Key-value pairs are parsed and merged into the request metadata
Existing metadata (from x-portkey-metadata header) takes precedence over baggage values
Values are URL-decoded automatically

Example: Using W3C Headers

cURL
Python
Node.js

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: openai" \
  -H "traceparent: 00-bad90143930ea019df8f681254fb2393-42df8ac2dde4ac52-00" \
  -H "baggage: userId=alice,environment=production" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user","content": "Hello!"}]
  }'

import requests

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {OPENAI_API_KEY}",
    "x-portkey-api-key": PORTKEY_API_KEY,
    "x-portkey-provider": "openai",
    "traceparent": "00-bad90143930ea019df8f681254fb2393-42df8ac2dde4ac52-00",
    "baggage": "userId=alice,environment=production"
}

response = requests.post(
    "https://api.portkey.ai/v1/chat/completions",
    headers=headers,
    json={
        "model": "gpt-4o",
        "messages": [{"role": "user", "content": "Hello!"}]
    }
)

const response = await fetch('https://api.portkey.ai/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${OPENAI_API_KEY}`,
    'x-portkey-api-key': PORTKEY_API_KEY,
    'x-portkey-provider': 'openai',
    'traceparent': '00-bad90143930ea019df8f681254fb2393-42df8ac2dde4ac52-00',
    'baggage': 'userId=alice,environment=production'
  },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello!' }]
  })
});

W3C Trace Context support is particularly useful when integrating Portkey into applications that already use OpenTelemetry for distributed tracing, as it allows seamless correlation between your existing traces and LLM calls.

Enabling Tracing

You can enable tracing by passing the trace tree values while making your request (or while instantiating your client). Based on these values, Portkey will instrument your requests, and will show the exact trace with its spans on the “Traces” view in Logs page.

NodeJS
Python
OpenAI NodeJS
OpenAI Python
cURL

Add tracing details to a single request (recommended)

const requestOptions = {
    traceId: "1729",
    spanId: "11",
    spanName: "LLM Call"
}

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-4o',
}, requestOptions);

Or, add trace details while instantiating your client

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider:"@PROVIDER",
    traceId: "1729",
    spanId: "11",
    spanName: "LLM Call"
})

completion = portkey.with_options(
    trace_id="1729",
    span_id="11",
    span_name="LLM Call"
).chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'gpt-3.5-turbo'
)

Pass Trace details while instantiating your client

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@PROVIDER",
    trace_id="1729",
    span_id="11",
    span_name="LLM Call"
)

import { createHeaders } from 'portkey-ai'

const requestOptions = {
    traceId: "1729",
    spanId: "11",
    spanName: "LLM Call"
}

const chatCompletion = await openai.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-3.5-turbo',
}, requestOptions);

from portkey_ai import createHeaders

req_headers = createHeaders(
    trace_id="1729",
    span_id="11",
    span_name="LLM Call
)

chat_complete = client.with_options(headers=req_headers).chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Say this is a test"}],
)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-trace-id: 1729"\
  -H "x-portkey-span-id: 11"\
  -H "x-portkey-span-name: LLM_CALL"\
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user","content": "Hello!"}]
  }'

If you are only passing trace ID and not the span details, you can set the trace ID while making your request or while instantiating your client.

NodeJS
Python
OpenAI NodeJS
OpenAI Python
cURL

const requestOptions = {traceID: "YOUR_TRACE_ID"}

const chatCompletion = await portkey.chat.completions.create({
  messages: [{ role: 'user', content: 'Say this is a test' }],
  model: 'gpt-4o',
}, requestOptions);

console.log(chatCompletion.choices);

Pass Trace ID while instantiating your client

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider:"@PROVIDER",
    traceID: "TRACE_ID"
})

completion = portkey.with_options(
    trace_id = "TRACE_ID"
).chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'gpt-3.5-turbo'
)

Pass Trace ID while instantiating your client

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@PROVIDER",
    trace_id="TRACE_ID"
)

import { createHeaders } from 'portkey-ai'

const reqHeaders = {headers: createHeaders({"traceID": "TRACE_ID"})}

const chatCompletion = await openai.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-3.5-turbo',
}, reqHeaders);

from portkey_ai import createHeaders

req_headers = createHeaders(trace_id="TRACE_ID")

chat_complete = client.with_options(headers=req_headers).chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Say this is a test"}],
)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-trace-id: TRACE_ID" \
  -d '{
    "model": "gpt-4-turbo",
    "messages": [{
        "role": "system",
        "content": "You are a helpful assistant."
      },{
        "role": "user",
        "content": "Hello!"
      }]
  }'

See Tracing in Action

Tracing in Langchain

Portkey has a dedicated handler that can instrument your Langchain chains and agents to trace them.

Trace Langchain Requests (Python)

First, install Portkey SDK, and Langchain’s packages

$ pip install langchain_OpenAI portkey-ai langchain_community

Import the packages

from langchain_openai import ChatOpenAI
from langchain.chains import LLMChain
from portkey_ai.langchain import LangchainCallbackHandler
from portkey_ai import createHeaders

Instantiate Portkey’s Langchain Callback Handler

portkey_handler = LangchainCallbackHandler(
      api_key="YOUR_PORTKEY_API_KEY",
      metadata={
            "user_name": "User_Name",
            "traceId": "Langchain_sample_callback_handler"
      }
)

Add the callback to the ChatOpenAI instance

llm = ChatOpenAI(
    api_key="OPENAI_API_KEY",
    callbacks=[portkey_handler],
)

Also add the callback when you define or run your LLM chain

chain = LLMChain(
    llm=llm,
    prompt=prompt,
    callbacks=[portkey_handler]
)

handler_config = {'callbacks' : [portkey_handler]}

chain.invoke({"input": "what is langchain?"}, config=handler_config)

Tracing Llamaindex Requests

Portkey has a dedicated handler to instrument your Llamaindex requests on Portkey.

Trace Llamaindex Requests

First, install Portkey SDK, and LlamaIndex packages

$ pip install openai portkey-ai llama-index

Import the packages

from llama_index.llms.openai import OpenAI
from portkey_ai.llamaindex import LlamaIndexCallbackHandler

Instantiate Portkey’s LlamaIndex Callback Handler

portkey_handler = LlamaIndexCallbackHandler(
      api_key="PORTKEY_API_KEY",
      metadata={
            "user_name": "User_Name",
            "traceId": "Llamaindex_sample_callback_handler"
      }
)

Add it to OpenAI llm class

llm = OpenAI(
    model="gpt-4o",
    api_key="OPENAI_API_KEY",
    callback_manager=[portkey_handler],
)

In Llama Index, you can also set the callback at a global level

from llama_index.core import Settings
from llama_index.core.callbacks import CallbackManager

Settings.callback_manager = CallbackManager([portkey_handler])
Settings.llm = llm

Inserting Logs

If you are using the Insert Log API to add logs to Portkey, your traceId, spanId etc. will become part of the metadata object in your log, and Portkey will instrument your requests to take those values into account. The logger endpoint supports inserting a single log as well as log array, and helps you build traces of any depth or complexity. For more, check here:

Insert a Log

Tracing for Gateway Features

Tracing also works very well to capture the Gateway behavior on retries, fallbacks, and other routing mechanisms on Portkey Gateway. Portkey automatically groups all the requests that were part of a single fallback or retry config and shows the failed and succeeded requests chronologically as “spans” inside a “trace”. This is especially useful when you want to understand the total latency and behavior of your app when retry or fallbacks were triggered. For more, check out the Fallback & Automatic Retries docs.

Why Use Tracing?

Cost Insights: View aggregate LLM costs at the trace level.
Debugging: Easily browse all requests in a single trace and identify failures.
Performance Analysis: Understand your entire request lifecycle and total trace duration.
User Feedback Integration: Link user feedback to specific traces for targeted improvements.

Capturing User Feedback

Trace IDs can also be used to link user feedback to specific generations. This can be used in a system where users provide feedback, like a thumbs up or thumbs down, or something more complex via our feedback APIs. This feedback can be linked to traces which can span over a single generation or multiple ones. Read more here:

Introduction

Product

Self-Hosting

Support

How Tracing Works

Trace Tree Structure

W3C Trace Context Support

traceparent Header

baggage Header

Example: Using W3C Headers

Enabling Tracing

Or, add trace details while instantiating your client

Pass Trace details while instantiating your client

Pass Trace ID while instantiating your client

Pass Trace ID while instantiating your client

See Tracing in Action

Tracing in Langchain

Tracing Llamaindex Requests

Inserting Logs

Insert a Log

Tracing for Gateway Features

Why Use Tracing?

Capturing User Feedback

Feedback

Introduction

Product

Self-Hosting

Support

​How Tracing Works

​Trace Tree Structure

​W3C Trace Context Support

​traceparent Header

​baggage Header

​Example: Using W3C Headers

​Enabling Tracing

​Or, add trace details while instantiating your client

​Pass Trace details while instantiating your client

​Pass Trace ID while instantiating your client

​Pass Trace ID while instantiating your client

​See Tracing in Action

​Tracing in Langchain

​Tracing Llamaindex Requests

​Inserting Logs

Insert a Log

​Tracing for Gateway Features

​Why Use Tracing?

​Capturing User Feedback

Feedback

How Tracing Works

Trace Tree Structure

W3C Trace Context Support

traceparent Header

baggage Header

Example: Using W3C Headers

Enabling Tracing

Or, add trace details while instantiating your client

Pass Trace details while instantiating your client

Pass Trace ID while instantiating your client

Pass Trace ID while instantiating your client

See Tracing in Action

Tracing in Langchain

Tracing Llamaindex Requests

Inserting Logs

Tracing for Gateway Features

Why Use Tracing?

Capturing User Feedback