Count Tokens

Estimate token usage for Google Vertex AI models before sending requests using Portkey’s CountTokens endpoint.

Portkey supports the Google Vertex AI CountTokens API, which returns the token count for a given input. This helps you estimate token usage before sending requests to models, allowing you to estimate costs, optimize prompts to fit within token limits, and plan for token usage in your applications.

The token count returned by this operation matches the token count that would be charged if the same input were sent to the model in a generation request.

Using Count Tokens with Portkey

Portkey supports Vertex AI’s CountTokens endpoint through the Anthropic-compatible format, allowing you to use the same API signature across Vertex AI, Bedrock, and Anthropic providers.

Python

import anthropic

client = anthropic.Anthropic(
    api_key="dummy", # we will use portkey's provider slug
    default_headers={"x-portkey-api-key": "YOUR_PORTKEY_API_KEY"},
    base_url="https://api.portkey.ai"
)

response = client.messages.count_tokens(
    model="@your-vertex-provider-slug/your-model-name",
    system="You are a scientist",
    messages=[{
        "role": "user",
        "content": "Hello, Claude"
    }],
)

print(response.json())

Use Cases

• Estimate costs before sending inference requests
• Optimize prompts to fit within token limits
• Plan for token usage in your applications

Related Documentation

• Google Vertex AI CountTokens API Reference
• Vertex AI Integration Guide

---

Portkey is now PRISMA AIRS AI Gateway. See it in action.

Contact Us