​
Functions Usage

Portkey supports the OpenAI signature to define functions as part of the API request. The tools parameter accepts functions which can be sent specifically for models that support function/tool calling. 
import Portkey from 'portkey-ai';

// Initialize the Portkey client
const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",  // Replace with your Portkey API key
    virtualKey: "VIRTUAL_KEY"   // Add your provider's virtual key
});

// Generate a chat completion with streaming
async function getChatCompletionFunctions(){
  const messages = [{"role": "user", "content": "What's the weather like in Boston today?"}];
  const tools = [
      {
        "type": "function",
        "function": {
          "name": "get_current_weather",
          "description": "Get the current weather in a given location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA",
              },
              "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["location"],
          },
        }
      }
  ];

  const response = await portkey.chat.completions.create({
    model: "gpt-3.5-turbo",
    messages: messages,
    tools: tools,
    tool_choice: "auto",
  });

  console.log(response)

}
await getChatCompletionFunctions();

​
API Reference

On completion, the request will get logged in the logs UI where the tools and functions can be viewed. Portkey will automatically format the JSON blocks in the input and output which makes a great debugging experience.

​
Managing Functions and Tools in Prompts

Portkey’s Prompt Library supports creating prompt templates with function/tool definitions, as well as letting you set the tool choice param. Portkey will also validate your tool definition on the fly, eliminating syntax errors.

​
Supported Providers and Models

The following providers are supported for function calling with more providers getting added soon. Please raise a request or a PR to add model or provider to the AI gateway.
ProviderModels
OpenAIgpt-4 series of modelsgpt-3.5-turbo series of models
Azure OpenAIgpt-4 series of modelsgpt-3.5-turbo series of models
Anyscalemistralai/Mistral-7B-Instruct-v0.1 mistralai/Mixtral-8x7B-Instruct-v0.1
Together AImistralai/Mixtral-8x7B-Instruct-v0.1 mistralai/Mistral-7B-Instruct-v0.1 togethercomputer/CodeLlama-34b-Instruct
Fireworks AIfirefunction-v1fw-function-call-34b-v0
Google Gemini / Vertex AIgemini-1.0-progemini-1.0-pro-001gemini-1.5-pro-latest

​
Cookbook

Here’s a detailed cookbook on function calling using Portkey.