Building production-grade chatbots comes with its share of challenges. From managing conversation context and engineering prompts to ensuring consistent responses across different scenarios – the development process can quickly become complex. Add in the need for version control, testing different configurations, and maintaining production stability, and you’ve got quite a puzzle to solve.
This is where Portkey’s prompt templates come in. More than just a development tool, Portkey provides a complete ecosystem for building, testing, and deploying chatbots with confidence. You’ll be able to:
- Experiment with different prompt configurations while maintaining version control
- Test your chatbot’s responses in real-time with an interactive playground
- Portkey’s robust versioning system ensures that you can experiment freely with your prompts, allowing for easy rollback.
- Experiment with different models and configurations to find the best fit for your use case
In this guide, we’ll walk through the process of building a production-ready chatbot using Portkey’s prompt templates. Whether you’re creating a customer service bot, a knowledge assistant, or any other conversational AI application, you’ll learn how to leverage Portkey’s features to build a robust solution that scales.
Here’s the link to the collab notebook of the chatbot-

Setting Up Your Chatbot
Go to Portkey’s Prompts dashboard. Click on the Create button. You are now on Prompt Playground.
Step 1: Define Your System Prompt
Start by defining your system prompt. This sets the initial context and behavior for your chatbot. You can set this up in your Portkey’s Prompt Library using the JSON View
[
{
"content": "You're a helpful assistant.",
"role": "system"
},
{{chat_history}}
]
Step 2: Create a Variable to Store Conversation History
In the Portkey UI, set the variable type: Look for two icons next to the variable name: “T” and ”{..}”. Click the ”{…}” icon to switch to JSON mode.
Initialize the variable: This array will store the conversation history, allowing your chatbot to maintain context. We can just initialize the variable with []
.
As your chatbot interacts with users, it will append new messages to this array, building a comprehensive conversation history.
Step 3: Implementing the Chatbot
Use Portkey’s API to generate responses based on your prompt template. Here’s a Python example::
from portkey_ai import Portkey
client = Portkey(
api_key="YOUR_PORTKEY_API_KEY" # You can also set this as an environment variable
)
def generate_response(conversation_history):
prompt_completion = client.prompts.completions.create(
prompt_id="YOUR_PROMPT_ID", # Replace with your actual prompt ID
variables={
"variable": conversation_history
}
)
return prompt_completion.choices[0].message.content
# Example usage
conversation_history = [
{
"content": "Hello, how can I assist you today?",
"role": "assistant"
},
{
"content": "What's the weather like?",
"role": "user"
}
]
response = generate_response(conversation_history)
print(response)
Step 4: Append the Response
After generating a response, append it to your conversation history:
def append_response(conversation_history, response):
conversation_history.append({
"content": response,
"role": "assistant"
})
return conversation_history
# Continuing from the previous example
conversation_history = append_response(conversation_history, response)
Implement a loop to continuously take user input and generate responses:
# Continue the conversation
while True:
user_input = input("You: ")
if user_input.lower() == 'exit':
break
conversation_history.append({
"content": user_input,
"role": "user"
})
response = generate_response(conversation_history)
conversation_history = append_response(conversation_history, response)
print("Bot:", response)
print("Conversation ended.")
Complete Example
Here’s a complete example that puts all these steps together:
from portkey_ai import Portkey
client = Portkey(
api_key="YOUR_PORTKEY_API_KEY"
)
def generate_response(conversation_history):
prompt_completion = client.prompts.completions.create(
prompt_id="YOUR_PROMPT_ID",
variables={
"variable": conversation_history
}
)
return prompt_completion.choices[0].message.content
def append_response(conversation_history, response):
conversation_history.append({
"content": response,
"role": "assistant"
})
return conversation_history
# Initial conversation
conversation_history = [
{
"content": "Hello, how can I assist you today?",
"role": "assistant"
}
]
# Generate and append response
response = generate_response(conversation_history)
conversation_history = append_response(conversation_history, response)
print("Bot:", response)
# Continue the conversation
while True:
user_input = input("You: ")
if user_input.lower() == 'exit':
break
conversation_history.append({
"content": user_input,
"role": "user"
})
response = generate_response(conversation_history)
conversation_history = append_response(conversation_history, response)
print("Bot:", response)
print("Conversation ended.")
Conclusion
Voilà! You’ve successfully set up your chatbot using Portkey’s prompt templates. Portkey enables you to experiment with various LLM providers. It acts as a definitive source of truth for your team, and it versions each snapshot of model parameters, allowing for easy rollback. Here’s a snapshot of the Prompt Management UI. To learn more about Prompt Management click here.