> ## Documentation Index > Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # October **Portkey in October 🎃 🪔** October was packed with treats (no tricks!) for Portkey. As we celebrate Halloween and Diwali, we're lighting up your AI infrastructure with some exciting updates. Let's dive in! ### Executive Summary | | | | :- | :- | | Guardrails GA Release | Production-ready guardrails to enforce LLM behavior in real-time, with support for PII detection, moderation, and more — are now generally available. [(*Docs*)](https://docs.portkey.ai/docs/product/guardrails.md) | | Enterprise Momentum | Refreshed Portkey's [enterprise offering](https://portkey.ai/docs/product/enterprise-offering) with enhanced security features, and support for [AWS Assume Role Auth](https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys/bedrock-amazon-assumed-role.md). Also [onboarded](https://x.com/PortkeyAI/status/1841292805454643393) one of the world's largest tech companies to Portkey. | | Provider Ecosystem | Added 7 new providers including [vLLM](https://docs.portkey.ai/docs/integrations/llms/vllm.md), [Triton](https://docs.portkey.ai/docs/integrations/llms/triton.md), [Lambda Labs](https://docs.portkey.ai/docs/integrations/llms/lambda.md), and more. | | Image Generation | Added support for Stable Diffusion v3 and Google Imagen. | | Integrations | Added [MindsDB](https://docs.portkey.ai/docs/integrations/libraries/mindsdb.md), [ToolJet](https://docs.portkey.ai/docs/integrations/libraries/tooljet.md), [LibreChat](https://docs.portkey.ai/docs/integrations/libraries/librechat.md), and [OpenWebUI](https://docs.portkey.ai/docs/integrations/libraries/openwebui.md). | | Prompt Caching | Anthropic's prompt caching feature is now available directly in prompt playground. [(*Docs*)](https://docs.portkey.ai/docs/integrations/llms/anthropic/prompt-caching.md#prompt-templates-support) | | .NET | You can now integrate Portkey with [your .NET app](https://docs.portkey.ai/docs/api-reference/inference-api/sdks/c-sharp.md) | | Agent Tooling Leadership | Portkey was recognized for providing [11 critical capabilities](https://x.com/PortkeyAI/status/1851596076488479001) for production-grade AI agents, leading the Agent Ops tooling benchmark. | | Featured Coverage | Our DevOps for AI vision featured in the [People+AI Newsletter](https://sreeramsridhar.substack.com/p/building-the-devops-for-ai) and [Pulse2 publication](https://pulse2.com/portkey-profile-rohit-agarwal-interview/). | ### Features * **AWS Assume Role Support**: Enhanced Bedrock authentication for enterprise security [(*Docs*)](https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys/bedrock-amazon-assumed-role.md) * **User Management API**: New API to resend user invites [(*Docs*)](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/user-invites/resend-a-user-invite.md). Also updated the API specs for Prompt Completions [API](https://docs.portkey.ai/docs/api-reference/inference-api/prompts/prompt-completion.md), Prompt Render [API](https://docs.portkey.ai/docs/api-reference/inference-api/prompts/render.md), and Insert Log [API](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/insert-a-log.md) * **New OpenAI Param**: OpenAI's `max_completion_tokens` is now supported across all providers (This means you can send either the `max_completion_tokens` or `max_tokens` param in your requests and Portkey will handle both correctly) * **Caching**: Improved cost calculations for OpenAI & Azure OpenAI cached responses, and Anthropic's prompt caching feature is now available directly in prompt playground * **Gemini Updates**: Added support for Gemini JSON mode and Controlled Generations along with Pydantic support * **Bedrock**: Integrated Converse API for `/chat/completions`. [(*Docs*)](https://docs.portkey.ai/docs/integrations/llms/aws-bedrock.md#bedrock-converse-api) * **Enterprise**: Refreshed Portkey's [enterprise offering](https://portkey.ai/docs/product/enterprise-offering) with enhanced security features. * **C# (.NET) Support**: You can now integrate Portkey in your .NET apps using the OpenAI official library. [(*Docs*)](https://docs.portkey.ai/docs/api-reference/inference-api/sdks/c-sharp.md) *** ### Models & Providers **7 New Providers**: Expanding your model hosting and deployment options. > [card] [Lemonfox](https://docs.portkey.ai/docs/integrations/llms/lemon-fox.md) > [card] [Lambda Labs](https://docs.portkey.ai/docs/integrations/llms/lambda.md) > [card] [Dashscope](https://docs.portkey.ai/docs/integrations/llms/dashscope.md) > [card] [Upstage](https://docs.portkey.ai/docs/integrations/llms/upstage.md) > [card] [Nvidia Triton](https://docs.portkey.ai/docs/integrations/llms/triton.md) > [card] Github > [card] [vLLM](https://docs.portkey.ai/docs/integrations/llms/vllm.md) **2 Image Generation Models**: Strengthening our multimodal capabilities with next-gen image models. > [card] [Stable Diffusion v3](https://docs.portkey.ai/docs/api-reference/inference-api/images/create-image.md) > > Now available across [Stability AI](https://docs.portkey.ai/docs/integrations/llms/stability-ai.md), [Fireworks](https://docs.portkey.ai/docs/integrations/llms/fireworks.md), [AWS Bedrock](https://docs.portkey.ai/docs/integrations/llms/aws-bedrock.md), and [Segmind](https://docs.portkey.ai/docs/integrations/llms/segmind.md) > [card] [Imagen on Google Vertex](https://docs.portkey.ai/docs/integrations/llms/vertex-ai.md#image-generation-models) > > Official support for Google's Imagen model through Vertex AI **2 New LLMs**: > [card] Llama 3.2 > > Now integrated with [Fireworks](https://docs.portkey.ai/docs/integrations/llms/fireworks.md), [AWS Bedrock](https://docs.portkey.ai/docs/integrations/llms/aws-bedrock.md), [Groq](https://docs.portkey.ai/docs/integrations/llms/groq.md), and [Together AI](https://docs.portkey.ai/docs/integrations/llms/together-ai.md) > [card] [Vertex Embeddings](https://docs.portkey.ai/docs/integrations/llms/vertex-ai.md#text-embedding-models) > > Added support for both `English` and `Multilingual` embedding models from Google Vertex AI *** ### Integrations **Model Management & Monitoring**: Enhance your AI infrastructure with enterprise-grade observability. > [card] [LibreChat](https://github.com/timmanik/librechat-for-portkey) > > You can now track costs per user on your LibreChat instance by forwarding unique user IDs from LibreChat to Portkey - thanks to [Tim](https://www.linkedin.com/in/tim-manik/)'s contribution! > [card] [OpenWebUI](https://docs.portkey.ai/docs/integrations/libraries/openwebui.md) > > Portkey is the only plugin you’ll need for model management, cost tracking, observability, metadata logging, and more for your Open WebUI instance. **Data & App Integration**: Connect your existing tools and databases to LLMs. > [card] [MindsDB](https://docs.portkey.ai/docs/integrations/libraries/mindsdb.md) > > Connect your databases, vector stores, and apps to 250+ LLMs with enterprise-grade monitoring and reliability built-in. > [card] [ToolJet](https://docs.portkey.ai/docs/integrations/libraries/tooljet.md) > > Add AI-powered capabilities such as chat completions and automations into your ToolJet apps easily. *** ### Guardrails The guardrails feature is now **generally available** - it brings production-ready content filtering and response validation to your LLM apps. **Updated Content Safety Guardrails:** > [card] [PII Detection](https://docs.portkey.ai/docs/product/guardrails/list-of-guardrail-checks.md#pro-llm-guardrails) > > Detect sensitive personal information in user messages > [card] [Content Moderation](https://docs.portkey.ai/docs/product/guardrails/list-of-guardrail-checks.md#pro-llm-guardrails) > > Automated content filtering and moderation **Updated Guardrails to Ensure Response Quality:** > [card] [Language Detection](https://docs.portkey.ai/docs/product/guardrails/list-of-guardrail-checks.md#pro-llm-guardrails) > > Automatically detect and validate response languages > [card] [Gibberish Detection](https://docs.portkey.ai/docs/product/guardrails/list-of-guardrail-checks.md#pro-llm-guardrails) > > Filter out nonsensical or low-quality responses **And More!** > [card] [Custom Webhooks](https://docs.portkey.ai/docs/integrations/guardrails/bring-your-own-guardrails.md) > > Metadata sent to the Portkey API will now be automatically forwarded to your custom webhook endpoint. > [card] [Lowercase Detection](https://docs.portkey.ai/docs/product/guardrails/list-of-guardrail-checks.md#portkeys-default-guardrail-checks) > > Check if the given string is lowercase or not. *** ### Resources **Quick Implementation Guides:** * [Guide to Prompt Caching](https://x.com/PortkeyAI/status/1843209780627997089): Learn how to optimize your LLM costs * [Production Apps with Vercel](https://x.com/PortkeyAI/status/1844675148609204615): Learn how to build prod-ready apps using Vercel AI SDK * [OpenAI Swarm Cheat Sheet](https://x.com/jumbld/status/1846909380354064526): Learn how OpenAI's new Swarm framework really works **Technical Deep Dives for Production Deployments:** > [card] [OpenAI Swarm + Portkey](https://www.youtube.com/watch?v=9qLkAEJol9A) > > Build and secure multi-agent AI systems using OpenAI Swarm and Portkey > [card] [RAG with Observability](https://github.com/Portkey-AI/gateway/blob/main/cookbook/use-cases/Contextual%20Embeddings%20Guide%20Anthropic%2C%20Cohere%2C%20Voyage.ipynb) > > Enhanced version of Anthropic's RAG Cookbook with unified API and monitoring **Latest insights on AI infrastructure and tooling**: * [Automated Prompt Engineering](https://portkey.ai/blog/what-is-automated-prompt-engineering/): Scale your prompt engineering workflow * [OpenAI's Prompt Caching](https://portkey.ai/blog/openais-prompt-caching-a-deep-dive/): Optimize costs and performance * [Complete Prompt Engineering Guide](https://portkey.ai/blog/the-complete-guide-to-prompt-engineering/): Best practices and patterns * [OpenTelemetry Guide](https://portkey.ai/blog/the-developers-guide-to-opentelemetry-a-real-time-journey-into-observability/): Real-time observability for AI systems Check out more technical content on our [Blog →](https://portkey.ai/blog). *** ### Fixes **Model & Provider Enhancements** Fixed core provider issues and improved reliability: * Enhanced streaming transformer for Perplexity * Fixed response transformation for Ollama * ⭐️ Added missing logprob mapping for Azure OpenAI (Thanks [Avishkar](https://www.linkedin.com/in/avishkar-gupta/)!) * Fixed token counting for Vertex embeddings (now using tokens instead of characters) * Added support for Bedrock cross-region model IDs with pricing * Fixed media file handling for Vertex AI & Gemini **Default Models** We've also reset the default model options (These are the models called by default when you do not pass a specific model slug in your request) for the following providers: * **Fireworks**: `accounts/fireworks/models/llama-v3p1-405b-instruct` * **Together AI**: `meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo` * **Gemini**: `gemini-1.5-pro` **Dev Ex Improvements** * Added support for `anthropic-beta` and `anthropic-version` headers in the Portkey API * In Portkey SDK, the Portkey API key is now optional when you're calling the self-hosted Gateway * Enhanced support for custom provider headers in SDK *** ### Community Updates **Upcoming Events** > [card] [LLMs in Prod Dinner Singapore](https://lu.ma/llms-in-prod-dinner) > > Join top tech leaders for a closed-door dinner around OpenAI Dev Day. [Register here](https://lu.ma/llms-in-prod-dinner) **Service Reliability** When OpenAI users were hitting usage limits earlier this month, [Portkey users remained unaffected](https://x.com/PortkeyAI/status/1841172271076954588) thanks to our built-in reliability features. **Industry Recognition** * Our DevOps for AI vision was featured in the [People+AI Newsletter](https://sreeramsridhar.substack.com/p/building-the-devops-for-ai) and [Pulse2 publication](https://pulse2.com/portkey-profile-rohit-agarwal-interview/). * Portkey was recognized for providing [11 critical capabilities](https://x.com/PortkeyAI/status/1851596076488479001) for production-grade AI agents. **Recent Events** We co-sponsored the [TED AI Hackathon](https://x.com/PortkeyAI/status/1847733473529999377)! Thanks to everyone who participated and built amazing projects. *** ### Support > [card] [Bug Report](https://github.com/Portkey-AI/gateway/issues) > > Found a bug or have a feature request? Open an issue on our GitHub repository. *** > [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact) > > Contact Us ## Related topics - [October](https://docs.portkey.ai/docs/changelog/2025/october.md) - [Enterprise Gateway](https://docs.portkey.ai/docs/changelog/enterprise.md) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.