This feature is available on all plans.

  • Developer: Access to BASIC Guardrails
  • Production: Access to BASIC, PARTNER, PRO Guardrails.
  • Enterprise: Access to all Guardrails plus custom Guardrails.

LLMs are brittle - not just in API uptimes or their inexplicable 400/500 errors, but also in their core behavior. You can get a response with a 200 status code that completely errors out for your app’s pipeline due to mismatched output. With Portkey’s Guardrails, we now help you enforce LLM behavior in real-time with our Guardrails on the Gateway pattern.

Use Portkey’s Guardrails to verify your LLM inputs AND outputs, adhering to your specifed checks. Since Guardrails are built on top of our Gateway, you can orchestrate your request - with actions ranging from denying the request, logging the guardrail result, creating an evals dataset, falling back to another LLM or prompt, retrying the request, and more.

​
Examples of Guardrails Portkey offers:

  • Regex match - Check if the request or response text matches a regex pattern
  • JSON Schema - Check if the response JSON matches a JSON schema
  • Contains Code - Checks if the content contains code of format SQL, Python, TypeScript, etc.
  • Custom guardrail - If you are running a custom guardrail currently, you can also integrate it with Portkey
  • …and many more.

Portkey currently offers 20+ deterministic guardrails like the ones described above as well as LLM-based guardrails like Detect Gibberish, Scan for prompt injection, and more. These guardrails serve as protective barriers that help mitigate risks associated with Gen AI, ensuring its responsible and ethical deployment within organizations.

List of supported Guardrail checks here

Portkey also integrates with your favourite Guardrail platforms like Aporia, SydeLabs, Pillar Security and more. Just add their API keys to Portkey and you can enable their guardrails policies on your Portkey calls! More details on Guardrail Partners here.

​
Using Guardrails

Putting Portkey Guardrails in production is just a 4-step process:

  1. Create Guardrail Checks
  2. Create Guardrail Actions
  3. Enable Guardrail through Configs
  4. Attach the Config to a Request

This flowchart shows how Portkey processes a Guardrails request:

Let’s see in detail below:

Portkey only evaluates the last message in the request body when running guardrails checks.

​
1. Create a New Guardrail & Add Checks

On the Guardrails page, click on Create and add your preferred Guardrail checks from the right sidebar.

On Portkey, you can configure Guardrails to be run on either the INPUT (i.e. PROMPT) or the OUTPUT. Hence, for the Guardrail you create, make sure your Guardrail is only validating ONLY ONE OF the Input or the Output.

Each Guardrail Check has a custom input field based on its usecase — just add the relevant details to the form and save your check.

  • You can add as many checks as you want to a single Guardrail.
  • A check ONLY returns a boolean (Yes/No) verdict.

List of Guardrail checks

​
2. Add Guardrail Actions

Define a basic orchestration logic for your Guardrail here.

Guardrail is created to validate ONLY ONE OF the Input or the Output. The Actions set here will also apply only to either the request or the response.

​
There are 6 Types of Guardrail Actions

ActionStateDescriptionImpact
AsyncTRUE This is the default stateRun the Guardrail checks asynchronously along with the LLM request.Will add no latency to your requestUseful when you only want to log guardrail checks without affecting the request
AsyncFALSEOn Request Run the Guardrail check BEFORE sending the request to the LLM On Response Run the Guardrail check BEFORE sending the response to the userWill add latency to the requestUseful when your Guardrail critical and you want more orchestration over your request based on the Guardrail result
DenyTRUEOn Request & Response If any of the Guardrail checks FAIL, the request will be killed with a 446 status code. If all of the Guardrail checks SUCCEED, the request/response will be sent further with a 200 status code.This is useful when your Guardrails are critical and upon them failing, you can not run the requestWe would advice running this action on a subset of your requests to first see the impact
DenyFALSE This is the default stateOn Request & Response If any of the Guardrail checks FAIL, the request will STILL be sent, but with a 246 status code. If all of the Guardrail checks SUCCEED, the request/response will be sent further with a 200 status code.This is useful when you want to log the Guardrail result but do not want it to affect your result
On SuccessSend FeedbackIf all of the Guardrail checks PASS, append your custom defined feedback to the requestWe recommend setting up this actionThis will help you build an “Evals dataset” of Guardrail results on your requests over time
On FailureSend FeedbackIf any of the Guardrail checks FAIL, append your custom feedback to the requestWe recommend setting up this actionThis will help you build an “Evals dataset” of Guardrail results on your requests over time

Set the relevant actions you want with your checks, name your Guardrail and save it! When you save the Guardrail, you will get an associated $Guardrail_ID that you can then add to your request.

​
3. “Enable” the Guardrails through Configs

This is where Portkey’s magic comes into play. The Guardrail you created above is yet not an Active guardrail because it is not attached to any request.

Configs is one of Portkey’s most powerful features and is used to define all kinds of request orchestration - everything from caching, retries, fallbacks, timeouts, to load balancing.

Now, you can use Configs to add Guardrail checks & actions to your request.

Portkey now offers a more intuitive way to add guardrails to your configurations:

Config KeyValueDescription
input_guardrails["guardrails-id-xxx", "guardrails-id-yyy"]Apply these guardrails to the INPUT before sending to LLM
output_guardrails["guardrails-id-xxx", "guardrails-id-yyy"]Apply these guardrails to the OUTPUT from the LLM
{
  "retry": {
    "attempts": 3
  },
  "cache": {
    "mode": "simple"
  },
  "provider":"@openai-xxx",
  "input_guardrails": ["guardrails-id-xxx", "guardrails-id-yyy"],
  "output_guardrails": ["guardrails-id-xxx", "guardrails-id-yyy"]
}

​
Option 2: Hook-Based Configuration (For Creating Raw Guardrails)

You can also continue to use the original hook-based approach:

TypeConfig KeyValueDescription
Before Request Hookbefore_request_hooks[{"id":"$guardrail_id"}]These hooks run on the INPUT before sending to the LLM
After Request Hookafter_request_hooks[{"id":"$guardrail_id"}]These hooks run on the OUTPUT after receiving from the LLM
{
  "retry": {
    "attempts": 3
  },
  "cache": {
    "mode": "simple"
  },
  "provider":"@openai-xxx",
  "before_request_hooks": [{
    "id": "input-guardrail-id-xx"
  }],
  "after_request_hooks": [{
    "id": "output-guardrail-id-xx"
  }]
}

Both configuration approaches work identically - choose whichever is more intuitive for your team. The simplified input_guardrails and output_guardrails fields are recommended for better readability.

​
Guardrail Behaviour on the Gateway

For asynchronous guardrails (async= TRUE), Portkey returns the standard, default status codes from the LLM providers — this is because the Guardrails verdict is not affecting how you orchestrate your requests. Portkey will only log the Guardrail result for you.

But for synchronous requests (async= FALSE), Portkey can orchestrate your requests based on the Guardrail verdict. The behaviour is dependent on the following:

  • Guardrail Check Verdict (PASS or FAIL) AND
  • Guardrail Action — DENY Setting (TRUE or FALSE)

Portkey sends different request status codes corresponding to your set Guardrail behaviour.

For requests where async= FALSE:

Guardrail VerdictDENY SettingReturned Status CodeDescription
PASSFALSE200Guardrails have passed, request will be processed regardless
PASSTRUE200Guardrails have passed, request will be processed regardless
FAILFALSE246Guardrails have failed, but the request should still **be processed.**Portkey introduces a new Status code to indicate this state.
FAILTRUE446Guardrails have failed, and the request should not **be processed.**Portkey introduces a new Status code to indicate this state.

​
Example Config Using the New 246 & 446 Status Codes

{
  "strategy": {
    "mode": "fallback",
    "on_status_codes": [246, 446]
  },
  "targets": [
    {"provider":"@openai-key-xxx"},
    {"provider":"@anthropic-key-xxx"}
  ],
  "input_guardrails": ["guardrails-id-xxx"]
}

Create these Configs in Portkey UI, save them, and get an associated Config ID to attach to your requests. More here.

​
4. Final Step - Attach Config to Request

Now, while instantiating your Portkey client or while sending headers, just pass the Config ID.

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    config: "pc-***" // Supports a string config id or a config object
});

For more, refer to the Config documentation.

​
Viewing Guardrail Results in Portkey Logs

Portkey Logs will show you detailed information about Guardrail results for each request.

​
On the Feedback & Guardrails tab on the log drawer, you can see

​
Guardrail Details

  • Overview: How many checks passed and how many failed
  • Verdict: Guardrail verdict for each of the checks in your Guardrail
  • Latency: Round trip time for each check in your Guardrail

​
Feedback Details

Portkey will also show the feedback object logged for each request

  • Value:** The numerical feedback value you passed
  • Weight: The numerical feedback weight
  • Metadata Key & Value:** Any custom metadata sent with the feedback
  • successfulChecks: Which checks associated with this request passed
  • failedChecks: Which checks associated with this request failed
  • erroredChecks: If there were any checks that errored out along the way

​
Understanding Guardrail Response Structure

When Guardrails are enabled and configured to run synchronously (async=false), Portkey adds a hook_results object to your API responses. This object provides detailed information about the guardrail checks that were performed and their outcomes.

​
Hook Results Structure

The hook_results object contains two main sections:

"hook_results": {
  "before_request_hooks": [...],  // Guardrails applied to the input
  "after_request_hooks": [...]    // Guardrails applied to the output
}

Each section contains an array of guardrail execution results, structured as follows:

​
Example Hook Results

Here’s a simplified example of what the hook_results might look like in a response:

"hook_results": {
    "before_request_hooks": [
        {
            "verdict": true,
            "id": "sentence_and_word_check_guardrail",
            "transformed": false,
            "checks": [
                {
                    "data": {
                        "sentenceCount": 1,
                        "minCount": 1,
                        "maxCount": 99999,
                        "not": false,
                        "verdict": true,
                        "explanation": "The sentence count (1) is within the specified range of 1 to 99999.",
                        "textExcerpt": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number,..."
                    },
                    "verdict": true,
                    "id": "default.sentenceCount",
                    "execution_time": 0,
                    "transformed": false,
                    "created_at": "2025-05-20T08:00:59.492Z",
                    "log": null
                },
                {
                    "data": {
                        "wordCount": 24,
                        "minWords": 1,
                        "maxWords": 99999,
                        "not": false,
                        "verdict": true,
                        "explanation": "The text contains 24 words, which is within the specified range of 1-99999 words.",
                        "textExcerpt": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number,..."
                    },
                    "verdict": true,
                    "id": "default.wordCount",
                    "execution_time": 0,
                    "transformed": false,
                    "created_at": "2025-05-20T08:00:59.492Z",
                    "log": null
                }
            ],
            "feedback": {
                "value": 5,
                "weight": 1,
                "metadata": {
                    "successfulChecks": "default.sentenceCount, default.wordCount",
                    "failedChecks": "",
                    "erroredChecks": ""
                }
            },
            "execution_time": 0,
            "async": false,
            "type": "guardrail",
            "created_at": "2025-05-20T08:00:59.492Z",
            "deny": false
        },
        {
            "verdict": true,
            "id": "character_check_guardrai",
            "transformed": false,
            "checks": [
                {
                    "data": {
                        "characterCount": 130,
                        "minCharacters": 1,
                        "maxCharacters": 9999999,
                        "not": false,
                        "verdict": true,
                        "explanation": "The text contains 130 characters, which is within the specified range of 1-9999999 characters.",
                        "textExcerpt": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number,..."
                    },
                    "verdict": true,
                    "id": "default.characterCount",
                    "execution_time": 0,
                    "transformed": false,
                    "created_at": "2025-05-20T08:00:59.492Z",
                    "log": null
                }
            ],
            "feedback": {
                "value": 5,
                "weight": 1,
                "metadata": {
                    "successfulChecks": "default.characterCount",
                    "failedChecks": "",
                    "erroredChecks": ""
                }
            },
            "execution_time": 0,
            "async": false,
            "type": "guardrail",
            "created_at": "2025-05-20T08:00:59.492Z",
            "deny": false
        }
    ],
    "after_request_hooks": [
        {
            "verdict": true,
            "id": "sentence_and_word_check_guardrail",
            "transformed": false,
            "checks": [
                {
                    "data": {
                        "sentenceCount": 2,
                        "minCount": 1,
                        "maxCount": 99999,
                        "not": false,
                        "verdict": true,
                        "explanation": "The sentence count (2) is within the specified range of 1 to 99999.",
                        "textExcerpt": "I'm unable to provide real-time flight information, such as specific flight times, numbers, or bagga..."
                    },
                    "verdict": true,
                    "id": "default.sentenceCount",
                    "execution_time": 0,
                    "transformed": false,
                    "created_at": "2025-05-20T08:01:02.229Z",
                    "log": null
                },
                {
                    "data": {
                        "wordCount": 46,
                        "minWords": 1,
                        "maxWords": 99999,
                        "not": false,
                        "verdict": true,
                        "explanation": "The text contains 46 words, which is within the specified range of 1-99999 words.",
                        "textExcerpt": "I'm unable to provide real-time flight information, such as specific flight times, numbers, or bagga..."
                    },
                    "verdict": true,
                    "id": "default.wordCount",
                    "execution_time": 0,
                    "transformed": false,
                    "created_at": "2025-05-20T08:01:02.229Z",
                    "log": null
                }
            ],
            "feedback": {
                "value": 5,
                "weight": 1,
                "metadata": {
                    "successfulChecks": "default.sentenceCount, default.wordCount",
                    "failedChecks": "",
                    "erroredChecks": ""
                }
            },
            "execution_time": 0,
            "async": false,
            "type": "guardrail",
            "created_at": "2025-05-20T08:01:02.229Z",
            "deny": false
        }
    ]
}
}

This example corresponds to a config like:

{
  "input_guardrails": [
    "sentence_and_word_check_guardrail",  // Contains sentenceCount and wordCount checks
    "characer_check_guardrail"  // Contains characterCount check
  ],
  "output_guardrails": [
    "sentence_and_word_check_guardrail"   // The same guardrail can be used for both input and output
  ]
}

​
Important Notes

  • If a guardrail is configured to run asynchronously (async=true), the hook_results will not appear in the API response. The results will only be available in the Portkey logs.
  • The data field varies based on the type of guardrail check being performed. Each guardrail type returns different information relevant to its function.
  • The overall verdict for a guardrail is true only if all individual checks pass. If any check fails, the verdict will be false.
  • When transformed is true, it indicates that the guardrail has modified the content (such as redacting PII).
  • If deny is true and the verdict is false, the request will be denied with a 446 status code.

​
Special Fields

  • Check-specific data: Each guardrail type provides different data in the data field. For example, a sentence count check provides information about the number of sentences, while a PII check might provide information about detected PII entities.
  • Feedback metadata: The metadata object within feedback keeps track of which checks were successful, failed, or encountered errors.

​
Defining Guardrails Directly in JSON

On Portkey, you can also create the Guardrails in code and add them to your Configs. Read more about this here:

Creating Raw Guardrails (in JSON

​
Bring Your Own Guardrails

If you already have a custom guardrail pipeline where you send your inputs/outputs for evaluation, you can integrate it with Portkey using a modular, custom webhook! Read more here:

Bring Your Own Guardrails

​
Examples of When to Deny Requests with Guardrails

  1. Prompt Injection Checks: Preventing inputs that could alter the behavior of the AI model or manipulate its responses.
  2. Moderation Checks: Ensuring responses do not contain offensive, harmful, or inappropriate content.
  3. Compliance Checks: Verifying that inputs and outputs comply with regulatory requirements or organizational policies.
  4. Security Checks: Blocking requests that contain potentially harmful content, such as SQL injection attempts or cross-site scripting (XSS) payloads.

By appropriately configuring Guardrail Actions, you can maintain the integrity and reliability of your AI app, ensuring that only safe and compliant requests are processed.

To enable Guardrails for your org, ping us on the Portkey Discord