ant_ai.hooks.builtins.llm_guardrail

GuardrailVerdict pydantic-model

Bases: BaseModel

Structured judge response consumed by LLMGuardrailHook.

Show JSON schema:

{
  "description": "Structured judge response consumed by ``LLMGuardrailHook``.",
  "properties": {
    "passed": {
      "description": "True if the text satisfies the guardrail criteria.",
      "title": "Passed",
      "type": "boolean"
    },
    "reason": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Explanation of the failure. Should be set when passed is False.",
      "title": "Reason"
    }
  },
  "required": [
    "passed"
  ],
  "title": "GuardrailVerdict",
  "type": "object"
}

Fields:

• passed (bool)
• reason (str | None)

Source code in src/ant_ai/hooks/builtins/llm_guardrail.py

class GuardrailVerdict(BaseModel):
    """Structured judge response consumed by ``LLMGuardrailHook``."""

    passed: bool = Field(
        description="True if the text satisfies the guardrail criteria."
    )
    reason: str | None = Field(
        default=None,
        description="Explanation of the failure. Should be set when passed is False.",
    )

passed pydantic-field

passed: bool

True if the text satisfies the guardrail criteria.

reason pydantic-field

reason: str | None = None

Explanation of the failure. Should be set when passed is False.

LLMGuardrailHook pydantic-model

Bases: AgentHook, BaseModel

Templated LLM-as-judge guardrail base class.

Sends the model's output to a configurable judge LLM and turns its verdict into a PostModelPass/PostModelRetry/PostModelBlock decision. Subclass and override criteria — or build_judge_messages for full control over the prompt — to guardrail on domain-specific rules (e.g. "no medical advice", "stay on topic about product X") without wrapping a third-party judge framework.

Only overrides after_model. The judge call is issued directly against judge_llm and does not go through hooks, so it never recurses into this guardrail (or any other hook) and cannot trigger its own retries.

Parameters:

Name	Type	Description	Default
judge_llm		Language model invoked as the judge.	required
criteria		Natural-language description of what the text must satisfy. Used by the default build_judge_messages; ignored if that method is overridden.	required
on_fail		"retry" (default) asks the agent to retry with the judge's reason as critique; "block" raises immediately.	required

Example:

```python
class NoMedicalAdviceGuardrail(LLMGuardrailHook):
    criteria: str = "The text must not contain medical advice or diagnoses."

hook = NoMedicalAdviceGuardrail(judge_llm=LiteLLMChat(model="gpt-4o-mini"))
agent = Agent(..., hooks=[hook])
```

Full prompt control:

```python
class JsonOnlyGuardrail(LLMGuardrailHook):
    def build_judge_messages(self, raw: str) -> list[Message]:
        return [
            Message(role="system", content="Reply with the requested JSON only."),
            Message(role="user", content=f"Is this valid JSON?\n\n{raw}"),
        ]
```

Show JSON schema:

{
  "description": "Templated LLM-as-judge guardrail base class.\n\nSends the model's output to a configurable judge LLM and turns its\nverdict into a ``PostModelPass``/``PostModelRetry``/``PostModelBlock``\ndecision. Subclass and override ``criteria`` \u2014 or ``build_judge_messages``\nfor full control over the prompt \u2014 to guardrail on domain-specific rules\n(e.g. \"no medical advice\", \"stay on topic about product X\") without\nwrapping a third-party judge framework.\n\nOnly overrides ``after_model``. The judge call is issued directly against\n``judge_llm`` and does not go through hooks, so it never recurses into\nthis guardrail (or any other hook) and cannot trigger its own retries.\n\nArgs:\n    judge_llm: Language model invoked as the judge.\n    criteria: Natural-language description of what the text must\n        satisfy. Used by the default ``build_judge_messages``; ignored\n        if that method is overridden.\n    on_fail: ``\"retry\"`` (default) asks the agent to retry with the\n        judge's reason as critique; ``\"block\"`` raises immediately.\n\nExample:\n\n    ```python\n    class NoMedicalAdviceGuardrail(LLMGuardrailHook):\n        criteria: str = \"The text must not contain medical advice or diagnoses.\"\n\n    hook = NoMedicalAdviceGuardrail(judge_llm=LiteLLMChat(model=\"gpt-4o-mini\"))\n    agent = Agent(..., hooks=[hook])\n    ```\n\n    Full prompt control:\n\n    ```python\n    class JsonOnlyGuardrail(LLMGuardrailHook):\n        def build_judge_messages(self, raw: str) -> list[Message]:\n            return [\n                Message(role=\"system\", content=\"Reply with the requested JSON only.\"),\n                Message(role=\"user\", content=f\"Is this valid JSON?\\n\\n{raw}\"),\n            ]\n    ```",
  "properties": {
    "judge_llm": {
      "description": "Language model invoked as the judge.",
      "title": "Judge Llm"
    },
    "criteria": {
      "description": "Natural-language description of what the text must satisfy.",
      "title": "Criteria",
      "type": "string"
    },
    "on_fail": {
      "default": "retry",
      "enum": [
        "retry",
        "block"
      ],
      "title": "On Fail",
      "type": "string"
    }
  },
  "required": [
    "judge_llm",
    "criteria"
  ],
  "title": "LLMGuardrailHook",
  "type": "object"
}

Config:

• arbitrary_types_allowed: True

Fields:

• judge_llm (SkipValidation[Any])
• criteria (str)
• on_fail (Literal['retry', 'block'])

Source code in src/ant_ai/hooks/builtins/llm_guardrail.py

class LLMGuardrailHook(AgentHook, BaseModel):
    """
    Templated LLM-as-judge guardrail base class.

    Sends the model's output to a configurable judge LLM and turns its
    verdict into a ``PostModelPass``/``PostModelRetry``/``PostModelBlock``
    decision. Subclass and override ``criteria`` — or ``build_judge_messages``
    for full control over the prompt — to guardrail on domain-specific rules
    (e.g. "no medical advice", "stay on topic about product X") without
    wrapping a third-party judge framework.

    Only overrides ``after_model``. The judge call is issued directly against
    ``judge_llm`` and does not go through hooks, so it never recurses into
    this guardrail (or any other hook) and cannot trigger its own retries.

    Args:
        judge_llm: Language model invoked as the judge.
        criteria: Natural-language description of what the text must
            satisfy. Used by the default ``build_judge_messages``; ignored
            if that method is overridden.
        on_fail: ``"retry"`` (default) asks the agent to retry with the
            judge's reason as critique; ``"block"`` raises immediately.

    Example:

        ```python
        class NoMedicalAdviceGuardrail(LLMGuardrailHook):
            criteria: str = "The text must not contain medical advice or diagnoses."

        hook = NoMedicalAdviceGuardrail(judge_llm=LiteLLMChat(model="gpt-4o-mini"))
        agent = Agent(..., hooks=[hook])
        ```

        Full prompt control:

        ```python
        class JsonOnlyGuardrail(LLMGuardrailHook):
            def build_judge_messages(self, raw: str) -> list[Message]:
                return [
                    Message(role="system", content="Reply with the requested JSON only."),
                    Message(role="user", content=f"Is this valid JSON?\\n\\n{raw}"),
                ]
        ```
    """

    model_config = ConfigDict(arbitrary_types_allowed=True)

    name: ClassVar[str] = "llm_guardrail"
    judge_llm: SkipValidation[Any] = Field(
        description="Language model invoked as the judge.",
    )
    criteria: str = Field(
        description="Natural-language description of what the text must satisfy.",
    )
    on_fail: Literal["retry", "block"] = "retry"

    def build_judge_messages(self, raw: str) -> list[Message]:
        """Build the messages sent to the judge LLM. Override for full prompt control.

        Args:
            raw: The candidate text being judged (model output, or the
                serialized tool-call arguments when the model produced no
                text).

        Returns:
            Messages sent to ``judge_llm`` with ``response_format=GuardrailVerdict``.
        """
        return [
            Message(role="system", content=_DEFAULT_JUDGE_SYSTEM_PROMPT),
            Message(
                role="user",
                content=f"Criteria:\n{self.criteria}\n\nText to judge:\n{raw}",
            ),
        ]

    async def after_model(
        self,
        result: StepResult,
        ctx: InvocationContext | None,
    ) -> PostModelDecision:
        raw = model_output_text(result)
        if raw is None:
            return PostModelPass(result=result)

        messages = self.build_judge_messages(raw)

        try:
            response = await self.judge_llm.ainvoke(
                messages, response_format=GuardrailVerdict
            )
            verdict = GuardrailVerdict.model_validate_json(
                response.message.content or ""
            )
        except Exception as exc:  # noqa: BLE001
            return PostModelRetry(reason=f"LLM guardrail judge error: {exc}")

        if verdict.passed:
            return PostModelPass(result=result)

        reason = verdict.reason or "LLM guardrail check failed"
        if self.on_fail == "block":
            return PostModelBlock(reason=reason)
        return PostModelRetry(reason=reason)

judge_llm pydantic-field

judge_llm: SkipValidation[Any]

Language model invoked as the judge.

criteria pydantic-field

criteria: str

Natural-language description of what the text must satisfy.

build_judge_messages

build_judge_messages(raw: str) -> list[Message]

Build the messages sent to the judge LLM. Override for full prompt control.

Parameters:

Name	Type	Description	Default
raw	str	The candidate text being judged (model output, or the serialized tool-call arguments when the model produced no text).	required

Returns:

Type	Description
list[Message]	Messages sent to judge_llm with response_format=GuardrailVerdict.

Source code in src/ant_ai/hooks/builtins/llm_guardrail.py

def build_judge_messages(self, raw: str) -> list[Message]:
    """Build the messages sent to the judge LLM. Override for full prompt control.

    Args:
        raw: The candidate text being judged (model output, or the
            serialized tool-call arguments when the model produced no
            text).

    Returns:
        Messages sent to ``judge_llm`` with ``response_format=GuardrailVerdict``.
    """
    return [
        Message(role="system", content=_DEFAULT_JUDGE_SYSTEM_PROMPT),
        Message(
            role="user",
            content=f"Criteria:\n{self.criteria}\n\nText to judge:\n{raw}",
        ),
    ]