Skip to content

safe flag never set to False — agent always reports safe=true even on successful attacks #3

Description

@Adityakk9031

What happened?

The safe flag in engine/agent.py is initialized to True at the start of every turn, but it is never updated to False when a tool call is allowed to execute.

In both chat() (L89–L109) and _process_response_stream() (L338–L342), the branch handling an unblocked tool call updates only the reason field:

if tool_call_result["blocked"]:
    safe = True
    reason = tool_call_result.get("reasoning", "Tool call blocked by guardrails")
else:
    safe = False
    reason = tool_call_result.get("reasoning", "Tool executed successfully")

However, the current implementation omits safe = False in the else branch, causing safe to remain True regardless of whether the guardrail blocked the tool call.

Expected: When a tool call is executed without being blocked, safe should be False.

Actual: safe is always True, so every COMPLETE SSE event and every PlaygroundChatResponse returned to the client reports "safe": true, even after a successful guardrail bypass.

Consequences

  • The frontend safety indicator always remains green and never reflects an unsafe execution.
  • Backend monitoring or alerting that relies on safe == False is never triggered.
  • Successful challenge completions report "success": true while simultaneously reporting "safe": true, producing contradictory state in the UI.

Steps to reproduce

1. Start a playground session against any challenge (for example, `access-code-001`).
2. Send a prompt that successfully bypasses the guardrail and causes the agent to invoke a protected tool (for example, `reveal_access_code`) without being blocked.
3. Observe the SSE `complete` event payload: `"success": true` is returned, but `"safe": true` is also reported.
4. Alternatively, place a breakpoint or log statement immediately after `_handle_tool_call_generic()` returns with `blocked=False` and inspect the `safe` variable—it still remains `True`.

**Minimal code path:**

- `agent.py::chat()`
  - `safe = True`
  - `_handle_tool_call_generic(...)`
  - `blocked == False`
  - `safe` is never updated
  - `result = {"safe": safe, ...}`

The same issue exists in `_process_response_stream()`, where the unblocked tool-call path likewise fails to set `safe = False`.

Browser / Device

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions