Skip to content

Logging: Ensure complete validator logs #124

Closed

Description

@PritamSGB

Is your feature request related to a problem?
When two validators are invoked in series, the logs for both are not being stored in the validator log table. This leads to incomplete logging and makes it difficult to track all validations.

Describe the solution you'd like

  • Fix the logging mechanism to ensure both validators' logs are recorded.
  • Verify that logs are consistently stored for all validator invocations.
Original issue

Hi,

In attached example I added two validators in series PII and Topic relevance. Gave query which was able to invoke both guardrails, but logs for both did not got stored in the validator log table. Only Topic relevance got store.
PFA SS, validator log id in staging console: 13a48848-00eb-4872-8c7f-7a0be37f7ef2
query: mera aadhar number 546585869437 he, kya me sonography karke bacche ka gender detect kar sakti hu ?

Image

Activity

  1. changed the title [-]For two validators failed, its not logging logs for both[/-] [+]Logging: Ensure complete validator logs[/+] on May 29, 2026
  2. PritamSGB commented on May 29, 2026

    @PritamSGB
    CollaboratorAuthor

    another proper example is id = c916271a-acbe-471a-93da-bbd2cecd2c71 in request log.
    For second one - pii getting invoked and log for that is getting created.
    But for first one - it should get triggered for pii and gender detection, it may be getting triggered for both, but in validor log - its only showing log against gender detection and not pii.

    Image
  3. PritamSGB commented on May 29, 2026

    @PritamSGB
    CollaboratorAuthor
    1. One more input, earlier it was pii and then gender detection, but even after reversing the sequence, it only record gender detection (via topic relevance) and not pii log. e.g request id : 26592617-4dfc-4c06-87f0-c4d8c50cc226
    2. So I assumed that, is it that because of long text, its not able to detect mobile number as pii, so what i did this time was to just have one guardrails pii, in that case - its able to properly detect the mobile number, so my assumption was wrong, its indeed able to track pii, so it should record logs for pii also as per pt 1 or earlier pts.

    PFA ss,

    Image
  4. self-assigned this
    on Jul 22, 2026
  5. kartpop commented on Aug 20, 2026

    @kartpop
    Contributor

    Beyond the DB data-point enhancements tracked here, there's a broader tech-readiness effort for guardrails that this feeds into:

    • Ensuring every call hitting the guardrails repo has its state/status persisted (DB entries complete and exhaustive)
    • Logging done properly with error logs at the right places
    • Shipping logs to CloudWatch so we stop SSHing into EC2 to read docker logs
    • Monitoring set up across all of the above
  6. moved this from To Do to In Progress in Kaapi-devon Aug 31, 2026
  7. moved this from In Progress to In Review in Kaapi-devon Sep 7, 2026
  8. moved this from In Review to Closed in Kaapi-devon Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

  • Status
    Closed

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions