Add AI Guard scenario documentation and cassette generation

[smola]

Motivation

APPSEC-61477

Changes

Add documentation for AI Guard scenario, and script to re-generate VCR cassettes.

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added, removed or renamed?
  • Get a review from R&P team

[github-actions]

CODEOWNERS have been resolved as:

docs/understand/scenarios/ai_guard.md                                   @DataDog/k9-ai-guard @DataDog/system-tests-core
tests/ai_guard/conftest.py                                              @DataDog/k9-ai-guard @DataDog/system-tests-core
utils/_context/_scenarios/ai_guard.py                                   @DataDog/system-tests-core
utils/scripts/generate-ai-guard-cassettes.sh                            @DataDog/k9-ai-guard @DataDog/system-tests-core
.cursor/rules/repository-structure.mdc                                  @DataDog/system-tests-core
.github/CODEOWNERS                                                      @DataDog/system-tests-core
docs/understand/scenarios/README.md                                     @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core
utils/_context/containers.py                                            @DataDog/system-tests-core
utils/build/docker/vcr/cassettes/aiguard/aiguard_evaluate_post_3156697a.json  @DataDog/k9-ai-guard @DataDog/system-tests-core
utils/build/docker/vcr/cassettes/aiguard/aiguard_evaluate_post_8919fde6.json  @DataDog/k9-ai-guard @DataDog/system-tests-core
utils/build/docker/vcr/cassettes/aiguard/aiguard_evaluate_post_ba6efcf0.json  @DataDog/k9-ai-guard @DataDog/system-tests-core
utils/build/docker/vcr/cassettes/aiguard/aiguard_evaluate_post_ee2b240f.json  @DataDog/k9-ai-guard @DataDog/system-tests-core
utils/build/docker/vcr/cassettes/aiguard/aiguard_evaluate_post_f2b74780.json  @DataDog/k9-ai-guard @DataDog/system-tests-core
utils/build/docker/vcr/cassettes/aiguard/aiguard_evaluate_post_f517ff03.json  @DataDog/k9-ai-guard @DataDog/system-tests-core

[datadog-prod-us1-5]

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 17 Tests failed

tests.appsec.test_blocking_addresses.Test_Blocking_client_ip.test_blocking_before[envoy] from system_tests_suite (Datadog) (Fix with Cursor)

ValueError: No appsec event validate this condition

self = <tests.appsec.test_blocking_addresses.Test_Blocking_client_ip object at 0x7f7b994fbad0>

    def test_blocking_before(self):
        """Test that blocked requests are blocked before being processed"""
        # second request should block and must not set the tag in span
        assert self.block_req2.status_code == 403
>       interfaces.library.assert_waf_attack(self.block_req2, rule="blk-001-001")

...

tests.appsec.test_blocking_addresses.Test_Blocking_client_ip.test_blocking[envoy] from system_tests_suite (Datadog) (Fix with Cursor)

ValueError: No appsec event validate this condition

self = <tests.appsec.test_blocking_addresses.Test_Blocking_client_ip object at 0x7f7b994fba10>

    def test_blocking(self):
        """Can block the request forwarded for the ip"""
    
        assert self.rm_req_block.status_code == 403
>       interfaces.library.assert_waf_attack(self.rm_req_block, rule="blk-001-001")

...

tests.appsec.test_blocking_addresses.Test_Blocking_client_ip_with_forwarded.test_blocking_before[envoy] from system_tests_suite (Datadog) (Fix with Cursor)

ValueError: No appsec event validate this condition

self = <tests.appsec.test_blocking_addresses.Test_Blocking_client_ip_with_forwarded object at 0x7f7b994fbe90>

    def test_blocking_before(self):
        """Test that blocked requests are blocked before being processed"""
        # second request should block and must not set the tag in span
        assert self.block_req2.status_code == 403
>       interfaces.library.assert_waf_attack(self.block_req2, rule="blk-001-001")

...

View all

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 26abb8c | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}