Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 85 additions & 91 deletions .github/workflows/run-end-to-end.yml

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Deep dives into each scenario type.
- **AWS SSI / Onboarding**: [full guide](understand/scenarios/onboarding.md) and [provision structure](understand/scenarios/onboarding_provision_section.md)
- **Docker SSI**: [guide](understand/scenarios/docker_ssi.md)
- **Kubernetes lib injection**: [overview](understand/scenarios/k8s_library_injection_overview.md), [details](understand/scenarios/k8s_lib_injection.md), and [injector dev](understand/scenarios/k8s_injector_dev.md)
- **Other scenarios**: [AWS Lambda](understand/scenarios/aws_lambda.md), [integration frameworks](understand/scenarios/integration_frameworks.md) (IPv6 and Go proxies are covered in the [scenarios overview](understand/scenarios/README.md))
- **Other scenarios**: [AWS Lambda](understand/scenarios/aws_lambda.md), [integration frameworks](understand/scenarios/integration_frameworks.md), and [IPv6](understand/scenarios/README.md#ipv6-scenario)

## CI integration

Expand Down
20 changes: 0 additions & 20 deletions docs/understand/scenarios/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,26 +66,6 @@ Please note that it requires the docker daemon to support IPv6. It should be ok

A user has seen his network function altered after running it on a linux laptop (to be investigated). If it happen, `docker network prune` may solve the issue.

### Go proxies (Envoy and HAProxy) scenario

```mermaid
flowchart LR
%% Nodes
A("Test runner")
B("Proxy (Envoy or HAProxy)")
C("Go security processor")
D("HTTP app")
E("Proxy")
F("Agent")
G("Backend")

%% Edge connections between nodes
A --> B --> D
B --> C --> B
C --> E --> F --> G
%% D -- Mermaid js --> I --> J
```

## Scenario lifecycle

System tests spawn several services before starting. Here is the lifecycle:
Expand Down
84 changes: 77 additions & 7 deletions manifests/envoy.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,40 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/DataDog/system-tests/refs/heads/main/utils/manifest/schema.json
---
manifest:
tests/appsec/api_security/test_endpoints.py: irrelevant (language not implementing this feature)
tests/ai_guard: irrelevant (no ai guard)
tests/appsec/api_security/test_api_security_rc.py: v1.72.0
tests/appsec/api_security/test_api_security_rc.py::Test_API_Security_RC_ASM_DD_processors: v2.0.0
tests/appsec/api_security/test_api_security_rc.py::Test_API_Security_RC_ASM_DD_scanners: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/api_security/test_apisecurity_telemetry.py::Test_API_Security_Telemetry_Metric: missing_feature (weblog does not support set 404 status code and /tag_value endpoint)
tests/appsec/api_security/test_custom_data_classification.py: v2.4.0
tests/appsec/api_security/test_custom_data_classification.py::Test_API_Security_Custom_Data_Classification_Multiple_Scanners: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/api_security/test_custom_data_classification.py::Test_API_Security_Custom_Data_Classification_Negative: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/api_security/test_custom_data_classification.py::Test_API_Security_Custom_Data_Classification_Scanner: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/api_security/test_schemas.py: v1.72.0
tests/appsec/api_security/test_schemas.py::Test_Schema_Request_FormUrlEncoded_Body: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/api_security/test_schemas.py::Test_Schema_Request_Json_Body: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/api_security/test_schemas.py::Test_Schema_Request_Path_Parameters: irrelevant (can only infer the path parameters from the request when using proxies)
tests/appsec/api_security/test_schemas.py::Test_Schema_Response_Body::test_request_method: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/api_security/test_schemas.py::Test_Schema_Response_Body_env_var: missing_feature (The endpoint /tag_value is not implemented in the weblog)
tests/appsec/api_security/test_schemas.py::Test_Schema_Response_Headers: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/api_security/test_schemas.py::Test_Schema_Response_on_Block::test_request_method: missing_feature (missing /tag_value endpoint in the weblog)
tests/appsec/iast: irrelevant (proxies does not support iast)
tests/appsec/test_alpha.py: v1.72.0
tests/appsec/test_asm_standalone.py: v1.73.0
tests/appsec/test_asm_standalone.py::Test_APISecurityStandalone: missing_feature
tests/appsec/test_asm_standalone.py::Test_AppSecStandalone_UpstreamPropagation_V2: missing_feature (weblog does not implement upstream request)
tests/appsec/test_asm_standalone.py::Test_UserEventsStandalone_Automated: irrelevant (Proxies doesn't support login events (go sdk))
tests/appsec/test_asm_standalone.py::Test_UserEventsStandalone_SDK_V1: irrelevant (Proxies doesn't support login events (go sdk))
tests/appsec/test_asm_standalone.py::Test_UserEventsStandalone_SDK_V2: irrelevant (Proxies doesn't support login events (go sdk))
tests/appsec/test_automated_login_events.py: irrelevant (proxies does not support login events)
tests/appsec/test_automated_login_events.py::Test_V3_Login_Events_Blocking: irrelevant (Proxies doesn't support login events (go sdk))
tests/appsec/test_automated_user_and_session_tracking.py: irrelevant (Proxies doesn't support login events (go sdk))
tests/appsec/test_blocking_addresses.py: v1.72.0
tests/appsec/test_blocking_addresses.py::Test_BlockingGraphqlResolvers: irrelevant
tests/appsec/test_blocking_addresses.py::Test_Blocking_client_ip_with_K8_private_ip: v2.2.0
tests/appsec/test_blocking_addresses.py::Test_Blocking_client_ip_with_forwarded: v2.3.0
tests/appsec/test_blocking_addresses.py::Test_Blocking_request_body: missing_feature
tests/appsec/test_blocking_addresses.py::Test_Blocking_request_body_multipart: irrelevant (Body blocking happens through SDK)
tests/appsec/test_blocking_addresses.py::Test_Blocking_request_body: missing_feature (the tests is made using raw text and not json, skiped in proxies)
tests/appsec/test_blocking_addresses.py::Test_Blocking_request_body_multipart: missing_feature
tests/appsec/test_blocking_addresses.py::Test_Blocking_request_cookies::test_blocking_before: missing_feature (The endpoint /tag_value is not implemented in the weblog)
tests/appsec/test_blocking_addresses.py::Test_Blocking_request_headers::test_blocking_before: missing_feature (The endpoint /tag_value is not implemented in the weblog)
tests/appsec/test_blocking_addresses.py::Test_Blocking_request_method::test_blocking_before: missing_feature (The endpoint /tag_value is not implemented in the weblog)
Expand All @@ -21,16 +47,60 @@ manifest:
tests/appsec/test_blocking_addresses.py::Test_Blocking_response_status: missing_feature
tests/appsec/test_blocking_addresses.py::Test_Blocking_user_id: irrelevant (not supported on proxies)
tests/appsec/test_blocking_addresses.py::Test_Suspicious_Request_Blocking: missing_feature (can't test with set tag)
tests/appsec/test_blocking_addresses.py::Test_Suspicious_Request_Blocking::test_blocking_without_path_params: v2.3.0
tests/appsec/test_client_ip.py: v1.72.0
tests/appsec/test_ip_blocking_full_denylist.py::Test_AppSecIPBlockingFullDenylist: v1.72.0
tests/appsec/test_conf.py::Test_ConfigurationVariables: v2.0.0
tests/appsec/test_conf.py::Test_ConfigurationVariables_New_Obfuscation: v2.1.0
tests/appsec/test_event_tracking.py: irrelevant (proxies does not support login events)
tests/appsec/test_extended_data_collection.py: v1.72.0
tests/appsec/test_extended_data_collection.py::Test_ExtendedDataCollectionCapability: missing_feature
tests/appsec/test_extended_data_collection.py::Test_ExtendedRequestBodyCollection: missing_feature
tests/appsec/test_extended_data_collection.py::Test_ExtendedRequestHeadersDataCollection: missing_feature
tests/appsec/test_extended_data_collection.py::Test_ExtendedResponseHeadersDataCollection: missing_feature
tests/appsec/test_extended_header_collection.py::Test_ExtendedHeaderCollection: missing_feature (weblog does not implement /headers endpoint)
tests/appsec/test_fingerprinting.py: v1.72.0
tests/appsec/test_fingerprinting.py::Test_Fingerprinting_Header_And_Network_Preprocessor::test_fingerprinting_header_blocking: missing_feature (proxies removes the 'connection' header, thus making fails the header fingerprinting rule)
tests/appsec/test_fingerprinting.py::Test_Fingerprinting_Session_Preprocessor::test_session_non_blocking: irrelevant (Proxies doesn't support session fingerprinting)
tests/appsec/test_ip_blocking_full_denylist.py: v1.72.0
tests/appsec/test_logs.py::Test_Standardization: missing_feature
tests/appsec/test_logs.py::Test_StandardizationBlockMode: missing_feature
tests/appsec/test_only_python.py::Test_ImportError: irrelevant (specific tests for python tracer)
tests/appsec/test_remote_config_rule_changes.py: v1.72.0
tests/appsec/test_remote_config_rule_changes.py::Test_AsmDdMultiConfiguration: v2.1.0
tests/appsec/test_reports.py: v1.72.0
tests/appsec/test_request_blocking.py::Test_AppSecRequestBlocking: v1.72.0
tests/appsec/test_request_blocking.py: v1.72.0
tests/appsec/test_shell_execution.py: irrelevant
tests/appsec/test_span_tags_headers.py::Test_Headers_Event_Blocking::test_content_type_event_blocking: missing_feature
tests/appsec/test_trace_tagging.py::Test_TraceTaggingRules: v2.1.0
tests/appsec/test_trace_tagging.py::Test_TraceTaggingRules::test_rule_with_attributes_no_keep_event: irrelevant (Proxies run with appsec standalone by default)
tests/appsec/test_trace_tagging.py::Test_TraceTaggingRules::test_rule_with_attributes_no_keep_no_event: irrelevant (Proxies run with appsec standalone by default)
tests/appsec/test_trace_tagging.py::Test_TraceTaggingRulesRcCapability: v2.1.0
tests/appsec/test_traces.py: v1.72.0
tests/appsec/test_traces.py::Test_AppSecEventSpanTags::test_header_collection: irrelevant (test)
tests/appsec/test_traces.py::Test_CollectRespondHeaders::test_header_collection: missing_feature (The endpoint /headers is not implemented in the weblog)
tests/appsec/test_user_blocking_full_denylist.py::Test_UserBlocking_FullDenylist: missing_feature (no endpoint to test user blocking - auto instrum irrelevant)
tests/appsec/test_versions.py::Test_Events: v1.72.0
tests/parametric/test_otel_span_methods.py::Test_Otel_Span_Methods::test_otel_record_exception_sets_handling_stack_in_go: irrelevant
tests/appsec/waf/test_addresses.py::Test_BodyUrlEncoded::test_body_value: missing_feature (no urlencoded body parsing implemented in proxies)
tests/appsec/waf/test_addresses.py::Test_BodyXml::test_xml_attr_value: missing_feature (no xml body parsing implemented in proxies)
tests/appsec/waf/test_addresses.py::Test_BodyXml::test_xml_content: missing_feature (no xml body parsing implemented in proxies)
tests/appsec/waf/test_addresses.py::Test_FullGrpc::test_main: irrelevant (proxies does not support grpc)
tests/appsec/waf/test_addresses.py::Test_PathParams::test_security_scanner: missing_feature
tests/appsec/waf/test_addresses.py::Test_ResponseStatus::test_basic: missing_feature
tests/appsec/waf/test_addresses.py::Test_gRPC::test_basic: missing_feature (proxies does not support grpc)
tests/appsec/waf/test_blocking.py: v1.72.0
tests/appsec/waf/test_blocking.py::Test_Blocking::test_accept_full_html: missing_feature
tests/appsec/waf/test_blocking.py::Test_Blocking::test_accept_partial_html: missing_feature (Support for partial html not implemented)
tests/appsec/waf/test_blocking.py::Test_Blocking::test_html_template_v2: missing_feature
tests/appsec/waf/test_blocking.py::Test_Blocking_strip_response_headers: missing_feature
tests/appsec/waf/test_blocking_security_response_id.py: v2.5.0
tests/appsec/waf/test_miscs.py::Test_404::test_404: missing_feature (weblog does not support 404 status code)
tests/integrations/test_dbm.py: irrelevant
tests/remote_config/test_remote_configuration.py: v1.72.0
tests/remote_config/test_remote_configuration.py::Test_RemoteConfigurationUpdateSequenceASMDDNoCache: irrelevant (cache is implemented)
tests/serverless: irrelevant
tests/test_config_consistency.py::Test_Config_UnifiedServiceTagging_CustomService: v1.72.0
tests/test_identify.py: irrelevant (proxies does not support custom sdk)
tests/test_protobuf.py: irrelevant
tests/test_resource_renaming.py: v2.4.0
tests/test_scrubbing.py: v1.72.0
tests/test_semantic_conventions.py: v1.72.0
tests/test_semantic_conventions.py::Test_Meta::test_meta_component_tag: v2.3.0
Expand Down
Loading
Loading