feat(traces): per-span limits and exception stacktraces

[turnipdabeets]

💡 Motivation and Context

Bounds what one span can hold, per the traces spec.

• At most max_attributes_per_span user attributes and max_events_per_span events (128 each, earliest-set wins); each event at most 128 attributes. Overflow is reported as droppedAttributesCount / droppedEventsCount, clamped to uint32. The posthogDistinctId and sessionId join keys are exempt.
• max_attribute_value_length (8192) bounds every string an attribute holds, nested ones included, plus span and event names, status messages and resource attributes, so one large value cannot get a span dropped as too large.
• Recorded exceptions now carry exception.stacktrace, keeping the tail of the traceback where Python puts the raising frame.

Not reachable from the client yet.

Stack (PR 7 of 9, based on traces/06-export):

1. feat(traces): W3C trace context ids and traceparent parsing #949 traces/01-ids-traceparent
2. feat(traces): OTLP span encoding and client-side validity #950 traces/02-otlp-encoding
3. feat(traces): span handles #951 traces/03-span-handles
4. feat(traces): span batch transport #952 traces/04-transport
5. feat(traces): span pipeline #953 traces/05-pipeline
6. feat(traces): span export queue with retries #954 traces/06-export
7. feat(traces): per-span limits and exception stacktraces #955 traces/07-span-limits ← this PR
8. feat(traces): before_span_send hook #956 traces/08-before-span-send
9. feat(traces): wire tracing into the client #957 traces/09-client-wiring

💚 How did you test it?

Unit tests in posthog/test/tracing/test_limits.py, plus additions to test_span.py, test_otlp.py, test_pipeline.py and test_export.py.

📝 Checklist

• I reviewed the submitted code.
• I added tests to verify the changes.
• I updated the docs if needed.
• No breaking change or entry added to the changelog.

If releasing new changes

• Ran sampo add to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with Claude Code (Claude Opus 5) against the traces spec, one commit per slice so each PR reviews on its own. Rebased onto main and opened as a stacked draft in a later Claude Code session (Claude Fable 5.1).

🤖 Generated with Claude Code

https://claude.ai/code/session_012o7CtHLfcypjmXL7g9ZGRC

[greptile-apps]

Prompt To Fix All With AI

### Issue 1
posthog/tracing/_limits.py:98
**Nested keys bypass limits**

A nested mapping with a very long key bypasses `max_attribute_value_length` because this stores the original key. The OTLP encoder later emits that key without truncation, so it can make the batch exceed the transport limit and be dropped. Store the truncated `key_str` instead.

```suggestion
        bounded[truncate_string(key_str, max_length)] = item
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "feat(traces): per-span limits and except..." | Re-trigger Greptile}

[github-actions]

posthog-python Compliance Report

Date: 2026-09-17 03:43:00 UTC
Duration: 256355ms

✅ All Tests Passed!

111/111 tests passed

---

Capture_V1 Tests

✅ 94/94 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	516ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	510ms
Required Headers.Has Authorization Bearer Header	✅	509ms
Required Headers.Has Content Type Json	✅	510ms
Required Headers.Has Posthog Sdk Info Format	✅	510ms
Required Headers.Has Posthog Attempt Header	✅	510ms
Required Headers.Has Posthog Request Id	✅	510ms
Required Headers.Has Posthog Request Timestamp	✅	510ms
Required Headers.Has User Agent	✅	509ms
Body Format.Body Has Created At And Batch	✅	509ms
Body Format.No Api Key In Body	✅	511ms
Body Format.No Sent At In Body	✅	510ms
Event Format.Event Has Required Root Fields	✅	510ms
Event Format.Event Uuid Is Valid	✅	510ms
Event Format.Event Timestamp Is Rfc3339	✅	510ms
Event Format.Distinct Id Is String	✅	510ms
Event Format.Distinct Id At Root Not Properties	✅	510ms
Event Format.Custom Properties Preserved	✅	510ms
Event Format.Set Properties Preserved	✅	510ms
Event Format.Set Once Properties Preserved	✅	510ms
Event Format.Groups Properties Preserved	✅	509ms
Event Format.Sdk Generates Uuid If Not Provided	✅	510ms
Event Format.Event Has Required Root Fields Batch	✅	513ms
Event Format.Event Uuid Is Valid Batch	✅	513ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	513ms
Event Format.Distinct Id Is String Batch	✅	512ms
Event Format.Distinct Id At Root Not Properties Batch	✅	514ms
Event Format.Custom Properties Preserved Batch	✅	513ms
Event Format.Set Properties Preserved Batch	✅	513ms
Event Format.Set Once Properties Preserved Batch	✅	513ms
Event Format.Groups Properties Preserved Batch	✅	513ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	516ms
Batch Behavior.Multiple Events In Single Batch	✅	518ms
Batch Behavior.Batch Envelope Smoke	✅	515ms
Batch Behavior.Flush With No Events Sends Nothing	✅	506ms
Batch Behavior.Flush At Triggers Batch	✅	1010ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	510ms
Deduplication.Generates Unique Uuids	✅	518ms
Deduplication.Different Events Same Content Different Uuids	✅	513ms
Deduplication.Preserves Uuid On Retry	✅	6518ms
Deduplication.Preserves Timestamp On Retry	✅	6515ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	6518ms
Deduplication.No Duplicate Events In Batch	✅	518ms
Header Behavior On Retry.Attempt Header Starts At One	✅	510ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	13522ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	6514ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	3018ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	6519ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	511ms
Response Format Validation.Success Response Has Ok For Each Event	✅	514ms
Response Format Validation.Success No Retry After When All Ok	✅	511ms
Response Format Validation.Success Retry After Present When Retry Events	✅	1516ms
Response Format Validation.Success No Retry After When Drop Only	✅	513ms
Response Format Validation.Response Echoes Request Id	✅	510ms
Retry Behavior.Retries On 408	✅	6518ms
Retry Behavior.Retries On 500	✅	6516ms
Retry Behavior.Retries On 503	✅	8523ms
Retry Behavior.Retries On 504	✅	6519ms
Retry Behavior.Retryable Errors Have Retry After	✅	3516ms
Retry Behavior.Respects Retry After On Retryable Error	✅	11523ms
Retry Behavior.Does Not Retry On 400	✅	2514ms
Retry Behavior.Does Not Retry On 401	✅	2513ms
Retry Behavior.Does Not Retry On 402	✅	2513ms
Retry Behavior.Does Not Retry On 413	✅	2513ms
Retry Behavior.Does Not Retry On 415	✅	2513ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	2513ms
Retry Behavior.Implements Backoff	✅	22524ms
Retry Behavior.Max Retries Respected	✅	22537ms
Partial Batch Handling.Handles 200 Full Success	✅	2511ms
Partial Batch Handling.Handles 200 With All Ok	✅	3515ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	3515ms
Partial Batch Handling.Does Not Retry Limited Events	✅	3516ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	6522ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	6519ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	6518ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	6521ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	6521ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	6522ms
Partial Batch Handling.Respects Retry After On Partial	✅	8517ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	3512ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry	✅	3519ms
Compression.Sends Gzip Content Encoding	✅	511ms
Compression.No Content Encoding When Disabled	✅	510ms
Compression.Compressed Body Is Decompressible	✅	511ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	2513ms
Event Options.Cookieless Mode Override	✅	510ms
Event Options.Disable Skew Correction Override	✅	510ms
Event Options.Process Person Profile Override	✅	509ms
Event Options.Product Tour Id Override	✅	510ms
Event Options.Unset Options Omitted	✅	510ms
Event Options.Options Override In Batch	✅	513ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties	✅	509ms
Geoip And Historical Migration.Historical Migration Set In Body	✅	509ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	510ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	11ms
Request Payload.Flags Request Uses V2 Query Param	✅	9ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	9ms
Request Payload.Flags Request Omits Authorization Header	✅	9ms
Request Payload.Token In Flags Body Matches Init	✅	9ms
Request Payload.Groups Round Trip	✅	9ms
Request Payload.Groups Default To Empty Object	✅	9ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	9ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	9ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	9ms
Request Lifecycle.No Flags Request On Init Alone	✅	3ms
Request Lifecycle.No Flags Request On Normal Capture	✅	509ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	14ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	9ms
Retry Behavior.Retries Flags On 502	✅	312ms
Retry Behavior.Retries Flags On 504	✅	312ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	511ms

[greptile-apps]

_{Reviews (2): Last reviewed commit: "feat(traces): per-span limits and except..." | Re-trigger Greptile}

[dustinbyrne]

Looks good overall. Two non-blocking inline notes on null-attribute drop counts and callable encoding; neither blocks this tracing slice.

Validation was static inspection and existing CI. The proposed regression cases were not executed.

Agent-led review, human-reviewed before posting.

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[jzhu13]

Reviewed against traces/06-export. Tests pass at the head. Limits are applied at write time, overwrites and None are accounted correctly, auto join keys are exempt, and the truncation walk mirrors the encoder's budgets in the safe direction. No blocking issues. Approving with non-blocking notes.

Non-blocking

1. posthog/tracing/_span.py:281 an empty-string key spends a span (or event) attribute slot that the encoder then discards. Reproduced with max_attributes=1: set_attribute("", 1).set_attribute("real", 2) stores {"": 1}, counts one drop, and ships zero attributes. _truncate_mapping already skips falsy keys for nested values, so the top level is the odd one out. Treat not key_str like None.
2. posthog/tracing/_span.py:190 tail truncation of a chained traceback keeps the outer wrapper and discards the root cause, since format_exception prints the cause first. Reproduced: raise RuntimeError("wrapper") from ValueError("ROOT CAUSE") at max_length=120 keeps wrapper, not ROOT CAUSE. The test test_bounds_the_stack_keeping_the_crash_site actually asserts the wrapper survives while the recursive crash site is cut off, contradicting its name. Either format with chain=False and prepend the innermost cause's tail, or rename the test and document that the tail is the outermost exception.
3. posthog/tracing/_limits.py:34 truncate_attribute_value allocates a _WalkState with a set and enters a try for every scalar. Measured: set_attribute("k", "value") spends 0.32 µs of 0.40 µs there versus 0.03 µs for a direct slice. A fast path for str, numbers, bool, and None before the walk removes it.
4. posthog/tracing/_span.py:317 add_event with non-mapping attributes no longer logs. The base went through copy_user_attributes, which debug-logs "expected a mapping"; bound_attributes returns ({}, 0) silently. Route through copy_user_attributes (which also removes the duplicated key walk in bound_attributes).
5. posthog/tracing/_span.py:281 same-key check-then-write across threads can permanently consume a slot. Not reproduced in 300 trials, but the _Activatable docstring invites multi-thread use. Guarding with the existing lock is cheap.
6. Nits: two tests (test_the_per_event_attribute_cap_is_opentelemetrys_default, the == 8192 literal in test_config.py) only fail on an intentional constant edit; test_a_self_referencing_attribute_ends_the_span asserts only the record count and would pass if the value were stored unbounded.

Two notes for #956: apply_span_limits inherits item 1 verbatim and re-walks every already-bounded value whenever a hook is configured, and #956 adds SpanRecord.auto_attribute_keys that this PR's end() does not populate, so the tip's end() must.

Reviewed with Claude Code (Claude Fable 5.1). Behaviors above were reproduced against this branch head.

[turnipdabeets]

Thanks. Fixed in 27b9229:

• 1: an empty key spends no slot, at the span and in the post-hook re-cap.
• 2: test renamed and docstring says the outermost exception survives the cut.
• 3: scalars skip the walk.
• 4: event attributes go through the shared copier, so a non-mapping logs.
• 6: the self-reference test checks the stored marker.
• 5 (locking writes): still open.

[turnipdabeets]

Follow-up, 1975532: item 5 is done. The cap check, the event count and the end() snapshot run under the handle's lock, with the value walk still outside it. Tests race eight threads on one key and on the event cap. Ready for another look.