Skip to content

obs: Add structured JSON logging throughout backend #81

@YaronZaki

Description

@YaronZaki

Problem Statement

All logging uses basic Python logging with f-string interpolation: logger.info(f"Processing position {position_id} for user {wallet_id}"). Logs are unstructured text, making them difficult to parse in log aggregators (Datadog, ELK, CloudWatch).

Evidence

# Throughout codebase — unstructured logging pattern
# quantara/web_app/api/position.py
logger.info(f"Processing position {position_id}")

# quantara/web_app/db/crud/position.py
logger.error(f"Failed to retrieve positions: {str(e)}")

# quantara/web_app/contract_tools/blockchain_call.py
logger.error("Network error fetching account %s: %s", holder_address, exc)

Impact

Medium — observability gap. Cannot filter logs by wallet_id, position_id, or request_id across services. Log aggregation tools cannot parse structured fields. Debugging production issues requires grep on raw text.

Proposed Solution

Add python-json-logger or structlog for structured JSON logging. Include request_id, wallet_id (masked), endpoint, and duration in every log entry. Configure JSON formatter for production, human-readable for development.

Acceptance Criteria

  • Structured JSON logging configured using structlog or python-json-logger
  • All log entries include: timestamp, level, logger, request_id, message
  • Sensitive fields (wallet_id) masked in production logs
  • Log format configurable: JSON in production, human-readable in dev
  • All existing log calls updated to use structured logger

File Map

  • quantara/web_app/utils/logger.pyNew: structured logging setup
  • quantara/web_app/api/ — update all log calls
  • quantara/web_app/db/ — update all log calls
  • quantara/pyproject.toml — add structlog or python-json-logger

Dependencies

  • Related: REPO-042 (Prometheus metrics complement structured logging)

Testing Strategy

  • Unit: Verify log output format in JSON mode and text mode
  • Integration: Run request through API, verify log output contains expected structured fields

Security Considerations

Ensure wallet_id is masked/hashed in production logs. Never log private keys, secrets, or full API responses. Structured logging should not increase PII leakage.

Definition of Done

  • Code implemented and peer-reviewed
  • All log calls updated
  • PR linked and merged

Labels: observability
Priority: Medium
Difficulty: Intermediate
Estimated Effort: 4h

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions