Support Agent — Transaction & Password-Reset Automation

An LLM-based support agent that reads unstructured customer/merchant/bank support emails, detects intent, extracts and validates identifiers, calls backend lookup tools, and returns a strict, schema-validated JSON response for downstream delivery. Built for a high-volume payment-gateway support inbox.

Note: This repo is a sanitized, illustrative reconstruction of a production agent I designed and shipped. Client name, API endpoints, headers, credentials, and real transaction data have been removed or replaced with mock equivalents. It's intended to demonstrate architecture and problem-solving, not to be a drop-in production system.

What it does

• Classifies inbound support requests into two intents: transaction status and password reset.
• Extracts and validates identifiers (reference numbers, order number + amount, user ID) directly from noisy, free-form email text.
• Calls the appropriate backend tool and treats the tool's output as the single source of truth — the orchestrator never authors, paraphrases, or edits response text.
• Returns a single strict JSON object per request, validated against a fixed schema, ready for a support system to send verbatim.
• Handles single and bulk (multiple reference numbers in one email) lookups in one call.
• Produces role-aware responses (customer / merchant / bank) without the orchestrator needing to know the business copy for each.

Why this design

The agent went through two architectural generations:

1. v1 — draft-in-prompt. Response copy for every role/scenario combination lived in the orchestrator's instructions. This was hard to maintain, drifted between response types, and bloated prompt size/cost.
2. v2 — draft-in-tool (current). Response drafting moved into the tool layer. The orchestrator's job shrank to: detect intent → validate/extract identifiers → call exactly one tool → return the tool's output unmodified. This made bugs much easier to triage (a wrong response is a tool bug, not a prompt bug) and cut both cost and inconsistency.

See docs/DESIGN_DECISIONS.md for the full problem log this evolution came from, and ARCHITECTURE.md for the request lifecycle.

Repo layout

.
├── README.md
├── ARCHITECTURE.md
├── docs/
│   └── DESIGN_DECISIONS.md      # postmortem-style log of design iterations
├── schemas/
│   └── response_schema.json     # strict output contract
├── src/
│   ├── orchestrator.py          # intent detection + passthrough logic
│   ├── tools/
│   │   ├── get_order_status.py       # mock transaction-lookup tool
│   │   └── resend_activation_link.py # mock password-reset tool
│   └── utils/
│       └── reference_extractor.py    # noisy-text reference number extraction
├── tests/
│   └── test_reference_extractor.py
└── examples/
    ├── sample_request_transaction.json
    ├── sample_request_password_reset.json
    └── sample_response.json

Key engineering problems solved

• Deterministic output — moved from freeform LLM prose to a strictly validated JSON schema, eliminating hallucinated/missing fields.
• Bulk reference handling — redesigned the tool to accept a list of reference numbers and return one consolidated response, instead of the orchestrator fanning out multiple tool calls and stitching results together.
• Robust identifier extraction — reference numbers embedded in noisy text (e.g. 112345678908/CCAVENUE, 118989098723-1089) were being silently dropped by exact-boundary matching. Rebuilt as a two-step scan-then-validate algorithm (see src/utils/reference_extractor.py) that finds every numeric substring first, then validates candidates independently against the format rule (12 digits, starts with 11).

Tech stack

• LLM served via a hosted inference endpoint (structured/JSON-only output mode)
• Python for the tool layer and extraction utilities
• JSON Schema for output contract validation
• Consumed as an API by a client backend (no direct end-user UI)

Business Analysis & Product Contributions

• Requirements & Use-Case Definition

  • Translated real-world support scenarios into defined use cases.
  • Covered reference-number lookup, order-number + amount fallback, bulk requests, refund status, and password reset.
  • Defined clear acceptance criteria for each use case.

• Process Mapping

  • Documented the end-to-end workflow: Intent → Data Extraction → Tool Call → Validation → Structured Response
  • Created a shared reference for engineering, QA, and support teams.

• Business Rules & Prioritization

  • Defined identifier priority based on data reliability and user experience.
  • Priority: Reference Number → Order Number + Amount → Fallback/Error
  • Worked with stakeholders to validate and finalize the business logic.

• Root-Cause Analysis

  • Investigated incorrect, missed, and inconsistent agent outputs.
  • Documented issues using: Problem → Root Cause → Solution → Impact
  • Used findings to drive iterative improvements to agent behavior.

• Quality Monitoring & Reporting

  • Maintained a daily quality scoreboard to track:
    • ✅ Correct
    • ❌ Wrong Output
    • ⚠️ Missed
    • 🔍 Message Not Found
  • Used quality trends to identify regressions and monitor post-launch reliability.

• Cross-Functional Collaboration

  • Acted as the bridge between client support/operations requirements and engineering implementation.
  • Ensured role-specific response requirements for customers, merchants, and banks were reflected in the product specification and agent behavior.

Status

Actively used in production for a live payment-gateway support inbox, processing requests across customer, merchant, and bank channels daily.