0→1 · Production AI · Visa
I shipped an AI agent
for fraud investigation
RAG + LLM orchestration on a 4.2B-transaction fraud platform. Production system with SLAs, human-in-the-loop, and enterprise governance.
Interface
Layer
💬
Natural Language Query
analyst input
🧠
Prompt Engine
context window mgmt
📐
Intent Routing
query classification
Intelligence
Layer
🔍
RAG Retrieval
vector + embeddings
⚙️
LLM Orchestration
tool invocation
🔗
Agent Tool-Use
multi-step reasoning
Data
Layer
🗄️
Fraud Data SSOT
4.2B txns / AWS
📊
Vector Store
embeddings index
🔄
Real-Time Pipeline
data freshness SLAs
Trust &
Safety
🛡️
Confidence Gate
threshold routing
👤
Human-in-the-Loop
escalation paths
📋
Audit + Compliance
full traceability
Fraud Investigation Agent
analyst > Anomalous decline rates for MCC 5411 in APAC, last 30 days
agent → Querying fraud SSOT via RAG retrieval...
found → 14,207 txns · decline rate 3.2× above baseline
pattern → Clustered across 3 acquirer endpoints in SG, MY, TH
conf → 0.87 · below auto-threshold
⚠ action → Escalating to analyst with full context + evidence package
Architecture Decision
RAG over fine-tuning
Fraud changes daily. Fine-tuned = stale in weeks. RAG retrieves from live SSOT — always current data.
Trust Decision
Human-in-the-loop by default
Agent accelerates judgment, doesn't replace it. False positives freeze real accounts.
Evaluation Decision
Production evidence, not intuition
Built measurable eval: quality, relevance, factuality, safety. Now used across Visa AI.
Governance Decision
Trust boundaries + access control
Not every data source is safe for agent access. Defined what the agent can and can't touch.
Confidence-Gated Response System
LOW < 0.6
MEDIUM 0.6–0.85
HIGH > 0.85
→ Escalate to human
→ Flag for review
→ Auto-surface
Iteration — What Failed & How I Fixed It
V1 — Failed
~40%
factuality issues on
multi-hop queries
→
V2 — Improved
~18%
curated eval datasets +
prompt refinement
→
V3 — Shipped
<5%
production-grade quality +
context window optimization
VIC
Fed Visa's AI Strategy
Production AI is a trust problem, not a demo problem. The model was 20% of the work. The other 80% was governance, safety design, and convincing a 30-person cross-functional team in one of the most regulated environments in the world.