Design a Fraud Detection System
Design a real-time fraud detection system: ML pipeline, feature engineering, and production deployment.
What is a fraud detection system?
A fraud detection system estimates whether a transaction is abusive, then turns that estimate into an action: allow, challenge, review, or block. The model supplies evidence, but a versioned decision policy owns the customer-facing outcome.
For this interview, design a payment-risk system that handles 10 billion transactions per day, reaches 120,000 transactions per second at peak, and returns a decision within 100 ms p99. Fraud affects about 1% of transactions. The central invariant is: every decision must be reproducible from the model, features, rules, and thresholds used at that moment.
Critical path
Spend latency deliberately
Authenticate, load fresh risk signals, score, apply policy, and record the decision before the payment deadline.
Business objective
Price both errors
A missed fraud loses money; a false block rejects a real customer and can cost the relationship.
Reliability
Degrade predictably
When a feature or model dependency fails, use a bounded, auditable fallback instead of silently allowing or blocking everything.
Define the decision before the model
The service receives a payment attempt and returns a risk score, an action, reason codes, and version lineage. The score is not the final product. A policy combines model output with hard rules, customer context, transaction value, and review capacity.
Functional requirements
- Score card-not-present payments in real time.
- Detect account takeover, stolen credentials, velocity abuse, and coordinated merchant or device attacks.
- Return an action with stable reason codes and the feature, model, rule, and policy versions.
- Accept delayed labels from disputes, chargebacks, investigations, and confirmed customer reports.
- Support a manual-review queue for valuable but ambiguous transactions.
Non-functional requirements
- Keep end-to-end scoring below 100 ms p99 at 120,000 peak transactions/s.
- Keep the legitimate-transaction false-positive rate below 0.8% on every critical market and payment slice.
- Make requests idempotent so retries cannot create conflicting decisions.
- Isolate payment data, encrypt sensitive fields, and retain only the evidence required for audit and learning.
- Continue making bounded decisions when an online feature or model dependency is unavailable.
Ask questions that change the architecture
- Which action is synchronous? A payment authorization needs an immediate answer; account suspension and network investigation can happen later.
- What does the label mean? A chargeback is delayed and noisy, while confirmed account takeover is stronger but rarer evidence.
- Who owns the threshold? Risk, payments, compliance, and customer support must agree on error costs and override rules.
- Which transactions deserve review? Human capacity should be reserved for high-value uncertainty, not used as an unlimited fallback.
State the initial scope
- Include payment authorization and post-transaction monitoring.
- Defer identity-document verification and anti-money-laundering case management.
- Use global models with market-specific calibration where data supports it.
- Require a human or a separate policy for permanent account actions.
Tuning the fraud scorer
Your model scores every transaction for fraud in real time. Fraud that slips through is expensive to reverse, but blocking a legitimate customer at checkout erodes trust. How do you tune and architect the scoring decision?
Constraint: Missed fraud is costly, but false positives that block real customers hurt trust just as much — protect recall without sacrificing the legitimate-customer experience.