Overview

Full-featured custodial cryptocurrency exchange for Blackcoin ($BLK) with a Central Limit Order Book (CLOB).

Trading Pairs: BLK/USDT, BLK/USDC, BLK/BTC, BLK/LTC
Platform: Web-based (Next.js frontend, NestJS backend)
Custody: Custodial (hot/warm/cold wallet hierarchy)
KYC: None (crypto-only, no fiat pairs)
Stack: TypeScript full-stack, PostgreSQL, Redis

System Architecture

Modular Monolith (NestJS)

┌──────────────────────────────────────────────────────────────────┐
│                      API Gateway (Fastify)                       │
│                REST + WebSocket + Rate Limiting                  │
└──────────────────────────────────────────────────────────────────┘
                              │
        ┌─────────────────────┼──────────────────────┐
        │                     │                      │
        ▼                     ▼                      ▼
┌──────────────┐   ┌────────────────────┐   ┌───────────────┐
│ Auth Module  │   │  Trading Module    │   │ Wallet Module │
│ JWT + RBAC   │   │  Orders + Matches  │   │Deposits/      │
│              │   │                    │   │Withdrawals    │
└──────────────┘   └─────────┬──────────┘   └───────────────┘
                             │
                   ┌─────────▼──────────┐
                   │  Matching Engine   │
                   │ (In-Memory CLOB)   |──── Single-threaded
                   └─────────┬──────────┘    execution queue
                             │
        ┌────────────────────┼────────────────────┐
        │                    │                    │
        ▼                    ▼                    ▼
┌──────────────┐    ┌────────────────┐    ┌────────────────────┐
│ PostgreSQL   │    │     Redis      │    │  Remote Providers  │
│ Persistence  │    │ Cache+Pub/Sub  │    │  ElectrumX + RPC   │
│ + Outbox     │    │ + Snapshots    │    │  + Circuit Breakers│
└──────────────┘    └────────────────┘    └────────────────────┘

Module Boundaries

CRITICAL: Transaction Atomicity & Crash Recovery

The Problem

If the matching engine produces trades but the database write fails, in-memory state diverges from persistent state. This is the #1 cause of exchange hacks and balance discrepancies.

Solution: Outbox Pattern + Two-Phase Persistence

                     ┌─────────────────────────────────────┐
                     │       Order Entry (Phase 1)         │
                     │  1. Validate order                  │
                     │  2. Lock balance (available→locked) │
                     │  3. Insert order (status: pending)  │
                     │  4. COMMIT transaction              │
                     └─────────────┬───────────────────────┘
                                   │
                                   ▼
                     ┌─────────────────────────────────────┐
                     │    Matching Engine (Phase 2)        │
                     │  5. Run matchOrder() in memory      │
                     │  6. Produce trades[] + bookUpdates[]│
                     │  7. Assign sequence number          │
                     └─────────────┬───────────────────────┘
                                   │
                                   ▼
                     ┌─────────────────────────────────────┐
                     │    Persist Results (Phase 3)        │
                     │  8. BEGIN transaction               │
                     │  9. Insert all trades               │
                     │ 10. Update all orders (filled, etc) │
                     │ 11. Update balances (locked→avail)  │
                     │ 12. Collect fees to platform account│
                     │ 13. Write to outbox for WS broadcast│
                     │ 14. Update order book snapshot in DB│
                     │ 15. COMMIT transaction              │
                     └─────────────┬───────────────────────┘
                                   │
                                   ▼
                     ┌─────────────────────────────────────┐
                     │    Post-Commit (Phase 4)            │
                     │ 16. Update in-memory order book     │
                     │ 17. Publish to Redis (WS broadcast) │
                     │ 18. Return order result to user     │
                     └─────────────────────────────────────┘

Key Rules:

1. In-memory order book is only updated AFTER successful DB commit. If Phase 3 fails, the order stays in pending status and the in-memory book never changes.
2. Outbox table ensures WebSocket events are eventually delivered even if Redis Pub/Sub fails.
3. If Phase 3 partially fails, a reconciliation job (runs every 30s) detects pending orders older than 10 seconds and re-processes them.
4. Balance updates use SELECT FOR UPDATE within the Phase 3 transaction to prevent concurrent modification.

Outbox Table

CREATE TABLE outbox_events (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  event_type VARCHAR(50) NOT NULL,  -- 'trade', 'order_update', 'balance_update'
  market VARCHAR(20),
  payload JSONB NOT NULL,
  sequence BIGINT NOT NULL,
  published BOOLEAN NOT NULL DEFAULT false,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  published_at TIMESTAMPTZ
);
CREATE INDEX idx_outbox_unpublished ON outbox_events(published) WHERE published = false;

A background worker polls published = false events and publishes them to Redis/WebSocket. This guarantees no events are lost even on crash.

Order State Machine

                    ┌───────────┐
          place     │  pending  │ ← Initial state after validation
         ──────────►│           │
                    └─────┬─────┘
                          │ matching engine picks up
                          ▼
                    ┌───────────┐
                    │   open    │ ← Fully resting on order book
                    └─────┬─────┘
                          │ partial fill
                          ▼
                    ┌───────────┐
                    │ partial   │ ← Some quantity filled, rest on book
                    └─────┬─────┘
                          │ full fill
                          ▼
                    ┌──────────┐
                    │  filled  │ ← Terminal state
                    └──────────┘

Transitions also available from any active state:
  open/partial → cancelled (user cancels)
  open/partial → expired (GTD order expires, or IOC/FOK with no match)
  market order with no match → cancelled (IOC semantics)

Crash Recovery Procedure

1. On startup, load last order book snapshot from PostgreSQL (orderbook_snapshots table)
2. Replay all trades with sequence > snapshot_sequence from the trades table
3. Rebuild in-memory order book from: snapshot + replayed trades
4. For any pending orders (stuck from crash), re-submit to matching engine
5. Verify balances match: SUM(available + locked) per asset across all users = expected total
6. If reconciliation fails, halt trading and alert admins

Snapshot frequency: Every 1000th sequence number, or every 60 seconds (whichever comes first).

Recovery Time Objective (RTO): < 30 seconds
Recovery Point Objective (RPO): 0 trades lost (every trade persisted before in-memory update)

CRITICAL: Wallet Rebalancing & Monitoring

Hot/Warm/Cold Wallet Architecture

Rebalancing Automation

// WalletMonitor — runs every 60 seconds
interface WalletMonitor {
  // Check hot wallet balance for each asset
  checkHotWalletBalances(): Promise<void>;

  // If hot wallet < HOT_WALLET_MIN (e.g., $50k equivalent),
  // initiate warm→hot transfer
  initiateWarmToHotTransfer(asset: string, amount: bigint): Promise<void>;

  // If warm wallet < WARM_WALLET_MIN (e.g., $100k equivalent),
  // alert admins for cold→warm manual transfer
  alertColdToWarmNeeded(asset: string): Promise<void>;
}

Rebalancing Rules by Asset:

Warm→Hot Transfer Process:

1. WalletMonitor detects hot wallet below minimum
2. Creates internal transfer request (status: pending_transfer)
3. Warm wallet signs and broadcasts transaction (via Vault Transit engine) to hot wallet address
4. Monitor confirms transaction reaches required confirmations
5. Hot wallet balance updated, transfer marked completed
6. All transfers logged in wallet_transfers table (immutable audit trail)

Cold→Warm Transfer Process:

1. WalletMonitor detects warm wallet below minimum
2. Alert sent to admins via Slack + email: "Cold→Warm transfer needed for {asset}"
3. Admin initiates manual multisig signing ceremony
4. 3-of-5 key holders sign transaction on air-gapped machine
5. Signed transaction broadcast to network
6. Warm wallet balance updated after confirmations

Wallet Monitoring & Alerting

// Alerts for wallet operations:
// - Hot wallet balance below minimum → auto-transfer from warm
// - Warm wallet balance below minimum → alert admins
// - Warm→hot transfer stuck (not confirmed in 30 min) → alert admins
// - Hot wallet balance > 200% of minimum → alert (possible security issue)
// - Any withdrawal > $5k → Slack notification to admins
// - Provider RPC failure → PagerDuty alert
// - Deposit confirmation anomaly → email alert

Wallet Transfers Table

CREATE TABLE wallet_transfers (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  asset VARCHAR(10) NOT NULL,
  from_wallet VARCHAR(20) NOT NULL CHECK (from_wallet IN ('hot', 'warm', 'cold')),
  to_wallet VARCHAR(20) NOT NULL CHECK (to_wallet IN ('hot', 'warm', 'cold')),
  amount NUMERIC(36,18) NOT NULL,
  txid VARCHAR(255),
  status VARCHAR(20) NOT NULL DEFAULT 'pending'
    CHECK (status IN ('pending', 'confirming', 'completed', 'failed')),
  initiated_by VARCHAR(20) NOT NULL CHECK (initiated_by IN ('auto', 'admin')),
  created_at TIMESTAMPTZ DEFAULT NOW(),
  completed_at TIMESTAMPTZ
);

CRITICAL: Provider Failure Handling

Circuit Breaker Pattern

interface ProviderCircuitBreaker {
  // States: CLOSED (healthy), OPEN (failing), HALF_OPEN (testing recovery)
  state: 'closed' | 'open' | 'half_open';
  
  // Failure threshold: 5 consecutive failures → OPEN
  failureThreshold: 5;
  
  // Recovery timeout: 30 seconds in OPEN before HALF_OPEN
  recoveryTimeout: 30000; // ms
  
  // Success threshold: 3 consecutive successes in HALF_OPEN → CLOSED
  successThreshold: 3;
}

Circuit Breaker per provider:

• blk-electrumx: CLOSED → health check every 30s
• btc-electrumx: CLOSED → health check every 30s
• ltc-electrumx: CLOSED → health check every 30s
• alchemy/infura: CLOSED → health check every 30s
• trongrid: CLOSED → health check every 30s

Deposit Scanner Resilience

interface DepositScanner {
  // Each chain gets its own scanner instance
  scanChain(chain: 'BLK' | 'BTC' | 'LTC' | 'ETH' | 'BSC' | 'TRON'): Promise<void>;
  
  // Last scanned block stored in DB (not just memory)
  getLastScannedBlock(chain: string): Promise<string>; // block hash
  
  // On provider failure:
  // 1. Circuit breaker opens
  // 2. Scanner pauses (no deposits missed — will resume from last block)
  // 3. Alert sent to monitoring
  // 4. On recovery, scanner resumes from last scanned block
  // 5. If gap detected (blocks missed), rescan from last known good block
  
  // Duplicate deposit prevention:
  // - txid UNIQUE constraint on deposits table
  // - Scanner is idempotent — safe to rescan same block range
}

Provider Health Monitoring

interface ProviderHealth {
  // Health check every 30 seconds per provider
  checkHealth(): Promise<{
    provider: string;
    connected: boolean;
    blockHeight: number;
    responseTime: number; // ms
    lastBlockTime: Date;
  }>;
  
  // Store health metrics in DB
  recordHealthMetrics(metrics: ProviderHealth): Promise<void>;
  
  // Alert triggers:
  // - connected: false → PagerDuty critical
  // - responseTime > 5000ms → Slack warning
  // - lastBlockTime > 30 min ago → Slack warning (chain stalled)
}

Withdrawal Failure Handling

Withdrawal Flow with Failure Recovery:

1. User requests withdrawal → DB (status: pending)
2. Validate balance, limits, address format
3. Check hot wallet balance → sufficient? proceed : queue warm→hot transfer first
4. Create raw transaction via provider RPC
   → RPC succeeds: sign and broadcast (status: processing)
   → RPC fails: circuit breaker check
     → If OPEN: queue for retry (status: queued), alert monitoring
     → If CLOSED: retry up to 3 times with exponential backoff
       → All retries fail: (status: failed), notify user, unlock balance
5. Monitor transaction confirmations
   → If TX stuck in mempool > 24h: alert admins, consider CPFP
   → If TX rejected by network: (status: failed), return funds to user
6. Required confirmations reached → (status: completed)

Matching Engine Design

Core Principles

1. Single-threaded, deterministic — No concurrency, no locks, no race conditions
2. Price-time priority — Best price first, then FIFO within price level
3. Pure function — No I/O, takes order book + incoming order, returns trades[] + bookUpdates[]
4. bigint for all monetary values — No floating point anywhere
5. Outbox-then-state — DB persistence before in-memory mutation

Order Types

Order Validation Rules

Data Structures

interface OrderBook {
  bids: Map<string, PriceLevel>;   // price key → orders at that price
  asks: Map<string, PriceLevel>;   // price key → orders at that price
  bidsByPrice: SortedSet;          // descending price order for fast best-bid lookup
  asksByPrice: SortedSet;           // ascending price order for fast best-ask lookup
  ordersById: Map<string, Order>;  // O(1) order lookup for cancels
  sequence: bigint;                 // Monotonically increasing, persisted in DB
}

interface PriceLevel {
  orders: Order[];                 // FIFO queue (time priority)
  totalVolume: bigint;
}

interface Order {
  id: string;                     // UUID
  userId: string;
  market: string;                  // e.g. "BLK/USDT"
  side: 'buy' | 'sell';
  type: 'limit' | 'market' | 'stop' | 'stop_limit' | 'post_only' | 'fok' | 'ioc';
  price: bigint | null;            // null for market orders
  stopPrice: bigint | null;        // for stop/stop_limit orders
  quantity: bigint;
  filled: bigint;                  // Running total of filled quantity
  fee: bigint;                     // Accumulated fee
  timestamp: number;               // Nanosecond precision (process.hrtime.bigint())
  status: OrderStatus;
  timeInForce: 'gtc' | 'ioc' | 'fok';  // Good-till-cancelled, immediate-or-cancel, fill-or-kill
  expiresAt: Date | null;          // For GTD (good-till-date) orders
}

type OrderStatus = 'pending' | 'open' | 'partial' | 'filled' | 'cancelled' | 'expired' | 'triggered';

interface Trade {
  id: string;
  market: string;
  makerOrderId: string;
  takerOrderId: string;
  makerUserId: string;
  takerUserId: string;
  price: bigint;
  quantity: bigint;
  makerFee: bigint;
  takerFee: bigint;
  side: 'buy' | 'sell';           // Taker side
  sequence: bigint;                // Matching engine sequence number
  timestamp: Date;
}

Matching Algorithm

1. Incoming order enters the matching engine via an in-process queue
2. Engine matches against opposite side using price-time priority
3. For buy orders: match against lowest asks first (asksByPrice ascending)
4. For sell orders: match against highest bids first (bidsByPrice descending)
5. Within a price level, match against oldest orders first (FIFO)
6. If incoming limit order has remaining quantity, add to order book as maker
7. Market orders with no match are cancelled (IOC semantics)
8. Post-only orders that would cross the spread are immediately cancelled
9. FOK orders that can't fully fill are immediately cancelled
10. IOC orders fill as much as possible, remainder cancelled
11. All trades are emitted as events for persistence and WebSocket broadcast
12. Each matching event gets a monotonically increasing sequence number

Stop Order Processing

Stop orders are NOT on the order book. They sit in a separate trigger queue:

interface StopOrderQueue {
  // Monitor mark price after each trade
  checkTriggers(markPrice: bigint, market: string): StopOrder[];
  
  // When mark price crosses stop_price:
  // - For stop orders: create market order
  // - For stop_limit orders: create limit order at specified price
  // - Remove triggered stop from queue
  // - Triggered order enters normal matching engine flow
}

Fee Model

• Maker fee: 0.10% (configurable per market via markets table)
• Taker fee: 0.20% (configurable per market via markets table)
• Fees collected in quote currency (USDT, USDC, BTC, LTC) to exchange platform account
• Fee calculation: fee = (price * quantity * feeRate) / 10000n (integer arithmetic, no float)
• Platform collects fees to a system account tracked in fee_revenue table
• Volume-based discounts: users trading >$1M/month get 25% fee reduction (configured in user_tiers table)

Order Processing Pipeline (Revised)

Phase 1: ORDER ENTRY (synchronous, REST thread)
  1. Validate order (size, price deviation, balance check)
  2. Lock user balance (available → locked) in DB transaction
  3. Insert order into DB (status: pending)
  4. Return order ID to user (HTTP 201 Created)

Phase 2: MATCHING (single-threaded queue)
  5. Dequeue pending order
  6. Run matchOrder() — produce trades[] and bookUpdates[]
  7. Assign sequence number

Phase 3: PERSIST (single DB transaction)
  8. BEGIN TRANSACTION
  9. Insert all trades into trades table
  10. Update all order statuses/filled quantities
  11. Update user balances (locked → available for counterparties, collect fees)
  12. Write to outbox_events for WS broadcast
  13. Update orderbook_snapshots (every 1000th sequence)
  14. COMMIT TRANSACTION
  15. If COMMIT fails → order status stays pending → reconciliation job will re-process

Phase 4: POST-COMMIT (async)
  16. Update in-memory order book (ONLY after successful commit)
  17. Publish outbox events to Redis → WebSocket broadcast
  18. Check stop order triggers against new mark price
  19. Return result to user if still waiting

Price Oracle (Mark Price)

Problem

Stop-loss orders, price deviation checks, and slippage protection all depend on a reliable "mark price." For BLK, there's no major external exchange to reference.

Solution: Internal Mark Price with Circuit Breakers

interface PriceOracle {
  // Primary: weighted volume average of last N trades on our exchange
  // Fallback: last trade price if volume too low
  // Circuit breaker: halt trading if mark price moves >30% in 5 minutes
  
  getMarkPrice(market: string): Promise<bigint>;
  
  // Sources (in priority order):
  // 1. Volume-weighted average price (VWAP) of last 100 trades on our exchange
  // 2. If our volume < 10 trades in last hour: use last trade price
  // 3. If extreme volatility (>30% in 5 min): trigger circuit breaker
  //    - Pause trading for 5 minutes
  //    - Alert admins
  //    - Resume with widened price deviation bounds (20% instead of 10%)
}

// Market configuration stored in DB:
interface MarketConfig {
  id: string;                    // e.g. "BLK/USDT"
  baseAsset: string;             // "BLK"
  quoteAsset: string;            // "USDT"
  pricePrecision: number;        // decimal places for price
  quantityPrecision: number;     // decimal places for quantity
  tickSize: bigint;              // minimum price increment (e.g., 10000 satoshis = $0.0001)
  minOrderSize: bigint;          // minimum order quantity
  maxOrderSize: bigint;          // maximum order quantity
  makerFee: bigint;              // maker fee rate (e.g., 10 = 0.10%)
  takerFee: bigint;              // taker fee rate (e.g., 20 = 0.20%)
  priceDeviationPercent: number; // max % from mark price (10)
  circuitBreakerPercent: number;  // max % move in 5 min (30)
  minConfirmations: Record<string, number>; // per asset
}

Blockchain Integration

Provider Configuration

Chain Client Abstraction (with Circuit Breaker)

All three UTXO chains (BLK/BTC/LTC) connect to remote ElectrumX servers — no self-hosted daemons. They share a common ChainClient interface so that the deposit scanner and withdrawal processor never know which protocol backend they're talking to:

abstract class ChainClient {
  protected circuitBreaker: CircuitBreaker;
  protected healthCheckInterval: NodeJS.Timer;

  abstract getBlockHeight(): Promise<number>;
  abstract getHistory(scriptHash: string): Promise<TxHistoryEntry[]>;
  abstract listUnspent(scriptHash: string): Promise<UTXO[]>;
  abstract broadcastTx(rawTx: string): Promise<string>;
  abstract estimateFee(targetBlocks: number): Promise<number>;
  abstract subscribeAddress(scriptHash: string): Promise<void>;

  // Circuit breaker wraps every call:
  // - CLOSED: pass through
  // - OPEN: throw CircuitBreakerOpenError immediately
  // - HALF_OPEN: allow one test request, if success → CLOSE
}

class ElectrumxClient extends ChainClient {
  // Uses electrum-client library over TCP
  // Supports: blockchain.scripthash.*, blockchain.transaction.*, blockchain.estimatefee
  // BLK network params: pubKeyHash=0x19, scriptHash=0x55, wif=0x99
}

class EVMProvider extends ChainClient {
  // Uses ethers.js for Ethereum + BSC
  // Alchemy primary, Infura fallback, QuickNode optional
  // ERC-20 Transfer event scanning
}

class TronProvider extends ChainClient {
  // Uses TronWeb (NOT ethers.js — Tron is NOT EVM)
  // TronGrid primary provider
  // Bandwidth + Energy gas model (different from EIP-1559)
  // Separate scanner module: deposit-scan-tron
}

Deposit Monitoring

UTXO Chains (BLK, BTC, LTC):

1. Scanner runs every 30 seconds per chain (configurable)
2. Queries ElectrumX via blockchain.scripthash.get_history for all tracked deposit addresses (with circuit breaker)
3. If provider unavailable → scanner pauses, resumes from lastBlockHash when provider recovers
4. Filter for incoming transactions to known deposit addresses
5. Track confirmations — credit deposit after required confirmations (6 for BLK/BTC/LTC, configurable per market)
6. Last scanned block hash stored in scanner_state table (survives restarts)

EVM Chains (USDT, USDC on Ethereum, BSC):

1. Use Alchemy/Infura as RPC provider (with circuit breaker and fallback provider)
2. Monitor ERC-20 Transfer events (Transfer(address from, address to, uint256 value))
3. Filter events where to is an exchange deposit address
4. Credit after required confirmations: 12 blocks (ETH), 30 blocks (BSC)
5. Support multi-chain: same deposit address derived from HD wallet per chain
6. USDT/USDC contract addresses configurable in system_config table

Tron Chain:

1. Tron is NOT EVM — uses separate TronWeb library and TronGrid provider
2. Dedicated deposit-scan-tron scanner job (separate from EVM scanner)
3. Queries TronGrid for USDT/USDC Transfer events to tracked addresses
4. Credit after 19 confirmations (Tron block speed: ~3s)
5. Address format: Base58 check with T prefix (34 chars) — validated with tronweb.isAddress()
6. Gas model: Bandwidth + Energy (free daily quota, stake for more) — not EIP-1559

Chain Reorg Handling

interface ReorgDetector {
  // On each scan, for each newly-confirmed block:
  // 1. Verify chain continuity (parent hash matches previous best hash)
  // 2. If parent hash doesn't match → reorg detected
  
  // On reorg detection:
  // 1. Find the common ancestor block
  // 2. Identify all deposits credited from reorged-out blocks
  // 3. Mark those deposits as status: 'reorged'
  // 4. Debit user balances for reorged deposits (reverse the credit)
  // 5. Log reorg event in reorg_events table
  // 6. Notify affected users via WebSocket
  // 7. Re-scan from common ancestor forward
}

CREATE TABLE reorg_events (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  chain VARCHAR(10) NOT NULL,
  depth INTEGER NOT NULL,           -- How many blocks were reorged
  old_tip VARCHAR(255) NOT NULL,    -- Previous best block hash
  new_tip VARCHAR(255) NOT NULL,    -- New best block hash
  common_ancestor VARCHAR(255) NOT NULL,
  affected_deposits UUID[] NOT NULL, -- Deposit IDs that were reversed
  created_at TIMESTAMPTZ DEFAULT NOW()
);

Withdrawal Processing (with UTXO Management)

1. User requests withdrawal → DB (status: pending)
2. Validate: sufficient balance, daily limit, address format check
3. Check circuit breaker → provider available? proceed : queue withdrawal
4. Below hot wallet threshold? → auto-process
   Above threshold? → queue for admin approval
5. UTXO Selection (coin selection algorithm):
   a. Gather all UTXOs from hot wallet via ElectrumX (blockchain.scripthash.listunspent)
   b. Select UTXOs using branch-and-bound algorithm (minimize fee):
      - Prefer UTXOs closest to withdrawal amount
      - Avoid creating dust change outputs
      - If no optimal selection, use all available UTXOs
   c. Calculate fee: blockchain.estimatefee × estimated vsize
   d. Create raw transaction with selected UTXOs as inputs (bitcoinjs-lib)
   e. Add change output back to hot wallet (new address for privacy)
6. Sign transaction via Vault Transit engine (app never sees raw key)
7. Broadcast via ElectrumX (blockchain.transaction.broadcast) with retry logic (3 attempts, exponential backoff)
8. Monitor transaction confirmations (6 required)
9. Update withdrawal status: completed
10. Move change UTXOs to tracking for future use

EVM/Tron Withdrawals:

1. EVM (ETH/BSC): Build via ethers.js, sign with Vault-stored key, broadcast via Alchemy/Infura
2. Tron: Build via TronWeb, sign with Vault-stored key, broadcast via TronGrid; gas uses Bandwidth + Energy model

UTXO Consolidation (background job, runs daily during low-activity hours):

1. Find all dust UTXOs (< 10% of average transaction size)
2. Create consolidation transaction: many small UTXOs → one large UTXO
3. Sign and broadcast consolidation transaction
4. Wait for confirmation before using consolidated UTXO

Fee Estimation:

interface FeeEstimator {
  // Estimate withdrawal fee based on current network conditions
  estimateFee(asset: 'BLK' | 'BTC' | 'LTC' | 'ETH' | 'BSC' | 'TRON', confTarget: number): Promise<bigint>;
  
  // UTXO chains: ElectrumX blockchain.estimatefee, fallback to fixed floor
  // EVM chains: ethers.js provider.estimateGas(), Alchemy gas oracle
  // Tron: TronWeb transactionBuilder.estimateEnergy()
  // Minimum fee floor per asset (configurable in system_config)
}

Deposit Address Generation:

• BLK/BTC/LTC: BIP32/BIP44 HD wallet with unique derivation per user
  • BLK: m/44'/10'/<userId>'/0/0 (coin type 10 for BLK, pubKeyHash: 0x19)
  • BTC: m/44'/0'/<userId>'/0/0
  • LTC: m/44'/2'/<userId>'/0/0
  • One address per user per asset (reuse is fine for deposit tracking)
  • New address generated on each deposit request for privacy (gap limit: 20)
• USDT/USDC on Ethereum/BSC: Single address per user per chain via HD derivation on EVM
  • Ethereum path: m/44'/60'/<userId>'/0/0
  • BSC path: m/44'/60'/<userId>'/0/1 (same coin type as ETH, different account index)
• USDT/USDC on Tron: Tron addresses derived from same HD seed using TronWeb
  • Tron is NOT EVM — separate address format (Base58 check, T prefix, 34 chars)
  • Path: m/44'/195'/<userId>'/0/0 (coin type 195 for Tron)

Deposit Address Policy

Address reuse: Each request generates a new address, but old addresses remain functional. The system watches all derived addresses up to the gap limit (20). This provides privacy while ensuring no deposits are missed.

Database Schema (Complete)

-- Markets configuration
CREATE TABLE markets (
  id VARCHAR(20) PRIMARY KEY,            -- e.g. "BLK/USDT"
  base_asset VARCHAR(10) NOT NULL,        -- "BLK"
  quote_asset VARCHAR(10) NOT NULL,       -- "USDT"
  price_precision INTEGER NOT NULL DEFAULT 8, -- decimal places for price
  quantity_precision INTEGER NOT NULL DEFAULT 8, -- decimal places for quantity
  tick_size NUMERIC(36,18) NOT NULL,      -- minimum price increment (in quote asset units)
  min_order_size NUMERIC(36,18) NOT NULL, -- minimum order quantity (in base asset)
  max_order_size NUMERIC(36,18) NOT NULL,  -- maximum order quantity
  maker_fee_bps INTEGER NOT NULL DEFAULT 10,  -- maker fee in basis points (10 = 0.10%)
  taker_fee_bps INTEGER NOT NULL DEFAULT 20,  -- taker fee in basis points (20 = 0.20%)
  price_deviation_percent INTEGER NOT NULL DEFAULT 10, -- max % from mark price
  circuit_breaker_percent INTEGER NOT NULL DEFAULT 30, -- max % move in 5 min
  status VARCHAR(20) NOT NULL DEFAULT 'active'
    CHECK (status IN ('upcoming', 'active', 'paused', 'delisting', 'delisted')),
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Seed data:
-- INSERT INTO markets VALUES ('BLK/USDT', 'BLK', 'USDT', 8, 8, 0.0001, 10, 10000000, 10, 20, 10, 30, 'active', NOW());
-- INSERT INTO markets VALUES ('BLK/USDC', 'BLK', 'USDC', 8, 8, 0.0001, 10, 10000000, 10, 20, 10, 30, 'active', NOW());
-- INSERT INTO markets VALUES ('BLK/BTC',  'BLK', 'BTC',  8, 8, 0.00000001, 1, 1000000, 10, 20, 10, 30, 'active', NOW());
-- INSERT INTO markets VALUES ('BLK/LTC',  'BLK', 'LTC',  8, 8, 0.000001, 1, 10000000, 10, 20, 10, 30, 'active', NOW());

-- Users
CREATE TABLE users (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  email VARCHAR(255) UNIQUE NOT NULL,
  password_hash VARCHAR(255) NOT NULL,
  two_factor_secret VARCHAR(255),
  role VARCHAR(20) NOT NULL DEFAULT 'user' CHECK (role IN ('user', 'admin', 'superadmin')),
  tier VARCHAR(20) NOT NULL DEFAULT 'basic' REFERENCES user_tiers(tier),
  is_active BOOLEAN NOT NULL DEFAULT true,
  withdrawal_locked_until TIMESTAMPTZ,      -- 24h lock after password change
  created_at TIMESTAMPTZ DEFAULT NOW(),
  last_login_at TIMESTAMPTZ,
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

-- User tiers (fee discounts)
CREATE TABLE user_tiers (
  tier VARCHAR(20) PRIMARY KEY,
  name VARCHAR(50) NOT NULL,
  maker_fee_discount_bps INTEGER NOT NULL DEFAULT 0,  -- basis points discount
  taker_fee_discount_bps INTEGER NOT NULL DEFAULT 0,
  daily_withdrawal_limit NUMERIC(36,18) NOT NULL,
  monthly_withdrawal_limit NUMERIC(36,18) NOT NULL,
  max_open_orders INTEGER NOT NULL DEFAULT 200
);
-- Seed: basic(0,0,$10k,$100k,200), silver(5,5,$50k,$500k,500), gold(10,10,$100k,$1M,1000), platinum(25,25,$500k,$5M,2000)

-- User balances (ACID: one row per user-asset pair)
-- Pessimistic locking only: all updates use SELECT FOR UPDATE (no version column)
-- subaccount_id: NULL = main account, non-NULL = subaccount balance
CREATE TABLE balances (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  asset VARCHAR(10) NOT NULL,
  subaccount_id UUID REFERENCES subaccounts(id),  -- NULL = main account
  available NUMERIC(36,18) NOT NULL DEFAULT 0,
  locked NUMERIC(36,18) NOT NULL DEFAULT 0,  -- Locked in open orders
  updated_at TIMESTAMPTZ DEFAULT NOW(),
  CONSTRAINT positive_balance CHECK (available >= 0 AND locked >= 0)
);
CREATE UNIQUE INDEX idx_balances_user_asset_sub ON balances (user_id, asset, COALESCE(subaccount_id, '00000000-0000-0000-0000-000000000000'));

-- Exchange platform account (where fees accumulate)
-- This is a special user with role='platform' or a separate table
CREATE TABLE platform_accounts (
  asset VARCHAR(10) PRIMARY KEY,
  balance NUMERIC(36,18) NOT NULL DEFAULT 0,  -- Accumulated fees
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

-- Orders
CREATE TABLE orders (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  market VARCHAR(20) REFERENCES markets(id) NOT NULL,
  side VARCHAR(4) NOT NULL CHECK (side IN ('buy', 'sell')),
  type VARCHAR(15) NOT NULL CHECK (type IN ('limit', 'market', 'stop', 'stop_limit', 'post_only', 'fok', 'ioc')),
  price NUMERIC(36,18),           -- NULL for market orders
  stop_price NUMERIC(36,18),     -- For stop/stop_limit orders
  quantity NUMERIC(36,18) NOT NULL,
  filled NUMERIC(36,18) NOT NULL DEFAULT 0,
  fee NUMERIC(36,18) NOT NULL DEFAULT 0,
  time_in_force VARCHAR(10) NOT NULL DEFAULT 'gtc' CHECK (time_in_force IN ('gtc', 'ioc', 'fok', 'gtd')),
  status VARCHAR(15) NOT NULL DEFAULT 'pending'
    CHECK (status IN ('pending', 'open', 'partial', 'filled', 'cancelled', 'expired', 'triggered')),
  expires_at TIMESTAMPTZ,        -- For GTD orders
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW(),
  cancelled_at TIMESTAMPTZ
);
CREATE INDEX idx_orders_user ON orders(user_id);
CREATE INDEX idx_orders_market_status ON orders(market, status);
CREATE INDEX idx_orders_expires ON orders(expires_at) WHERE status = 'open';

-- Trades (immutable — never updated, only inserted)
CREATE TABLE trades (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  market VARCHAR(20) NOT NULL,
  maker_order_id UUID REFERENCES orders(id),
  taker_order_id UUID REFERENCES orders(id),
  maker_user_id UUID REFERENCES users(id),
  taker_user_id UUID REFERENCES users(id),
  side VARCHAR(4) NOT NULL,       -- Taker side
  price NUMERIC(36,18) NOT NULL,
  quantity NUMERIC(36,18) NOT NULL,
  maker_fee NUMERIC(36,18) NOT NULL,
  taker_fee NUMERIC(36,18) NOT NULL,
  sequence BIGINT NOT NULL UNIQUE, -- Monotonically increasing
  executed_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_trades_market ON trades(market, executed_at DESC);
CREATE INDEX idx_trades_maker ON trades(maker_user_id);
CREATE INDEX idx_trades_taker ON trades(taker_user_id);

-- Deposits
CREATE TABLE deposits (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  asset VARCHAR(10) NOT NULL,
  address VARCHAR(255) NOT NULL,
  txid VARCHAR(255) UNIQUE,
  amount NUMERIC(36,18) NOT NULL,
  confirmations INTEGER NOT NULL DEFAULT 0,
  required_confirmations INTEGER NOT NULL DEFAULT 6,
  status VARCHAR(20) NOT NULL DEFAULT 'pending'
    CHECK (status IN ('pending', 'confirming', 'credited', 'failed', 'reorged')),
  created_at TIMESTAMPTZ DEFAULT NOW(),
  credited_at TIMESTAMPTZ
);
CREATE INDEX idx_deposits_txid ON deposits(txid);
CREATE INDEX idx_deposits_user ON deposits(user_id, status);

-- Withdrawals
CREATE TABLE withdrawals (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  asset VARCHAR(10) NOT NULL,
  address VARCHAR(255) NOT NULL,
  amount NUMERIC(36,18) NOT NULL,
  fee NUMERIC(36,18) NOT NULL,
  txid VARCHAR(255) UNIQUE,
  status VARCHAR(20) NOT NULL DEFAULT 'pending'
    CHECK (status IN ('pending', 'approved', 'processing', 'completed', 'failed', 'rejected', 'queued')),
  requires_approval BOOLEAN DEFAULT false,
  approved_by UUID REFERENCES users(id),
  approved_at TIMESTAMPTZ,
  failure_reason TEXT,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  completed_at TIMESTAMPTZ
);
CREATE INDEX idx_withdrawals_status ON withdrawals(status);
CREATE INDEX idx_withdrawals_user ON withdrawals(user_id);

-- Deposit addresses (HD wallet derivation)
CREATE TABLE deposit_addresses (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  asset VARCHAR(10) NOT NULL,
  chain VARCHAR(20) NOT NULL DEFAULT 'native',  -- native, ethereum, tron, bsc
  address VARCHAR(255) UNIQUE NOT NULL,
  derivation_path VARCHAR(255),
  derivation_index INTEGER,
  is_active BOOLEAN NOT NULL DEFAULT true,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  UNIQUE (user_id, asset, chain)
);

-- Order book snapshots (for crash recovery)
CREATE TABLE orderbook_snapshots (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  market VARCHAR(20) NOT NULL,
  sequence BIGINT NOT NULL,
  snapshot JSONB NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_snapshots_market_seq ON orderbook_snapshots(market, sequence DESC);

-- Outbox events (for reliable WebSocket broadcast)
CREATE TABLE outbox_events (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  event_type VARCHAR(50) NOT NULL,  -- 'trade', 'order_update', 'balance_update', 'orderbook_delta'
  market VARCHAR(20),
  payload JSONB NOT NULL,
  sequence BIGINT NOT NULL,
  published BOOLEAN NOT NULL DEFAULT false,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  published_at TIMESTAMPTZ
);
CREATE INDEX idx_outbox_unpublished ON outbox_events(published) WHERE published = false;

-- Wallet transfers (hot↔warm↔cold movement audit trail)
CREATE TABLE wallet_transfers (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  asset VARCHAR(10) NOT NULL,
  from_wallet VARCHAR(20) NOT NULL CHECK (from_wallet IN ('hot', 'warm', 'cold')),
  to_wallet VARCHAR(20) NOT NULL CHECK (to_wallet IN ('hot', 'warm', 'cold')),
  amount NUMERIC(36,18) NOT NULL,
  txid VARCHAR(255),
  status VARCHAR(20) NOT NULL DEFAULT 'pending' 
    CHECK (status IN ('pending', 'confirming', 'completed', 'failed')),
  initiated_by VARCHAR(20) NOT NULL CHECK (initiated_by IN ('auto', 'admin')),
  created_at TIMESTAMPTZ DEFAULT NOW(),
  completed_at TIMESTAMPTZ
);

-- Chain reorg events
CREATE TABLE reorg_events (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  chain VARCHAR(10) NOT NULL,
  depth INTEGER NOT NULL,
  old_tip VARCHAR(255) NOT NULL,
  new_tip VARCHAR(255) NOT NULL,
  common_ancestor VARCHAR(255) NOT NULL,
  affected_deposits UUID[] NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Scanner state (last processed block per chain)
CREATE TABLE scanner_state (
  chain VARCHAR(20) PRIMARY KEY,   -- 'BLK', 'BTC', 'LTC', 'ETH', 'TRON', 'BSC'
  last_block_hash VARCHAR(255) NOT NULL,
  last_block_height INTEGER NOT NULL,
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

-- System configuration (exchange-wide settings)
CREATE TABLE system_config (
  key VARCHAR(100) PRIMARY KEY,
  value JSONB NOT NULL,
  description TEXT,
  updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- Seed: withdrawal_limits, fee_rates, confirmation_requirements, provider_urls, etc.

-- Provider health metrics
CREATE TABLE provider_health (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  provider VARCHAR(20) NOT NULL,    -- 'blk-electrumx', 'btc-electrumx', 'ltc-electrumx', 'alchemy', 'infura', 'trongrid'
  connected BOOLEAN NOT NULL,
  block_height INTEGER,
  response_time_ms INTEGER,
  last_block_at TIMESTAMPTZ,
  checked_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_provider_health_provider ON provider_health(provider, checked_at DESC);

-- Audit log (immutable, no updates or deletes)
CREATE TABLE audit_log (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID,
  action VARCHAR(50) NOT NULL,
  entity VARCHAR(50) NOT NULL,
  entity_id UUID,
  details JSONB,
  ip_address VARCHAR(45),
  created_at TIMESTAMPTZ DEFAULT NOW()
);
-- NO indexes for updates/deletes — this is append-only

-- Fee revenue tracking
CREATE TABLE fee_revenue (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  market VARCHAR(20) NOT NULL,
  asset VARCHAR(10) NOT NULL,       -- Asset the fee was collected in
  user_id UUID REFERENCES users(id),
  trade_id UUID REFERENCES trades(id),
  fee_type VARCHAR(10) NOT NULL CHECK (fee_type IN ('maker', 'taker')),
  amount NUMERIC(36,18) NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_fee_revenue_market ON fee_revenue(market, created_at DESC);
CREATE INDEX idx_fee_revenue_asset ON fee_revenue(asset, created_at DESC);

-- API keys (for programmatic/bot trader access)
CREATE TABLE api_keys (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  name VARCHAR(100) NOT NULL,
  key_hash VARCHAR(255) NOT NULL,       -- SHA-256 hash of API key (for lookup)
  secret_hash VARCHAR(255) NOT NULL,     -- bcrypt hash of API secret
  permissions JSONB NOT NULL,            -- {"read": true, "trade": true, "withdraw": false}
  ip_whitelist INET[],                   -- NULL = any IP, array = whitelist
  rate_limit INTEGER NOT NULL DEFAULT 600, -- Requests per minute
  last_used_at TIMESTAMPTZ,
  expires_at TIMESTAMPTZ,
  is_active BOOLEAN NOT NULL DEFAULT true,
  created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_api_keys_user ON api_keys(user_id);
CREATE INDEX idx_api_keys_hash ON api_keys(key_hash) WHERE is_active = true;

-- API key usage tracking
CREATE TABLE api_key_usage (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  api_key_id UUID REFERENCES api_keys(id) NOT NULL,
  endpoint VARCHAR(100) NOT NULL,
  method VARCHAR(10) NOT NULL,
  status_code INTEGER NOT NULL,
  response_time_ms INTEGER,
  created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_api_key_usage_key ON api_key_usage(api_key_id, created_at DESC);

-- Withdrawal address whitelist (24h lock after adding)
CREATE TABLE withdrawal_addresses (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  asset VARCHAR(10) NOT NULL,
  chain VARCHAR(20) NOT NULL DEFAULT 'native',
  address VARCHAR(255) NOT NULL,
  label VARCHAR(100),
  is_confirmed BOOLEAN NOT NULL DEFAULT false,
  confirmation_token VARCHAR(255),        -- Email confirmation token
  whitelisted_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  first_withdrawal_allowed_at TIMESTAMPTZ NOT NULL,  -- whitelisted_at + 24h
  last_used_at TIMESTAMPTZ,
  is_active BOOLEAN NOT NULL DEFAULT true,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  UNIQUE (user_id, asset, chain, address)
);
CREATE INDEX idx_withdrawal_addresses_user ON withdrawal_addresses(user_id, asset);

-- Notification preferences per user
CREATE TABLE notification_preferences (
  user_id UUID PRIMARY KEY REFERENCES users(id),
  email_enabled BOOLEAN NOT NULL DEFAULT true,
  push_enabled BOOLEAN NOT NULL DEFAULT true,
  sms_enabled BOOLEAN NOT NULL DEFAULT false,
  in_app_enabled BOOLEAN NOT NULL DEFAULT true,
  security_alerts_email BOOLEAN NOT NULL DEFAULT true,  -- Can't disable
  security_alerts_sms BOOLEAN NOT NULL DEFAULT false,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

-- Notifications (in-app + email/push/sms delivery tracking)
CREATE TABLE notifications (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  type VARCHAR(50) NOT NULL,             -- 'withdrawal_requested', 'deposit_credited', 'login_new_ip', etc.
  channel VARCHAR(20) NOT NULL CHECK (channel IN ('email', 'push', 'sms', 'in_app')),
  title VARCHAR(255) NOT NULL,
  body TEXT,
  data JSONB,                            -- Structured payload (orderId, txid, etc.)
  read BOOLEAN NOT NULL DEFAULT false,
  sent_at TIMESTAMPTZ,
  read_at TIMESTAMPTZ,
  created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_notifications_user ON notifications(user_id, created_at DESC);
CREATE INDEX idx_notifications_unread ON notifications(user_id) WHERE read = false;

-- Refresh token families (for rotation with theft detection)
CREATE TABLE refresh_token_families (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  current_token_hash VARCHAR(255) NOT NULL,  -- Only the latest is valid
  created_at TIMESTAMPTZ DEFAULT NOW(),
  expires_at TIMESTAMPTZ NOT NULL,
  is_revoked BOOLEAN NOT NULL DEFAULT false,
  revoked_reason VARCHAR(50),   -- 'logout', 'reuse_detected', 'password_change', 'admin_action'
  revoked_at TIMESTAMPTZ
);
CREATE INDEX idx_refresh_families_user ON refresh_token_families(user_id);

-- Revoked tokens blacklist (for immediate JWT revocation)
CREATE TABLE revoked_tokens (
  token_hash VARCHAR(255) PRIMARY KEY,  -- SHA-256 hash of token
  user_id UUID REFERENCES users(id) NOT NULL,
  reason VARCHAR(50) NOT NULL,
  expires_at TIMESTAMPTZ NOT NULL,       -- Auto-cleanup after token expiry
  created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_revoked_tokens_expiry ON revoked_tokens(expires_at);

-- Sessions (active user sessions with device tracking)
CREATE TABLE sessions (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  refresh_token_family_id UUID REFERENCES refresh_token_families(id),
  device_info JSONB NOT NULL,       -- {userAgent, browser, os, deviceType}
  ip_address INET NOT NULL,
  last_active_at TIMESTAMPTZ DEFAULT NOW(),
  expires_at TIMESTAMPTZ NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_sessions_user ON sessions(user_id, last_active_at DESC);

-- Candles / K-lines (OHLCV time-series data for charts)
CREATE TABLE candles (
  market VARCHAR(20) NOT NULL,
  interval VARCHAR(5) NOT NULL,    -- '1m', '5m', '15m', '1h', '4h', '1d', '1w'
  open_time TIMESTAMPTZ NOT NULL,
  open NUMERIC(36,18) NOT NULL,
  high NUMERIC(36,18) NOT NULL,
  low NUMERIC(36,18) NOT NULL,
  close NUMERIC(36,18) NOT NULL,
  volume NUMERIC(36,18) NOT NULL,
  quote_volume NUMERIC(36,18) NOT NULL,
  trades INTEGER NOT NULL DEFAULT 0,
  closed BOOLEAN NOT NULL DEFAULT false,
  PRIMARY KEY (market, interval, open_time)
);
CREATE INDEX idx_candles_market_interval ON candles(market, interval, open_time DESC);

-- Withdrawal IP history (for IP-based withdrawal restrictions)
CREATE TABLE withdrawal_ip_history (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  ip_address INET NOT NULL,
  first_used_at TIMESTAMPTZ DEFAULT NOW(),
  last_used_at TIMESTAMPTZ DEFAULT NOW(),
  withdrawal_count INTEGER NOT NULL DEFAULT 1,
  is_whitelisted BOOLEAN NOT NULL DEFAULT false,
  whitelisted_at TIMESTAMPTZ,
  UNIQUE (user_id, ip_address)
);

-- Subaccounts (for institutional users, multiple trading accounts per user)
CREATE TABLE subaccounts (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  name VARCHAR(100) NOT NULL,
  type VARCHAR(20) NOT NULL DEFAULT 'trading' CHECK (type IN ('trading', 'savings', 'defi')),
  is_active BOOLEAN NOT NULL DEFAULT true,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Internal transfers between own subaccounts (no blockchain fee)
CREATE TABLE internal_transfers (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID REFERENCES users(id) NOT NULL,
  from_subaccount_id UUID REFERENCES subaccounts(id),
  to_subaccount_id UUID REFERENCES subaccounts(id),
  asset VARCHAR(10) NOT NULL,
  amount NUMERIC(36,18) NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

Real-Time Architecture (WebSocket)

Channels

ws.subscribe('orderbook:BLK/USDT')    // Order book depth updates
ws.subscribe('trades:BLK/USDT')       // Public trade feed
ws.subscribe('ticker:BLK/USDT')       // 24h ticker stats
ws.subscribe('user:orders')           // User's order updates (authenticated)
ws.subscribe('user:trades')           // User's trade executions (authenticated)
ws.subscribe('user:balances')        // Balance changes (authenticated)
ws.subscribe('user:deposits')         // Deposit status updates (authenticated)
ws.subscribe('user:withdrawals')      // Withdrawal status updates (authenticated)

Sequence Numbers for Consistency

Every matching engine event gets a monotonically increasing sequence number (stored in PostgreSQL trades.sequence and outbox_events.sequence).

Client-side reconciliation:

class OrderBookManager {
  private lastSequence: bigint = 0n;
  
  onConnect(): void {
    // Always request full snapshot on connect
    this.requestFullSnapshot();
  }
  
  onWebSocketUpdate(update: { sequence: bigint; type: string; data: any }): void {
    if (update.sequence <= this.lastSequence) return; // Stale, discard
    if (update.sequence !== this.lastSequence + 1n) {
      // Gap detected — resubscribe with full snapshot request
      this.requestFullSnapshot();
      return;
    }
    this.applyUpdate(update);
    this.lastSequence = update.sequence;
  }
  
  requestFullSnapshot(): void {
    // GET /api/v1/orderbook/:market?sequence=true
    // Response includes current sequence number
    // Client can then safely apply deltas starting from that sequence
  }

  onPlaceOrder(response: OrderResponse): void {
    // Track that our order was placed at sequence X
    // Wait for WS event with sequence >= response.sequence to confirm fills
  }
}

Snapshot API: GET /api/v1/orderbook/:market?depth=20&sequence=true returns full order book snap with current sequence number.

Gap recovery: Client requests GET /api/v1/orderbook/:market/snapshot?since=<lastKnownSequence> which returns missed events from outbox_events table (kept for 24 hours).

Front-Running & Manipulation Defense

Order Flow

1. No order book peeking: REST endpoint GET /api/v1/orderbook/:market returns a snapshot, but with 1-second staleness. Real-time updates come only through authenticated WebSocket (requires login).
2. Rate limiting:
   • 10 orders/second per user (prevents order flooding)
   • 100 requests/minute per user (general API)
   • 10 requests/minute for order book snapshots per IP
3. Price deviation circuit breaker: If mark price moves >30% in 5 minutes, trading halts for that market for 5 minutes. All open orders cancelled. Trading resumes with widened price deviation bounds (20% instead of 10%).
4. Wash trade detection: Flag accounts that consistently trade with themselves (same IP, same deposit address pattern, circular fund flows). Alert admins for manual review.

Batch processing: Orders are batched in 100ms windows. All orders in a batch are processed together in timestamp order. This prevents HFT advantage from faster API connections.

⚠️ Latency impact: An order submitted mid-batch waits until the next batch boundary (~50ms average, 100ms maximum additional latency). The 100ms window is a deliberate trade-off: front-running protection vs. slight latency increase. This is acceptable for non-HFT retail spot markets.

Testnet & Development Environment

Mock Providers for Development

// Mock provider implementations for local development and testing
interface MockProvider {
  chain: string;
  start(): Promise<void>;
  stop(): Promise<void>;
  generateBlocks(n: number): Promise<void>;  // Generate n blocks instantly
  sendToAddress(address: string, amount: number): Promise<string>; // Mock transaction
  getBlockHeight(): number;
  listUnspent(scriptHash: string): UTXO[];
}

• UTXO chains (BLK/BTC/LTC): Mock ElectrumX server or direct unit test mocks (no testnet daemon)
• EVM chains: Sepolia testnet for USDT/USDC on Ethereum, BSC testnet
• Tron: Shasta testnet via TronGrid test endpoint

Docker Compose

# docker-compose.yml (core services: 6 containers + Nginx on host)
services:
  postgres: ...
  pgbouncer: ...
  redis: ...
  vault: ...
  backend: ...
  frontend: ...

No self-hosted blockchain daemons in any profile — all chains connect to remote providers.

Integration Test Fixtures

• Pre-generated HD wallet with funded test addresses
• Mock blockchain with controllable block generation
• Deterministic test scenarios: deposit, confirm, trade, withdraw
• All tests run against mock providers, never requiring real funds

Scaling Path

Current (Single Node)

• Single NestJS instance with matching engine
• PostgreSQL on same or dedicated server
• Redis for caching, pub/sub, snapshots
• Handles 10k+ orders/second on single thread

Level 1 Scale (10k+ concurrent users)

• Multiple API instances behind load balancer
• Matching engine still single-threaded (one per market)
• Redis pub/sub broadcasts matching events to all API instances
• PostgreSQL with read replicas for trade history queries
• WebSocket connections distributed across API instances

Level 2 Scale (100k+ concurrent users)

• Extract matching engine to dedicated microservice per market
• Separate read-heavy APIs (order book, trades) from write-heavy (order placement)
• Redis Cluster for pub/sub and caching
• PostgreSQL partitioning by market for trades table
• CDN for static frontend assets

Level 3 Scale (When Needed)

• Dedicated matching engine server per market
• Market sharding: different markets on different servers
• Kafka for event streaming (replaces Redis pub/sub)
• Separate read replicas for each API type
• This architecture would be overkill for initial launch

Admin Security

Role-Based Access Control

interface AdminRole {
  'superadmin': {
    canApproveWithdrawals: true;
    canManageUsers: true;
    canManageMarkets: true;
    canViewSystemStatus: true;
    canManageWallets: true;    // Initiate cold wallet transfers
    canViewAuditLog: true;
  };
  'admin': {
    canApproveWithdrawals: true;
    canManageUsers: true;
    canManageMarkets: false;
    canViewSystemStatus: true;
    canManageWallets: false;
    canViewAuditLog: true;
  };
  'support': {
    canApproveWithdrawals: false;
    canManageUsers: true;  // View-only, no password changes
    canManageMarkets: false;
    canViewSystemStatus: false;
    canManageWallets: false;
    canViewAuditLog: false;
  };
}

Withdrawal Approval Flow (Multi-Stage for Large Amounts)

Amount < $1k/user/day
  → Auto-process from hot wallet

Amount < $10k
  → Auto-process, requires warm→hot transfer if hot wallet low
  → Logged in audit_log

Amount >= $10k AND < $50k
  → Requires single admin approval (any admin role)
  → 2FA confirmation required from approver
  → 10-minute delay before processing (cancellation window)
  → Auto-process from warm wallet

Amount >= $50k
  → Requires superadmin approval (any superadmin)
  → 2FA confirmation required from approver
  → 30-minute delay before processing
  → May require cold wallet access (3-of-5 multisig)
  → Slack + email notification to all admins

Admin Actions Audit

All admin actions are logged in the immutable audit_log table with:

• Admin user ID
• Action taken (approve_withdrawal, freeze_user, etc.)
• Target entity and entity ID
• Full details as JSONB (before/after state)
• IP address and timestamp
• Cannot be deleted or modified (trigger prevents UPDATE/DELETE on audit_log)

Implementation Phases

Updated to incorporate all identified gaps. Details now live in docs/backend.md (implementation specs) and docs/frontend.md (frontend specs). Phase numbering reorganized to include operational and security requirements alongside core features.

Phase 1: Foundation (Weeks 1-4)

Goal: Matching engine works, orders persist, basic balance tracking, health checks, scheduled jobs, and idempotency.

1. Monorepo scaffolding
   • Turborepo workspace: apps/backend, apps/frontend, packages/shared
   • TypeScript, ESLint, Prettier across all packages
   • Docker Compose for PostgreSQL + Redis
   • .env.example with all configuration
2. Shared types package (packages/shared)
   • All TypeScript types: Order, Trade, Balance, Market, Deposit, Withdrawal
   • Zod validation schemas for all API inputs
   • Constants: market definitions, fee rates, asset configs
   • Order type enum: limit, market, stop, stop_limit, post_only, fok, ioc
3. Database setup (Prisma migrations)
   • ALL tables from complete schema above (including new tables: api_keys, withdrawal_addresses, notification_preferences, notifications, refresh_token_families, sessions, candles, api_key_usage, revoked_tokens)
   • Seed markets, user tiers, system config
   • Connection pooling with PgBouncer
   • Remove version column from balances table (use pessimistic locking only — see matching engine design above)
4. Matching engine (pure function, zero dependencies)
   • OrderBook class with bids/asks Maps, SortedSets for price levels
   • matchOrder() — all order types (limit, market, stop, stop_limit, post_only, fok, ioc)
   • Order validation: tick size, min/max size, price deviation, max open orders
   • Self-trade prevention
   • Fee calculation (maker/taker)
   • Stop order trigger queue
   • Exception handling: try/catch in matchOrder, mark failed orders for reconciliation, halt trading after 3 consecutive errors (per Matching Engine Design section above)
   • Unit tests: all order types, partial fills, self-trade prevention, edge cases
   • Benchmark: 10k+ orders/second throughput
   • Sequence number generation
5. Transaction-atomic REST API
   • NestJS with Fastify adapter
   • API versioning: all endpoints prefixed with /api/v1/ (per H4)
   • POST /api/v1/orders with 4-phase pipeline (order entry → matching → persist → post-commit)
   • Idempotency: Idempotency-Key header on order placement (per C10 — see backend.md)
   • GET /api/v1/orders with cursor-based pagination
   • DELETE /api/v1/orders/:id — cancel order with race condition handling (per Critical Constraints item #16 below)
   • Balance service: lock on order placement, unlock on cancel, update on trade
   • All balance updates use SELECT FOR UPDATE within the persistence transaction
   • Outbox events written in same transaction as trade/order persists
6. Health check infrastructure (per C9 — see backend.md)
   • GET /api/v1/health/live — Liveness probe (process running)
   • GET /api/v1/health/ready — Readiness probe (DB, Redis, matching engine)
   • GET /api/v1/health/detailed — Component-by-component status (admin only)
7. Scheduled jobs infrastructure (per C5 — see backend.md)
   • BullMQ job queue (Redis-backed) for all recurring and event-driven tasks
   • Job definitions: order expiry, outbox publishing, wallet rebalancing, deposit scanning, candle generation, fee snapshots
   • Bull Board dashboard for admin monitoring
   • Dead-letter queue for failed jobs with retry and manual review
8. Graceful shutdown handler (per C15 — see backend.md)
   • Stop accepting new connections
   • Wait for in-flight orders (30s timeout)
   • Flush outbox events to Redis
   • Persist in-memory order book to DB
   • Close DB and Redis connections
   • PM2 kill_timeout: 60000

Deliverables: Working matching engine with transaction-atomic REST API, health checks, scheduled jobs, idempotent order placement, graceful shutdown

Phase 2: Auth, Security & Core Services (Weeks 5-8)

Goal: Complete authentication, API keys, rate limiting, notifications, withdrawal whitelisting, and session management.

1. Authentication system
   • JWT access tokens (15 min, RS256) stored in-memory (Zustand), sent via Authorization: Bearer header (NOT httpOnly cookies — in-memory storage prevents XSS theft; per C12 — see backend.md)
   • Refresh tokens: httpOnly Secure sameSite=strict cookies, 7-day lifetime
   • Refresh token rotation with token family tracking (per C12 — see backend.md)
   • Detect token reuse → revoke all sessions for user
   • CSRF protection via double-submit cookie pattern (per C11 — see backend.md)
   • POST /api/v1/auth/login, POST /api/v1/auth/register, POST /api/v1/auth/refresh
   • TOTP 2FA via otplib (enable, verify, backup codes)
2. Session management (per C13 — see backend.md)
   • sessions table tracking all active sessions per user
   • Maximum 5-10 concurrent sessions per user (configurable per tier)
   • "Logout all devices" functionality
   • Session invalidation on password change
   • New device/IP email notification
   • Device info tracking (browser, OS, IP)
3. API key management (per C2 — see backend.md)
   • api_keys table with hashed key + bcrypt secret
   • HMAC-SHA256 signature authentication (not JWT)
   • Permissions: {read, trade, withdraw} (withdraw requires per-request 2FA)
   • IP whitelisting per API key
   • Rate limit per API key
   • Key expiration dates
   • Activity logging in api_key_usage table
4. Rate limiting service (per C4 — see backend.md)
   • Redis-based sliding window rate limiter
   • Per-user, per-API-key, per-IP limits
   • Separate limits per endpoint category (orders, withdrawals, read endpoints)
   • Rate limit headers on all responses (X-RateLimit-Limit, X-RateLimit-Remaining)
   • NestJS guard integration
5. Notification service (per C1 — see backend.md)
   • Email provider: SendGrid or AWS SES
   • Push notifications: Firebase Cloud Messaging (FCM)
   • SMS (security alerts only): Twilio
   • In-app notifications via WebSocket channel user:notifications
   • notification_preferences table per user (opt-in/out)
   • notifications table for history and read status
   • Critical security notifications cannot be disabled (email always on)
6. Withdrawal address whitelist (per C3 — see backend.md)
   • withdrawal_addresses table with 24h lock after adding
   • Email confirmation required to add/remove addresses
   • Withdrawals only to whitelisted addresses (after lock period)
   • Address book with labels/nicknames per user
7. Withdrawal IP restrictions (per H8 — see backend.md)
   • withdrawal_ip_history table tracking IPs used for withdrawals
   • Optional per-user IP restriction: only whitelisted IPs can withdraw
   • New IP → 24h lock + email confirmation to whitelist
8. CORS & security headers (per H5 — see backend.md)
   • Fastify CORS configuration (only frontend origin)
   • Helmet: CSP, HSTS, X-Frame-Options, X-Content-Type-Options
   • Security headers on all responses

Deliverables: Complete auth with 2FA, API keys for bots, rate limiting, notifications, withdrawal whitelisting, session management

Phase 3: Wallet Integration (Weeks 9-12)

Goal: Deposits and withdrawals work for BLK, BTC, LTC, and stablecoin assets on EVM + Tron.

1. Remote provider clients (with circuit breakers)
   • ElectrumxClient for BLK/BTC/LTC (single abstraction, electrum-client library over TCP)
   • EVMProvider for Ethereum/BSC (Alchemy primary, Infura fallback, QuickNode optional)
   • TronProvider for Tron (TronWeb library, TronGrid provider — Tron is NOT EVM)
   • Circuit breaker per provider: 5 failures → OPEN, 30s recovery, 3 successes → CLOSE
   • Health monitoring: 30s check interval, store in provider_health table
   • Alert on: provider down, stale blocks, high response time
2. Deposit address generation
   • HD wallet master key stored in HashiCorp Vault (never in env vars)
   • Derive unique deposit address per user per asset per chain
   • Address format per chain: P2PKH (BLK), P2PKH (BTC), P2SH (LTC), hex (ETH/BSC), Base58 T-prefix (Tron)
   • Store address + derivation path in deposit_addresses table
   • API: GET /api/v1/deposit-address/:asset
3. Deposit scanning service (with crash resilience)
   • UTXO scanner: polls ElectrumX blockchain.scripthash.get_history every 30s per chain
   • EVM scanner: Alchemy/Infura ERC-20 Transfer event logs
   • Tron scanner: dedicated deposit-scan-tron job via TronWeb (separate from EVM)
   • Last scanned block hash stored in scanner_state table (survives restarts)
   • Idempotent: txid unique constraint prevents double-crediting
   • Chain reorg detection and reorg_events table
   • Resume from last block hash on restart (no deposits missed)
   • Credit deposit only after required confirmations (configurable per asset)
4. Withdrawal service (with UTXO coin selection)
   • UTXO coin selection: branch-and-bound algorithm to minimize fees
   • Fee estimation: ElectrumX blockchain.estimatefee for UTXO chains, ethers.js gas oracle for EVM, TronWeb estimateEnergy() for Tron
   • Change address management: generate new change address per withdrawal
   • UTXO consolidation background job (daily, low-activity hours)
   • Withdrawal flow: validate → coin select → create TX → sign → broadcast → monitor
   • Circuit breaker: if provider unavailable, queue withdrawal (status: queued)
   • Failed withdrawal handling: retry 3x, then mark failed, unlock user balance
   • Signing via Vault Transit engine (UTXO chains: app never sees raw keys; EVM keys are ephemeral in memory, never persisted to disk)
5. Wallet rebalancing service
   • WalletMonitor: check hot/warm balances every 60s per asset
   • Auto-transfer warm→hot when hot wallet below minimum
   • Alert admins when warm wallet below minimum (cold→warm needed)
   • wallet_transfers table for audit trail
6. Deposit/withdrawal notifications
   • Email notification on deposit credited
   • Email + in-app notification on withdrawal requested, processing, completed, failed
   • Admin notification for large withdrawals (>$5k)

Deliverables: Full deposit/withdrawal across all 6 chains (BLK, BTC, LTC, Ethereum, BSC, Tron) with remote providers, circuit breakers, wallet rebalancing, notifications

Phase 4: Real-Time & Market Data (Weeks 13-16)

Goal: WebSocket feeds, market data, tickers, candles, trade history.

1. WebSocket server (NestJS + Socket.io + Redis adapter)
   • Authentication: JWT verification on WS connection
   • Channel subscription management
   • Outbox worker: polls published = false events, publishes to Redis, marks published
   • Outbox worker implementation details (per Critical Constraints items #11, #18 below): batch size 100, dead-letter queue for failed publishes, cleanup job for events older than 24h
2. Order book WebSocket (with sequence numbers)
   • Full snapshot on subscribe (includes current sequence number)
   • Delta updates on every matching engine event (with sequence number)
   • Gap detection on client side → request full snapshot
   • Client can request missed events: GET /api/v1/orderbook/:market/events?since=<seq>
   • Throttled updates (max 10/sec per market)
3. Trade feed, balance updates, deposit/withdrawal status
   • Public trades stream per market
   • User's private: order updates, trade executions, balance changes, deposit/withdrawal status
   • All events routed through outbox for guaranteed delivery
4. Market data service (per C6 — see backend.md)
   • 24h rolling statistics per market (high, low, volume, VWAP, price change)
   • candles table for OHLCV data (1m, 5m, 15m, 1h, 4h, 1d, 1w intervals)
   • BullMQ scheduled jobs for candle generation and ticker updates
   • Redis caching: tickers (updated on every trade), recent trades (last 100 per market)
   • REST API: tickers, candles, order book snapshots, recent trades
5. Trade history & reporting (per C7 — see backend.md)
   • Paginated trade history API (cursor-based)
   • Paginated order history API
   • Deposit/withdrawal history API
   • CSV export for trade history
   • P&L tracking per market (realized/unrealized)
   • Fee report API
6. Ticker WebSocket
   • 24h ticker stats: high, low, volume, last price, change
   • Updated on every trade or every 1 second (whichever is less frequent)
7. Crash recovery service
   • On startup: rebuild order book from last snapshot + trade replay
   • Re-process any pending orders (stuck from crash)
   • Verify balance consistency: SUM(available + locked) per asset matches expected total
   • If inconsistent: halt trading, alert admins, manual intervention required

Deliverables: Full real-time pipeline with guaranteed delivery, crash recovery, sequence numbers, market data, trade history

Phase 5: Frontend (Weeks 17-22)

Goal: Production-grade trading interface with full security hardening, critical UX patterns, and mobile support.

1. Next.js app scaffolding (dark theme, Tailwind CSS 4.3)
2. Auth pages (register, login, 2FA setup, 2FA verify, access token in Zustand + refresh token in httpOnly cookie)
3. Trading interface (/trade/:market) — order book, price chart, trade form, open orders, recent trades, balance panel, market selector
4. Wallet pages (deposit address + QR code, withdrawal form + 2FA + address whitelist, transaction history)
5. Account pages (profile, 2FA management, session management, API keys, order/trade history with pagination, CSV export, P&L)
6. Admin panel (user management, withdrawal approval queue, system status, fee revenue dashboard, market management, maintenance mode toggle)
7. Frontend security hardening (see frontend.md — CSP, DOMPurify, input sanitization, error codes, token management)
   • Content Security Policy (see frontend.md for CSP details): strict CSP with default-src 'self', script-src, style-src, connect-src for XSS prevention; dev-friendly WebSocket entries (ws://localhost:*); production WSS domains. Also: Permissions-Policy with camera/mic/geolocation/payment/fullscreen restricted.
   • HTML sanitization with DOMPurify (see frontend.md for DOMPurify): all dangerouslySetInnerHTML wrapped in DOMPurify sanitization. Added to tech stack as dompurify + @types/dompurify.
   • Amount input sanitization (see frontend.md for input sanitization): sanitizeAmountInput() utility strips non-numeric characters, removes extra decimals, trims input to 20 chars before bigint parsing.
   • Error code mapping (see frontend.md for error codes): shared ERROR_CODES and ERROR_MESSAGES in packages/shared/src/errors.ts, used by both frontend and backend for consistent user-facing messages.
8. Critical UX patterns (see frontend.md — withdrawal UX, address UX, session UI, WS connection)
   • Withdrawal confirmation modal (see frontend.md for withdrawal modal): amount, address with checksum highlighting, fee, total deduction, 2FA code input, confirm button. Required for ALL withdrawals.
   • Address whitelisting UX (see frontend.md for address whitelisting UX): /account/addresses page with add/confirm/remove flow, 24h countdown timer, email confirmation, 2FA required.
   • Session management UI (see frontend.md for session UI): active sessions list in /account/security with device/browser/IP info, per-session revoke, "Logout All Devices" button, current session indicator.
   • WebSocket connection UX (see frontend.md for WS connection UX): connection status indicator (connected/connecting/disconnected), reconnect attempt counter, warning banner on disconnection: "Do not place new orders until reconnected."
9. Deposit address UX (see frontend.md for deposit address UX)
   • /wallet/deposit/:asset shows deposit address, QR code, asset logo
   • Confirmation copy button, address display in monospace font
   • Minimum deposit threshold warning (e.g., "0.001 BLK minimum")
   • Pending/confirmed/credited status with progress indicator (confirms: 0/6, 2/6, 6/6)
   • Email notification on first deposit to new address
10. SSR hydration strategy & mobile support (see frontend.md for SSR strategy and mobile support)
    • Deferred WebSocket subscription: 100ms delay after hydration to prevent React hydration mismatches on wallet/account SSR pages.
    • Mobile input handling: inputMode="decimal" on all amount inputs, pattern="[0-9]*[.]?[0-9]*" for mobile numeric keyboard with decimal.
    • Touch targets: minimum 44×44px for all interactive elements on touch devices.
    • PWA configuration: next-pwa for service worker, offline fallback page, install prompt.
11. Performance budget (see frontend.md for performance budget)
    • Initial JS: < 200KB gzipped | Total page weight: < 1MB | TTI: < 3s on 4G
    • Dynamic imports for chart, admin, wallet pages
    • Bundle analysis via @next/bundle-analyzer

Deliverables: Fully functional trading frontend with all trading pairs, security features, admin panel, CSP/XSS protection, withdrawal confirmation UX, session management UI, mobile support, and PWA

Phase 6: Security Hardening (Weeks 23-26)

Goal: Production-ready security posture.

1. Anti-fraud system (per H1 — see backend.md)
   • Velocity checks per user (orders/sec, orders/min, withdrawals/day)
   • Suspicious pattern detection (same IP across accounts, circular funds, wash trading)
   • Risk scoring per order and withdrawal
   • Admin flagging and review queue
   • New account delay: 24h before first withdrawal
2. Configuration management (per H2 — see backend.md)
   • All configuration stored in system_config table
   • Hot reload without restart (WebSocket broadcast to all instances)
   • Feature flags for gradual rollouts (e.g., feature.stop_orders.enabled)
   • Configuration audit log (who changed what, when)
3. Maintenance mode (per H3 — see backend.md)
   • Global and per-market maintenance mode toggle
   • New orders rejected during maintenance (503 + reason)
   • Existing orders can remain on book or be force-cancelled
   • WebSocket broadcasts maintenance event to all clients
   • Admin UI toggle with confirmation dialog
4. Audit logging (append-only, no UPDATE/DELETE)
   • All order events, deposits, withdrawals, logins, failed auth, admin actions
   • API key usage logging
   • Database trigger prevents modification of audit_log table
5. Input validation hardening
   • Zod schemas on every endpoint
   • Reject orders >10% from mark price (configurable per market)
   • Reject orders below minimum size
   • SQL injection: parameterized queries only (Prisma default)
   • XSS: sanitize all user inputs
6. Infrastructure security
   • HTTPS everywhere (Let's Encrypt or Cloudflare)
   • Secret management: HashiCorp Vault (NOT env vars for hot wallet keys)
   • Database encryption at rest
   • Daily encrypted backups of PostgreSQL (pgBackRest to S3) (per C8 — see backend.md)
   • Weekly backup verification (restore to staging, verify integrity) (per H11 — see backend.md)
   • Monitoring: Prometheus + Grafana for system metrics, Sentry for error tracking
   • Monitoring alerts with thresholds (per H10 — see backend.md): provider RPC > 5s → warning, > 30s → critical; hot wallet < minimum → critical; outbox backlog > 1000 → warning; matching engine errors ≥ 3 → auto-halt
7. Wash trade detection
   • Flag accounts trading with same IP, same deposit address, circular fund flows
   • Alert admins for review (no auto-ban)
8. Database backup & recovery (per C8 — see backend.md)
   • pgBackRest with WAL archiving for point-in-time recovery
   • Daily full backups to S3 (encrypted AES-256)
   • Incremental backups every 4 hours
   • 30-day retention policy
   • RTO: < 1 hour, RPO: < 5 minutes
   • Weekly automated restore verification to staging environment

Deliverables: Production-ready exchange with defense-in-depth security, anti-fraud, monitoring, alerting, and verified backups

Phase 7: Final Hardening & Launch Prep (Weeks 27-30)

Goal: Production-hardened deployment, graceful matching engine restart with RTO < 30s, load testing, incident runbooks, Redis HA, trading pair lifecycle (H7), and CSP/frontend security (M3/F1).

1. Production deployment (per C14 — see backend.md)
   • DB layer: expand/contract migrations (zero-downtime via CREATE INDEX CONCURRENTLY, column additions, view swaps)
   • API layer: PM2 fork-mode restart (~5-10s downtime per deploy with pm2 reload). Stateless — no session data lost. Load balancer health checks route traffic away during restart.
   • Matching engine: PM2 fork-mode restart with snapshot + replay for RTO < 30s. Controlled restart: snapshot to Redis → stop → start → load snapshot → replay from outbox_events.
   • Future path: Separate API and matching engine into different processes (see scaling path) to enable zero-downtime API deploys via rolling restart behind Nginx.
   • Health check integration with load balancer
   • Rollback procedure documented and tested
2. Capacity planning & load testing (per H12 — see backend.md)
   • Artillery load testing scripts (10k+ orders/second target)
   • Maximum concurrent WebSocket connections benchmark
   • Database connection limits and scaling triggers
   • Define when to scale horizontally (add API instances) vs. vertically
3. Redis HA (per M1 — see backend.md) post-launch
   • Redis Sentinel (3 nodes) for production
   • Fallback to in-memory rate limiting if Redis is unavailable
   • Order book snapshots rebuilt from PostgreSQL on Redis restart
4. Incident response runbooks (per M7 — see backend.md)
   • Provider failure (ElectrumX / Alchemy / TronGrid): circuit breaker → scanner pauses → withdrawals queue → alert admins → monitor recovery
   • Database corruption: stop trading → PITR recovery → verify integrity → resume
   • Hot wallet compromise: freeze withdrawals → transfer to cold wallet → new hot wallet → audit
   • DDoS: Cloudflare auto-mitigation → enable challenge-response → monitor
   • Matching engine crash: PM2 restart → rebuild from snapshot → replay → verify balances
5. Trading pair lifecycle management (per H7 — see backend.md)
   • Market states: upcoming | active | paused | delisting | delisted
   • Adding a new market: INSERT INTO markets (status: 'upcoming') → admin toggles to 'active'
   • Pausing a market (emergency): reject all new orders, broadcast market_paused WS event
   • Delisting: 7-day grace period → force-cancel open orders → set 'delisted' → hide from UI
   • Deposits disabled for delisted base assets
6. USDT/USDC stablecoin integration
   • Ethereum + BSC (EVM): Alchemy/Infura providers, ethers.js, ERC-20 Transfer event scanning, gas estimation with price floor, 12 block confirmations (ETH) / 30 (BSC)
   • Tron (NOT EVM): Dedicated TronWeb-based scanner (deposit-scan-tron), TronGrid provider, Bandwidth + Energy gas model, 19 block confirmations
   • Multi-chain deposit address generation (one per user per chain)
   • Circuit breaker per provider (Alchemy, Infura, TronGrid)
7. Stop orders and mark price oracle
   • Internal VWAP from last 100 trades on our exchange
   • Volume-weighted mark price with circuit breaker (>30% in 5 min → halt)
   • Stop order trigger service (checks after every trade execution)
   • Stop-limit orders (trigger creates limit order at specified price)
8. Fee model and user tiers
   • Configurable maker/taker fees per market (from markets table)
   • Volume-based discounts via user_tiers table
   • Fee revenue tracking in fee_revenue table
   • Revenue dashboard in admin panel
9. KMS key rotation procedures (per M6 — see backend.md) (post-launch)
   • HD wallet key rotation: generate new master seed in Vault, derive new addresses, monitor old addresses for 90 days
   • API key rotation: user-initiated, 24h overlap period
   • JWT signing key rotation: new RSA key pair in Vault, JWKS endpoint, 7-day overlap
10. Subaccount / multi-address support (per H6 — see backend.md)
    • subaccounts table for institutional users (trading, savings, defi sub-accounts)
    • internal_transfers table for fee-free transfers between own subaccounts
    • Balances track subaccount_id (NULL = main account)
    • Multiple deposit addresses per user per asset (privacy)
    • Post-launch feature, but schema tables added now for forward compatibility
11. Application-layer DDoS protection (per M2 — see backend.md) (post-launch)
    • Request fingerprinting to identify bot patterns
    • Progressive delays for repeated failed requests
    • CAPTCHA challenge for suspicious traffic on order placement
    • IP-based temporary bans for abuse patterns (auto-expire after 1 hour)
    • Integrate with Cloudflare challenge-response for known attack patterns
12. Scheduled maintenance windows (per M4 — see backend.md) (post-launch)
    • maintenance_schedule rows in system_config with planned start/end times
    • Frontend shows banner N days before: "Scheduled maintenance: BLK/USDT on Jan 15, 02:00-04:00 UTC"
    • 30 minutes before window: email notification to active users of affected markets
    • At start time: market pauses automatically
    • At end time: market resumes automatically (or admin re-enables manually)
13. Database connection exhaustion prevention (per M5 — see backend.md) (post-launch)
    • Prisma connection pool: connection_limit=30&pool_timeout=10 in DATABASE_URL
    • Database circuit breaker in NestJS: 5 consecutive failures → OPEN, 30s recovery
    • PgBouncer: max_client_conn=1000, default_pool_size=50
    • Monitor pg_stat_activity_count metric with alert at 80 connections
    • Connection quota per service module to prevent any single module from monopolizing connections

Deliverables: Production-hardened exchange ready for launch with deployment automation, load testing benchmarks, incident runbooks, Redis HA, EVM chain integration, and operational resilience

Technology Stack Summary

Project Structure

exchange/
├── apps/
│   ├── backend/                    # NestJS application
│   │   ├── src/
│   │   │   ├── main.ts
│   │   │   ├── app.module.ts
│   │   │   ├── core/              # @exchange/core (backend-only, NOT cross-project)
│   │   │   │   ├── backend.types.ts       # NestJS-specific types (controllers, services)
│   │   │   │   ├── backend.constants.ts   # Backend-only constants (rate limits, intervals)
│   │   │   │   ├── backend.errors.ts      # HTTP error subclasses, NestJS exceptions
│   │   │   │   ├── backend.utils.ts       # BigInt conversion, price formatting helpers
│   │   │   │   └── graceful-shutdown.ts   # NestJS shutdown hooks + disconnect cleanup
│   │   │   ├── matching-engine/   # @exchange/matching-engine
│   │   │   │   ├── order-book.ts
│   │   │   │   ├── matcher.ts
│   │   │   │   ├── stop-order-queue.ts
│   │   │   │   ├── price-oracle.ts
│   │   │   │   ├── fee-calculator.ts
│   │   │   │   ├── order-validator.ts
│   │   │   │   ├── circuit-breaker.ts
│   │   │   │   └── engine.test.ts
│   │   │   ├── trading/           # @exchange/trading
│   │   │   │   ├── trading.module.ts
│   │   │   │   ├── order.service.ts
│   │   │   │   ├── trade.service.ts
│   │   │   │   ├── balance.service.ts
│   │   │   │   ├── outbox.service.ts
│   │   │   │   ├── outbox-worker.service.ts  # Outbox polling & publishing
│   │   │   │   ├── recovery.service.ts     # Crash recovery
│   │   │   │   ├── sequence.service.ts     # Sequence number generation
│   │   │   │   └── idempotency.interceptor.ts  # Idempotent order placement
│   │   │   ├── wallets/           # @exchange/wallets
│   │   │   │   ├── wallets.module.ts
│   │   │   │   ├── deposit.service.ts
│   │   │   │   ├── withdrawal.service.ts
│   │   │   │   ├── withdrawal-address.service.ts  # Address whitelist
│   │   │   │   ├── utxo-scanner.ts
│   │   │   │   ├── evm-scanner.ts
│   │   │   │   ├── coin-selector.ts       # UTXO coin selection
│   │   │   │   ├── consolidation.service.ts # UTXO consolidation
│   │   │   │   ├── address-generator.ts
│   │   │   │   ├── wallet-monitor.ts       # Hot/warm/cold rebalancing
│   │   │   │   ├── price-oracle.service.ts
│   │   │   │   ├── reorg-detector.ts
│   │   │   │   └── providers/
│   │   │   │       ├── base-chain-client.ts    # Circuit breaker base
│   │   │   │       ├── electrumx-client.ts     # BLK/BTC/LTC via ElectrumX
│   │   │   │       ├── evm-provider.ts         # Alchemy/Infura with fallback (ETH/BSC)
│   │   │   │       └── tron-provider.ts        # TronGrid + TronWeb (NOT ethers.js)
│   │   │   ├── auth/              # @exchange/auth
│   │   │   │   ├── auth.module.ts
│   │   │   │   ├── auth.service.ts
│   │   │   │   ├── jwt.strategy.ts
│   │   │   │   ├── two-factor.service.ts
│   │   │   │   ├── session.service.ts      # Session management
│   │   │   │   ├── csrf.guard.ts            # CSRF protection
│   │   │   │   └── rate-limiter.ts
│   │   │   ├── notifications/      # @exchange/notifications
│   │   │   │   ├── notifications.module.ts
│   │   │   │   ├── notification.service.ts
│   │   │   │   ├── email.provider.ts        # SendGrid/SES integration
│   │   │   │   ├── push.provider.ts         # Firebase FCM
│   │   │   │   ├── sms.provider.ts          # Twilio (security alerts)
│   │   │   │   └── in-app.provider.ts       # WebSocket channel
│   │   │   ├── api-keys/          # @exchange/api-keys
│   │   │   │   ├── api-keys.module.ts
│   │   │   │   ├── api-key.service.ts
│   │   │   │   ├── api-key.guard.ts          # HMAC authentication
│   │   │   │   └── api-key.interceptor.ts   # Usage tracking
│   │   │   ├── market-data/       # @exchange/market-data
│   │   │   │   ├── market-data.module.ts
│   │   │   │   ├── ticker.service.ts
│   │   │   │   ├── candle.service.ts
│   │   │   │   └── orderbook-snapshot.service.ts
│   │   │   ├── reporting/         # @exchange/reporting
│   │   │   │   ├── reporting.module.ts
│   │   │   │   ├── trade-history.service.ts
│   │   │   │   ├── pnl.service.ts
│   │   │   │   └── export.service.ts         # CSV/JSON export
│   │   │   ├── fraud/             # @exchange/fraud
│   │   │   │   ├── fraud.module.ts
│   │   │   │   ├── fraud-detection.service.ts
│   │   │   │   └── velocity-check.service.ts
│   │   │   ├── config/            # @exchange/config
│   │   │   │   ├── config.module.ts
│   │   │   │   ├── config.service.ts          # Hot-reload from system_config
│   │   │   │   └── feature-flags.service.ts   # Feature flag management
│   │   │   ├── jobs/              # @exchange/jobs
│   │   │   │   ├── jobs.module.ts
│   │   │   │   ├── order-expiry.worker.ts
│   │   │   │   ├── wallet-rebalance.worker.ts
│   │   │   │   ├── deposit-scan.worker.ts
│   │   │   │   ├── candle.worker.ts
│   │   │   │   ├── ticker.worker.ts
│   │   │   │   ├── outbox-publisher.worker.ts
│   │   │   │   ├── fee-snapshot.worker.ts
│   │   │   │   ├── utxo-consolidation.worker.ts
│   │   │   │   ├── orderbook-snapshot.worker.ts
│   │   │   │   └── stuck-withdrawal.worker.ts
│   │   │   ├── api/               # @exchange/api
│   │   │   │   ├── api.module.ts
│   │   │   │   ├── controllers/
│   │   │   │   │   ├── public.controller.ts      # Tickers, orderbook, trades
│   │   │   │   │   ├── auth.controller.ts         # Login, register, 2FA, refresh
│   │   │   │   │   ├── order.controller.ts       # Place, cancel, list orders
│   │   │   │   │   ├── wallet.controller.ts      # Deposit addresses, withdrawals
│   │   │   │   │   ├── account.controller.ts      # Sessions, API keys, preferences
│   │   │   │   │   ├── reporting.controller.ts    # Trade history, P&L, exports
│   │   │   │   │   └── admin.controller.ts        # User mgmt, market mgmt, maintenance
│   │   │   │   ├── gateways/
│   │   │   │   │   ├── market.gateway.ts    # Orderbook + trade WS
│   │   │   │   │   └── user.gateway.ts      # Private user WS
│   │   │   │   ├── guards/
│   │   │   │   │   ├── jwt-auth.guard.ts
│   │   │   │   │   ├── api-key.guard.ts
│   │   │   │   │   ├── rate-limit.guard.ts
│   │   │   │   │   ├── csrf.guard.ts
│   │   │   │   │   ├── idempotency.interceptor.ts
│   │   │   │   │   └── maintenance.guard.ts        # Block orders during maintenance
│   │   │   │   └── dto/
│   │   │   └── admin/
│   │   │       ├── admin.module.ts
│   │   │       ├── admin.service.ts
│   │   │       ├── maintenance.service.ts         # Global/market maintenance mode
│   │   │       └── withdrawal-approval.service.ts
│   │   ├── prisma/
│   │   │   ├── schema.prisma
│   │   │   └── migrations/
│   │   ├── test/
│   │   │   ├── matching-engine/   # Engine stress tests
│   │   │   ├── trading/           # Integration tests
│   │   │   ├── wallets/           # Wallet integration tests
│   │   │   ├── auth/              # Auth integration tests
│   │   │   ├── api-keys/          # API key tests
│   │   │   ├── notifications/     # Notification tests
│   │   │   ├── fraud/             # Fraud detection tests
│   │   │   ├── load/              # Artillery load tests
│   │   │   └── e2e/              # End-to-end tests
│   │   ├── nest-cli.json
│   │   ├── tsconfig.json
│   │   └── package.json
│   │
│   └── frontend/                  # Next.js application
│       ├── src/
│       │   ├── app/               # Next.js App Router pages
│       │   ├── components/
│       │   │   ├── trading/
│       │   │   │   ├── OrderBook.tsx
│       │   │   │   ├── PriceChart.tsx
│       │   │   │   ├── TradeForm.tsx
│       │   │   │   ├── OpenOrders.tsx
│       │   │   │   ├── RecentTrades.tsx
│       │   │   │   ├── BalancePanel.tsx
│       │   │   │   └── MarketSelector.tsx
│       │   │   ├── wallet/
│       │   │   ├── auth/
│       │   │   ├── admin/
│       │   │   └── ui/
│       │   ├── hooks/
│       │   │   ├── useWebSocket.ts
│       │   │   ├── useOrderBook.ts
│       │   │   ├── useBalances.ts
│       │   │   └── useAuth.ts
│       │   ├── stores/
│       │   ├── lib/
│       │   └── types/
│       ├── package.json
│       ├── next.config.js
│       └── tsconfig.json
│
├── packages/
│   └── shared/                    # Shared types and schemas
│       ├── src/
│       │   ├── types.ts
│       │   ├── schemas.ts         # Zod validation schemas
│       │   ├── constants.ts       # Markets, assets, fees, order type enums
│       │   ├── formatters.ts      # formatSatoshi(), parseToSatoshi(), formatAmount()
│       │   ├── errors.ts          # ERROR_CODES + ERROR_MESSAGES mapping
│       │   └── index.ts           # Re-exports
│       ├── package.json
│       └── tsconfig.json
│
├── docker/
│   ├── docker-compose.yml         # PostgreSQL, Redis, PgBouncer, Vault, backend, frontend
│   ├── docker-compose.dev.yml     # Dev overrides (credentials, hot-reload)
│   ├── backend.Dockerfile
│   └── frontend.Dockerfile
│
├── scripts/
│   ├── setup.sh
│   ├── seed-markets.ts
│   ├── seed-test-data.ts          # Development seed data
│   ├── benchmark-matcher.ts
│   └── verify-backup.sh           # Weekly backup verification
│
├── tests/
│   └── load/
│       └── artillery-config.yml   # Load testing configuration
│
├── .env.example
├── turbo.json
├── package.json
└── README.md

Critical Constraints

1. No floating point — All monetary values use bigint (satoshi precision) or Prisma.Decimal
2. No as any / @ts-ignore — Full type safety, zero suppression
3. Balance atomicity — SELECT FOR UPDATE within matching transaction, never outside. No optimistic locking (removed version column from balances)
4. Trade immutability — Trades are never updated, only inserted
5. Idempotent deposits — txid unique constraint prevents double-crediting
6. Self-trade prevention — Reject orders where maker and taker are the same user
7. Order validation — Reject prices >10% from mark price, quantities below minimum, max 200 open orders per user per market
8. Crash recovery — Order book snapshots every 1000 sequences or 60 seconds; replay from snapshot + trade log; reconciliation job for stuck pending orders
9. Chain reorg handling — Detect reorgs, reverse credited deposits, rescan from common ancestor
10. No env var secrets for hot wallet keys — HashiCorp Vault only
11. Outbox pattern — All WS events written to outbox_events table in same transaction as trade persist; background worker publishes to Redis; no events lost on crash
12. Circuit breakers — Every provider RPC call wrapped in circuit breaker; 5 failures → OPEN; scanner pauses on provider failure; withdrawals queue on provider unavailability
13. Wallet rebalancing — Automatic warm→hot transfers when hot wallet below minimum; cold→warm requires manual multisig approval
14. Front-running mitigation — 100ms order batching, 1-second-stale order book snapshots, rate limiting per user
15. Idempotent order placement — Idempotency-Key header prevents duplicate orders from network retries
16. Cancel-order race condition — Cancel acquires same row lock as matching engine; pending orders can be cancelled directly in DB; open/partial orders go through matching engine
17. Matching engine exception handling — Wrap matchOrder() in try/catch; failed orders marked for reconciliation; auto-halt after 3 consecutive matching errors
18. Outbox event staleness — Events older than 24h are cleaned up; client reconnecting with stale sequence gets full snapshot
19. Pessimistic locking only — Use SELECT FOR UPDATE for balance updates; removed optimistic locking version column
20. Withdrawal whitelist — 24h lock after adding new withdrawal address; withdrawals only to whitelisted addresses
21. API key authentication — HMAC-SHA256 signature (not JWT) for programmatic access; separate scopes for read/trade/withdraw
22. Notification service — All security events (login, withdrawal, 2FA change) generate notifications; email always on for security events