MatLogica | Blog Post

Blog Post

From Batch Risk to Real-Time Live Risk with Automated IFT

Transform overnight batch risk processing into real-time Live Risk using Automatic Adjoint Differentiation and Automated Implicit Function Theorem. No multi-year rewrite required.

Executive Summary

Transform overnight batch risk into real-time Live Risk in weeks, not years. Using Automated Implicit Function Theorem (AIFT) with modern AAD enables automatic differentiation of model calibrations without code refactoring.

Proven Result: QuantLib integration achieves 20ms pricing + complete Greeks for 1000 IR swaps including curve calibration on single CPU.

The Batch Risk Problem

Batch processing for portfolio risk has been standard for decades with risk reports produced overnight. But traders now demand real-time risk visibility, and legacy quant libraries weren't designed for this.

Why Batch Risk Is Problematic

Batch Processing Limitations:

  • Slow and dangerous: Several hours overnight execution
  • Corner-cutting: When systems are slow, people don't compute all scenario risks
  • Unmonitored risks: Some exposures left unchecked between batch runs
  • Missed opportunities: Can't respond to intraday market moves
  • Competitive disadvantage: Lagging competitors with real-time risk

The Korean Autocallable Crisis Example

Real-world consequences of slow risk systems: Korean autocallable notes pushed over trillion Won ($810 million) of equity-linked securities into the red in Q3 2022. Better intraday risk monitoring could have enabled earlier risk mitigation.

What Live Risk Delivers

  • Real-time visibility: Risk updates with every market tick
  • Faster decision-making: Respond to market moves immediately
  • Complete scenario coverage: No corner-cutting when computation is fast
  • Better returns: Capitalize on opportunities others miss
  • Reduced operational risk: Comprehensive risk monitoring
  • Regulatory compliance: More frequent and complete risk reporting

The Traditional Barrier

Transitioning to Live Risk typically requires multi-year IT transformation involving major rewrites of quant libraries. This is expensive, risky, and often abandoned halfway through.

This article presents an alternative: Achieve Live Risk in weeks using Automated Implicit Function Theorem (AIFT) and modern AAD—no major rewrite required.

Live Risk in Action: QuantLib Example

We demonstrate with QuantLib—a real-life production library—integrated with MatLogica AADC (modern AAD tool).

The Test Case

Interest rate swap portfolio with complete workflow:

  1. Curve fitting using Levenberg-Marquardt optimization
  2. Pricing 1000 12-year IR swaps with random parameters
  3. Computing complete Greeks via AAD
  4. Transforming calibrated curve sensitivities → tradeable market rates via AIFT

Performance Results

20 Milliseconds Total

Complete pricing + Greeks + calibration for 1000 swaps on single budget CPU with every market data update

Try It Yourself

Experience the Live Risk demo on MatLogica.com:

This demonstrates how existing batch analytics transition to Live Risk—making traders happy with real-time risk visibility!

The Technology Behind Live Risk

Two Key Technologies

1. Automated Implicit Function Theorem (AIFT)

The challenge: Model calibrations (implicit functions solved by minimizers/solvers) are common in quant finance but difficult to differentiate automatically.

Traditional belief: Manual intervention required to enable AAD through calibration routines—expensive and time-consuming.

AIFT breakthrough: Published in Risk.NET 2022 by Dmitri Goloubentsev, Evgeny Lakshtanov, and Vladimir Piterbarg—enables fully automated differentiation of calibration routines:

  • Exact-fit calibration: Fully automated (e.g., yield curve bootstrap)
  • Nearly-exact calibration: Approximate solutions (e.g., volatility surface fitting)
  • No hidden variable tracking: Works without identifying dependencies
  • No large Jacobians: Avoids computing massive matrices

Read the Risk.NET paper

2. High-Performance AAD (AADC)

Live Risk requires speed. MatLogica's AADC achieves adjoint factor less than 1—meaning all risks compute faster than price alone.

Key performance characteristics:

  • Records computation once at start of trading day
  • Generates optimized binary kernel with AVX2/AVX512 vectorization
  • Processes market updates in milliseconds including calibration
  • Always-hot state: Ready for instant computation

Live Risk System Architecture

Start of Trading Day

AADC generates binary kernels for forward and adjoint execution in "always-hot" state on the grid:

Live Risk System Actions at Start of Trading Day

Figure: System initialization at trading day start

Intraday Operation

Kernel receives market data updates, runs calibration solvers, executes pricing + adjoints, returns prices and risks to traders:

Live Risk System Intraday Actions

Figure: Continuous intraday risk updates

Workflow Summary

  1. Start of day: Record computation, generate optimized kernel
  2. Market update: Receive new rates/prices
  3. Calibration: Run solvers to fit market data
  4. Execution: Price portfolio + compute all Greeks via AAD
  5. AIFT transformation: Convert calibrated sensitivities → market rates
  6. Distribution: Send live risk feed to traders
  7. Repeat: Process next market update (steps 2-6)

The Mathematics Behind AIFT

Note: This section provides mathematical foundations for technical readers. Decision-makers can skip to Conclusions section below.

1. The Calibration Problem

Setup: We have an objective function Ω(X,C) where:

  • X(x): Market observable data in market variables x (rates, spreads, etc.)
  • C: Model coefficients to be calibrated

Calibration equation: Ω(X,C) = 0

Goal: Use calibrated coefficients C(x) to compute value of interest U(x) = V(C(x)) (prices, XVA, etc.)

Challenge: We need sensitivities ∂U/∂x (risk with respect to market rates), but calibration creates implicit dependencies.

2. Multiple Calibration Steps

Real-world complexity: Calibration often uses multiple solvers in sequence:

  • Ω₁(C₁) = 0
  • Ω₂(C₂) = 0
  • ...
  • U(x) = V(C₁, C₂, ...)

The hidden variable problem:

Code example showing hidden variables in calibration

Figure: Virtual function hides dependencies on X and C₁

Virtual function GoalFunc2 (implementing Ω₂) can't see dependence on X and C₁ - these are hidden variables. In practice, GoalFunc may be buried deep in the library.

3. Naive IFT Approach (Why It Fails)

Traditional approach requires:

  1. Identify all hidden variables Wᵢ in each Ωᵢ
  2. Compute large Jacobians ∂Ωᵢ/∂Wᵢ
  3. Extensive code refactoring to track dependencies

Problems: Large Jacobians, expensive computation, major code changes required.

4. Automated IFT Solution

Key insight: When backpropagating with AAD, we can choose adj(Ω₂) to cancel out adj(C₂) and automatically propagate adj(W₂) without identifying W₂ explicitly!

The correct choice:

adj(Ω₂) := −(∂V/∂C₂)(∂Ω₂/∂C₂)⁻¹

Both quantities readily available:

  • ∂V/∂C₂ obtained from straightforward AAD application
  • ∂Ω₂/∂C₂ already computed during forward pass (Newton/Levenberg-Marquardt solvers)

Benefits:

  • No need to identify hidden variables W
  • No large Jacobian ∂Ωᵢ/∂Wᵢ computation
  • Works via checkpointing without going into solvers
  • Extends naturally to multiple calibration steps

Mathematical details: See Risk.NET paper equations (12) and (20)

Benefits and Implementation Steps

AIFT Key Advantages

  • Automatic solver/minimizer handling with no codebase refactoring
  • No hidden variable identification or tracking required
  • No large Jacobian computation (∂Ωᵢ/∂Wᵢ) needed
  • Much faster backpropagation than naive IFT
  • Straightforward integration into legacy quant libraries

Transition Steps: Batch → Live Risk

Transform your batch system into "always-on" Risk Server in weeks:

  1. Integrate AADC: Add MatLogica's AADC to existing quant library
  2. Initialize kernels: Send PV requests to grid at trading day start
  3. Record once: Capture market rates, execute through solvers, generate kernel
  4. Receive updates: Get new market data ticks
  5. Run calibration: Fit solvers to new market data
  6. Execute pricing + AAD: Run pricers and adjoint code in kernel
  7. Distribute results: Send live risk feed to traders
  8. Repeat: Return to step 4 throughout trading day

No Multi-Year IT Transformation Required

Unlike traditional approaches requiring complete quant library rewrites, AIFT with AADC enables Live Risk transition through targeted integration—typically weeks to months instead of years.

Ready to Transition to Live Risk?

Talk to us about integrating AIFT and AADC into your quant library

MatLogica assists with integration every step of the way

Schedule Consultation

info@matlogica.com

Related topics: batch risk to live risk transition, automated implicit function theorem, model calibration AAD, solver differentiation automatic, hidden variable problem AAD, QuantLib AAD integration, real-time risk calculation, overnight risk to intraday, curve fitting AAD, Levenberg-Marquardt differentiation, checkpointing AAD calibration, multi-step calibration differentiation