Asking a QuantLib tape questions¶
Record a calculation with source tracing on, and every operation on the tape remembers
the file and line that produced it. trace_inspect() then answers questions about the tape,
one plain-text request in, one plain-text answer out:
f.start_recording(trace=True)
... # ordinary QuantLib code
f.stop_recording()
f.trace_inspect("source root=/project/work/QLAADC=/path/to/quantlib") # once
f.trace_inspect("find value=101.20511929299454") # where did my program compute this?
f.trace_inspect("origin #134") # what formula produced it, from what?
f.trace_inspect("replay set=#5=0.05") # what would it be at another input?
This tutorial runs a real QuantLib bond and a real barrier option and uses every request:
| question | request |
|---|---|
| What am I looking at? | stats |
| Where is the source? (set once) | source root=… |
| Which source lines did the work? | sites, at |
| Where is the number my program printed? | find |
| What formula produced it, and who uses it? | origin, uses |
| Which numbers went into the model, including ones nobody passed in? | leaves |
| What would the price be at other inputs, without repricing? | replay |
Which ifs were decided at record time, and how close were they? |
branches |
| What can this value depend on? | cone |
Requirements: Linux x86-64, Python 3.10–3.14, and two packages from PyPI:
pip install numpy scipy
pip install "aadc==2.25.2" "aadc-quantlib-tracing==1.41.1"
aadc-quantlib-tracing is QuantLib built with the AADC compiler, so every QuantLib
operation on the tape carries its QuantLib file and line.
1. Setup¶
import os, re, pathlib
from importlib.metadata import version
import aadc
import aadc_quantlib_tracing as ql
print("aadc ", version("aadc"))
print("aadc-quantlib-tracing ", version("aadc-quantlib-tracing"))
def ask(tape, request, max_lines=0):
# Send one request to a recorded tape and print the answer.
print(tape.trace_inspect(request, max_lines=max_lines))
aadc 2.25.2 aadc-quantlib-tracing 1.41.1
The pricers¶
The next cell writes two ordinary QuantLib pricers to a file. Each market input is created
as an aadc.idouble on its own line, so the tape can name the line that declared it.
The code is in a real file, not a notebook cell, so answers can show its source lines.
%%writefile pricers.py
"""Two small QuantLib pricers used by the tape-inspection tutorial.
Ordinary QuantLib code. The only AADC-specific step is that each market
input is created as an ``aadc.idouble`` so that the calculation is recorded.
Each input sits on its own line, so the tape can name that line as the
place where the input was declared.
"""
import aadc
import aadc_quantlib_tracing as ql
TODAY = ql.Date(15, 6, 2026)
CAL = ql.TARGET()
DC = ql.Actual365Fixed()
def bond_npv(flat_yield=0.042):
"""NPV of a 7Y annual 4.5% fixed-rate bond on a flat yield curve."""
ql.Settings.instance().evaluationDate = TODAY
rate = aadc.idouble(flat_yield) # market input
curve = ql.YieldTermStructureHandle(
ql.FlatForward(TODAY, ql.QuoteHandle(ql.SimpleQuote(rate)), DC))
issue = CAL.advance(TODAY, ql.Period("2D"))
maturity = CAL.advance(issue, ql.Period("7Y"))
schedule = ql.Schedule(issue, maturity, ql.Period("1Y"), CAL,
ql.Unadjusted, ql.Unadjusted,
ql.DateGeneration.Backward, False)
bond = ql.FixedRateBond(2, 100.0, schedule, [0.045],
ql.ActualActual(ql.ActualActual.ISMA, schedule))
bond.setPricingEngine(ql.DiscountingBondEngine(curve))
return bond.NPV()
def barrier_npv(spot=100.0, vol=0.20, rate=0.03, strike=105.0, barrier=130.0):
"""Up-and-out call, 1Y, analytic Black-Scholes barrier engine."""
ql.Settings.instance().evaluationDate = TODAY
s = aadc.idouble(spot) # market input
v = aadc.idouble(vol) # market input
r = aadc.idouble(rate) # market input
process = ql.BlackScholesProcess(
ql.QuoteHandle(ql.SimpleQuote(s)),
ql.YieldTermStructureHandle(
ql.FlatForward(TODAY, ql.QuoteHandle(ql.SimpleQuote(r)), DC)),
ql.BlackVolTermStructureHandle(
ql.BlackConstantVol(TODAY, CAL, ql.QuoteHandle(ql.SimpleQuote(v)), DC)))
expiry = CAL.advance(TODAY, ql.Period("1Y"))
option = ql.BarrierOption(ql.Barrier.UpOut, barrier, 0.0,
ql.PlainVanillaPayoff(ql.Option.Call, strike),
ql.EuropeanExercise(expiry))
option.setPricingEngine(ql.AnalyticBarrierEngine(process))
return option.NPV()
Writing pricers.py
import importlib, pricers
importlib.reload(pricers); # pick up the file just written, also on a re-run
2. Record a fixed-rate bond¶
start_recording(trace=True) turns on source tracing. Every number the calculation starts
from becomes a leaf of the tape: the yield created in pricers.py, and every number
QuantLib uses internally. Each leaf can be changed later on replay.
bond = aadc.Functions()
bond.start_recording(trace=True)
npv = pricers.bond_npv(flat_yield=0.042)
npv.mark_as_output()
bond.stop_recording()
NPV = float(npv)
print("NPV =", NPV)
NPV = 101.20511929299454 [aadc] Community Edition -- licensed for non-commercial and academic use only. For commercial use or advanced features contact info@matlogica.com
stats: what am I looking at?¶
ask(bond, "stats")
# aadc-tape/2 verb=stats status=ok
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
entries 422 attributed 421 (99%)
sites 54 files 19
inputs 45 hardcoded 0
outputs 1 constants 13 leaf/leaves (one per source site), each a replay input (#N)
replay set EVERY leaf, inputs AND constants (AADCFunctions::seedMarkedLeaves(ws) seeds all
at their recorded values); an unset Random leaf reads an unset cell, not the literal
frozen branches 8 site(s), 96 conversion(s) !! this tape is valid only at its recording point
recorded switches 0 (recording did not set CAADNG_RecOpt_RegisterSwitches)
dead entries 31 tape_ir 20
ids one plane -- every value is an entry #N (an AADC_TRACE tape, or CAADNG_RecOpt_TraceIds = 1)
slots stable (recording option: recycling off)
attributedis the first number to check. Close to 100% means almost every operation knows its source line. A low number means part of the code was not built with source tracing, and answers about that part can't name a line.ids one plane: every value on the tape has an id of the form#N: tape entry N and the value it computed. That covers inputs, constants, intermediate results and outputs. A bare number in a request or an answer is always a value, never an id.inputs/constants: the leaves.inputsare values the program created (rateinpricers.py, and QuantLib's own variables);constantsare literals written inside formulas.frozen branches:ifstatements on a recorded value, decided once at record time. For this bond they are QuantLib's own validity checks (is the quote set, is the date in range). Section 9 shows branches that matter.
Take ids from answers. Don't guess them. This notebook pulls every id it uses out of an earlier answer.
3. Point the tape at the source, once¶
The tape records each file at the path where it was compiled. QuantLib was compiled in a build directory that isn't on your machine. Tell the tape once where the source is now, and every later answer shows the real source lines.
Set QL_SOURCE to a QuantLib 1.41 source tree (the directory that contains ql/):
QL_SOURCE = pathlib.Path(os.environ.get("QL_SOURCE", "quantlib-src")).absolute()
print("QuantLib source:", QL_SOURCE, "(found)" if (QL_SOURCE / "ql").is_dir() else "(NOT found)")
ask(bond, f"source root=/project/work/QLAADC={QL_SOURCE}")
QuantLib source: <tutorial>/quantlib-src (found) # aadc-tape/2 verb=source status=ok action=set added=1 replaced=0 unchanged=0 roots=1 files=19 mapped=14 readable=14 # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. # source roots remembered on this tape. Every request on it reads source through them # without root=; a request's own root= items add to them and win on the same FROM; # the LONGEST matching FROM wins (whole path components, '/' and '\' equal). root=/project/work/QLAADC=<tutorial>/quantlib-src resolves=14/14 # resolves=K/M: M recorded files map through this root, K of them are readable at # the mapped path (a mapped file that is missing is never read from the recorded path) # files: 19 recorded, 14 mapped (14 readable), 5 unmapped (1 readable as recorded) # forms: source root=FROM=TO [root=FROM=TO ...] | source remove=FROM [remove=FROM ...] | source clear | source # the table lives on this tape object until it is freed; saveTape does not write it
resolves=K/M says how many of the recorded files this mapping covers (M) and how many
of those exist at the new location (K). You can call source root= again for other
projects whose code is on the tape (boost, the C++ standard library, your own libraries).
source alone lists the mappings, source remove=<FROM> drops one, and source clear
drops them all. The mappings belong to this tape object; they are not saved with the tape.
The last line of the answer counts the files this mapping doesn't cover: here boost,
the C++ standard library and the SWIG wrapper. Add a source root= for each of them if you
have their source.
4. Where did the work happen? sites and at¶
sites lists every source location that recorded something, busiest first. None of these
are in your code. They are QuantLib's own day counter, compounding and cash-flow loop:
ask(bond, "sites limit=12")
# aadc-tape/2 verb=sites sites=54 order=entries shown=12 limit-hit=true # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. first entries cmps a2p switches where #48 52 25 25 0 interestrate.cpp:47:9 in compoundFactor #45 51 25 25 0 interestrate.cpp:46:9 in compoundFactor #61 37 0 0 0 actual365fixed.hpp:57:42 in yearFraction #63 37 18 18 0 termstructure.cpp:66:9 in checkRange #77 19 0 0 0 interestrate.hpp:70:23 in discountFactor #68 18 18 18 0 termstructure.cpp:68:9 in checkRange #75 18 0 0 0 interestrate.cpp:54:31 in compoundFactor #76 18 0 0 0 interestrate.cpp:54:20 in compoundFactor #79 16 0 0 0 cashflows.cpp:443:41 in npv #80 16 0 0 0 cashflows.cpp:443:26 in npv #60 10 0 0 0 actual365fixed.hpp:57:24 in yearFraction #30 8 7 7 0 actualactual.cpp:114:35 in yearFractionWithReferenceDates<QuantLib::ActualActual::ISMA_Impl>
at <file>:<line> shows what one line recorded, with five lines of source either side.
cashflows.cpp:443 is QuantLib's cash-flow loop. It ran many times, so at summarises
the shape of each run: one shape means every run did the same thing.
ask(bond, "at cashflows.cpp:443")
# aadc-tape/2 verb=at file=cashflows.cpp lines=443-443 context=5 sections=16 entries=32 anchored-on=cashflows.cpp:443 mode=summary(auto: sections>8 -- pass id= for one section) shapes=1 scanned=16 source=ok path=<tutorial>/quantlib-src/ql/cashflows/cashflows.cpp compiled-as=/project/work/QLAADC/ql/cashflows/cashflows.cpp hash=ok
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
# source: <tutorial>/quantlib-src/ql/cashflows/cashflows.cpp (compiled as /project/work/QLAADC/ql/cashflows/cashflows.cpp; root /project/work/QLAADC=<tutorial>/quantlib-src)
# shape (identical across the 16 scanned section(s)):
totalNPV = totalNPV + (in0 * in1)
# anchors: #79=4.31392110552587 #112=4.135060891775504 #142=3.965441201042605 #172=3.802779289224533 #202=3.646370142624317 #232=3.4959918368858216 #262=3.3522009255179044 #276=74.49335390039799 #304=4.31392110552587 #318=4.135060891775504 #332=3.965441201042605 #346=3.802779289224533 #360=3.646370142624317 #374=3.4959918368858216 #388=3.3522009255179044 #402=74.49335390039799
438 |
439 | Real totalNPV = 0.0;
440 | for (const auto& i : leg) {
441 | if (!i->hasOccurred(settlementDate, includeSettlementDateFlows) &&
442 | !i->tradingExCoupon(settlementDate))
443 | totalNPV += i->amount() * discountCurve.discount(i->date());
444 | }
445 |
446 | return totalNPV/discountCurve.discount(npvDate);
447 | }
448 |
The anchors are the discounted cash flows, in two runs of eight: seven coupons and the
redemption. QuantLib's DiscountingBondEngine prices the bond twice: once for the NPV
and once for the settlement value.
5. Where is the number my program printed? find¶
found = bond.trace_inspect(f"find value={NPV!r}")
print(found)
# aadc-tape/2 verb=find value=101.20511929299454 rtol=1e-12(default) matches=4 exact=4 unevaluated=0 # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. #277 read by 1 produced #277 cashflows.cpp:443:26 in npv = 101.20511929299454 #291 read by 2 produced #291 cashflows.cpp:446:24 in npv = 101.20511929299454 #403 read by 1 produced #403 cashflows.cpp:443:26 in npv = 101.20511929299454 #421 sink, marked OUTPUT produced #421 cashflows.cpp:446:24 in npv (= #291) = 101.20511929299454 # `=` exact (bitwise), listed FIRST; `~` within the tolerance, nearest first # (d = row - value). A value is not unique -- the count above is complete, # never assume the first row is THE answer.
Exact (bitwise) matches are listed first, and the answer says the count is complete. The
default tolerance is relative (rtol=1e-12 in the header), so a printed number is found
whatever its size. The marked OUTPUT is a copy of the entry that computed the price
((= #…)). The other rows are running totals from the two pricing runs.
6. What produced it? origin and uses¶
origin folds the arithmetic behind an entry into one formula over its inputs, and shows
the source around it:
PRICE = re.search(r"\(= (#\d+)\)", found).group(1)
print("the price is computed by", PRICE, "\n")
ask(bond, f"origin {PRICE}")
the price is computed by #291
# aadc-tape/2 verb=origin found=yes context=5 source=ok path=<tutorial>/quantlib-src/ql/cashflows/cashflows.cpp compiled-as=/project/work/QLAADC/ql/cashflows/cashflows.cpp hash=ok
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
#291 DivTT at cashflows.cpp:446:24 in npv
# source: <tutorial>/quantlib-src/ql/cashflows/cashflows.cpp (compiled as /project/work/QLAADC/ql/cashflows/cashflows.cpp; root /project/work/QLAADC=<tutorial>/quantlib-src)
441 | if (!i->hasOccurred(settlementDate, includeSettlementDateFlows) &&
442 | !i->tradingExCoupon(settlementDate))
443 | totalNPV += i->amount() * discountCurve.discount(i->date());
444 | }
445 |
> 446 | return totalNPV/discountCurve.discount(npvDate);
^
447 | }
448 |
449 | Real CashFlows::bps(const Leg& leg,
450 | const YieldTermStructure& discountCurve,
451 | const ext::optional<bool>& includeSettlementDateFlows,
#291 = totalNPV@#277 / (1@#77 / (exp((#0 * #279)))) = 101.20511929299454
reads: totalNPV@#277, #290
read by: #419 (CmpNE, instrument.hpp:170), #421 (CopyT, output copy: cashflows.cpp:446:24 in npv (= #291))
leaves with fn=bond_npv keeps the leaves declared in our pricer: the yield. uses
then shows which QuantLib line reads it:
mine = bond.trace_inspect("leaves fn=bond_npv")
print(mine)
YIELD = re.search(r"^\s+(#\d+)\s.*pricers\.py", mine, re.M).group(1)
ask(bond, f"uses {YIELD} by=site")
# aadc-tape/2 verb=leaves status=ok leaves=1 rows=1 input=1 hardcoded=0 constant=0 unread=0 by=site fn=bond_npv suspicious=0 value-class=0 no-literal=0
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
var name value times reads kind declared
#0 - 0.042 37 37 input pricers.py:19 in bond_npv
# var = the leaf's id (its copy entry #N -- the first of the row). ONE leaf per source
# site (and value) when the op carries a site: a literal in a loop is one leaf,
# shared by every execution; a literal with NO site (code not compiled through the
# plugin) is one leaf per use;
# times = executions reading it = reads = tape entries that consume the value
# THROUGH #N (x*x counts once; the copy itself never counts); unread = consumed by
# no entry. kind: input = a variable the program declared, constant = an inline
# constant -- both are replay inputs with their own adjoint (summed over the reads);
# a replay must SET every one (AADCFunctions::seedMarkedLeaves(ws) seeds all at their
# recorded values); `replay set=#N=V` and AADC_TRACE_SUBST #N=V take either.
# aadc-tape/2 verb=uses context=5 source=ok hash=unverified status=ok readers=37 sites=3
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
#0 produced by #0 CopyT at pricers.py:19 in bond_npv = 0.042
14 |
15 |
16 | def bond_npv(flat_yield=0.042):
17 | """NPV of a 7Y annual 4.5% fixed-rate bond on a flat yield curve."""
18 | ql.Settings.instance().evaluationDate = TODAY
> 19 | rate = aadc.idouble(flat_yield) # market input
20 | curve = ql.YieldTermStructureHandle(
21 | ql.FlatForward(TODAY, ql.QuoteHandle(ql.SimpleQuote(rate)), DC))
22 |
23 | issue = CAL.advance(TODAY, ql.Period("2D"))
24 | maturity = CAL.advance(issue, ql.Period("7Y"))
reads first# where
1 #70 simplequote.hpp:69:22 in isValid
18 #73 interestrate.cpp:47:9 in compoundFactor
18 #75 interestrate.cpp:54:31 in compoundFactor
7. The numbers that went into the model: leaves¶
fn=!bond_npv keeps every leaf declared outside our pricer: the numbers QuantLib
supplied itself.
ask(bond, "leaves fn=!bond_npv")
# aadc-tape/2 verb=leaves status=ok leaves=57 rows=43 input=44 hardcoded=0 constant=13 unread=16 by=site fn=!bond_npv suspicious=3 value-class=0 no-literal=3 shown=40 limit-hit=true # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. var name value times reads kind declared #1 arg1 0 0 0 input wrap_part1.cxx:19043:16 in _wrap_new_SimpleQuote__SWIG_0 unread #2 this 3.4028234663852886e+38 0 0 input interestrate.cpp:31:10 in InterestRate unread #3 this 0 0 0 input interestrate.cpp:30:19 in InterestRate unread #4 arg2 0 0 0 input wrap_part10.cxx:6312:8 in _wrap_new_FixedRateBond unread #5 arg7 100 1 1 input wrap_part10.cxx:6317:15 in _wrap_new_FixedRateBond #6 - 0 0 0 input vector.tcc:863:3 in _M_default_append unread #7 this 3.4028234663852886e+38 0 0 input instrument.cpp:27:12 in Instrument unread #8 this 3.4028234663852886e+38 0 0 input instrument.cpp:27:42 in Instrument unread #9 - 0 0 0 input bond.cpp:38:11 in Bond unread #10 - 0 0 0 input interestrate.cpp:33:19 in InterestRate unread #11 - 3.4028234663852886e+38 0 0 input coupon.cpp:37:51 in Coupon unread #12 - 0 0 0 input fixedratecoupon.cpp:45:22 in FixedRateCoupon unread #13 - 0 1 1 input bond.cpp:394:30 in calculateNotionalsFromCashflows #14 - 100 1 1 constant bond.cpp:315:29 in addRedemptionsToCashflows #16 - 100 1 1 constant bond.cpp:315:53 in addRedemptionsToCashflows #19 - 3.4028234663852886e+38 1 1 input simplecashflow.cpp:31:9 in SimpleCashFlow #22 this 0 0 0 input pricingengine.hpp:63:11 in GenericEngine unread #24 result 0 0 0 input wrap_part6.cxx:18079:8 in _wrap_Instrument_NPV unread #25 - 3.4028234663852886e+38 0 0 input pricingengine.hpp:68:42 in reset unread #26 - 3.4028234663852886e+38 0 0 input instrument.hpp:117:37 in reset unread #27 totalNPV 0 2 2 input cashflows.cpp:439:25 in npv #28 - 0 7 7 input actualactual.cpp:168:32 in yearFraction #29 referenceDayCount 365 .. 366 14 14 input actualactual.cpp:111:38 in yearFractionWithReferenceDates<QuantLib::ActualActual::ISMA_Impl> #30 - 16 7 7 input actualactual.cpp:114:35 in yearFractionWithReferenceDates<QuantLib::ActualActual::ISMA_Impl> #32 - 365 .. 366 7 7 input actualactual.cpp:34:55 in findCouponsPerYear<QuantLib::ActualActual::ISMA_Impl> #33 - 12 7 7 constant actualactual.cpp:34:53 in findCouponsPerYear<QuantLib::ActualActual::ISMA_Impl> #35 - 365 7 7 constant actualactual.cpp:34:93 in findCouponsPerYear<QuantLib::ActualActual::ISMA_Impl> #37 - 12 7 7 input actualactual.cpp:35:48 in findCouponsPerYear<QuantLib::ActualActual::ISMA_Impl> #38 - 12 7 7 constant actualactual.cpp:35:46 in findCouponsPerYear<QuantLib::ActualActual::ISMA_Impl> #40 - 365 .. 366 7 7 input actualactual.cpp:121:20 in yearFractionWithReferenceDates<QuantLib::ActualActual::ISMA_Impl> #41 - 1 7 7 constant actualactual.cpp:121:68 in yearFractionWithReferenceDates<QuantLib::ActualActual::ISMA_Impl> suspicious=no-literal #45 - 0 25 25 input interestrate.cpp:46:9 in compoundFactor #48 - 0.045 .. 3.4028234663852886e+38 32 32 input interestrate.cpp:47:9 in compoundFactor #52 - 0.045 7 7 constant interestrate.cpp:50:28 in compoundFactor #54 - 1 7 7 constant interestrate.cpp:50:24 in compoundFactor #56 - 1 7 7 constant fixedratecoupon.cpp:69:85 in performCalculations suspicious=no-literal #58 - 100 7 7 constant fixedratecoupon.cpp:68:29 in performCalculations suspicious=no-literal #60 - 0 .. 63386 36 36 input actual365fixed.hpp:57:24 in yearFraction #61 - 365 36 36 constant actual365fixed.hpp:57:42 in yearFraction #63 - 0 18 18 input termstructure.cpp:66:9 in checkRange ... 43 rows, list capped at 40 (limit=N raises it) # suspicious = a value the code may never have WRITTEN: subnormal (an integer or # pointer bit pattern read as a double), nan / nan-payload / inf (bits shown), or # no-literal (a constant whose source line has no numeric literal -- read from memory: # a plain-double variable, a named constant, or an out-of-bounds / uninitialised read). # A lead to check, not a verdict; `leaves suspicious=on` lists only these. # var = the leaf's id (its copy entry #N -- the first of the row). ONE leaf per source # site (and value) when the op carries a site: a literal in a loop is one leaf, # shared by every execution; a literal with NO site (code not compiled through the # plugin) is one leaf per use; # times = executions reading it = reads = tape entries that consume the value # THROUGH #N (x*x counts once; the copy itself never counts); unread = consumed by # no entry. kind: input = a variable the program declared, constant = an inline # constant -- both are replay inputs with their own adjoint (summed over the reads); # a replay must SET every one (AADCFunctions::seedMarkedLeaves(ws) seeds all at their # recorded values); `replay set=#N=V` and AADC_TRACE_SUBST #N=V take either.
For model validation this is the list of numbers your code never passed in, each with the
line that declared it: day counts (365 .. 366), the coupon period in months (12), the
default redemption 100, and QuantLib's Null<Real> sentinel 3.4028234663852886e+38
used in validity checks. Rows marked unread are values QuantLib set up but this pricing
never used.
Three rows carry suspicious=no-literal (the header counts them: suspicious=3). Each
is a constant whose source line contains no number, so the value came from memory rather
than from a literal. Here all three are harmless: fixedratecoupon.cpp:68 multiplies by
nominal() (the 100), actualactual.cpp:121 by couponsPerYear (the 1), and the
1.0 charged to fixedratecoupon.cpp:69 is written on the next line. The flag is a lead,
not a verdict. The same check flags a NaN, an infinity or a subnormal number on any leaf,
the trace of an uninitialised or out-of-bounds read; leaves suspicious=on lists only the
flagged rows.
8. What-if without repricing: replay¶
replay set=<leaf>=<value> re-evaluates the recorded tape at a new value. It doesn't run
QuantLib again, so it is cheap. Move the yield from 4.2% to 5% and compare with QuantLib
repricing the bond:
def replayed(tape, settings, out):
# The replayed value of entry `out`, or None if replay withheld it.
answer = tape.trace_inspect(f"replay {settings} out={out}")
m = re.search(rf"^\s*{out}\s.*->\s*(\S+)\s*$", answer, re.M)
return answer, (None if m is None or m.group(1) == "SUSPECT" else float(m.group(1)))
OUT_BOND = re.search(r"^\s+(#\d+)\s+.*marked OUTPUT", found, re.M).group(1)
answer, value = replayed(bond, f"set={YIELD}=0.05", OUT_BOND)
print(answer)
print("QuantLib at 5%:", float(pricers.bond_npv(flat_yield=0.05)))
# aadc-tape/2 verb=replay set=1 flips=0 crossings=0 status=ok # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. set: #0 (#0) = 0.042 -> 0.05 outputs at the new inputs: (1 picked by out=; the tape has 1 marked output(s)) #421 = 101.20511929299454 -> 96.33676425183306 # answering: the recorded path SURVIVES at these inputs (no recorded # comparison flipped, no COUNTED active->passive value changed), so the # re-evaluated values above ARE the tape's values there. A SILENT # extraction (AAD_PASSIVE_IO / AAD_PASSIVE_SAFE / .value()) is not recorded # and cannot be checked. # 84 active->passive conversion(s) re-evaluated: unchanged at these inputs. QuantLib at 5%: 96.33676425183306
Same number, to the last digit. The answer gives the output first, then the verdict
(SURVIVES). out= picks the output to show; without it, replay lists the declared
outputs first.
Marked inputs work the same way¶
For derivatives you mark inputs with mark_as_input(). Under trace=True the mark keeps
the value and the line, and it stays the same leaf: one variable, one id.
f = aadc.Functions()
f.start_recording(trace=True)
x = aadc.idouble(2.0)
x_in = x.mark_as_input()
y = x * x + 1.5
y.mark_as_output()
f.stop_recording()
leaf = f.trace_inspect("leaves")
print(leaf)
X = re.search(r"^\s+(#\d+)\s", leaf.split("declared", 1)[1], re.M).group(1)
print(f.trace_inspect(f"replay set={X}=3"))
# aadc-tape/2 verb=leaves status=ok leaves=2 rows=2 input=1 hardcoded=0 constant=1 unread=0 by=site suspicious=0 value-class=0 no-literal=0 literal-unchecked=1 # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. var name value times reads kind declared #0 - 2 1 1 input 1797724670.py:4 in <module> #2 - 1.5 1 1 constant 1797724670.py:5 in <module> # var = the leaf's id (its copy entry #N -- the first of the row). ONE leaf per source # site (and value) when the op carries a site: a literal in a loop is one leaf, # shared by every execution; a literal with NO site (code not compiled through the # plugin) is one leaf per use; # times = executions reading it = reads = tape entries that consume the value # THROUGH #N (x*x counts once; the copy itself never counts); unread = consumed by # no entry. kind: input = a variable the program declared, constant = an inline # constant -- both are replay inputs with their own adjoint (summed over the reads); # a replay must SET every one (AADCFunctions::seedMarkedLeaves(ws) seeds all at their # recorded values); `replay set=#N=V` and AADC_TRACE_SUBST #N=V take either. # aadc-tape/2 verb=replay set=1 flips=0 crossings=0 status=ok # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. set: #0 (#0) = 2 -> 3 outputs at the new inputs: #4 = 5.5 -> 10.5 # answering: the recorded path SURVIVES at these inputs (no recorded # comparison flipped, no COUNTED active->passive value changed), so the # re-evaluated values above ARE the tape's values there. A SILENT # extraction (AAD_PASSIVE_IO / AAD_PASSIVE_SAFE / .value()) is not recorded # and cannot be checked.
9. An option with real branches: branches and replay¶
An up-and-out barrier option on QuantLib's analytic engine:
barrier = aadc.Functions()
barrier.start_recording(trace=True)
bnpv = pricers.barrier_npv(spot=100.0, vol=0.20, rate=0.03, strike=105.0, barrier=130.0)
bnpv.mark_as_output()
barrier.stop_recording()
print("NPV =", float(bnpv))
ask(barrier, f"source root=/project/work/QLAADC={QL_SOURCE}");
NPV = 1.8686223378903692 # aadc-tape/2 verb=source status=ok action=set added=1 replaced=0 unchanged=0 roots=1 files=25 mapped=20 readable=20 # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. # source roots remembered on this tape. Every request on it reads source through them # without root=; a request's own root= items add to them and win on the same FROM; # the LONGEST matching FROM wins (whole path components, '/' and '\' equal). root=/project/work/QLAADC=<tutorial>/quantlib-src resolves=20/20 # resolves=K/M: M recorded files map through this root, K of them are readable at # the mapped path (a mapped file that is missing is never read from the recorded path) # files: 25 recorded, 20 mapped (20 readable), 5 unmapped (1 readable as recorded) # forms: source root=FROM=TO [root=FROM=TO ...] | source remove=FROM [remove=FROM ...] | source clear | source # the table lives on this tape object until it is freed; saveTape does not write it
branches order=margin lists the ifs decided at record time, closest first. The
margin is left - right of the comparison at record time: a small margin means a
small input move could flip that branch.
ask(barrier, "branches order=margin limit=12")
# aadc-tape/2 verb=branches sites=33 rows=240 frozen=33-sites/248-conversions source=tape-scan order=margin # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. # no switch registry on this tape (record with CAADNG_RecOpt_RegisterSwitches for exact # per-occurrence g); margins below are record-time re-evaluation where possible entry count kind op margin(first) where #1069 1 FROZEN < 0 analyticbarrierengine.cpp:244:22 in F ibool->bool x1 #783 1 FROZEN == 0.004273115088376578 analyticbarrierengine.cpp:209:61 in C ibool->bool x1 #174 3 FROZEN < 0.004277411874295244 errorfunction.cpp:210:19 in operator() ibool->bool x3 #801 1 FROZEN == 0.007555832953428043 analyticbarrierengine.cpp:210:61 in C ibool->bool x1 #108 12 FROZEN == 0.030454533953516938 interestrate.cpp:79:21 in impliedRate ibool->bool x12 #1042 1 FROZEN == 0.059165043626815295 analyticbarrierengine.cpp:222:61 in D ibool->bool x1 #1060 1 FROZEN == 0.08662714037521507 analyticbarrierengine.cpp:223:61 in D ibool->bool x1 #412 5 FROZEN < -0.35775758632940957 errorfunction.cpp:221:15 in operator() ibool->bool x5 #213 8 FROZEN <= 0.5024132486078934 normaldistribution.cpp:36:19 in operator() ibool->bool x8 #172 8 FROZEN < -0.8394725844004145 errorfunction.cpp:209:15 in operator() ibool->bool x8 #454 3 FROZEN >= -0.8922424136705904 errorfunction.cpp:225:17 in operator() ibool->bool x3 #642 2 FROZEN < -0.9976699360834274 errorfunction.cpp:234:15 in operator() ibool->bool x2 ... capped at 12 site(s) -- raise limit= to see the rest # margin is SIGNED g (left - right) at record time; a site taken more than once # has a margin PER OCCURRENCE -- `branches at=<file>:<line>` lists them # a margin of 0 (or nan) sits ON the threshold: the sign does not say the outcome: # < > != are false at 0, <= >= == true, so a move off 0 can flip it (nan: all but != false)
Two branches matter for pricing: one in QuantLib's piecewise approximation of erf
(errorfunction.cpp), and the knock-out test itself (barrieroption.cpp:133). The rest
are validity checks. Here is the knock-out test with its source:
ask(barrier, "at barrieroption.cpp:133")
# aadc-tape/2 verb=at file=barrieroption.cpp lines=133-133 context=5 sections=1 entries=2 anchored-on=barrieroption.cpp:133 anchor=#55 section=1-of-1 source=ok path=<tutorial>/quantlib-src/ql/instruments/barrieroption.cpp compiled-as=/project/work/QLAADC/ql/instruments/barrieroption.cpp hash=ok
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
# anchors: #55
-- IN -- read here, produced elsewhere ------------------------------
#54 130 constant leaf (a replay input), at barrieroption.cpp:133:31 in triggered
#0 100 <- #0 pricers.py:37 in barrier_npv
# source: <tutorial>/quantlib-src/ql/instruments/barrieroption.cpp (compiled as /project/work/QLAADC/ql/instruments/barrieroption.cpp; root /project/work/QLAADC=<tutorial>/quantlib-src)
128 | case Barrier::DownIn:
129 | case Barrier::DownOut:
130 | return underlying < arguments_.barrier;
131 | case Barrier::UpIn:
132 | case Barrier::UpOut:
133 | return underlying > arguments_.barrier;
|- #55 = #0 > 130@#54 = 0 [#55]
!! FROZEN here: ibool->bool, 1 conversion(s). This tape recorded ONE arm;
the other is absent. margin = -30 (left - right at record time)
134 | default:
135 | QL_FAIL("unknown type");
136 | }
137 | }
138 |
-- OUT -- produced here, read elsewhere (or marked output) -----------
(nothing crosses out and nothing here is a marked output)
Replay at other market data¶
The three market inputs are the leaves declared in barrier_npv:
inputs = barrier.trace_inspect("leaves fn=barrier_npv")
print(inputs)
SPOT, VOL, RATE = re.findall(r"^\s+(#\d+)\s.*pricers\.py", inputs, re.M)
OUT_BARRIER = re.search(r"^\s+(#\d+)\s+.*marked OUTPUT",
barrier.trace_inspect(f"find value={float(bnpv)!r}"), re.M).group(1)
print("spot", SPOT, " vol", VOL, " rate", RATE, " price", OUT_BARRIER)
# aadc-tape/2 verb=leaves status=ok leaves=3 rows=3 input=3 hardcoded=0 constant=0 unread=0 by=site fn=barrier_npv suspicious=0 value-class=0 no-literal=0 # ids: #N = tape entry N and the value it computed -- this tape names EVERY value # (input, constant, output, intermediate) this way. A bare number is always a # VALUE, never an id. var name value times reads kind declared #0 - 100 21 21 input pricers.py:37 in barrier_npv #1 - 0.2 36 36 input pricers.py:38 in barrier_npv #2 - 0.03 21 21 input pricers.py:39 in barrier_npv # var = the leaf's id (its copy entry #N -- the first of the row). ONE leaf per source # site (and value) when the op carries a site: a literal in a loop is one leaf, # shared by every execution; a literal with NO site (code not compiled through the # plugin) is one leaf per use; # times = executions reading it = reads = tape entries that consume the value # THROUGH #N (x*x counts once; the copy itself never counts); unread = consumed by # no entry. kind: input = a variable the program declared, constant = an inline # constant -- both are replay inputs with their own adjoint (summed over the reads); # a replay must SET every one (AADCFunctions::seedMarkedLeaves(ws) seeds all at their # recorded values); `replay set=#N=V` and AADC_TRACE_SUBST #N=V take either. spot #0 vol #1 rate #2 price #1075
Each replay below is checked against QuantLib repricing the option with the same inputs:
def check(settings, **market):
answer, value = replayed(barrier, settings, OUT_BARRIER)
flips = re.search(r"flips=(\d+)", answer.splitlines()[0]).group(1)
try:
quantlib = repr(float(pricers.barrier_npv(**market)))
except RuntimeError as e: # QuantLib refuses: spot beyond the barrier
quantlib = f"refuses ({e})"
verdict = ("withheld: a branch flips" if value is None
else "same" if repr(value) == quantlib else "DIFFERENT")
shown = "SUSPECT" if value is None else repr(value)
print(f"{settings:<34} flips={flips} replay {shown:<20} QuantLib {quantlib:<24} {verdict}")
check(f"set={SPOT}=101", spot=101.0)
check(f"set={VOL}=0.21", vol=0.21)
check(f"set={RATE}=0.035", rate=0.035)
check(f"set={SPOT}=101,{VOL}=0.21", spot=101.0, vol=0.21)
check(f"set={SPOT}=110", spot=110.0)
check(f"set={SPOT}=135", spot=135.0)
set=#0=101 flips=0 replay 1.8996278525395591 QuantLib 1.8996278525395591 same set=#1=0.21 flips=0 replay 1.7291409611132473 QuantLib 1.7291409611132473 same set=#2=0.035 flips=0 replay 1.8939296859360053 QuantLib 1.8939296859360053 same set=#0=101,#1=0.21 flips=1 replay SUSPECT QuantLib 1.7504945353414048 withheld: a branch flips set=#0=110 flips=1 replay SUSPECT QuantLib 1.8408999962344375 withheld: a branch flips set=#0=135 flips=1 replay SUSPECT QuantLib refuses (barrier touched) withheld: a branch flips
With flips=0 the replay is exactly QuantLib's price. Moving spot and vol together
flips a branch that neither move flips alone, and so does spot 110 on its own. The answer
names it:
for settings in (f"set={SPOT}=101,{VOL}=0.21", f"set={SPOT}=110"):
answer, _ = replayed(barrier, settings, OUT_BARRIER)
print(settings)
print("\n".join(l for l in answer.splitlines() if l.lstrip().startswith("FLIPPED")))
set=#0=101,#1=0.21 FLIPPED #410 frozen < margin 0.04849241367059043 -> -0.02059711802749664 errorfunction.cpp:209:15 in operator() set=#0=110 FLIPPED #410 frozen < margin 0.04849241367059043 -> -0.28847995860814546 errorfunction.cpp:209:15 in operator()
Both flip errorfunction.cpp:209: an argument of erf crosses the boundary between two
pieces of QuantLib's piecewise approximation (its margin changes sign, from 0.048 at record time).
The price itself is smooth there, but the tape holds only the piece used at record time,
so replay withholds the number. At spot 135 the knock-out test flips: the spot is
already above the barrier, and QuantLib itself refuses to price the option there. The tape
holds only the arm recorded at spot 100, so replay withholds the price instead of
returning a wrong one. To price there, record again at the new inputs.
The full answer at spot 135:
answer, _ = replayed(barrier, f"set={SPOT}=135", OUT_BARRIER)
print(answer)
# aadc-tape/2 verb=replay set=1 flips=1 crossings=0 unevaluated-branches=240 unevaluated-passive=93 status=ok
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
set: #0 (#0) = 100 -> 135
outputs (values withheld past the flip):
(1 picked by out=; the tape has 1 marked output(s))
#1075 = 1.8686223378903692 -> SUSPECT (downstream of flipped #55 -- the recorded branch's
arithmetic, NOT the value at these inputs)
FLIPPED #55 frozen > margin -30 -> 5 barrieroption.cpp:133:31 in triggered
downstream-unknown=240 (first #57 at simplequote.hpp:69:22 in isValid, last #1073 at instrument.hpp:170:9 in NPV)
not compared: past flipped #55 their margins are the recorded branch's;
unknown=on lists them
downstream-passive=93 active->passive conversion(s) past flipped #55 not checked;
unknown=on lists them
!! the recorded path is no longer the path this program would take at these
inputs; values downstream of flipped #55 are the RECORDED path's and are
not the answer at this point. Re-record at these inputs to price here.
# answering: "did the recorded path survive?" -- NO. Values are reported
# only for entries BEFORE the first decision; the rest are withheld.
# `path=recorded` returns the recorded path's values at these inputs,
# labelled RECORDED-PATH (an explicit approximation, e.g. a converged
# solver's stopping test; tolerate=#N[,#M] limits it to the listed flips).
# NOTE: 240 branch site(s) could not be compared (operands this walk cannot
# evaluate, or downstream of a decision) -- "no flip" is proven only for the
# compared sites.
The output comes first, withheld. Then the branch that flipped, barrieroption.cpp:133,
and two summaries of what comes after it: downstream-unknown=240 branches and
downstream-passive=93 active-to-passive conversions that can't be judged past the flip
(unknown=on lists them).
cone: what can the price depend on?¶
cone sorts the leaves by how they can affect a value. Data leaves have a derivative.
Control-only leaves change the price only by flipping a recorded branch, which no
derivative shows; the barrier level is one of them. The header gives the counts:
print(barrier.trace_inspect(f"cone {OUT_BARRIER}").splitlines()[0])
# aadc-tape/2 verb=cone dir=back leaves=83 of=157 entries=587 control=347 control-only=40 unreachable=34 birth=#1075 control-scope=whole-tape
10. Save a tape, inspect it later¶
A tape is a byte string. Save it, send it to a colleague or to MatLogica support, and query it later. Source mappings are not saved with it, so set them again after loading. Then ask the loaded tape the replay that section 9 answered at spot 101:
pathlib.Path("barrier.tape").write_bytes(barrier.save_tape())
later = aadc.Functions()
later.load_tape(pathlib.Path("barrier.tape").read_bytes())
later.trace_inspect(f"source root=/project/work/QLAADC={QL_SOURCE}")
print(later.trace_inspect(f"replay set={SPOT}=101 out={OUT_BARRIER}"))
# aadc-tape/2 verb=replay set=1 flips=0 crossings=0 passive-changed=0 unchecked=1 unevaluated-branches=248 status=ok
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
set: #0 (#0) = 100 -> 101
outputs (values withheld past the decision):
(1 picked by out=; the tape has 1 marked output(s))
#1075 = 1.8686223378903692 -> SUSPECT (downstream of an active->passive conversion recorded without its operand -- the recorded branch's
arithmetic, NOT the value at these inputs)
UNCHECKED - ibool->bool operand not recorded (no per-event record on this tape) errorfunction.cpp:200
downstream-unknown=248 (first #23 at normaldistribution.hpp:266:9 in NormalDistribution, last #1073 at instrument.hpp:170:9 in NPV)
not compared: past an active->passive conversion recorded without its operand their margins are the recorded branch's;
unknown=on lists them
!! the recorded path could not be CHECKED at these inputs: a decision
(an active->passive conversion recorded without its operand)
cannot be re-evaluated: its operand is not on record (a loaded tape
keeps no per-conversion operand; a stale handle names none),
so values downstream of it are withheld. Re-record at these inputs to
price here.
# answering: "did the recorded path survive?" -- UNKNOWN. Values are
# reported only for entries BEFORE the first decision.
# `path=recorded` returns the recorded path's values at these inputs,
# labelled RECORDED-PATH (tolerate=#N[,#M] limits it to the listed decisions).
# An active->passive conversion (AAD_PASSIVE, idouble->int, a bool branch,
# a markAsDiff freeze) is a decision the tape cannot replay: the program may
# have used the value for a loop count or a branch. PASSIVE-CHANGED = the
# extracted value differs at these inputs at all (the tape does not record
# whether only its integer part was used, so this is conservative);
# UNCHECKED = it could not be evaluated or its operand was not recorded.
# NOTE: 248 branch site(s) could not be compared (operands this walk cannot
# evaluate, or downstream of a decision) -- "no flip" is proven only for the
# compared sites.
The loaded tape withholds the price that the in-memory tape returned (1.8996…, equal
to QuantLib). The reason is in the UNCHECKED row: one if in QuantLib's erf
(errorfunction.cpp:200) was decided on a recorded value, and the saved tape has no
per-event record of its operand, so the loaded tape cannot check that this decision still
goes the same way. It says so instead of assuming it.
To get the number anyway, ask for the recorded path. It comes back labelled
RECORDED-PATH, and here it equals the in-memory replay and QuantLib:
answer = later.trace_inspect(f"replay set={SPOT}=101 out={OUT_BARRIER} path=recorded")
row = re.search(rf"^\s*{OUT_BARRIER}\s.*->\s*(\S+)\s+RECORDED-PATH", answer, re.M).group(1)
print("loaded tape, path=recorded:", row)
print("in-memory replay :", repr(replayed(barrier, f"set={SPOT}=101", OUT_BARRIER)[1]))
print("QuantLib :", repr(float(pricers.barrier_npv(spot=101.0))))
loaded tape, path=recorded: 1.8996278525395591 in-memory replay : 1.8996278525395591 QuantLib : 1.8996278525395591
11. The rules of the road¶
A malformed request returns text, not an exception: check the header for
status=error. A bare number is a value, never an id:
ask(bond, "not-a-verb")
ask(bond, "origin 134")
# aadc-tape/2 verb=not-a-verb status=error reason=unknown-verb
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
error: unknown verb `not-a-verb`.
verbs: stats | sites | at | branches | origin | cone | find | replay | uses | leaves | source
# aadc-tape/2 verb=origin status=error reason=bare-number
# ids: #N = tape entry N and the value it computed -- this tape names EVERY value
# (input, constant, output, intermediate) this way. A bare number is always a
# VALUE, never an id.
error: origin `134` is a bare number, and a bare number is a VALUE, never an id.
did you mean #134 (tape entry 134 and the value it computed)?
ids: #N = tape entry N and its value; RD#k / D#k / ... = a location; c#k =
a constants-pool cell.
Asking before a tape exists is a programming error, so it raises:
try:
aadc.Functions().trace_inspect("stats")
except RuntimeError as e:
print("RuntimeError:", e)
RuntimeError: trace_inspect: this Functions has no sealed tape (stop_recording() first)
12. Cheat sheet¶
<id> is #N, taken from an earlier answer.
| request | arguments | answers |
|---|---|---|
stats |
outputs=on dead=on constants=on a2p=on |
overview; each =on adds the list behind one count |
source |
root=FROM=TO …, remove=FROM, clear |
where to read source; set once per tape object |
sites |
file=NAME fn=FUNCTION limit=N |
source locations that recorded something |
at |
<file>:<lo>[-<hi>] or id=<id>, context=N (default 5), summary=on |
what a line range recorded, with source |
origin |
<id> context=N |
the formula behind a value |
uses |
<id> by=entry\|site |
everything that reads a value |
leaves |
kind=input\|constant fn=NAME\|fn=!NAME at=FILE suspicious=on limit=N |
every leaf, with its declaration |
find |
value=X rtol=R tol=EPS at=FILE |
every entry equal to a number you saw, exact matches first |
replay |
set=<id>=<value>[,…] out=<id> limit=N unknown=on path=recorded tolerate=<id> |
re-evaluate at new values: outputs first, then flipped branches |
branches |
order=margin\|count at=FILE[:LINE] limit=N |
every recorded decision with its margin |
cone |
<id> |
data / control-only / unreachable leaves of a value |
Habits that pay off:
- Check
attributedinstatsfirst. - Set
source root=once per tape. - Take ids from answers; never type them by hand.
- Before trusting a replayed value, check
flips=0; a flip means record again.path=recordedgives a labelled approximation, not an answer: check it against a reprice. - A tape describes one run with one set of inputs. Code that didn't run is not on it.