How We Know
The last lesson is not another signal. It is the question that sits under all of them: how do you know any of this is true? The answer is a method — backtest like a fund, publish like a regulator, keep a graveyard, and show the misses — and that method is the same thing that makes the whole apparatus able to teach.
You have learned to read loans, structures, cycles, systems, geographies, and books. A fair and necessary question, the one a good student asks last and a bad one never asks at all: why should I believe any of these signals actually work? The honest disciplines that answer it are the real subject of this course, more than any single metric.
Backtest like a fund. A fund does not get to grade its own homework with hindsight. It asks the brutal question: if I had followed this signal out of sample — on data it was never fit to — would it have caught distress before the fact, cohort after cohort, cycle after cycle? A signal that only “works” on the data used to build it works on nothing. The reason the 30–89 early-warning signal earns its place is that it holds its lift across four decades of cohorts it never saw during construction. Stability out of sample is the difference between a finding and a story you told yourself.
Publish like a regulator. A regulator does not report a point estimate and a swagger. It reports a range, its confidence, and its uncertainty — because it is accountable for being wrong. Adopt that posture: surface the gap and the inputs behind a read, never a bare verdict; give a range, not a false point; say how sure you are. This is why the platform’s charter is awareness, not judgment — it shows you the gap between what a loan claims and what its numbers imply, and the ingredients of that gap, and then it stops, because the verdict is yours to render with context the tool doesn’t have. A tool that hands you a confident number robs you of the judgment that is the actual job.
Keep a graveyard. This is the most unusual discipline and the most honest. Every signal that was tried and failed is kept — not deleted, not buried — in a graveyard, along with the conditions under which it might be reconsidered. A refuted signal is not “false forever”; it is false point-in-time, under a frame, and frames change. Keeping the failures does two things: it stops you from re-running dead ideas as if they were new, and it turns the record of what didn’t work into a curriculum. The graveyard is arguably the most valuable thing the platform holds, because anyone can show you what worked; almost no one will show you, honestly and in detail, what didn’t. Where you can see this discipline made public is the validation scorecard — the live track record with its misses shown alongside its hits, not buried in fine print.
Show the misses. Pull it together and you have the throughline of the entire course: the honesty that makes a signal credible is the same honesty that makes it teach. A black box that is always confidently right teaches nothing and, worse, cannot be trusted, because you can’t see where it would break. An apparatus that shows its work — the backtest, the range, the graveyard, the misses — is both more believable and more educational, and that is not a coincidence. It is the same property viewed from two sides. The platform’s Methodology is where this is laid out in the open: how the signals are tested, what confidence they carry, where they have failed. Read it not as a disclaimer but as the point.
That is the course. You came in able to build a model. You leave able to read one — a loan, a structure, a cycle, a system, a market, a book — and, most importantly, able to ask of any signal the question that separates an analyst from a technician: how do we know? Keep asking it. The answer is always the method, shown in the open, misses and all.