Koraki is an instrument for answering a question that ordinary trading cannot answer: do my rules help or hurt?
It runs six versions of the same position simultaneously — five defined by rules, one recording what actually happened — over an identical set of stock picks and an identical price history. The only thing that differs between them is the rule. What is left after everything else cancels is the effect of the rule itself.
Every trade produces exactly one observed outcome. A position is cut at −8% and what holding would have done is never learned. A winner is held through a drawdown and what a stop would have saved is never learned. The money cannot be in two places, so the counterfactual is permanently unavailable.
This breaks the feedback loop that is supposed to make a trader better. Twenty years of experience accumulates as stories rather than evidence — "I get shaken out too early," "my stops save me" — and because those beliefs are unfalsifiable, they harden with age instead of improving.
Koraki manufactures the missing counterfactual. Every pick is held six different ways at once. Five of those ways are roads not taken.
A pick arrives. It opens a position in all six books on the same day, at the same price, against the same benchmark. Four books apply trend rules of varying construction. One simply holds. The sixth records real fills from a real account.
If two different baskets of stocks were compared — one with stops, one without — the noise would swamp the signal. Single-stock outcomes vary enormously; some names triple and some go to zero, and that variance is vastly larger than the effect any stop rule has. Hundreds of observations would be needed to see through it.
Because every book holds the identical stock over the identical days, that variance cancels completely. A brilliant pick appears in all six books equally and subtracts out. What remains is only what the rule did.
This is the same reasoning behind crossover designs in clinical trials: comparing a patient to themselves beats comparing them to a stranger. It reduces the sample requirement from hundreds to dozens, and it is the entire reason the experiment is feasible for one person.
| Comparison | Question |
|---|---|
| hold vs everything | Does stopping out help at all, or is it pure friction on research that was already good? |
| fixed vs ratchet | Does the pyramiding flaw matter? Under a fixed stop, risk grows as the position grows, which is inverted from what pyramiding is supposed to achieve. |
| fixed vs trailing | Patience or protection? A fixed stop gives a long-horizon thesis room; a trailing stop protects gains but gets shaken out on ordinary chop. |
| hybrid | Is the answer both, at different stages of a position's life? |
| actual vs fixed | Does discretion add or subtract? The same rule in spirit, one executed by a machine and one by a person who gets tired and reads the tape. |
Everything is measured relative to SPY, never in raw price terms. A stock up 8% while the index rose 8% has done nothing. Relative return is defined as the log return of the position minus the log return of the benchmark over the same days.
For each completed pick, the dollars each book finished ahead of the same money in SPY is computed. Subtracting the baseline book's result gives one paired difference. Those are averaged across picks and reported with a confidence interval.
Stated plainly: "Across N picks, the rules added $X per $10,000 versus simply buying the pick and sitting on it."
If X is negative, the rules are taxing the research, and the correct response is to stop using them.
The "right but early" count — how often a stop fired on a name that subsequently recovered above its entry. That is the specific mechanism by which good research loses money, and if it is frequent it explains whatever the headline figure says.
The paired difference is zero whenever no stop ever fires, and large when one does. Estimating roughly half of picks touch a stop, with differences of ±20–30 points on those, puts the standard deviation of the paired difference somewhere near 15 relative-return points. For 80% power at conventional significance:
| Effect size to detect | Completed positions needed |
|---|---|
| 8 points per pick | ~28 |
| 5 points per pick | ~70 |
| 3 points per pick | ~195 |
At a realistic 40 picks a year — and noting that losers complete in weeks while winners take the full twelve months, so completions lag entries badly in year one — this is a one-to-two-year experiment. It is research, not a business, and the timeline is written down on purpose so that nobody is surprised in month four.
The standard deviation above is an estimate. After fifteen completed positions it should be recomputed from real data and this table revised. If the true figure is 8, everything gets much faster; if it is 25, much slower.
"The interval contains zero" is not by itself a finding. Consider three outcomes with the same average:
| Mean | 90% CI | Interpretation | |
|---|---|---|---|
| A | +$8 | ±$400 | The truth could be −$392 or +$408. Nothing has been learned. |
| B | +$8 | ±$40 | Whatever the rules do is bounded to a few tens of dollars per pick. A finding. |
| C | +$8 | ±$4 | Real, and small. |
A wide interval around zero means the measurement failed. A narrow interval around zero means the measurement succeeded and the answer is "nothing much" — which is genuinely useful, because it says stop running rules and do the simpler thing.
An effect can never be proven to be exactly zero. What can be done is squeezing the interval until every effect size worth caring about has been ruled out.
Written into the project README before the first line of code, which is the only moment they can be written honestly.
| Condition | Trigger |
|---|---|
| Attention | Thirty days without opening the dashboard. Not looking is the earliest and cheapest kill signal there is, so days-since-viewed sits on the front page. |
| Throughput | Fewer than fifteen completed positions by month twelve. Too few observations to conclude anything, and patience does not fix it. |
| Null result | At forty positions, every interval containing zero and narrow enough to matter. A real finding, and an actionable one. |
| Broken record | Missed entries, wrong timestamps, unadjusted splits. Better to restart clean than to publish a number built on a corrupt log. |
Not on the list: a losing paper account. A negative result is a successful experiment with an unwelcome answer, and it gets published either way.
The system is split into immutable inputs and derived state, and that split is the most consequential decision in the design.
Because state is recomputed rather than incremented, a missed day costs nothing. This matters because the operator travels for work and a laptop scheduler will be closed for a week at a time. It also enables a nightly integrity check: rebuild into a scratch database, diff against the live one, and any disagreement surfaces the day it appears rather than in month eight.
| Raw bars, adjusted at read time | Vendor-adjusted prices change retroactively after a split, which would silently rewrite past results. Storing raw prices plus a separate corporate-actions table means any calculation from six months ago reproduces exactly today. |
| Signed lot ledger | Fills are positive on a buy and negative on a sell, and a position stays open while net shares are positive. This is what makes partial exits — scaling out, selling part and letting the rest run — representable at all. |
| Proportional benchmark reduction | Every dollar into the stock is matched by a hypothetical dollar into the benchmark. A sell removes the same fraction from both, so remaining alpha is measured only on what is still at risk. |
| Sanity guard | A single-bar relative move beyond ±25% halts the position rather than triggering. Almost always an unhandled corporate action or a bad bar, not a real move. |
| Staging before intake | Any automated extraction proposes; a human confirms. Automation is good at the tedious part and bad at the integrity boundary. |
| Reasons captured live | A reason written a month after a trade is a rationalisation. Records are flagged as logged-live or backfilled so the two are never confused. |
Effectively nothing. No language model sits in the evaluation loop — the six state machines are arithmetic. Price data is free at this volume, hosting is free, and compute is a laptop. Cost scales with picks received, roughly forty a year, rather than with universe size.
Open bug — alpha dollars are not comparable to the recorded book. The five simulated books deploy fixed unit sizes. The recorded book deploys whatever was actually traded, which can be several times larger. Comparing them in raw dollars would credit position size as skill. Every comparison involving the recorded book must run on relative return, which is scale-free, or on alpha normalised per dollar deployed. This is the first task of the next work session.
Also outstanding: a single scheduled job chaining data fetch, rebuild and render; live broker execution for the primary book, which measures execution friction rather than the experiment itself and can wait; and optional automated extraction of new picks, which stages for confirmation rather than writing directly.
Nothing. That is the discipline, and it is the hardest part of the design.
At a dozen observations the intervals will be enormous and one book will be ahead. That book is ahead by chance. The single most likely way this fails is not a bug — it is watching one variant lead early and changing how real money is traded on the strength of noise. Every element of the design exists to resist that.
Log the picks. Log real trades honestly, including the unflattering reasons. Look at the board. Change nothing until the intervals stop straddling zero.
The instrument is finished. The experiment has not started. What decides whether any of this produces an answer is a single line in a file, on mornings when it does not feel important.