The Cancel-Stream Gap: Why Your Signal Stack Is Building on 35% of the Order Book

If I were rebuilding the signal stack today, the first dollar would go to the data line.

$2,388 a year buys the layer that closes roughly 30 R-squared points on the price-prediction model. The FPGA spend — the co-location, the production-grade hardware deployment — comes after. The same desk that commits seven figures to hardware infrastructure is often running on a data layer that cannot see the dominant event type in the book it is trying to predict.

That inversion is what this article is about.

The research is clear. The data access has never been cheaper or more standardized. The barrier is architectural, not financial. And with Rule 605 amended compliance landing in August 2026, execution quality at 50-millisecond resolution is about to become public record. Desks running on incomplete order flow data will have that visibility gap reflected in their published realized spread figures. The window for a quiet fix is closing.

Table of Contents

1. The 30-Point Gap: What R-Squared Tells You About Your Data Layer
2. Why 97% of the Order Book Is Invisible to Most Signal Stacks
3. The Cost Arithmetic
4. Where L3 Lives Now: The Venue Access Map
5. Running the Test: Three Failure Modes
6. Practical Framework: The Data Layer Diagnostic

---

The 30-Point Gap: What R-Squared Tells You About Your Data Layer

In 2014, Cont, Kukanov, and Stoikov published “The Price Impact of Order Book Events” in the Journal of Financial Econometrics. They measured unified order flow imbalance at the best bid and ask against price changes on 50 NYSE stocks over 10-second windows. The unified OFI model produced an average R-squared of 0.65.

For comparison: a model built on signed trade volume — the trade-tape proxy most signal stacks still use as their foundation — produces R-squared in the 0.32 to 0.35 range on the same data (figures from the paper’s results tables; verify at arxiv.org/abs/1011.6402).

These are two independently measured models. The comparison is not a before-and-after decomposition of a single model. It is two separate constructs measured against the same price-change outcome. One includes the cancel stream at the quote. One does not. The gap between them is approximately 30 R-squared points.

The practical interpretation: trade-only OFI captures roughly half the predictive signal that unified OFI captures. The other half sits in the limit order activity — the additions, modifications, and cancellations — that never produces a fill and therefore never appears on the trade tape.

The 2024 paper by Lu, Reinert, and Cucuringu (“Trade co-occurrence, trade flow decomposition, and conditional order imbalance in equity markets,” Quantitative Finance 24(6):779-809, online June 2024) provides the most direct quantification of what skipping decomposition costs in live-strategy terms. Across 457 US stocks over four years, strategies using conditional OFI — which distinguishes between order event types including cancellations — achieved a Sharpe ratio of 1.79. Strategies using undifferentiated OFI without decomposition produced negative Sharpe.

That result is worth sitting with. The difference between decomposing and not decomposing is the difference between a profitable signal and a loss-generating one, measured over four years and 457 names. The 30-point R-squared gap from the Cont et al. paper has a live-strategy correlate.

Cont, Cucuringu, and Zhang (2023), in Quantitative Finance 23(10), add a further layer: incorporating multiple price levels beyond the best quote produces meaningful R-squared improvement over best-quote-only OFI in the paper’s reported results (figures in the 0.80-0.87 range appear in the paper’s results tables; verify against the full text at doi.org/10.1080/14697688.2023.2236159). That is a separate improvement from cancel-event decomposition — better depth coverage rather than better event classification. Both matter, but they operate on different dimensions of the signal problem and require different data capabilities. Multi-level L2 OFI addresses depth. Cancel-event decomposition addresses event classification. Neither is a substitute for the other.

---

Why 97% of the Order Book Is Invisible to Most Signal Stacks

Khomyn and Putniņš (2021), “Algos gone wild: What drives the extreme order cancellation rates in modern markets?” (Journal of Banking and Finance, 129, Article 106170), measured order fate across US equities: 97% of limit orders cancel before executing. That figure is not a curiosity. It is the structural fact that explains why trade-only OFI consistently underperforms.

The trade tape captures post-execution fills. Every fill represents an order that survived the cancel cycle and crossed the spread. What the tape cannot see is the mass of orders that arrived, adjusted price expectations, and then withdrew before executing. Those withdrawn orders carry information. The price level they targeted, the aggressiveness of the placement, the timing of the cancellation relative to quote movement — all of that is prior information about directional pressure in the book. The trade tape sees the outcome of one outcome. The full order book feed sees the intentions of all participants.

Sitaru, Calinescu, and Cucuringu (2023), “Order Flow Decomposition for Price Impact Analysis in Equity Limit Order Books” (ACM ICAIF ’23, pages 637-645, DOI:10.1145/3604237.3626874), decomposed OFI into its constituent event types using market-by-order data. In forward-looking predictive scenarios, the decomposed model showed statistically and economically significant improvement over undifferentiated OFI. (Sample size, dataset window, and exact magnitude live behind the ACM paywall — verify the precise figures at doi.org/10.1145/3604237.3626874 against the full paper text.) The mechanism is the cancel stream: when cancel events are classified and incorporated as a distinct signal component rather than collapsed into net imbalance, forward predictive power improves.

The signal half-life question is where execution architecture enters. Kolm, Turiel, and Westray (2023), “Deep order flow imbalance” (Mathematical Finance, 33(4):1044-1081), found that the effective horizon of stock-specific forecasts is approximately two average price changes. The paper’s term is “effective horizon,” not “half-life” — the decay mechanics are analogous but not identical, and the framing here treats them as practitioner-equivalent. In liquid US equities, two average price changes maps to roughly 10 to 30 seconds — a practitioner inference from the paper’s finding, not a stated wall-clock result. The regime matters: thin books decay faster, deep books slower. Building a cancel-OFI predictor is only half the problem. The other half is confirming your execution layer can act within the window the predictor opens.

---

The Cost Arithmetic

Before running the test, the capital math is worth establishing.

Start with the strongest evidence. Lu, Reinert, and Cucuringu (2024), in Quantitative Finance, ran 457 US stocks over four years and measured live-strategy outcomes: conditional OFI (which classifies cancel events as a distinct component) produced a Sharpe ratio of 1.79. Undifferentiated OFI without decomposition produced negative Sharpe. That is not a backtest artifact or an in-sample R-squared improvement — it is live-strategy outcome over four years on 457 names. The gap between decomposing and not decomposing is not marginal.

The dollar figure is harder to pin precisely, and honesty requires saying so.

Scenario: liquid US equities, $50M average daily volume, $10B annual notional. Cartea and Jaimungal (2016), “A Closed-Form Execution Strategy to Target Volume Weighted Average Price” (SIAM Journal on Financial Mathematics, 7(1):760-785, SSRN:2542314), measured OFI-informed execution against a standard VWAP benchmark on five Nasdaq names. The published improvement range was 0.1 to 8 basis points, strategy-dependent. That paper is specifically about VWAP-targeting simulations — not an isolated measurement of cancel-event contribution. On $10B notional, that range translates to $100K at the floor (0.1 bps) and $8M at the ceiling (8 bps). The conservative practitioner anchor I use is 0.5 bps — $500K annually — but the honest framing is that your number sits somewhere within the published range, and where it lands depends on your instruments, regime, and execution path.

That estimate is conditional on two things: the signal has predictive power on your instruments, and your execution layer can act inside the signal half-life. Strip either condition and the $500K anchor does not materialize. If you already have L3 access through a prime broker or direct exchange feed, the data cost argument below may not move the needle — the relevant question is whether you are actually decomposing cancel events as a distinct signal component, not just receiving discrete cancel messages.

Note on scope: Rule 605’s 50ms realized spread disclosure applies to US equities and certain equity options through SEC jurisdiction. It does not directly govern crypto spot or DeFi perpetuals. For crypto-native desks, the analogous pressure point is execution quality transparency at the CEX venue level, where maker-taker fee structures and fill-rate benchmarking serve a similar forcing function.

The data cost to close the signal gap:

Databento Standard plan at $199 per month ($2,388 per year) includes one month of DBEQ.BOOK historical L3 data, with additional historical billed at $0.40 per GB. Live L3 access is not in the Standard tier — live data requires the Plus plan at $1,399 per month. The Standard tier is the right entry point for backtesting and signal research; Plus is the threshold for production deployment that depends on live event-level cancel data. Verify current rates at databento.com/pricing before commitment. Databento closed a $10M Series A+ in October 2024 with reported 985% revenue growth and over 7,000 new customers added that year, signaling that L3 access has crossed from specialist capability to commodity infrastructure layer.

Coinbase Full Channel WebSocket is available to authenticated Coinbase Exchange API users. As of April 2026, no additional data fee is documented for this feed in Coinbase’s official API documentation; verify current access terms at docs.cdp.coinbase.com/exchange before production implementation. The Full Channel covers crypto spot L3 — the per-order event lifecycle (received, open, match, done, change, activate) — for signal research extending into that venue class.

Hyperliquid L4 via 0xArchive offers a free tier for DeFi perpetual order book data. Coverage starts March 2026 and runs through real-time. Specific API rate limits and credit allocations are not publicly documented; verify directly at 0xarchive.io before building a production dependency on it.

CME MBO covers all CME Globex futures and options via MDP 3.0 — true L3 with order IDs and priority tracking. Pricing requires direct commercial engagement with CME’s market data team. This is not a self-serve entry point comparable to Databento.

The ROI arithmetic is straightforward when the data costs $2,388 and the conservative estimate sits at $500,000. The analytical question is whether the conditions for that $500K are actually in place: correct data feed, decomposition logic, and an execution path that acts inside the signal decay window. Each of those conditions is testable.

---

Where L3 Lives Now: The Venue Access Map

Five years ago, market-by-order data was expensive, difficult to access, and operationally complex to ingest. The vendor landscape has converged.

For US equities, Databento DBEQ.BOOK is the clearest entry point. The 985% revenue growth and over 7,000 new customers added in 2024 confirm that the access curve has shifted. If your team is still treating L3 as a specialist capability, that framing is approximately three years behind where the market actually is.

For US futures, CME MBO via MDP 3.0 is the standard. Full order-level data with priority IDs across all Globex products. The pricing model is commercial rather than self-serve — build that into infrastructure budget assumptions.

For crypto spot, Coinbase Full Channel via the Exchange API gives authenticated L3 at no additional data cost.

For DeFi perpetuals, Hyperliquid L4 via 0xArchive covers the space with a free tier, March 2026 coverage start running through real-time. Verify rate limits and credit allocation at 0xarchive.io before deployment.

For European equities and futures, the picture is less clean. ICE and Euronext do not offer clearly self-serve L3 equivalents at a comparable entry-price tier. Confirm feed specifications with each venue’s market data team before building signal research on that assumption.

One clarification worth making explicit: multi-level L2 OFI and cancel-stream decomposition are different capabilities requiring different data. Adding price depth beyond the best quote (L2 depth) lifts R-squared from 0.65 to 0.80-0.87 per Cont, Cucuringu, and Zhang (2023). Cancel-event decomposition, the improvement measured by Sitaru et al. (2023) and Lu et al. (2024), requires event-level L3 with discrete add/modify/cancel message classification. These are additive layers. Building multi-level L2 OFI captures the depth dimension; cancel-stream L3 adds the event-classification dimension on top.

---

Running the Test: Three Failure Modes

The directional test is straightforward in design and demanding in execution.

Run three models on the same instrument and the same execution stack: trade-only OFI, full unified OFI with cancels, and the combined multi-level model. Measure arrival-price slippage on a fixed flow profile. Run it long enough for statistical separation — 60 trading days gives you enough variance to distinguish signal from noise. The question is whether the combined model measurably dominates the trade-only baseline in execution quality, not just in in-sample R-squared.

Three failure modes explain why it often does not, even when the signal research suggests it should.

Failure Mode 1: Snapshot rate aliasing the cancel stream. L2 venues deliver periodic snapshots of the order book state. Cancel events between snapshots are compressed into the delta between two book states. If your cancel-OFI is derived from L2 snapshots rather than discrete L3 cancel messages, you are inferring event classifications from net changes. That inference loses precision precisely when cancel activity is highest — rapid cancellation cycles during aggressive market movement. The fix is event-level L3 with discrete message classification.

Failure Mode 2: Latency budget exceeding the signal half-life. The signal half-life is approximately two average price changes in liquid equities — 10 to 30 seconds by practitioner estimate. If your execution path from signal generation to fill confirmation takes longer than that window, the predictor has expired before the order reaches the market. This is an execution architecture problem that surfaces as a data problem in the test results. Log execution lag at P50 and P95 and compare against the half-life estimate for your instrument.

Failure Mode 3: Execution layer cannot act inside the predictor’s window. The model opens a window. The order management system, risk layer, and routing path have to fit inside it. In many production environments, the strategy model is faster than the execution infrastructure surrounding it. The predictor fires, the OMS queue introduces latency, and the fill arrives after the price has moved. The test will show this as the combined model failing to outperform despite better signal construction.

The regulatory context adds urgency.

SEC Rule 605 amendments (effective June 14, 2024; compliance deadline extended to August 1, 2026 via Federal Register 2025-19316) require realized spread reporting at 50ms, 1 second, 15 seconds, 1 minute, and 5 minutes after execution. Time-to-execution in millisecond-or-finer increments. The scope expands to broker-dealers with 100,000 or more customer accounts.

The 50ms realized spread horizon is structurally equivalent to a public signal-decay disclosure requirement. Every routing venue’s execution quality will be visible at 50ms resolution against published market center averages. Desks running trade-only OFI are not just leaving signal on the table — they are building toward a public comparison where their execution quality will be benchmarked against peers who have the fuller picture.

NYSE’s order-entry ratio fee structure has two tiers per the February 2026 Price List: $0.005 per excess order for ratios between 100:1 and less than 1,000:1, and $0.01 per excess order at 1,000:1 and above. The economic significance of cancel volume is already priced into exchange fee schedules at both tiers, with the 1,000:1 level being the threshold associated with aggressive HFT patterns.

A structural analogy worth noting: divergent state in a signal pipeline follows the same failure pattern as divergent state in any distributed system. The Knight Capital incident of August 1, 2012 (approximately $440M pre-tax loss per Knight’s own press release of August 2, 2012; 154 stocks; 45 minutes) was triggered by one server running a stale code path while the rest of the deployment ran the updated version. That was not a cancel-stream story. The structural analogy is the failure pattern itself: one component of the pipeline operating on a different view of state than the others. Signal stack architectures that mix L2-inferred cancel events on some instruments with L3 discrete events on others create exactly this class of hidden divergence. The failure mode is detectable before the P&L impact arrives.

---

Practical Framework: The Data Layer Diagnostic

A 5-point diagnostic for a CTO reviewing their current signal stack.

1. Feed classification query. Query your venue feed specification: does it expose Add, Modify, and Cancel as discrete message types, or does it deliver periodic book snapshots with net deltas? If your pipeline receives snapshots, your cancel-OFI is inferred from net state changes, not event-level classifications. That is the snapshot aliasing risk from Failure Mode 1.

2. Execution latency comparison. Measure your execution path from signal generation to fill confirmation at P50 and P95. Compare against the signal half-life estimate for your instrument (two average price changes; 10-30 seconds in liquid US equities by practitioner estimate). If P95 execution latency exceeds the half-life estimate, the predictor’s window is regularly expiring before you act in it.

3. Controlled signal comparison. Run trade-only OFI against combined OFI (with cancel decomposition) on a single instrument, same execution stack, minimum 60 trading days. Measure arrival-price slippage on a fixed flow profile, not just in-sample R-squared or fill rate. Slippage is what maps to P&L.

4. Failure mode attribution. If combined does not dominate trade-only in the slippage test, log which failure mode explains it. Failure Mode A: confirm cancel events are arriving as discrete messages. Failure Mode B: log execution lag P50/P95 against the signal decay window. Failure Mode C: confirm the execution layer receives and acts on the signal within the predictor’s window.

5. Rule 605 benchmark tracking. Once amended Rule 605 filings go live in August 2026, your 50ms realized spread is on public record. Pull the published market center averages for your primary routing venues. Compare your 50ms realized spread against them. Consistent underperformance against the venue average points to signal quality, execution path latency, or routing logic — the diagnostic above attributes which.

---

Conclusion

The core finding, stated plainly: a signal stack built on trade-only OFI is modeling approximately 35% of the events in the book it is trying to predict. The research from Cont et al. (2014) through Lu et al. (2024) traces a consistent line from the 30 R-squared point gap to a live-strategy Sharpe differential that runs from positive to negative depending on whether event-type decomposition happens or not.

The data to close that gap costs $2,388 a year at the Databento Standard tier for US equities. For crypto spot and DeFi perpetuals, the entry cost is zero.

Where I have not fully closed the loop: the conditions that determine whether the $500K execution-quality estimate materializes are not always under the data team’s control. Snapshot aliasing, execution path latency, and OMS architecture each sit in different parts of the organization. The signal research can be correct and the P&L improvement can still fail to appear if the execution layer cannot act inside the window the predictor opens.

Run the directional test: trade-only OFI, combined OFI, same instrument, same execution stack, 60+ trading days, arrival-price slippage as the measure. If combined does not dominate, the diagnostic above tells you which of the three failure modes is absorbing the signal. Once you have that attribution, the fix is a scoped engineering project, not an open research question.

---

This article expands on a LinkedIn post originally published April 30, 2026. View the original post