QB.
GitHubLinkedInContact
All projects

Case 06 / Intraday ranking under leakage constraints

Four Sigma

A hackathon model for ranking 1,000 anonymized US stocks by their next 30-minute return, built around metric alignment, walk-forward validation, and explicit promotion gates.

Write-up only. Research results and methodology are included below; no public demo.

Role
Competitor; modeling and evaluation
Status
SubmittedSubmitted hackathon model; write-up only
Period
Apr - May 2026
Key decision
Train on per-timestamp percentile ranks to match the competition metric, then promote changes only when they remain positive across walk-forward quarters.
Evidence
An intraday ranking model tested across four walk-forward quarters, with explicit leakage and promotion checks.
Canonical validation / 2024 walk-forwardPASS
Mean IC+0.05344Spearman rank correlation
4 / 4
positive quarters
10
seeded models
13
features
Q10.0478
Q20.0528
Q30.0505
Q40.0642
Values come from the submitted canonical metrics artifact; they are validation rank correlations, not trading returns.
01 / Problem

A signal is useful only if it survives the next quarter.

The task was to rank an anonymized 1,000-stock universe every 30 minutes using only historical OHLCV data. The score was mean Spearman information coefficient against realized rankings.

The central engineering risk was not fitting a model. It was preventing future or overnight information from leaking into features and labels while searching enough variants to overfit the validation period by accident.

02 / Architecture

Time-aligned features to a ranked prediction.

The pipeline aggregates five-minute bars into within-bar shape features, joins them to 30-minute OHLCV, removes first-bar overnight contamination, cross-sectionally standardizes each timestamp, and averages ten seeded LightGBM regressors. A canonical validation file audits timestamps, dynamic leakage, latency, coverage, and every promoted candidate.

System map / Four Sigma
Four Sigma training, prediction, and promotion pathAnonymized five-minute and thirty-minute market bars pass leakage and session rules, then become thirteen cross-sectionally standardized features over 8.40 million usable rows. Ten seeded LightGBM models produce timestamp rankings. Four walk-forward validation quarters, leakage scans, and a declared promotion gate determine whether a candidate replaces the incumbent.Validation and promotionMarket bars5m + 30m OHLCV8.40m usable rowsLeakage-safe datasession bounds, clean labels13 featurescross-sectional scaling10 LightGBMstimestamp rankingsWalk-forward4 positive quartersPromotion gatedeclared before the run
  1. 015m + 30m OHLCVAnonymized market bars
  2. 02Leakage policySession bounds and clean labels
  3. 0313 featuresReturns, shape, and liquidity
  4. 0410 LightGBMsRank-label bagged ensemble
  5. 05Submission APILoad, prepare, predict
The system optimizes rank correlation under an audited temporal boundary; it does not estimate trading returns.
03 / Decisions

The trade-offs.

01

Train in the metric's geometry

Instead of
Regress raw forward returns and rely on the final ranking to align the model with Spearman IC.
Choice
Use each timestamp's percentile rank of the forward return as the training label.
Why it holds
The evaluation rewards ordering, not return calibration. Rank labels improved walk-forward IC with the same model and features.
Cost
The model no longer estimates a meaningful return magnitude; it produces relative scores only.
02

Null cross-day labels

Instead of
Let the final bar of one day use the next day's opening bar as its 30-minute target.
Choice
Remove 766,976 cross-day labels and exclude raw rows outside the 09:30-15:30 prediction timestamps.
Why it holds
That target contains an overnight move and is not the intended next intraday bar.
Cost
About 7.7% of otherwise available supervision is discarded.
03

Keep a strong incumbent

Instead of
Adopt the best inner-validation hyperparameters or blend whenever the headline IC ticks upward.
Choice
Require robust walk-forward evidence and keep defaults when tuned models or candidate blends fail the outer gate.
Why it holds
Optuna's best inner-validation parameters underperformed out of sample, and added candidates reduced robustness.
Cost
The final system is less novel than several rejected experiments, but its evidence is cleaner.
04 / Build & evidence

Four positive quarters. No claim of trading returns.

Built

  • A feature pipeline over 9.95 million loaded rows and 8.40 million usable training rows.
  • Thirteen cross-sectionally standardized features spanning short returns, bar shape, intrabar acceleration, volatility, and liquidity.
  • A ten-seed LightGBM ensemble plus the required load, batch prepare, and fast timestamp lookup contract.
  • Leakage scanners, submission smoke tests, a walk-forward harness, immutable experiment logs, and a canonical metrics artifact.

Evaluation

  • Walk-forward validation covered four quarterly folds in 2024. Each fold was positive: 0.04777, 0.05281, 0.05047, and 0.06420 mean IC.
  • Aggregate 2024 performance was 0.05344 mean IC with 0.621 information ratio. The last-90-day slice measured 0.06115 mean IC.
  • Timestamp, dynamic leakage, and latency audits passed. The recorded mean predict lookup was 0.023 ms after batch preparation.

Supported result

  • The final artifact preserved positive rank correlation in all four validation quarters rather than relying on a pooled split.
  • The experiment journal records rejected rank blends, time-of-day variants, regime features, and tuned parameters alongside promoted work.
05 / Limits

What this does not solve.

Current limitations

  • These are local validation results on the hackathon data, not trading returns and not evidence of live profitability.
  • The universe is anonymized and constrained to OHLCV, so company fundamentals, costs, capacity, and market impact are outside scope.
  • No public competition placement is claimed in this case study.

Next improvements

  • Evaluate stability beyond a single calendar year and under explicit transaction-cost assumptions.
  • Calibrate experiment multiplicity so the promotion threshold reflects how many candidates were tested.
  • Profile full-batch preparation on the organizer's hardware, since lookup latency excludes feature construction.