A 2015 MSc Thesis, and a Real Betting Account
Most academic work on trading and betting models never leaves the backtest. A strategy gets validated against historical data, the out-of-sample numbers look good, and the paper gets published. Whether it would have actually made money with real prices, real latency, and real order books that don't wait for you is left as an exercise for someone else.
The timing wasn't a coincidence. In December 2013, DeepMind published "Playing Atari with Deep Reinforcement Learning", showing that a single Q-learning agent paired with a convolutional neural network could learn to play seven different Atari games directly from raw screen pixels, beating existing benchmarks on six of them and humans on three, without changing the network's architecture between games. Reinforcement learning went, almost overnight, from a niche academic method to something that looked like it could learn to act well in almost any domain, given enough trial and error.
In 2015, my MSc thesis followed that same approach: trading the Betfair exchange with reinforcement learning. I trained agents to trade greyhound racing markets automatically, validated them thoroughly on historical data, and then did the part almost nobody does: deployed the winning agents on Betfair with real money and let them trade for real.
"We could not find any papers in the literature where seemingly successful out-of-sample results were actually validated in the real world."
The agents made money. Over £2,000 of it, across two separate live trials in 2015. But they made only about 15% of what the historical backtest said they should. That gap, and what caused it, is a far more useful result than the profit number itself, and it's the part of this thesis worth revisiting more than a decade later.
Teaching an Agent to Trade a Fly (Before a Horse)
Before touching a real betting market, I built the reinforcement learning framework on a much simpler problem: a "grid world" where an agent has to learn to catch a fly.
The setup is deliberately trivial. A frog sits on one of five squares in a row; a fly sits on another. The agent can move left or right. Every time it reaches the fly it gets a reward, and a new fly appears. There's no strategy here beyond trial and error. But trial and error is exactly what reinforcement learning is built on.
This is Q-learning in its purest form: the agent keeps a running estimate, called a "Q-value", of how good each action is in each state, and nudges that estimate toward reality every time it takes an action and sees what happens.
Q(state, action) ← Q(state, action) + α [ reward + γ · max(Q(next_state, a)) − Q(state, action) ]
It looks abstract, but the intuition is simple: if an action leads somewhere good, raise its value a little. If it leads somewhere bad, lower it. Repeat that thousands of times and the agent starts consistently choosing the action that leads toward the fly, without ever being told the rules of the grid.
The real work was scaling this idea up: replacing the lookup table with artificial neural networks, testing how network width and depth affected learning, and then swapping the fly-catching grid for something with actual financial stakes: the live order book on Betfair.
From a Grid to a Betting Exchange
The trading version of the problem keeps the same shape. The "state" becomes a simplified read of the current order book: how much money is waiting to back a runner versus lay it, and where. The "actions" become open a back bet, open a lay bet, close the position, or do nothing. The "reward" is straightforwardly profit and loss.
I trained agents on historical greyhound racing markets, then validated them on races the agents had never seen. The results were genuinely strong: wider networks outperformed narrower ones, and, encouragingly for a real deployment, the best agents were consistently profitable across thousands of out-of-sample races.
"Increasing the width of the neural networks increased profits whereas increasing the network depth did not add to profitability on the races tested."
On paper, this is where most research in this area stops. The model works out-of-sample; write it up. Instead, I pulled the best-performing agents out of the backtest and pointed them at a live Betfair account.
Two configurations went forward to real money, about five months apart. The first, NN1, used just 2 hidden units in a single layer. The second, NN2, used 14. On the validation data NN2 beat NN1 by 40% on profit per race, 55p against 35p, which matches the width result above: more neurons in that single layer kept helping, at least up to 14. Both agents were deployed live, which is why most of the numbers below come in pairs.
Then Real Money Hit the Market
Two live trials followed: one running for 28 days from February to March 2015, a second for 24.5 days that August. Real bets, a live Betfair account, a hard cap of £50 risk per runner.
The first agent made £931 after commission across 2,632 markets. The second made £1,222 across 2,209 markets. Both periods were net profitable, which is more than most backtested strategies can say once they touch a real market. But two separate comparisons against the model both show the same gap, at two different sizes.
The first comparison is the one in the chart above. Alongside the live agent, I ran a parallel simulation of the exact same trading decisions assuming perfect fills and zero latency, on the same 28 days. That "model" line averaged £69.44 a day, about £1,944 over the full month. The live agent's real profit, after commission, came to £931 over the same period. Real profit ran at roughly 48% of this same-day model: a real gap, but a much smaller one than the headline number.
The second comparison is the stricter one, and it's where the abstract's "15%" actually comes from. The live agents were staking up to £50 a bet; the original historical validation in the previous chapter had only ever tested staking £10 a bet, on a completely different stretch of historical data. Scale that validation result up five times for the bigger stake, and the two agents should have made £2.55 and £3.45 profit per race respectively. What they actually made, live, was 35p and 55p per race: just 14% and 16% of that projection. This is the tougher test, because it isn't just asking whether the same decisions would have filled perfectly today. It's asking whether an edge measured on one dataset survives being deployed, months later, on a market the model has never seen.
Where the Other 85% Went
I don't wave this gap away. I go looking for it, and find several concrete, unglamorous causes rather than one clean explanation:
- Latency. The backtest assumed the agent's decision hit the market instantly, at the price it saw. In reality, 10 to 20% of bets weren't matched immediately and had to be cancelled. A follow-up experiment that deliberately added 100ms of latency into the backtest cut its profit by roughly 75% on its own, squarely in the range needed to explain the live shortfall.
- Partial fills. The backtest assumed every bet could be placed in full, at the best available price. Real order books don't always have that much money sitting there waiting.
- Minimum bet size and commission. Betfair's £2 minimum bet makes it harder to trade out of a position for exactly zero, and every winning market carries a real commission of 2 to 5%. The word "commission" never appears anywhere in the thesis before real-money trading starts, which suggests, though it's never stated outright, that the original per-race validation figures never accounted for it.
It's also worth being honest about where my own methodology could have been tighter: the architecture comparisons that decided which network width "won" were run once per configuration rather than averaged over multiple trials, and I selected the final live agent as the single best performer against one validation set, with no separate holdout to confirm that pick before risking real money. Neither of those flaws explains the 85% gap, but they're exactly the kind of thing worth tightening before trusting a backtest with real capital.
It Didn't Work Everywhere
One more result is worth including, because it's the kind of negative finding papers in this area rarely publish. Using the exact same agent structure, unchanged, I tested the framework on horse racing markets beyond greyhounds: win markets in the USA, Australia, Great Britain and Ireland, and place markets in Great Britain and Ireland.
It worked cleanly almost everywhere. The Australian and American win markets were profitable both in and out of sample. The GB and Ireland place markets did even better, up to 60p a race. The one place it failed outright was the single most competitive market of the set: GB and Ireland win markets, where the agent lost money on both the training and validation data.
The likely explanation is liquidity. Those markets are traded so heavily that prices barely move, and the agent's own behaviour reflected it: it recommended an average of two bets a race, against twenty for the place-market agent trained the exact same way. An agent built to always take the best available price, rather than sit in the order book waiting for a better one, has very little to work with in a market that's already this efficient.
I only had time to run one trial per market, so this isn't a result I'd hang a strong conclusion on. But it's an honest shape: the same framework, completely unchanged, was strong in every market except the single most competitive one, and that's a more useful place to end than pretending it generalised everywhere.
Why This Still Matters
This result is nearly a decade old, but the lesson hasn't aged at all. It's the same discipline behind every model on SharpModels: an out-of-sample backtest is a hypothesis, not a guarantee, and the only way to find out how much of it survives contact with a real market is to actually test it there.
"This shows how important real world testing is to validate academic research in this field. It brings into doubt the potential profitability of previous academic research on trading markets that has only been validated using historical data."
It's also part of why the current approach I use favours simpler, interpretable models over black-box complexity. A reinforcement learning agent that is profitable in backtesting but opaque about why is much harder to debug when live results disappoint, and, as this thesis shows, they often will, for reasons that have nothing to do with the model's core logic and everything to do with the market's plumbing.
Key Takeaways
- A reinforcement learning agent trained to trade Betfair markets was profitable both in backtesting and, more rarely, with real money.
- Against a same-day, perfect-fill simulation of the same trades, real profit ran at about 48%.
- Against the original historical validation, scaled up for the bigger live stake, real profit ran at just 14 to 16%.
- Latency was the single largest identified cause of the shortfall: adding just 100ms of it to the backtest cut profit by roughly 75%.
- Partial fills, minimum bet sizes, and commission accounted for most of the rest.
- The same framework, unchanged, worked on horse racing markets across the USA, Australia, GB and Ireland, except the single most liquid one: GB and Ireland win markets.
- A backtest can correctly rank strategies against each other while still being badly wrong about the absolute profit any of them will produce live.