Why Deep Reinforcement Learning Might Change Portfolio Management

Ask a finance student how to build a portfolio, and chances are you'll hear some version of the same answer that Harry Markowitz gave us in 1952: estimate the expected return and risk of each asset, understand how the assets move together, and find the combination that gives you the best return for a given level of risk.

It's elegant. It's foundational. And it has served portfolio theory remarkably well.

But there is one quiet limitation: it doesn't learn.

Mean-variance optimization treats each portfolio decision as a new optimization problem. When you rebalance, you feed the model updated inputs and solve the problem again. Yesterday's decision doesn't teach the model anything about tomorrow's decision.

In that sense, it's a snapshot rather than a strategy that develops through experience.

Deep reinforcement learning approaches the problem from a very different direction. Instead of explicitly solving for an optimal portfolio using a predefined mathematical relationship, it tries to learn a policy — a rule for deciding what to do next — from repeated interaction with an environment.

And that difference is what makes it interesting.

What does "learning" actually mean here?

Strip away the jargon for a moment, and a deep reinforcement learning portfolio agent works a little like learning to navigate a maze through trial and error.

• It observes a state — perhaps recent prices, returns, volatility, technical indicators, or information about the current portfolio.
• It takes an action — in portfolio management, this usually means changing the allocation across assets.
• It receives a reward, typically related to portfolio performance and sometimes adjusted to account for risk or trading costs.

The process is repeated over and over using historical data. Through those interactions, the agent gradually updates its policy to favour actions that have produced better outcomes.

Nobody explicitly tells it:

Sell when volatility increases.

Instead, if reducing exposure during certain conditions repeatedly produces better rewards, the learning process can eventually lead the agent toward behaviour that reflects that relationship.

That is one of the genuinely interesting differences from traditional portfolio models.

The agent isn't handed a complete model of how markets work. It is given an objective, an environment, and a feedback mechanism, and it learns a decision-making policy through experience.

In principle, this gives DRL the ability to capture relationships that can be difficult to express neatly in a conventional formula. Volatility, momentum, correlations, market conditions, and other features may interact in complicated ways. Rather than specifying exactly how those variables should interact beforehand, a neural network can learn a representation of those relationships from the data.

Of course, "can learn" is doing a lot of work in that sentence.

Whether it actually learns something useful is an empirical question.

Where could it actually help?

The potential advantage becomes more interesting when markets stop behaving the way we expect them to.

Many traditional portfolio models rely, explicitly or implicitly, on assumptions about the behaviour of returns, risks, and correlations. Those relationships can become much less reliable when markets move from a relatively calm period into a period of extreme volatility or when correlations suddenly rise across assets.

Think about what happens during a major market selloff.

Assets that appeared reasonably diversified under normal conditions can suddenly start moving together. Volatility increases. Correlations change. The relationships that looked stable yesterday may not look particularly stable anymore.

A DRL agent trained across different market conditions has the potential to learn different responses to different states of the market. Rather than applying exactly the same decision rule regardless of the environment, it may learn to alter its portfolio behaviour when the conditions it observes change.

But there is an important distinction here.

Adaptability is not the same thing as successful adaptation.

An agent might appear to behave differently across market conditions simply because it has learned patterns that happen to work in the training data. The real question is whether that behaviour continues to be useful when the agent encounters data it has not seen before.

That question is at the centre of one of my own research projects.

In Volatility Regime-Dependent Portfolio Optimization: Empirical Analysis of Deep Reinforcement Learning Agents, I examine how different deep reinforcement learning agents perform across different volatility conditions and whether their portfolio behaviour changes as market conditions change.

The interesting part isn't simply asking which algorithm produces the highest return.

It is asking a more fundamental question:

What does a portfolio agent actually learn when the market environment changes?

Does it genuinely respond to different volatility regimes? Or does it simply discover a strategy that performs reasonably well on average and continue doing roughly the same thing regardless of the underlying conditions?

The answer, as is often the case with financial machine learning, is more nuanced than either extreme.

Where does it fall short?

This is also where I think it is important not to oversell reinforcement learning.

DRL can sound almost magical when described only in terms of "learning from experience." But an agent doesn't suddenly acquire financial intuition just because it uses a neural network.

• It doesn't know that a central bank meeting is happening tomorrow.
• It doesn't understand that a geopolitical event could fundamentally change the market.
• It doesn't know that a particular regulatory announcement might make yesterday's relationships irrelevant.

A human portfolio manager can bring contextual knowledge and judgment to a genuinely unfamiliar situation. A DRL agent trained on historical market data cannot do that in the same way.

Its adaptability is constrained by the experiences available to it.

There is another problem that is arguably even more dangerous: overfitting.

An agent can produce an impressive backtest. High returns. Strong Sharpe ratios. Controlled drawdowns. Consistent performance.

And it can still be completely wrong.

The agent may have discovered a pattern that existed in that particular period of history but has no reason to persist in the future.

This is the classic problem of backtesting, but deep learning makes it harder to see. Instead of having a small number of transparent rules that you can inspect, you may have millions of learned parameters interacting in ways that are much harder to interpret.

A beautiful backtest can therefore be evidence of a useful strategy.

It can also be evidence that your model has become extremely good at remembering the past.

Those are not the same thing.

So, does DRL actually "change" portfolio management?

I don't think the answer is that deep reinforcement learning replaces traditional portfolio theory.

Mean-variance optimization isn't going anywhere. And there are good reasons for that. It is interpretable, mathematically grounded, well understood, and has decades of research behind it.

What DRL changes is the class of strategies we can explore.

Instead of asking only:

What is the optimal portfolio given these estimates of return, risk, and covariance?

we can also ask:

What decision-making policy should an agent learn when it repeatedly observes the market, takes portfolio actions, and receives feedback from the outcomes?

That's a fundamentally different question.

It opens the possibility of portfolio strategies that respond to changing conditions rather than applying one fixed allocation rule everywhere. It also shifts some of the burden of specifying relationships in advance from the researcher to the learning process.

But that doesn't make the problem disappear.

It changes the problem.

Now we have to ask:

• whether the agent has learned something meaningful,
• whether that behaviour survives outside the training period,
• whether transaction costs make the strategy feasible,
• whether the results are robust to different market conditions, and
• whether the apparent adaptability is actually genuine.

And ultimately, whether we would trust the strategy with real money.

That last question is probably the most important one.

A strategy that looks impressive inside a backtest is interesting.

A strategy that continues to work when the market does something it hasn't seen before is much more interesting.

That is where I think the real research challenge lies.

My paper, Volatility Regime-Dependent Portfolio Optimization: Empirical Analysis of Deep Reinforcement Learning Agents, explores this question through the performance and behaviour of deep reinforcement learning agents across different volatility regimes.

The aim isn't to prove that DRL is the future of portfolio management.

It's to understand when it works, why it works, and what it actually learns along the way.

The rest of my publications are on the Research page.