In case you missed it - Direct Construction through Reinforcement Learning and Interpretable AIPrize Winners
For those who were unable to attend, we compiled the key takeaways of the webinar featuring professor Lin William Cong (Cornell University).
The foundations of this research lie not with fancy AI or machine learning tools, but with a classic problem in financial economics: portfolio construction. According to Cong, the traditional approach, at least in academic work, has been a two-step paradigm. This is done by estimating distributions of returns, risk premia or pricing currents so that assets can be priced very accurately. Then the portfolio is optimized. This can be done using mean variance done with supervised learning in the second step.
“However, we all know that the first step isn’t easy and can be problematic. Estimation errors are huge. We have taken different approaches, using risk parity for example, but attempts have only achieved moderate success,” says Cong.
Can the reward be maximized, using whatever performance measure available, in a more direct way?
“To this end we attempted to find a direct optimization,” explains Cong. “We created a model that maximizes rewards. The issue is, the optimally constructed portfolio – or portfolio z – is not observed in historical data. We cannot label it, because there are an infinite number of potential combinations. Moreover, our construction to date may affect the portfolio going forward. Today’s performance will affect the budget for the next period. There could be transaction costs – if I trade more today, I will have a lower return. All of this will impact the strategy and environment going forward. The key elements of reinforcement learning are exactly this kind of trial-and-error search and delayed reward.”
The researchers’ primary objective was to carry out direct construction through supervised learning. “It is a very different approach. The difficultly is again that the optimal portfolio (z) doesn’t exist in the data, meaning that we have to take some proxies and create test portfolios. Supervised learning could work, but it is hard to compute and to choose the ‘z’ to label carefully. More importantly it doesn’t incorporate the dynamic interactions with the environment, in the way that reinforced learning does.”
Cong draws upon AI and deep learning tools and tweaks them to be used for finance. These tools were originally developed in science and engineering fields. They help to deal with non-linearity, high dimensionality, but in science and engineering, typically the signal- to- noise is reasonable, whereas in finance it is very low. The variables in social sciences that interact a lot faster, and are considered non-stationary dynamics. Unlike physical laws of science which are stagnant, investor preferences and government policies can change quite rapidly.
“The modules I use in this research have been used in self driving car simulation, but typically they deal with single sequence. But in finance, it is crucial that we think about multiple assets. Making cross-asset or cross-sequence analysis very important. That is missing from AI tools currently, and that is part of our contribution with this research.”
View the webinar via: https://vimeo.com/500829813