Deep Learning : The Greatest Difficulties Deep Learning Comes Across & how Deep Learning Applies in Financial Services.

Numerous methods are boosting deep learning (artificial intelligence) and solving challenging problems. Reinforcement learning, graph neural network, deep neural networks, representation learning are the top 4 topics in ICLR conference, displaying as the hottest methods in this field, while Transformer, contrastive learning, adversarial training are swiftly developing. 

With more and more supportive methods pouring into deep learning, diving into fancy techniques is not always the best choice. Rather, we first need to carefully analyze and structure the problems, otherwise, the algorithm will probably come up with poor results. Also, it should be noticed that deep learning is heavily based on the quality of databases and can be vulnerable to uncertainties in real cases. François Chollet argues that “There are no fixes for the fundamental brittleness of deep neural networks”, 

Here I would love to talk about three factors that curse the performance of deep learning: 

(1) The performance heavily depends on the quality of databases. For automated driving image recognition, models sometimes get fooled by blur images captured by the car cameras at high speed once they are brought out from the laboratory to the real world. 

(2) Uncertainty lies everywhere in practical scenarios.  Ie distribution shifts.  

For example, the prediction result can easily change if some simple perturbations are introduced. (3) Black boxes of deep learning. As deep learning comes with numerous parameters and nodes, even the developers find it hard to interpret exactly how the machine figures the prediction out.  

The inexplicability of models prevents them from being applied when some simple machine learning models like linear regression and decision trees are more welcomed. It is even more essential the interpretability when deep learning is applied in quantitative trading, investment, etc. I used to come across failure also. 

Our team is interested in a new method to predict garment companies’ stock returns with time-series factors using Temporal Convolution Neural Network, which is tested to outperform conventional recurrent networks like LSTM, GRU, RNN in popular cases including MINST, Word-level PTB, etc. However, with our poorly chosen factors, only including the daily market order fluctuation in the global cotton market, the result turns out that none of the networks are satisfying our needs. 

Deep learning indeed has amazing power in discovering underlying patterns in data, however, more focus should be paid to structuring the problems and improving data quality. Expert knowledge is urgently needed to understand the problem before the model construction. Going back to my own experience, poor predictive factors might be the reason why those neural networks failed. First, not every garment company relies on a global value chain to produce their products, rather, companies are more likely to place their orders in a certain area, especially the emerging countries like Vietnam and Indonesia.  

Also, as companies tend to apply contracts or futures to prevent themselves from material cost, the daily price of cotton may have little explaining ability of the stock returns. More reasonable models should have more focus on the problem itself, and for this question, an alternative can use information from the cotton market where the manufacturer is located. Another example to be emphasized is the graph.

And that’s why a great deal of effort in AI-based investment has been placed on factor engineering. Making a different and meaningful factor sometimes is way more important than a network framework. Any new factor can boost the prediction power and help investors generate higher alpha. However, overfitting and black box have been troubling deep learning applications in stock prediction a lot together with its development. 

Apart from return predictors, deep learning is also used in customer relationship management, portfolio management, risk management, and communication and reporting. For example, compliance and supervision institutions can apply deep learning to detect uncommon patterns and anomalies; emerging risks could be better identified and mitigated with large and highly related datasets. The booming progress in NLP draws a promising future of deep learning in finance. The biggest NLP model ‘Megatron-Turing’, with more than 530 billion parameters, has just been released by NVIDIA and Microsoft. 

More challenges lie in the application of deep learning. One of the challenges comes with the data infrastructure that as most of the companies cannot afford to handle data with their own structure and feels uncomfortable transferring their sensitive data onto the cloud platform, the data infrastructure should be carefully designed to maintain both the algorithm’s functionality and low cost development, and cross border data flow also needs to be monitored under the tightening data protection laws.