Applying Natural Language Process For Investing
Why can we apply the Natural Language Process (NLP) in quantitative analysis?
Traditional quantitative analysis is based on numbers, including macro data, industry data, enterprise fundamental data and market data. The data source used by each quant is basically the same, and strategies on quantitative ideas and algorithms are only things they can explore.
However, experience tells us that market sentiment is largely influenced by public opinions. Financial news reflects to a certain extent whether people are pessimistic or optimistic about the market.
The degree of attention to the industry, the sentiment of the wording, and the comments of economists all affect market trends.
In short, public opinion contains valuable information.
If such information can be extracted using NLP, it will expand the data source of quantitative analysis and increase the analysis dimension, which would undoubtedly be meaningful in analyzing the market's direction.
How can we apply the NLP in quantitative analysis?
There are many ways to apply Natural Language Processing to quant analysis, for example, word classification.
Bayes' theorem, building a corpus, or using GRU, LSTM and other neural network algorithms for sentiment analysis allow us to classify words based on their meaning and tone.
But before we take any method, we should first have an idea of the NLP pipeline.
After the processor carries out these steps, we can now analyze the text.
Where can we apply the NLP in quantitative analysis?
One of the most important resources we can apply the NLP to is financial statements released by the SEC. These documents have long been used as a valuable source of information for making investment decisions.
But it is undeniable that for investors, sorting out these reports is often tedious.
In some cases, financial disclosures are used by companies to hide the fact and the effect of changing accounting rules, which might hurt stock prices. Having the ability to detect these warning signs in financial reports sets apart the elite investors from the average ones.
Through Natural Language Processing however, investors can quickly and efficiently catch these obscure points and get an idea of the current situation and the expectation of future performance from management teams.
Another resource we can use NLP on is the daily market news. Big news usually causes large price movement instantly, and sometimes due to overshoot, it reverses later. Thus, NLP is a perfect tool to analyze the news within milliseconds and make trading decisions instantly.
However, unlike financial statements which are well-structured, these multimedia contents are unstructured data and even harder to be understood directly by computer. To process unstructured data, sentiment analysis (a subfield of natural language processing) is the best method to estimate it.
Simply speaking, sentiment uses the emotion of different words to measure the quality of the news.
The basic sentiment looks at the polarity of the news: good, bad, or neutral.
More advanced sentiment analysis can further express more sophisticated emotional details, such as “anger,” “surprise,” or “beyond expectation.” Some typical trading strategies could be following the sentiment directly.
News sentiment is just a fact. In order to pass to the market, they need to be processed by human beings.
Thus, public sentiment may also play an equally important role. We know from psychological research that emotions play an important role in human decision-making processes.
Behavioral finance further proves that financial decision-making is largely driven by emotions. So we have reason to assume that public sentiment can drive stock market prices like news. This is seen in a recent study where analysts were able to use the mood of Twitter by using NLP on tweets and predicting the stock market.
Written by Harrison Pan, Edited by Han Cui & Alexander Fleiss
‘Finding Alpha: A Quantitative Approach to Building Trading Strategies’, Igor Tulchinsky, P50 & P91