Stock market and data analytics: How machine learning helps to reduce trading costs

Technology is scaling at an exponential rate and today we are processing vast loads of data if numbers are to be believed. A recent report reveals that the total data existing in the world will grow at a CAGR of 61% to 175 zettabytes by 2025 from 33 zettabytes in 2018. These numbers are huge!

Today, Machine Learning and Data Analytics are making trading much more efficient. Together, they complement each other and act as catalysts towards improved ability to identify opportunities and reduce trading costs. Leveraging data in trading at present comes mostly in two flavours. During the research phase, the ability to analyze and learn effectively from the vast amount of data gives the strategies an edge. Here the analysis is mostly exploratory and speed is important but not critical. During the execution phase, these same tools can be used to gain the ability to quickly react to changing market conditions. Here speed is critical to success.

Also read: Share market LIVE: Sensex down 340 points, Nifty below 11,400; Sun Pharma, Tata Motors among top losers

Data and analysis have always been an integral part of trading and investing. A discretionary investor goes through volumes of company annual reports, news and price charts. The analysis tools are mostly limited (like spreadsheets) augmented by the most powerful data analytics engine today, the human brain. In fact, compared to the algos, the biggest edge of a discretionary trader is the deep proprietary insights developed by repeatedly going through this process of research.

However, the markets are becoming increasingly more complex. The globalized nature of our economies makes everything connected – a plane crash in Africa impacts the price of the manufacturer’s stock on American exchanges immediately. The sheer volume of price sensitive data that we generate today – tick-by-tick price, the real-time news feed of central bank meetings, or even a tweet – are simply too much for a single person to completely process and understand.

This is where the algos kick in, armed with machine learning and data analytics. Machines can tirelessly monitor data and news feeds, learn from them and act upon them. They are yet nowhere near perfect. But with advancement in natural language processing, learning algorithms, big data analytics and hardware technologies, today’s machines already surpass human level accuracies in many tasks. For an algo trader, this is the primary edge – the ability to process a vast amount of data at speed and scale.

Today, there are at least three areas where machine learning and data analytics have very promising applications in trading and investing. Trade execution, the process to complete a given trade order at the best price under constraints, already is an industry norm. Based on parameters set by humans, algos analyze the order book in real time, slice and route orders, and execute them to minimize impacts costs and slippages.

The second area is the application of natural language processing (NLP). Using NLP, machines can analyze and learn from unstructured data and texts, like using it to create strategies based on sentiment analysis for trading. Finally, alternative datasets are making information discovery, which was hitherto impossible, like estimating sales at supermarkets before quarterly results announcements, by counting footfalls through analysis of satellite images of parking lots.

In all these cases, the use of machine learning and data analytics provides an edge to the algos over their human counterparts. The ability to monitor and learn from the vast number of instruments and react quicker brings in many benefits – like reduced trading costs, better signal generations, better diversification and risk management etc. And these benefits are now not only restricted to institutional money.

New financial technology innovations are bringing these sophisticated tools and know-how to individual investors and traders. This is very much visible in the urgency that we see amongst individual investors & traders to learn about algorithmic & quantitative trading. Big data and machine learning have already conquered many arenas of human excellence. It is unwise to assume trading and investment will be any exceptions. Investing and trading are slowly, but surely, becoming more and more data-driven.

Humans have a superior ability to analyze context and develop the hypothesis, while machine learning allows us to validate, test and execute them more efficiently. But it is not a question of the man vs. the machine. The future of trading is the man and the machine.

(The author is Mr Prodipta Ghosh, Vice President, QuantInsti. Views expressed in the article are author’s own.)