Max Bowie: Filtering Out the Noise
The Dow Jones Industrial Average’s recent rollercoaster ride—which, as of August 11, had changed direction each day for the prior seven trading days and posted a net change of at least 400 points for four consecutive days for the first time in history—has left analysts baffled. At the same time, investors are worried about the worth of their savings and retirement funds. But the situation has also left the data industry scrutinizing the excessive volumes generated by the market volatility and no doubt wondering whether their existing infrastructures will be able to handle a repeat event.
Affordable bandwidth, high-performance processors and data-compression technologies have largely overcome capacity issues, but the huge volumes of data created by the volatile swings have brought the issue back to the fore. According to marketdatapeaks.com—the volume-monitoring joint venture between Exegy, Essex Radez and the Financial Information Forum—US data rates hit 5.25 million messages per second (MPS) on August 4 and 4.9 million twice on August 10, edging perilously close to the capacity of 5.257 million mps recommended by the Options Price Reporting Authority. (These figures are only for options data, while marketdatapeaks’ figures include equities markets.)
According to figures collected by low-latency ticker plant vendor SpryWare from the FIF Market Data Capacity Working Group, consolidated US equity messages increased by almost 90 percent to 1.03 billion messages per day (MPD) between June 2009 and June 2011, while OPRA messages rose by more than 140 percent to 7.14 billion MPD. In comparison, total equity messages on August 5 were almost 2.18 billion—an increase of 110 percent over the peaks set two months before, while OPRA was 77 percent higher with 12.6 billion messages.
Excess Volume
This still leaves sufficient headroom to not hit OPRA’s bandwidth recommendations for July to support 17.3 billion messages per day, but certainly exceeds the rule of thumb that firms generally like to build in capacity of 200 percent over and above the latest peaks—an expensive proposition, but one that doubtless kept the markets moving during recent weeks.
Although data vendors seem to have handled the high volumes, the exchanges didn’t cope as well. On August 10, Nasdaq had to reset the Securities Information Processor for Channel 6 of its UTP Quotation Data Feed—which carries data on securities starting with the letters S through Z, and includes symbols that experienced heavy trading activity—after maxing out the volume of messages it could support in one day (100 million).
Andy Nybo, principal and head of derivatives at research firm Tabb Group, says firms must balance whether to build in additional capacity to support peaks that may or may not occur, even if it strains their budgets. “Some firms impacted will re-evaluate their technical infrastructures that manage their data activities, and will evaluate how much they need to invest and re-invest,” he says, adding that whatever the cost, not being able to handle peak volumes could prove even more costly.
Over the Limit
Whenever vendors have battled volumes with conflation techniques, consumers—insistent on receiving the full picture of the marketplace—have turned to vendors who can provide every tick. However, as that becomes less practical, we may see some firms balancing the needs of their traders against the limits of their infrastructure and demanding that data sources provide more sophisticated ways to filter out noise without diluting the value of data.
Ticker plants have allowed firms to do this on their own site, but that still requires the firm to have sufficient capacity to process the full incoming feeds. So I expect to see clients shift this burden on to the exchanges themselves, demanding custom-tailored feeds that supply only the information they need. Deutsche Börse has allowed clients to filter out the data they don’t want from its CEF Alpha feed for several years—look for others to do so too.
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Printing this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Copying this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
More on Emerging Technologies
This Week: ION/LuxSE, BNY Mellon, Nasdaq, and more
A summary of the latest financial technology news.
Nasdaq to market new options strike listing tech to other exchanges
The exchange operator is experimenting with emerging technologies to determine which options strike prices belong in a crowded market, with hopes to sell the tech to its peers.
Former Goldman analyst aims to blend GenAI and synthetic data with start-up
Synthera.ai is taking a novel approach to calculating risk. While promising, industry observers are skeptical.
Waters Wavelength Podcast: Bloomberg’s Tony McManus
Tony McManus, global head of enterprise data division at Bloomberg, joins the podcast to talk about the importance of data in the context of AI and GenAI.
Devil’s Bargain: Closed architecture systems will derail AI ambitions
Rob Flatley explains why closed-off systems will fall flat when it comes to AI adoption.
This Week: First Trust/Bloomberg/New Constructs, Cboe/Metaurus, LTX/MultiLynq, and more
A summary of the latest financial technology news.
Waters Wavelength Podcast: S&P’s CTO on AI, data, and the future of datacenters
Frank Tarsillo, CTO at S&P Global Market Intelligence, joins the podcast to discuss the firm’s approach to AI, the importance of data, and what might be in store for datacenters in the coming years.
BMO’s cloud migration strategy eases AI adoption
The Canadian bank is embracing a more digital future as its cloud strategy makes gains and it looks to both traditional machine learning and generative AI for further augmentation.
Most read
- Waters Wavelength Podcast: Bloomberg’s Tony McManus
- IMD & IRD Awards 2024: All the winners
- Waters Wavelength Podcast: S&P’s CTO on AI, data, and the future of datacenters