Data Volume Explosion Continues

Industry observers cite numerous reasons for the growth in message rates-including increased electronic trading, the introduction of penny-increment options prices, fragmenting liquidity across exchanges, cross-asset algorithms that increased the quote-to-trade ratio, and the requirements of Reg NMS.

Message rates from the Options Price Reporting Authority showed the most dramatic growth, with the number of messages per second sent by Opra more than doubling from 200,000 to 440,000 between October 2006 and October 2007, according to data from the Financial Information Forum. By comparison, message rates for the previous year grew by 60 percent from 120,000 mps to 200,000 mps, FIF reported.

Recent years have also seen an increase in the quote-to-trade ratio of Opra messages-from 3,543 quotes for every option trade in 2005 to 4,243 this year, according to the FIF.

The Chicago Board Options Exchange reached an all time high this year of 72,169 mps on Nov. 2, according to the exchange. Equities markets saw similar rises, with Nasdaq's TotalView ITCH 2.0 feed growing from around 60,000 mps to a total of 175,000 mps during 2007, according to the FIF.

Rising message rates prompted exchanges and trading venues to introduce datafeeds utilizing the FIX FAST (FIX Adapted for Streaming data) protocol, which compresses data by up to 70 percent and reduces bandwidth requirements. For example, Opra's projected bandwidth recommendations for July 2008 are 517,000 megabits per second (Mbps) using the legacy ASCII network, compared to 155,100 Mbps using FAST (IMD, July 16).

In the past year, Opra, the International Securities Exchange, the American Stock Exchange, Swiss exchange operator SWX, the Nordic exchange group OMX, the European trading facility Turquoise, Deutsche Börse, the CME Group, the BATS Trading ECN and Instinet's Chi-X platform have announced plans to introduce FAST-encoded data feeds.

The bandwidth savings made possible by FAST have prompted Opra and the CME Group to migrate off old data protocols in favor of FAST. Opra will eliminate its ASCII network and only support FAST as of Jan. 25, 2008, while CME Group has instructed members to implement FAST as of Oct. 17, 2008, and has said it will impose surcharges for each month that members continue to use CME's legacy RLC format. (IMD, Oct. 1).

But the proliferation of FAST-encoded datafeeds has generated concerns from the authors of the format about the potential for FAST implementations that are not fully consistent with the standard's technical specifications, said Richard Shriver, co-chair of FIX Protocol Limited's market data optimization working group and managing director at consultancy Jordan & Jordan. To ensure a consistent standard, FPL created an interoperability portal online to allow users to verify the consistency of their FAST implementation (IMD, Nov. 19).

Complex event processing has emerged as a technology to help firms perform analysis on the increasing volumes of real-time data. For example, Hartford, Conn.-based investment manager George Weiss Associates is implementing a CEP engine from Aleri as part of a systems re-engineering project to enable the firm to handle rising data volumes, which had begun to tax the firm's servers. (IMD, Dec. 3). The platform will allow Weiss to perform profit and loss calculations in real time, irrespective of higher data rates.

Other firms have elected to craft their own solutions to cope with data volumes. For example, in reaction to record data volumes this summer, Credit Suisse undertook a four-week project in August to increase its market data capacity (IMD, Nov. 19). The effort focused on improving the ability of the firm's high-volume execution systems to accommodate high data rates, and involved adding further mainframe capacity, servers for order routing and notification servers to distribute load, and also involved adding capacity to its order management system application server.

Hardware acceleration ticker plants and messaging systems, unveiled this year by Exegy, Activ Financial and Terevela, provide an alternative to firms seeking to handle higher data rates without adding servers, though the technologies still remain in the realm of early adopters (see story, page 15).

The SEC's pilot phase of quoting options in penny increments, initiated by US exchanges in January, tripled the overall quote traffic for penny-listed issues. Quote mitigation strategies implemented by the exchanges to reduce message rates have limited the data growth, but could not prevent significant rises overall.

The initial pilot, which lasted from January to July and involved 13 option classes, still resulted in net quote-traffic increases of more than 200 percent, even after the effect of quote mitigation strategies. The continuation of the pilot until September resulted in further quote traffic increases, despite beefed-up mitigation strategies. Amex, for example, reported an 89 percent increase in average daily quotes per series between May and September compared to the pre-pilot period of November through January, while quote rates of the QQQQ-series exchange-traded fund (now known as the PowerShares QQQ) increased by 227 percent.

With the next phase of the pilot lasting until March 2008, the unprecedented rises look set to continue: In anticipation of even greater growth next year, Opra has recommended that subscribers to its consolidated feed of US options trade and quote data should increase their bandwidth to accept 907,000 mps by July 2008 (IMD, July 16).