Tick History – Query: Looking back to the future

The advantages of cloud-based services is well documented, from reduced upfront and ongoing operating and infrastructure costs to improved time-to-market for new services and datasets. Here, Tim Anderson, head of Tick History at London Stock Exchange Group (LSEG) explains how the benefits of the service model are greater than initially meets the eye, including dramatically enhanced analytics performance and the ability to look to the past as a means of shaping the future. 

The advent of cloud-based data services fundamentally changed the relationship between data producers and consumers. Traditionally, producers offered customers bulk downloads of data and left them with the heavy lifting associated with storage, maintenance and interrogation of that data. 

Now, however, the relationship is more consultative and collaborative. With LSEG’s Tick History – Query, customers consume only the data they require and can analyse it in the cloud. “It’s the LSEG cloud coming to the customer, where we snap into their cloud and share the activities between the two organisations,” explains Tim Anderson, head of Tick History, at LSEG.

“In an on-demand model, when it comes to market Tick History and market data, we’re dealing with petabytes [PB] of data—we have 20PB of normalised Tick History data—which is not something you can easily manoeuvre around an organisation. When a customer wants a specific part of Tick History, they can navigate to the on-demand service, query it and take away whatever they need. The old model was to take everything and then start looking for the needle in the haystack.”   
 

Problem solving 

Aside from the usual advantages of cloud-based services, Anderson explains that Tick History – Query delivers additional benefits to users that solve day-to-day challenges. The first, he says, addresses the issue of storage, which is often overlooked by financial services firms. Storage of data can often prove prohibitively expensive for firms to manage in-house—not just in terms of pure storage technology but also with the infrastructure around storage technology, such as capacity and resilience planning, cabling and cooling. 

The second benefit is that traditional on-premises systems are not good at querying or analyzing large data volumes or datasets—a challenge mitigated by LSEG’s partnership with Google to power its Tick History – Query solution. “We store all the data in Google BigQuery™ where we leverage Google’s database technology,” Anderson explains. “Customers can run a single query across 27 years of historical tick data across multiple venues and exchanges without having to break that data up into chunks and batch process it overnight. That allows them to get their responses from the service in seconds.” 
 

Making the business case

Making the business case to financial services firms for subscribing to cloud-based services is simple: it allows them to focus on core competencies, none of which entail building, operating or maintaining in‑house data centers. 

“When you think of an investment bank, a hedge fund, a fund of funds or a fund manager, their core business is not technology and maintaining data centers or even storing and maintaining data—their core business is investing,” says Anderson.

“For an on-premises system, it costs about $110 per gigabyte (GB) to store data, accounting for everything from cabling and racks to salaries. Typically, that figure drops to about $4/GB with a cloud-based service. When you get into the territory of terabytes and petabytes of data, that is a significant cost saving. 

“That allows firms to get on with doing what they do best, which is the analytics on behalf of their customers to be first to market and get results for them.” 
 

Not just any service 

Pretty much any data vendor worth their salt serving the capital markets either already has a cloud-based service or is in the throes of developing one. However, the cloud is not the differentiator—the data is. LSEG boasts close to 600 venues and exchanges stored in its Tick History database, covering the best part of 27 years of history, and featuring every tick and data point that has occurred across the capital markets for the past quarter of a century—a track record and pedigree impossible for other providers to match. 

“Essentially, we have this point-in-time database on which customers can conduct back-testing,” Anderson continues. “For example, we haven’t experienced double-digit inflation since the early 1990s. Customers can go back to that point and see how the markets reacted during that period and then redefine their trading strategies based on that information.

“The fact you have all that data on demand, available through an application programming interface or shared directly into your cloud (without the need to store data) with a high-performance database sitting behind it means it is very powerful but also very light.”   
 

Artificial intelligence (AI) to the fore 

According to Anderson, various types of AI are set to be suffused into the Tick History service, which is likely to impact how customers use the service, most notably around identifying trends and outliers in market behavior. 

“Machine learning will start to understand what is occurring in the markets and tell customers what it is seeing,” he explains. “It will start understanding the cadence of the market and patterns that occur and it will notify customers. when it identifies anomalies.”

Another area that may be impacted is helping customers marry additional content and datasets with Tick History as a means of identifying signals when multiple data points are combined. For example, with machine-readable news where headlines, events and sentiment can influence market behavior and vice versa.

“With machine learning, because we have so much data, it has become very good at understanding the way the markets behave under certain conditions and can inform customers on that,” Anderson says. 

The final area in which AI could help customers is writing queries into the Tick History – Query service as efficiently as possible.

They might write code that is bloated, where the AI would suggest, for example, five issues that would improve its efficiency.

“Ultimately, we would like customers to be able to type natural language into the system, enabling someone with business logic to write their understanding of the market into the system and receive a natural-language response, as the coding is taken care of by the system,” Anderson says. 
 

Too important to ignore

Clearly, cloud-based managed data services have the potential to deliver game-changing enhancements compared with on-premises systems. From a performance perspective, the improvements are nothing short of astonishing: in the past, financial services firms might have run queries on very large datasets overnight due to the sheer volumes of data involved. But, with cloud-based services, they can run such queries on the fly and expect results to be delivered in little more than a few seconds. 

The cost benefits are similarly impressive, especially when storing, managing and sharing large volumes of complex data. The cloud is appreciably more cost effective than in-house developed and maintained data warehouses, not to mention more operationally robust, flexible and infinitely scalable. Those benefits are simply too important for financial services firms to ignore.   

Sponsored content