Weaving an Enterprise Data Fabric

In September, sibling publication Waters magazine hosted a breakfast briefing in London sponsored by Gemstone Systems, entitled "Enterprise Data Fabric: Low-latency infrastructures for trading and grid-based analytics," which outlined the challenges of delivering a highly responsive data backbone. The panel was moderated by Victor Anderson, editor of Hedge Fund & Investment Technology, another sibling newsletter of Inside Market Data.

Similarly, look at the market data space. There's obviously a huge amount of innovation going on there among various companies, such as Wombat, 29West and others, which are emerging at the moment.

The challenge is how to integrate them. How do you enable your existing systems to take advantage and to best use these emerging technologies? How do you keep far enough up the curve without falling off the top? You don't want to adopt these too early before they are mature.

You should carefully plan the rollout, and carefully choose the areas where you have a real problem to solve rather than just say, "I'm going to mandate a global standard and we'll roll it out everywhere." You have to choose particular areas where you think there is real need before you can deploy some of these solutions.

As far as other market data is concerned, even for start-of-day data to get accurate data for start-of-day positioning … the big firms have got multiple sources of data coming in. It's not consolidated in one particular place. The data coming through from market vendors is actually not accurate so they have to scrub the data before fund managers can use it.

On the buy side, we're not even thinking like the sell side in many respects. With over-the-counter instruments coming through, the buy side is trying to make sure that the price from the sell side is accurate. Now the question is whether the buy side has to get tick data. If so, where is it from? Where will it be stored?

We have clients who are buying derivatives instruments. They don't even have a system to actually import the trade yet. If you can't put the trade in, forget about analytics, risks, real times, and so on.

So it comes down to the sell side creating the products and trying to stay competitive. On the buy side, we're years behind.

One of the considerations is something like a data fabric. How do you fit in? It turns out we had to fit it in at multiple levels. You have to fit in right, starting from the technology stack at the processor level. We're working very closely with Intel to take advantage of some of the processor optimization.

You eventually come to the final question that people always ask: What level of intrusiveness does this impose? What level of changes do I have to make at the application level? That's when you have to try to fit it in as naturally as possible.

That drove the technology problem, which means if you're not fast enough and your technology isn't up to it, you lose an awful lot of money. I'll be honest: a while back, our technology was not up to it.

The emergence of a reliable—or a number of possibly reliable—industry standard technologies to do this saves you the huge cost of having to do it yourself. You save the opportunity and hard-dollar cost to the business of the thing breaking every five minutes.

What is the penalty for data not being available? What's the penalty for data availability being unpredictable when you want to execute a trade or compute a risk position? The cost can be monumental. In some sense the return on investment in technology is huge if you measure it in those terms.

A natural business benefit that drives the adoption of this technology is the aim to reduce latency when accessing data to certain downstream applications. That drives the ability to execute things faster and beat the market, and it leads to a lot of profit.

I think we're sitting on something that is fundamentally going to change the way we can manage data across the firm. In that way, it's really exciting.

How do we do this? Many audience members here work for big firms. The way the vendor-buyer relationships work in the technology industry, you've got to exercise your muscle back on us as vendors to get these projects working.

You've got to come back to us and actually force these ecosystems to work. We have done a great job with the technology stack, making it open so you can interchange all the different layers. That is fantastic progress in terms of open systems.

You have to set your strategy down, and ask, "What am I going to do around the chip level, operating system, virtualization, grid, EDF, and so on?" Then, come back to us to form alliances to fix some of these problems. If the problem is a niche issue, such as accelerating a particular app like a FIX engine, it's relatively easy: point and fix.

That allows you to say, "We need ultra-low latency here," or "We actually need volume and scale for our retail business here." And those are some of the challenges we address in the next release.