CEP Comes of Age

During the course of 2007, complex event processing (CEP) software emerged from the stage of people saying, "Gee, let's take a look at this technology," to a stage where people began to say, "Gee, this stuff really works." In the process, CEP has become a valuable engine driving a number of strategic applications.

While much of the early attention around CEP centered on applications for algorithmic trading, complex event processing has since expanded well beyond this domain and is now used in a much broader array of financial trading applications. CEP has dramatically changed the application development landscape in financial trading, enabling the creation of new applications that were previously impossible.

As one would expect, the top-tier investment banks have all "kicked the tires" of CEP, with some progressing more aggressively than others. These firms are expanding their use of CEP globally with applications that perform tick-by-tick portfolio and risk analysis, examine granular market trends and trading activity in real time, and ensure the health of their trading infrastructure.

But CEP is not just for the large institutional firms. Smaller, more nimble trading firms have used CEP as a competitive weapon to build new trading backbones in record time, enabling them to reach new markets faster than their competitors. And the list goes on:

• Private trading networks use CEP to monitor illicit behavior by traders and capture best execution analytics.

• Buy-side firms and hedge funds dynamically price the value of their instruments to make instantaneous trading decisions.

• Alternative investment trading firms allow traders to run personalized, dynamic, real-time trading strategy analysis.

Clearly, the appeal of CEP lies not just with large firms, but with all companies that strive to create competitive advantage by tapping new markets, simplifying strategies, and providing better service to customers. So, what does this mean for CEP in 2008?

We've established that early adopters are now comfortable relying on CEP technology for business critical applications, to the extent that adoption has now reached a critical mass. What happens when critical mass begins to build? Adoption expands in three dimensions.

First, wide-scale, tick-by-tick complex analysis becomes a reality.

Many CEP engines are great at demonstrating tick-by-tick VWAP calculations. All the firms I've encountered yawn at such simplicity. They describe the complex, real-time, tick-by-tick analyses they want to perform, and challenge the CEP engine to perform the work.

Each time, we've taken on these sophisticated analyses and proven that CEP can handle both the complexity and speed. In 2008, with these proof points in hand, development teams will take advantage of the agile programming language and deployment environments of CEP to quickly deploy a larger number of highly complex real-time analyses. Some will take advantage of the ability of CEP engines to dynamically run parameterized instances of these queries, allowing traders to run their own, personalized real-time trading strategies.

Second, CEP will grow to encompass more applications across more data sources.

In 2008, investment firms will build CEP applications that tap a greater array of real-time information sources. Now comfortable that CEP can handle the "garden hose" that is the equities market data sources, firms will open CEP to the "fire hoses" such as options market data. In addition, the ability for CEP engines to natively analyze XML data will enable applications to analyze complex, nested data sources, such as news feeds.

Third, existing applications will go "enterprise."

Once "division-scale" CEP applications have proven their value in analyzing portfolios and risk at a tick-by-tick granularity, these results can feed larger, enterprise-scale trading and risk monitoring systems. The network-oriented nature of certain CEP engines allows them to be "chained" in a pyramid-style application, configuration for real-time, event-driven rollups and monitoring at an enterprise level. Investment firms will truly start to embrace the power of CEP at the enterprise level.

With the Société Générale rogue trading scandal fresh in our minds, it is a reasonable assumption that many large investment firms will look for ways to monitor their trading and market risk at a both a granular level within divisions, and at a global enterprise level.

An exciting phenomenon fueling the rise of CEP is the rapid growth of a CEP developer community. We are witnessing more companies sharing their experiences with CEP technology and exchanging ideas-for example, when one firm discusses where and how they used CEP, it then spurs ideas inside other firms on how they can use this technology also. Or one division of a firm shows how they used CEP, which in turn generates ideas within other divisions of the same firm. This growing culture of knowledge networking will enable a large number of creative developers to invent even more applications for CEP.

So, how will CEP be viewed at the end of 2008, versus the end of 2007? By the end of 2008, CEP will not be considered a new technology any longer, but will be a critical part of the overall trading, intelligence and decision-making fabric. By the end of 2008, we will not be wondering what the limitations of CEP are, but rather where the ceiling lies.