For the love of grid – Joel Clark investigates the rise in popularity of computer grids supporting a variety of compute-intensive processes on the sell side and looks at how buy-side firms might benefit by deploying such technologies.

Grid has proven itself to be a significant enabler of business on the sell side and there are few tier-one banks that have not yet implemented a grid of some kind to harness their latent computing power. But as the banks have scrambled into the ring, the buy side has been lying in wait – no surprise there – and watching the technology evolve. Some funds and managers however, are now starting to dip their toes into the water and consider grid implementations. Experts are certain that the buy side will be the next stop on this hardware innovation's journey through the financial services world.

The need for grid computing has appeared over the last decade as financial operations have become more complex, requiring increasing amounts of computing power. In the 1990s, calculating prices for complex financial instruments or running risk simulations on large portfolios was taking unwieldy lengths of time and slowing down the business of supposedly fast-moving investment banks and brokerages. The initial idea of grid computing was to harness all available compute power to speed up such processes. A grid would essentially allow firms to access and harness any idle computers on a network to complete calculations.

Barry Childe, director of high-performance computing at Barclays Capital in London, manages his firm's grid initiatives and has been at the forefront of the technology since its inception. He remembers that in the 1990s, the sheer compute power required to price exotic instruments such as collateralised debt obligations (CDOs) meant customising platforms and setting the speed of processors beyond their stated maximum for improved performance. Consequently, the CPUs (central processing units) were continually overheating and would therefore not last long. "In those days we had lots of unique, customised systems to provide pricing that ran at ridiculous temperatures," he says. "The number of Monte Carlo iterations we needed in order to produce risk pricing was getting too big for what we could fit on a single machine; the systems had a shelf life of six months, after which the processors just melted from constant usage." Childe says grid came as a lifeline to banks when it first arrived in the late 1990s. He has championed its use ever since.

Investment banks such as Barclays Capital have yielded significant benefits from a relatively low investment in grid technology, massively reducing the time to complete calculations through simple allocation of resources. Adam Vile, head of grid, high-performance, and technical computing at London-based consultancy Excelian, says that in many cases the use of grid technology has actually allowed banks to trade instruments they would not otherwise have been able to. He cites one example in which the pricing and risk calculations on a single structured product were reduced from 32 hours to two minutes by using grid. "This technology essentially allows banks to trade more rapidly and dynamically because they can revalue portfolios every few seconds and make investment decisions more quickly," he says.

But technology is not the only challenge for a firm looking to use grid. There are also cultural and organisational barriers that need to be overcome to allow for the sharing of resources. William Fellows, principal consultant at technology research firm the 451 group, says this is 80% of the challenge: "Grid is still in the early stages, but I'd say that technology is only 20% of it; the biggest thing is really creating a shared internal enterprise utility where resources can be shared across departments and groups."

While large sell-side organisations were the earliest adopters of grid technology, few are in any doubt that, given time, the technology will ultimately trickle down and permeate the buy side. Investment banks were the first users mainly because the size of their operations created the greatest need. "The sheer volume of their books has meant that they needed that compute power to manage their risk," says Childe. But he adds that the scope of grid technology is certainly not limited to the large investment banks. "The common misconception is that you have to build a space shuttle to use grid, but grid is actually very good for managing your bicycles rather than having to build that space shuttle."

Many of the buy-side bicycles to which Childe refers are certainly keen to embrace grid and the need is set to grow stronger. "We're on the cusp with the buy side," says Vile. "They've got to the point now where their existing technology can't handle the kind of things they want to do. They can either just not do it or they can invest in this technology."

The vendors are certainly waiting to welcome the buy-side firms when they are ready to take the grid route. "It's still early days but in the 2008 financial year, we would like to see a doubling of penetration in hedge funds and traditional fund managers," says Charles Jarvis, vice-president for EMEA financial services at Platform Computing, a grid provider. Like Childe, he says grid is as effective for a small operation as it is for the largest banks. While a sell-side outfit might use the technology across thousands of CPUs, Jarvis says one of his clients, an energy trading firm, derives a proportionally equal benefit across just 50 CPUs.

One significant pool of interest on the buy side is among the larger hedge funds which trade exotic instruments. As regulatory pressure continues pushing all financial institutions to provide better risk analysis and greater transparency, grid is becoming the most viable option to manage those calculations, even for technology-averse hedge funds. "It all boils down to the fact that it could take hedge funds several hours or even days to run a calculation, but if you parallelise it across all their available computing power, you can get it done in minutes," says Jarvis.

Although hedge funds and institutional buy-side firms are still in the early stages of looking at grid technology, there is little doubt that it is coming soon. "We do see more and more hedge fund representatives at our grid events," says Childe. "They are a culture of people who we would very much like to encompass into our space because they bring a lot of smart talent into a world which is very exciting."

Hedge funds may accept that they need to break down business lines and harness computer power to run their simulations, but many remain unwilling to invest in more computers to speed up that process. An option which many say will be popular on the buy side is to rent computers by the hour from the handful of hardware vendors which offer the service, including IBM and HP. "This may be where you'll see it break through because buy-side firms won't make the same massive capital infrastructure investments that the sell side has, but they may be willing to pay when they use it," says Tony Bishop, senior vice-president and head of the architecture and engineering group at US-based financial services company, Wachovia. But Fellows says the 'compute-on-demand' model still has some way to go. "So far the evidence suggests that the usage of this notion of on-demand or utility computing hasn't worked all that well with the vendors who have branded it. Users want to have licences supplied to them along with compute capacity and that's certainly not part of anyone's offering yet. These are baby steps but it's definitely moving forward."

Yet as the notion of renting compute resources gathers pace, Bishop says that several high-end US hedge funds already have their own grid capabilities, although they are keeping them hidden from public view. "The visionaries are already there and I'd predict that 2008 will be the year when it starts to become more generally accepted and buy-side firms will need it for competitive advantage," he says.

Once hedge funds and asset managers have access to enough CPUs, whether in-house or through the compute-on-demand model, the implementation of the grid itself must be done through an independent software vendor. One such vendor, DataSynapse, is beginning to see some level of interest in buy-side deployment. "We already deal with the investment management arms of big banks who have enterprise licence agreements with us," says Willy Ross, senior vice-president of DataSynapse, whose GridServer product has had high penetration on the sell side. "It's inevitable that this will take off more on the buy side because the rate at which the sell side is introducing new structures and products is increasing rapidly. The buy side has got to be able to price and get their risk profile right so they can keep up with the sell side or they won't know what they're buying."

Ross says that several DataSynapse clients have used GridServer to accelerate the performance of high-powered risk management products provided by Algorithmics, a Toronto-based provider with a large buy-side client base. John Macdonald, executive vice-president of Algorithmics, agrees that grid is becoming a sensible way to get the best out of his products. "DataSynapse is harnessing a lot of the technology that already exists, enabling our clients to do lots of the processing that traditionally could only be done on much bigger, more complex machines." He stresses that interest in the technology is on the rise among his buy-side clients who are starting to understand that it will drive better performance of Algorithmics' products. "We're definitely receiving more enquiries around this as a lot more processing and computing power is needed for the kind of software we provide that allows them to improve their decisions, look at what-if calculations and rebalance their portfolios," he says.

Grid technology may be a revolutionary way of speeding up processing, but firms should be wary of thinking it will remove all bottlenecks. For most large financial institutions, deploying computer grids is not enough – they need to implement additional technology to harness all the data required for the running of calculations. "In a grid, the bottleneck becomes access to the database, particularly if you have thousands of computers trying to get data at the same time," says Vile. Typically banks have abstracted data from a database and put it into a cache which sits underneath the grid."

Data cache systems are provided by vendors such as Gemstone, Gigaspaces and Tangosol, but buy-side firms will need to make their own decisions about whether they need a data cache vendor as well as a grid vendor. "It depends on the scale of the firm," says Bishop. "If you have multiple areas of operation and multiple data sets, you may need a data cache, but the beauty of this technology is that you can crawl, walk or run with it. So if you want to use it to price a deal versus a portfolio and start with simple products, you can go through a natural evolution."

The simplicity and effectiveness of grid technology is certainly appealing to buy-side firms as their need for compute power grows exponentially. But despite its simplicity, the implementation is complex and experts stress the importance of working with the right people. "The danger is you can get it wrong very easily," says Childe. "It's quite hard to tell people where to start with grid because it's been such an evolving and fast-running technology."

But Childe says if buy-side firms can find a good consultant or analyst to help with the implementation, they will yield the benefits. "I'm a huge evangelist of this technology and I think if you can get the right person who understands it up front, with a fairly minimal level of effort you can achieve some massive performance goals, even within smaller organisations." >

- Hedge funds and asset managers are starting to see the benefit of grid technology on a smaller scale, although adoption is yet to become widespread

- Although the concept of grid is simple, the implementation is complex and firms need to find good consultants before they start deployment