More on CEP and Complexity

I must admit that while writing my last post I was hoping it'd start a discussion going. So I'm very grateful to Greg Reemler for his reply. While Greg makes some interesting points, I, naturally, respectfully disagree with some of them.

It's easy to argue against small points in Greg's post. For example, Greg writes "Coral8 sounds more like a lab tool for the engineering department of Caltech or Stanford than a tool for everyday business users." Well, everybody knows that Coral8 Engine, as well as other CEP platforms, as opposed to CEP applications, are tools for developers, and not for business users.

Or, it's easy to point to other very successful examples of enterprise software products that are more complex than the Coral8 Engine. But I don't want to do this. Because Greg has an excellent point: if a product is needlessly complex, nobody is going to use it!

The place where Greg and I disagree is whether our product is needlessly complex. Here is my claim: the list in the previous post covers the features and skills needed to write sophisticated real-world CEP applications. Those are the applications that process tens and hundreds of thousands of events per second, coming from a number of very different data sources, integrating with databases, message buses, and numerous other third party systems, and performing highly complex analysis with millisecond latency. All this while being highly scalable, available, and secure.

Unfortunately, most of discussion around CEP today focuses on very simple use cases. Everybody publishes their implementation of VWAP (Volume-Weighted Average Price) and some benchmarks for VWAP. Well, if CEP was limited to such use cases, people would be right to consider CEP just hype. Writing a simple CEP application is indeed quite easy in C or Java, as Greg correctly notes. It's when the complexity of the application grows that CEP engines start to shine.

How do I know this? We have about 50+ customers now, and good 90% of them had started writing their applications using traditional languages, such as C or Java. And they all were happy for a while. But then they needed to add more streams, more data sources, and more complex queries. They needed to talk to databases and to Web Services. They needed to handle XML events and complex event patterns. And most of all, they needed to make sure their application would never fail.

That's when they start looking around, find this whole field of CEP, and realize that the Coral8 Engine is indeed a great choice for writing highly sophisticated CEP applications. Our product makes accomplishing all the tasks listed in the last post easy. Is there a learning curve? Sure. It does not take a week, as Greg suggests, to learn each of the 60 topics. It takes maybe an hour, give or take. Which means that in under two weeks customers can become true CEP experts, if their applications demand this level of expertise. If you consider what they get for their effort, you'll understand why they feel the effort is well worth it.

So, if I may offer Greg some friendly advice, here it is: if your CEP application is simple, by all means use your C or Java programmers. I'm sure they'll do a fine job. But if or when you realize that the requirements are more complex, feel free to give us a call. We'll be happy to talk to you and your team about our product.

Mark Tsimelzon, President & CTO, Coral8