Gregor's Ramblings

Are "Pattern" and "Component" Antonyms?

October 15, 2004

Gregor HohpeHi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I like to work on and write about asynchronous messaging systems, service-oriented architectures, and all sorts of enterprise computing and architecture topics. I am also an Enterprise Strategist at AWS.
ALL RAMBLINGS  Architecture (12)  Cloud (10)  Conversations (8)  Design (26)  Events (27)  Gregor (4)  Integration (19)  Messaging (12)  Modeling (5)  Patterns (8)  Visualization (3)  WebServices (5)  Writing (12) 

My blog posts related to IT strategy, enterprise architecture, digital transformation, and cloud have moved to a new home:

EAI Patterns Toolkit

After last year's OOPSLA my goal was to develop a messaging toolkit that allows people to run the patterns from our book and observe a live messaging system in action. Well, it took a bit longer than originally expected but I now have a kit that implements about a dozen key patterns as individual executables. These executables can be composed into messaging solutions straight from the command line or with simple batch scripts. Most components have a cute little GUI as shown in the following figure.

The example domain for the messaging toolkit is a coffee shop where customers place orders that are then fulfilled by baristas. Along the way orders can be enriched, split, aggregated, resequenced etc. The diagram shows a simple example of a customer placing an order, which is split into individual order items by a Splitter. Two baristas act as Competing Consumers to process orders. A Logger displays messages on the orderCompletedChannel. The scenario can be scripted in a simple batch file:

Customer orderChannel
Splitter orderChannel orderItemChannel "/Order/Item"
Barista orderItemChannel orderCompletedChannel
SecondBarista orderItemChannel orderCompletedChannel
Logger orderCompletedChannel

These scripts make up a nice Domain-Specific Language which is based on the vocabulary in our book. The free composability makes it interesting in fun to build larger messaging solutions from these relatively simple building blocks.

Codifying Patterns

Typically, when people ask me about "codifying" or "toolifying" patterns my first reaction is one of skepticism. Patterns are meant to be a human-to-human communication mechanism, not a human-to-machine mechanism. After all, I have pointed many people to the fact that a pattern is not just the piece of code in the example section. It's the context-problem-forces-solution combination that makes patterns so useful. A pattern applies most of the time, but not always. Patterns can be combined and morphed. Most patterns lead to multiple different implementation strategies. And last but not least a pattern has to be realized, i.e. mapped into a specific programming environment to be used.

This does not mean that patterns cannot be incorporated into a tool. Many tools and libraries embody many successful patterns. Almost every messaging library supports Competing Consumers, Event-driven Consumers and many more patterns. Of course patterns are meant to be included in all kinds of software, including libraries and tools. My skepticism usually arises when someone mentions notion of programming in patterns a la "right click and make a factory" or when someone plans to map patters to a strict metamodel like the OMG's MOF (meta-object facility). The power of patterns lies in the fact that they can be a little bit fuzzy around the edges and constraining them into a specific model seems to me like sticking a wild animal into a cage. They'll likely survive but they will definitely not flourish.

With all this in the back of my mind, now I sit here with a whole collection of codified patterns and am having a ball with it. Did I forget to eat my proverbial dog food or did I have pre-established opinions that were not grounded in actual experience?

Patterns to Components

There is no doubt that implementing the patterns in my messaging toolkit required me to tighten up the "fuzziness" of the patterns. Each component has to have a very specific (read embedded in code) behavior and has to have well defined input and output ports. There is also a specific command line syntax for each component.

So what is the secret sauce here? I think there are a number of specific reasons that the patterns I chose (only about 10 of 65 patterns are implemented) are easily translated into components. First, I fixed a lot of the technical environment parameters: the components only run on a .NET platform (most likely Windows), require MSMQ, can only execute on a single machine, and process only XML messages. As all pattern components share these properties a consistent realization of a collection of patterns seems a lot more likely.

But I believe that other forces are at work. One definite contributor to the easy conversion of these patterns into components is the Pipes-and-Filters architectural style. This style's strongest asset is its simplicity and the easy composability of individual components. This composability allows us to create larger solutions once we have morphed a set of base patterns into components. Object-oriented systems expose much more complex (and also more powerful) ways of composing patterns that do not necessary make it that easy to combine patterns without overlap between them.

I believe one last reason the conversion from pattern to component worked out well in this case is due to the fact that I deliberately choose a family of patterns with well-defined variability points. Each pattern exhibits certain givens while other elements are up to the specific instantiation of the pattern. We call these elements variability points.

Many of the routing patterns are very close descendants of the basic router. They look at a message, make a simple decision and republish the message. The variability point is how the router makes the decision, for example which field of the incoming message it looks at. It turns out that the variability points for most routing patterns are relatively simple expressions. Fixing the document format to XML allows us to codify the expressions in XPath. This means that the behavior of the pattern can be fixed as long as the expressions are variable. This is exactly what happens in case of a Splitter or Content-Based Router. Essentially, we use a general purpose programming language (C# in this example) to codify the general behavior of the pattern (and all the fluff like cutesy user interfaces). Then we rely on a more domain-specific programming (XPath in this example) to codify the variability points. Because the domain-specific language is interpreted it is easy to allow it to be specified on the command line. Voila, we have converted a pattern into a generic enough component.

Likely, we could have created our own domain-specific language to express the variability points or use a popular scripting language like Ruby. However, since our XML documents are simple, using XPath to express the variability seemed simple and convenient.

Example: Splitter

To demonstrate how this reasoning translated into practice, let's look at the (abbreviated) implementation of the Splitter component. The core code of the Splitter is quite compact, once again benefiting from imposing the proper constraints (XML only, .NET only, using a highly abstracted messaging library):

public class SplitterComponent : SimpleFilter
    protected string nodePath;

    public SplitterComponent(IMessageReceiver input, IMessageSender output, string nodePath) : base (input, output)
        this.nodePath = nodePath;

    protected override void OnMessage(XmlDocument message)
        XmlNodeList nodeList = message.SelectNodes(nodePath);
        foreach (XmlNode node in nodeList)
            XmlDocument splitDoc = new XmlDocument();
            splitDoc.AppendChild(splitDoc.ImportNode(node, true));

The run-time environment provides the Splitter class with an input channel, an output channel and an XPath expression that specifies where to slice the incoming document. The remainder of the code is little more than a loop over an XML node set.

Now is this the Splitter pattern? Not really. It is one possible implementation of a Splitter in a specific run-time environment. Consequently, if I constrain my world to match the specific constraints of the run-time environment assumptions baked into this specific implementation of a Splitter, I can then view the executable component as my "Splitter". This is what happened in case of the messaging toolkit.

Mixed Success

Not all routing patterns were that easily converted into components. For example, an Aggregator has very well defined variability points (Correlation, Completeness Condition and Aggregation Algorithms), but not all of these are as easily expressed in a higher-level domain-specific language. Correlation can easily be expressed using XPath if all documents are XML, but the completeness condition and the aggregation algorithm pretty much have to be expressed in a more general purpose programming language. Because I am using a compiled language, I had to hard-code this part of the logic. I still isolated it from the core Aggregator logic via a generic interface (any comments abut the I... naming convention for interface belong into another forum...).

public interface IAggregate
    void AddMessage(XmlDocument message);
    bool IsComplete();
    XmlDocument GetResultMessage();

It is easy to see that the interface represents the remaining variability points of an Aggregator (Correlation is taken care of by an XPath command line parameter). However, the relationship between Aggregator and Completeness Condition is now made at compile-time as opposed to composition time.

Next Steps

I hope this example sheds some light on the sometimes controversial topic of codifying patterns. Given the right constraints and a limited scope (a the messaging toolkit is only for educational purposes and contains only a subset of patterns) turning patterns into composable components can be a great learning tool. However, using them as a generic programming construct is more dangerous. There is a reason these elements are patterns and not a coding library. Granted, as patterns mature, they ultimately tend to be incorporated into libraries. This usually happens to specific patterns in a narrow domain. For example, many UI frameworks are based on the Model-View Controller pattern. However, using patterns as a high-level generic programming construct a la MDA is in my opinion still fraught with many problems and unanswered questions. Alas, I have received a request from the Integration Consortium to map the patterns in out book to the MOF. My general attitude is that skepticism is healthy but you never know until you try. Now I just need to find someone who understands the meta-meta model to do the work. That sounds like a meta-meta-meta problem :-)


Follow:       Subscribe  SUBSCRIBE TO FEED


Gregor is an Enterprise Strategist with Amazon Web Services (AWS). He is a frequent speaker on asynchronous messaging, IT strategy, and cloud. He (co-)authored several books on architecture and architects.