Enterprise Integration Patterns
Gregor's Ramblings

Sync or Swim

FEB 15, 2015

Gregor HohpeHi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I like to work on and write about asynchronous messaging systems, service-oriented architectures, and all sorts of enterprise computing and architecture topics. I am also an Enterprise Strategist at AWS.
ALL RAMBLINGS  Architecture (12)  Cloud (11)  Conversations (8)  Design (26)  Events (28)  Gregor (4)  Integration (20)  Messaging (12)  Modeling (5)  Patterns (8)  Visualization (3)  WebServices (5)  Writing (12) 

My blog posts related to IT strategy, enterprise architecture, digital transformation, and cloud have moved to a new home: ArchitectElevator.com.

Our visual pattern language has turned out to be a major feature of EIP: what started as an attempt to provide an Alexandrian "sketch" for each pattern ended up becoming a useful design and communication tool. The pattern icons have since been implemented in Visio and Omnigraffle stencils, and, thanks to the great folks at FuseSource (now Red Hat), compiled into a deck of cards (see right). Also, subsequent pattern authors have adopted the idea of defining an iconic language for their patterns, for example the Cloud Computing Patterns. Lastly, the patterns have also been incorporated into a variety of tools, for example the Fusesource IDE.

When talking about tooling, the question often comes up whether it's useful to extend or formalize the visual language in a way that facilitates code generation or immediate execution. Because we have seen the idea of visual programming languages go wrong too many times we tended to shy away from turning the patterns into a programming language, with the notable exception of representing the patterns as components. However, the problems with visual programming may be rooted in the attempt to make a generic visual language. With messaging being a fairly specific and well defined domain and relatively constrained domain, we may just get away with it.

Polling vs. Event-driven Consumers

EIP defines two patterns that describe how a component consumes and processes events. A Polling Consumer consumes messages at the preferred rate of the application by having the application explicitly request the next message. The Event-driven Consumer on the other hand processes messages as soon as they arrive on the channel. The pattern description also mentions the alternative names synchronous receiver for the Polling Consumer and asynchronous receiver for the Event-driven Consumer. For example, message processing in the Polling Consumer happens synchronously with the program execution. We did not detail the synchronous or asynchronous aspect for message producers as we assume they are always synchronous, meaning they fire messages off at the convenience of the executing program. After all, this is one of the benefits of asynchronous messaging: the producer can fire off messages at will and the consumer can process them when it feels like doing so. If you put the asynchronous channel aside for a moment (and accept somewhat tighter coupling), you could connect a synchronous sender directly, i.e. without a queuing channel, to an asynchronous receiver: a message created in the context of the producing code is processed by the consumer whenever it is received. If both producer and consumer are synchronous you do need a buffer or a queue in between as both sides want to produce and consume messages at their leisure.

Sink or Swim

My former co-worker Ivan Gevirtz described exactly these concepts in his (now somewhat dated, but still very relevant) blog post Sink or swim, which was inspired by his work in media processing pipelines: "A side which gets told when to act is considered to be asynchronous. It is waiting for the appropriate trigger or event to activate it. Otherwise, it is dormant. The side which tells another side to act will be called the synchronous side. The synchronous side is what determines the timing of the interaction."

Ivan also defines a visual notation that expresses whether a consumer is synchronous, i.e. polling, or asynchronous, i.e. even-driven: the synchronous consumer has a "nose" and the asynchronous consumer a matching indentation. This visual style makes it clear that an asynchronous consumer can only be directly connected to a synchronous producer and vice versa. At the very start, I thought assigning the "nose" to the synchronous consumer is somewhat arbitrary, but if we remind ourselves that a synchronous consumer is actively looking for messages from a channel (or another component), the "nose" is a fitting visual representation: the control flow of this consumer extends "into" the other component. As long as we assume messages flow left to right, the notation feels very natural.

Augmenting EIP Icons

One nice property of Ivan's notation is that it overlays nicely with the EIP icons. For example, the endpoint patterns are essentially orthogonal to the routing patterns or transformation patterns, so a single icon could express both aspects: the main function of the pattern and the synchronicity of message consumption and generation. We could even think to extend the notation to include properties such as whether the endpoint is transactional or not. One would have to be careful, though that the language remains intuitive and avoids getting messy. So let's stick with sync/async for now.

Considering that most routing and transformation patterns both consume and publish messages, we can use the four combinations as defined by Ivan. The diagrams on the right shows what the augmented pattern icons would look like (reading the message flow from left to right):

Typical messaging systems as discussed in EIP mostly consist of Pushers, Drivers, and Pools (channels) as the control flow follows the messages. Synchronous consumers cannot request generation of messages through a Message Channels as the channel decouples the sender from the receiver. However, as our patterns are used inside systems that allow control flow in both directions, all forms of interaction models are relevant.

Looking for Integration Architects

I am actively looking for an architect to shape our service landscape at Allianz in Munich. please have a look at the position profile on LinkedIn.


Follow:       Subscribe  SUBSCRIBE TO FEED


Gregor is an Enterprise Strategist with Amazon Web Services (AWS). He is a frequent speaker on asynchronous messaging, IT strategy, and cloud. He (co-)authored several books on architecture and architects.