Enterprise Integration Patterns Enterprise Integration Patterns
HOME    PATTERNS    RAMBLINGS    ARTICLES    TALKS    DOWNLOAD    BOOKS    CONTACT

My latest thoughts on integration architecture:
Serverless Loan Broker @ GCP
Serverless Loan Broker @ AWS, Part 5: Integration Patterns with CDK
Serverless Loan Broker @ AWS, Part 4: Automation
Software Architecture Alliance
Oct 11, 2022
Munich
Software Architecture Gathering
Nov 14, 2022
Online
A Decade of Enterprise Integration Patterns
(IEEE Software)
20 Years of Patterns' Impact
(IEEE Software)
Conversations Between Loosely Coupled Services
(Video on InfoQ)
Developing in a Service-oriented World
(Video on InfoQ)
SOA Patterns - New Insights or Recycled Knowledge?
(Whitepaper)
Let's Have a Conversation
(IEEE Internet Computing)
Programming Without a Call Stack - Event-driven Architectures
(ObjektSpektrum, 2006)
Your Coffee Shop Does Not Use Two-Phase Commit
(IEEE Software)
Developing in a Service-Oriented World
(ThoughtWorks Whitepaper)
An Asynchronous World
(Software Development)
Nearfield Communication (NFC) in Japan
(OOP 2012)
Embedded OSS
(OOP 2012)
Distributed Programming the Google Way
(Devoxx 2009)
Developing in a Service-Oriented World
(SOACon 2007)
Programming without a Call Stack: Event-driven Architectures
(SOACon 2007)
Software Visualization and Model Extraction
(TheServerSide)
Where did all my beatiful code go?
(SpringOne, 2006)
Conversations Between Loosely Coupled Systems
(SD West, 2006)
Enterprise Integration Patterns
(JAOO, 2003)
Microsoft MVP
 
Hillside
 
IEEE Software Board Member

Patterns and Best Practices for Asynchronous Messaging

Buy on

Today's applications rarely live in isolation. Users expect instant access across all functions, which may reside in disparate applications or third-party services. However, building distributed and integrated applications requires developers to tackle asynchrony, partial failures, incompatible data models, API drift, and much more.

This pattern language consisting of 65 integration patterns helps developers design and build distributed applications or integrate existing ones. The patterns provide a technology-independent vocabulary and visual notation harvested from proven solutions to recurring problems. They also share common "gotchas" and design considerations. Besides receiving numerous accolades, the patterns spurred the development of a whole generation of open source Enterprise Service Bus (ESB) implementations, such as Apache Camel, Mule, WSO2, Oracle Service Bus, Open ESB, SonicESB, or Fuse ServiceMix.

Buy the book Enterprise Integration Patterns or read a sample chapter first. Find the most recent content in my blog or articles.

Messaging Patterns in Today's World: Microservices and Serverless

When Bobby Woolf and I started to document the patterns 20 years ago, the key technologies for distributed applications were JMS, SOAP, MSMQ, and .NET WCF. Large-scale application integration was done with integration tools and platforms, such as IBM WebSphere MQ, TIBCO, WebMethods (now Software AG), or Microsoft BizTalk. Although technology has evolved, many of these products still form the backbone of modern enterprises.

Cloud platforms and deployment automation have laid the foundation for a new generation of distributed systems: microservices and serverless architectures. Those applications rely on a smooth interconnect between components, giving rise to Service Meshes, Serverless Orchestrators, and Event Buses. Amazingly, we find the same integration patterns in those systems! That's why this site contains many modern examples for integration patterns:

New Book: The Software Architect Elevator

Modern architects must do more than draw UML diagrams and recite architectural styles. Rather than focus on technical decisions alone, architects and senior technologists need to combine organizational and technical knowledge to help transform their company’s structure and processes. That's why successful architects are those who connect the IT engine room, where the technical reality is defined, to the organization's penthouse, where the business strategy is set.

This books equips architects and IT leaders with the technical, communication, and organizational skills to be successfully in modern enterprises. Available now on Amazon and anywhere where books are sold.

New Book: Cloud Strategy

Most books on cloud computing either stay at a very high level, offer simplistic recipes, or dive deep into vendor-specific product details. This book fills the giant space in between, helping you aligning technology change with organizational transformation, make architectural decisions, and communicate trade-offs to diverse stakeholders.

Harvested from over half a decade of defining and implementing cloud strategies, this books equips architects and IT leaders with the technical, communication, and organizational skills to be successfully in modern enterprises. Available now on Amazon or as ebook from Leanpub.

Work-in-progress: Conversation Patterns

Asynchronous messaging is the foundation for most integration solution because its architectural style acknowledges the challenges of distributed communication, such as latency or partial failure. However, many interactions between systems extend beyond sending a single, stateless message: a request may expect a response; a handshake or authentication are needed first; a reservation is confirmed or expires. Such conversations, stateful exchanges between participants, present new design challenges and patterns. I therefore started documenting Conversation Patterns, which are the starting point for Enterprise Integration Patterns 2.

What Makes Integration Difficult?

Architecting integration solutions is a complex task. There are many conflicting drivers and even more possible 'right' solutions. Whether the architecture was in fact a good choice usually is not known until many months or even years later, when inevitable changes and additions put the original architecture to test. Unfortunately, there is no "cookbook" for enterprise integration solutions. Most integration vendors provide methodologies and best practices, but these instructions tend to be very much geared towards the vendor-provided tool set and often lack treatment of the bigger picture, including underlying guidelines, principles and best practices.

Asynchronous Messaging Architectures

Asynchronous messaging architectures have proven to be the best strategy for enterprise integration because they allow for a loosely coupled solution that overcomes the limitations of remote communication, such as latency and unreliability. That's why most EAI suites and ESB's are based on asynchronous messaging. Unfortunately, asynchronous messaging is not without pitfalls. Many of the assumptions that hold true when developing single, synchronous applications are no longer valid. Vendor-independent design guidance helps developers avoid these pitfalls so they can build robust integration architectures based on asynchronous messaging.

How can Patterns Help?

Patterns are a proven way to capture experts' knowledge where no simple “one size fits all” answers exist, for example in application architecture, object-oriented design, or message-oriented integration . Each pattern tackles a specific problem by discussing design considerations and presenting an elegant solution that balances often conflicting forces. The solution is not the first approach that comes to mind, but one that has evolved through actual use over time, capturing the experience that senior developers and architects have gained by repeatedly building solutions and learning from their mistakes.

What am I Reading Right Now?

Modern Software Engineering: Doing What Works to Build Better Software Faster, Dave Farley, 2021

The "father of continuous delivery" distills the concepts behind modern software delivery, such as modularity, feedback, coupling. A must-read for every modern software engineer.

Practical Process Automation: Orchestration and Integration in Microservices and Cloud Native Architectures, Ruecker, 2021

Process automation is no longer the domain of large-scale business processes but an integral part of modern application architecture. This books shows you where the two meet.

System Architecture: Strategy and Product Development for Complex Systems, Crawley, Cameron, Selva, Addison-Wesley 2015

A tour de force on designing complex systems with a special emphasis on decomposition and logical relationships, augmented by a list of System Architecture Principles. Most examples drawing on non-IT domains helps convey the concepts without getting lost in technology debates and makes it worth the price.

Table of Contents
Preface
Introduction
Solving Integration Problems using Patterns
Integration Styles
File Transfer
Shared Database
Remote Procedure Invocation
Messaging
Messaging Systems
Message Channel
Message
Pipes and Filters
Message Router
Message Translator
Message Endpoint
Messaging Channels
Point-to-Point Channel
Publish-Subscribe Channel
Datatype Channel
Invalid Message Channel
Dead Letter Channel
Guaranteed Delivery
Channel Adapter
Messaging Bridge
Message Bus
Message Construction
Command Message
Document Message
Event Message
Request-Reply
Return Address
Correlation Identifier
Message Sequence
Message Expiration
Format Indicator
Interlude: Simple Messaging
JMS Request/Reply Example
.NET Request/Reply Example
JMS Publish/Subscribe Example
Message Routing
Content-Based Router
Message Filter
Dynamic Router
Recipient List
Splitter
Aggregator
Resequencer
Composed Msg. Processor
Scatter-Gather
Routing Slip
Process Manager
Message Broker
Message Transformation
Envelope Wrapper
Content Enricher
Content Filter
Claim Check
Normalizer
Canonical Data Model
Interlude: Composed Messaging
Synchronous (Web Services)
Asynchronous (MSMQ)
Asynchronous (TIBCO)
Messaging Endpoints
Messaging Gateway
Messaging Mapper
Transactional Client
Polling Consumer
Event-Driven Consumer
Competing Consumers
Message Dispatcher
Selective Consumer
Durable Subscriber
Idempotent Receiver
Service Activator
System Management
Control Bus
Detour
Wire Tap
Message History
Message Store
Smart Proxy
Test Message
Channel Purger
Interlude: Systems Management Example
Instrumenting Loan Broker
Integration Patterns in Practice
Case Study: Bond Trading System
Concluding Remarks
Emerging Standards
Appendices
Bibliography
Revision History