January 1, 2008
Hi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I like to work on and write about asynchronous messaging systems, service-oriented architectures, and all sorts of
enterprise computing and architecture topics. I am also an Enterprise Strategist at
AWS.
My blog posts related to IT strategy, enterprise architecture, digital transformation, and cloud have moved to a new home: ArchitectElevator.com.
The end of the year is always the time to reflect on the past happenings. My friends in Japan often send a New Year's card with 12 pictures, each showing the significant event during the month. I am not sure something significant related to Enterprise Integration Patterns happened each month, but I think it's still nice to reflect a little bit on the history and current state of the patterns. After all, the patterns have matured from a draft paper to being adopted as the lingua franca for a number of open source messaging projects.
I have been working in integration since the ancient days, when "EAI" was still a cool term. Even my very first client project was an integration project! I had also used Design Patterns extensively when building large applications (I remember it took me a few attempts to understand the Visitor pattern).
Working with EAI tools of the day, it did not take long for me to figure out that integration is difficult. Besides building solutions with the tools, I started to focus on best practices to help clients better understand the development process and architectural trade-offs. When I started working with multiple integration technologies (my company figured that someone who knows Vitria should also understand TIBCO), it occurred to me that deep beneath the surface (and a pile marketing material) these tools do share similarities. There are message queues, transformations, process editors and engines, message metadata, routing, and so on. But it had not dawned on me to document some of these insights as patterns.
The meeting between integration and patterns happened when Martin Fowler was putting the finishing touches on PofEAA. Kyle Brown from IBM pointed out that enterprise applications often rely on messaging (and IBM happens to sell a J2EE container including JMS), and suggested Martin should include messaging patterns in his book. Martin, knowing a thing or two about publishing books, quickly sensed the dangers of scope creep and decided to stick to patterns inside the application (with a few cliff hangers like Gateway.
With everyone agreeing on messaging being a relevant, but under served topic, in early 2002 Kyle started a series of meetings in RTP to talk about messaging patterns. I remember that on one of our first meetings, North Carolina was blessed with very rare snow fall, pretty much crippling all forms of transportation. RTP is also the place of the "shotgun wedding" (Martin's choice of words) between Bobby and myself to take on the task of documenting patterns related to asynchronous messaging and enterprise integration.
The first milestone was the Pattern Languages of Programming conference in summer of 2002. Bobby and Kyle submitted one large paper titled Patterns of System Integration with Enterprise Messaging, while I submitted a paper on Enterprise Integration Patterns (it looks like the title ultimately stuck). Together this already accounted for some 100 pages of material and 27 + 17 = 44 patterns (with a small amount of overlap). Both papers are still available in the PLoP 2002 Proceedings. We were lucky enough to have a great set of workshop participants, including Martin Fowler, John Crupi, Brian Marick, Ali Arsanjani, and many more.
After we looked at the volume of work contained in the submitted papers, we decided (with a lot of nudging from Martin) that the patterns deserve a more permanent home on the Internet. Hence, the domain with the long name www.enterpriseintegrationpatterns.com was born. Because I am a poor typist (and the name is too long), I also registered www.eaipatterns.com, and Kyle got www.messagingpatterns.com (we actually "lost" that domain to squatters for some time, but it is back in good hands now). All domains point to the same content. We now receive a steady stream of about a thousand visitors a day.
The next few months were characterized by a lot of feedback, revisions, and editing. Our revision history captures some of this activity. A lot of discussion also occurred on our Yahoo! mailing list. The list has been overrun by spammers, but we'll try to clean it up so we can preserve this historical record. After the "open" review, we submitted the manuscript to select reviewers, which included Luke Hohmann, David Rice, Richard Helm, and the Silicon Valley Patterns Group. Based on the detailed feedback (and to our shock), the manuscript grew from some 400 pages to 700.
Incorporating the technical feedback was followed by tedious months of proof reading, image editing, and copy editing. That's when Martin Fowler's voice appeared to us like Obiwan reminding us that every time you think you are done with the book, there's more work left than you did up to that point. Even the force can't help there.
Our lives returned somewhat to normal in summer of 2003 when the final manuscript when to the printer. Finally, at OOPSLA 2003 in Anaheim, Enterprise Integration Patterns hit the shelves. At the show, we were outsold by Eric Evans' Domain Driven Design, but we were still mighty proud to see people actually buy the book. We even got to autograph a good handful, much to Martin's astonishment.
Of course, the book has come a long way since then. We have well over 35,000 copies in print by now, not counting the international versions. We still look at our Amazon Sales Rank every once in a while. We are also most pleased to report that 25 out of 29 reviews give us 5 stars. Since feedback is important to us, I even created an RSS feed for our book reviews using Yahoo! Pipes. Interestingly, EIP also put on some weight over the years. No, we did not add anything (I am busy posting ramblings and Bobby runs a blog at Developerworks), but it's simply the magic of paper weight.
Naturally, as soon as a book goes to the printer, the authors can't think of anything else but all the things they would have liked to include in the book. There is always more to say than fits in a single book. Thus, in October of 2003, these Ramblings started. Instead of having a traditional blog, my intention was to have start a collection of small "articles", which cover relevant topics in the integration space. I was envisioning something like my own monthly column. It turns out that I have not even managed to post once a month on average, but I think I managed to cover some interesting topics. The ramblings is also where we first learned about the relationship between making coffee and transactions, the rambling that still gets the most hits and was featured in Joel Spolsky's Best Software Writing.
Unlike most blogs, mine does not allow readers to post of comments. It's not really by intention, it's mainly because my blog system is hand-rolled in XSLT (argh!) and it was too much work to implement a comment-feature with spam detection and such. A full-text RSS feed is also still missing. At least I made my feeds recognizable by FireFox.
Speaking of spam, it's the thing that killed the Enterprise Integration Pattern Wiki. As we often point out, patterns are not invented, but harvested from practice. Therefore, it seemed great to have a Wiki with a page for each pattern that allows readers to add their own experience using the pattern, or even add new patterns. This was quite successful for a while – it was great to see people's input on our patterns. For example, some readers added implementation tips or words of caution when applying certain patterns. Sadly, spam wiped out all topics on the Wiki because I used a standard wiki with no spam protection. I think I still have a backup. I hope to be able to find time to reinstate at least some of the content.
One of the greatest things about writing the book is the group of truly great people I met through the work. All of these people are extremely bright, great communicators, and simply fun to hang out with. The list of people I met is simply too long to be fair to everyone, but I want to rattle a few folks off. Even in the early stages I got to meet the folks from the Silicon Valley Patterns Group. You'll see their names in the "Acknowledgment" section of many, many good books. Then of course, I got in close touch with many other popular pattern authors, including Ralph Johnson, Martin Fowler, the John Crupi, Danny Malks, Deepak Alur triumvirate, the Siemens Gang (Frank Buschmann, Michael Kircher, and many more), and words-do-not-suffice folks like Kevlin Henney. The Hillside folks are a group of really neat people, including Linda Rising, Joe Yoder, and Brian Foote (the last two of Big Ball of Mud fame). Of course, patterns are also a popular teaching tool, so I like to connect with folks like Uwe Zdun and Schahram Dustdar from the TU Vienna, Wil van der Aalst in Eindhoven, folks from the Enterprise Architecture Center at the TU Berlin, Gustavo Alonso at the ETH Zurich, and of course Frank Leymann in Stuttgart. Frank is a convert from another great group of people, the often underappreciated folks who work on standards. Even I have dished out the occasional joke at the standards bodies, but they include a long list of very bright folks who truly try to make solution development less painful. I certainly can't list everyone, but Frank Leymann (WSFL -> BPEL), Satish Thatte (XLANG -> BPEL), Sanjiva who-can-tell-me-how-many-vowels-his-last-name-has-wins-a-price (WSDL and many more), John Evdemon (BPEL), Jean Jacques Dubrai (ebXML, WS-CDL) spring immediately to mind. Open source is a major driver of innovation these days, so I am very happy to have met many of the great minds behind the most significant software movement of the last decade. Again, many omissions, but of course the LogicBlaze / Iona folks (James, Bruce), the Mule team (Ross, Dan), and the Interface 21 gang (Rod, Rob, Alef, Arjen, Adrian, and many more) have to be near the top of the list (it's particularly fun, because on one hand they are fierce competitors, but they are also all great guys). Then there are the illustrious personalities like Hani and Crazybob, plus the Fowlbots, including Paul Hammant, Alsak, Erik, Jon Tirsen, and many more. Oh, I am tired of typing (I can’t touch type), so please fill in all the missing folks. I am also too lazy to create links to everyone, but I can recommend a decent search engine to do the job for you ;-)
New Year is certainly not the time to brag, but I think mentioning proud moments is OK. I mean the graduation day type of proud moment. Without a doubt, this must include OOPSLA 2005 when Grady Booch included our book in the list of most significant software patterns books.
I often find myself explaining that patterns are more than a solution (the canonical example are GOF's State and Strategy patterns, which have exactly the same solution structure, but very different intents). Still, it is quite normal for frequently used patterns to make their way into common libraries and platforms. Naturally, a number of our patterns were already implemented in messaging middleware before we even documented them. Still, I am very happy that a number of (commercial) open source projects have adopted our patterns. The most relevant ones (in no particular order are):
If you know of other open source frameworks that use our patterns, let me know.
Enterprise Integration Patterns is also no longer restricted to English. We already have a Russian and Chinese copy out, with a Korean one in the works, and a Japanese version under negotiation. If all goes according to plan, EIP will soon be available in five languages for you to enjoy.
People sometimes ask whether we'll do a second edition since many of the code examples are now also four years old. My prediction is that the chances for a second edition are slim. Almost all code examples use languages that are still in very popular use (mainly Java and C#) and should compile with the latest versions of the respective languages and frameworks. Yes, we are missing out on some neat language features (annotations / attributes, generics, etc), but for our small code examples those would not really add a whole lot to the pattern. We would be happy to make all code examples available in open source, which also allows readers to help maintain or update them. If I ever have a spare minute, I'll set that up.
Of course, there are many more integration patterns to be harvested and documented. Sadly, my progress on this front is extremely slow (it turns out you can't use 20% time to work on personal book projects). Last year, I held a workshop in Dagstuhl on conversation patterns, and just last month I workshopped a like titled paper at OOPSLA, which documents over a dozen new patterns. With a little luck, we'll be seeing some of the new patterns on this site soon.
Gregor is an Enterprise Strategist with Amazon Web Services (AWS). He is a frequent
speaker on asynchronous messaging, IT strategy, and cloud. He (co-)authored several
books on architecture and architects.