WebServices dot org

BPM And SOA Are Natural Allies, Not Enemies

Does anyone care about BPM? If Google searches are a good measure, “ nobody cares about BPM ” (SOA in comparison shows a doubling ). But Google searches are not a good measure and enterprises do care. BPM is rising up the priority ladder and for a few good reasons. The increased focus on business agility, the control of outsourced business processes and of course tighter regulatory scrutiny (Basel II and Sarbanes-Oxley for example) all play a part. Organizations are closely considering the adoption of “process improvement” workstreams and process driven methodologies such as Six Sigma key drivers for BPM. Reclassification of “processes” as business assets, which of course inevitably need managed.

BPM is a management discipline. It has been around for quite some time with roots in business process reengineering which (per wikipedia) first introduced way back in the 1900s in this article The Principles of Scientific Management . The article illustrates how to implement the principles. Reading the article gives a real sense of history, and I assume I replace for “bricklayers” with “developers”. But what does BPM mean now to a typical enterprise and does SOA play a part? I’ll attempt to answer both questions since I believe they are interrelated.

BPM is business process management based on modeling and performance measurement from an end to end perspective. To understand how it is implemented we need to move to the next step down, that is design, where the actual implementation is constructed. Design is where the integration problem comes to the forefront. The integration problem is solved in the main by what I would call the “ad hoc” architecture. Let me explain. BPM is process centric, and processes are about the way in which organizations operate. They involve interactions of systems, information, policies and most importantly, people, coupled with a well defined set of outcomes. From general experience it is easy to imagine how processes get implemented in a typical organization. The “ad hoc” approach based on a collection of apps, user knowledge, human procedures, some middleware and maybe some workflow. It somehow hangs together and a few key people will know how (and probably charge). Everyone will know their own silo. The outcome is that dependencies between different components are not clear, and as the business strives to make changes, they are meeting with “organizational process impedance”.

Here is where SOA has a part to play. The “ad hoc” approach doesn’t cut the mustard these days. SOA can alleviate the problem. Note the word alleviate, I don’t say the answer. There won’t be a total solution to integration soon (we’ll be in employment for a while yet).

Currently BPM related technology on the market comes under the heading of “Business Process Management Suites” (BPMS), a vendor/Gartner driven concept. Described by Bruce Silver as “a unified suite of tools that link modeling, design/execution, and performance management”, it has features you would expect including graphical modeling tools, orchestration engines, rules engines, registries, analysis tools, monitoring and …integration. The integration was however always the weak point. The outcome has been BPMS vendors have found it hard to make market ground, but those times could be changing. SOA can strengthen this weak point by being the bottom up answer to integration.

In layman terms, SOA allows organisations to map a process that is essentially a set of “tasks” to actual “endpoints”, services that execute. Those tasks effectively take on the form of exposed business functions through clearly defined loosely coupled interoperable interfaces. Ah Ha! Web Services. Regardless of platform, API, data model, SOA via Web services enables that interoperability. The benefits of the SOA roll up to the BPMS, the reuse, the interoperability, the clean interface driven change management aspects, the SOA governance and policy enforcements etc. SOA can provide the basis to separate out the process from the implementation. It can help organizations make the leap between the business driven requirements and the technology driven resources (tools, methodologies, consultancies). Fantastic. Point and click modeling functionality for business analysts suddenly starts to become a reality. Processes can be created, modified, tested and executed at will. IT will forever be talked of in positive glowing terms (pinch of salt). The potential for Point and click is discussed further in this wonderfully titled blog entry What BPM can learn from a Spreadsheet . Lets not get to carried away however with point and click. I am still to be decided on this topic. Tom Baeyens does bring us down to earth however, when he reminds us , “In some limited number of cases, you can stay within the boundaries of what the tool offers with forms around the graphical diagram. But this can never be taken to the level of flexibility to implement a complete typical project.” I totally agree with Tom when he says, “So my main point is that I believe BPM Suites approach can be used IN COMBINATION with plain programming.”

Of course, there are a few caveats to the ideal of SOA and BPM. Rather enormous caveats in fact.

• Firstly, you need an SOA. An SOA that is “layered” with your BPMS. Layered in the sense that SOA should be designed to work in a BPMS environment. For example, you’ll need to expose course grained services that a BPMS (and therefore a business analyst) would want to invoke. But of course, this is more to do with good practice SOA and not some failing of the basic concept.

• Secondly, an SOA is not about people, and processes often are. A very important point that I’ll come back to this at the end.

• Thirdly, there is either a technology overlap or gap between BPMS and SOA. Either there is a gap where two product sets can’t work together somehow or an overlap where both perform similar functions producing a split brain type scenario. Imagine for example an organization that says it has an SOA via an ESB type solution (by the way I don’t mandate ESB as the way to go for SOA, a topic for another blog). The ESB performs some translations, routing, the BPMS also. Where is the process, who owns it? Etc.

Could it be that the issue is BPMS and SOA vendors have tackled a similar problem from different directions. Jesper Joergensen says “The BPM camp has traditionally claimed that SOA was not necessary to implement BPM. Just deploy a BPM suite and you can achieve your goals much faster with less complexity…The SOA camp has focused on how to fix the complexity of enterprise IT in general. This camp has often claimed that BPM is a feature of SOA that is part of the solution but not a separate thing.”

Bruce Silver is the industry expert and uniquely sees the problem from both a technology and business perspective. In his superb writings at BPMS Watch he catches onto the gulf between SOA and BPM, he says , “I’ve been struggling for a while trying to understand why attendees at the Brainstorm BPM conference and the SOA conference in the next room have so little to say to each other, i.e. what really is the relationship between BPM and ESB (and all the other SOA goodies). I just put out how I’ve made sense of it, and so far no one’s said I’m wrong. Sure, BPMS pure plays may say “loose coupling” as part of their SOA shtick, but please expand on what that means. As I explained it in the post, it means the logical-to-physical mapping happens in the ESB not in the process model. Conceptually that computes for me, although I noted the problem with it (logic is split into 2 places). If that’s not correct, please set me straight.”

Bruce is fundamentally questioning the role SOA (incarnated as an ESB in his eyes) plays have in a BPM environment. Bruce asks, “In fact, in some ways the functions of an ESB are at cross-purposes with those of the BPMS process engine. For example, ESBs provide data transformation and content-based routing of messages from one service to another. Wait a minute; isn’t the process engine supposed to do that?”

To be honest, I believe at this point mixing up SOA with ESB is in fact the issue. A pure play SOA does not mandate an ESB (and routing, transformations etc)..and in fact many companies are staying well clear of ESB for a host of reasons (future blog). Bruce disagrees and thinks fundamentally it is about how we define SOA. I make my stance here. It comes down to a disagreement over loose coupling. Loose coupling goes to the heart of good SOA design, the data centric design, granularity, semantics. It is not mediation performed by a “fabric”. Ultimately, if done correctly, I think BPMS solutions have a great future ahead of ESB. Some of the vendors like BEA I think understand this.

Bruce picks this up on this point also, “Today, if you want a true BPMS layered on true SOA, it’s easiest to get it pre-integrated from a single vendor. There are a few vendors providing it now, including IBM, Cordys, and webMethods, with BEA and Oracle, among others, waiting in the wings… While BPMS layered on SOA is today the exception or special case, that won’t always be so. As BPM merges into the mainstream IT infrastructure, this layering will become the norm, and BPMSs will likely be able to work with any of the popular ESBs of the day. But we’re not there yet.” Bruce does conclude generally however that BPM and SOA are natural allies, not enemies.

What about standards, BPEL? I hear you ask. Surely this is the linkage between SOA and BPMS. True, but only part of the linkage and your orchestration layer should be independent of the SOA (ah, loose coupling once again). Most BPMS products now support standards like BPEL, BPMN (business process management notation) and lots of others namely bpel4people, bpel4-j, bpel sub processes, XPDL. More and more these standards are becoming unified and transferable. If you want to read more, one very good article considers “The BPMN-XPDL-BPEL value chain” . However, no mention of BPEL can be made without giving both sides, as David Chappell points out all is not perfect, The Case Against BPEL: Why the Language is Less Important Than You Think . Most of David’s issues with BPEL, it reliance on Web services, its reliance on XML, I actually view as hindrances that are long term positives. However, the biggest issue comes from the fact BPEL is pure Web services. The input and output from a BPEL is a Web service. David writes, “Business processes commonly involve people. BPEL was designed for system-to-system interactions, not interactions that involve human beings. Accordingly, it doesn’t define mechanisms focused on interacting with people“. I don’t necessarily think this is an issue specific to BPEL. History teaches us a valuable lesson; a Wikipedia entry gets right to the point, “In order to reanalyze BPR, it is being replaced by Business Process Management (BPM). BPM is presently taking a similar road toward many failures by focusing too heavily on automation and failing to consider people in processes”.

One initiative by the way is BPEL4People. Bruce has concerns, “Will BPEL4People succeed? The authors' current position that a BPMS must support all five interaction patterns to be considered "compliant" is, in my view, overly ambitious and unlikely to be adopted beyond IBM and SAP themselves – if even they can achieve it.” However, is it really possible to include humans anyhow? As another commenter to this blog puts it , “Retro-fitting human elements into BPMS are just another IT legacy that persists when IT mindset prevails. As long as users are always outside the boundary of a system, that is what one would expect. Not everything can be or should be automated”.

As a footnote to this article, I wanted to quote a paragraph from Sandy Kemsley’s weblog commenting on a recent survey of BPM trends , “Not surprisingly to me, although it may come as a bit of a smack to BPMS vendors, is that over half consider their most important BPM software tool to be a modeling tool, either a purely graphical tool such as Visio, a repository-based tool such as ProVision or an organizational modeling environment. In fact, BPMS only scored 12% of the votes on that question.”

Room for growth then.