On BPMN
Thursday, July 20th 2006 | Assaf Arkin
Over the past couple of years, the Business Process Modeling Notation (BPMN) has established itself as the de-facto standard for process modeling. Nevertheless, an impedance mismatch exists between BPMN and BPEL, one of its target execution languages. What follows is the view of Assaf Arkin, Intalio CTO and co-author of the BPML and BPEL specifications, on this critical subject.
One of my favorite quotes from Sam Ruby, on how WS-* became what it is today:
- The first thing they did was to standardize on XML
- The second thing they did was to abstract away the XML
BPMN had a similar challenge. It’s easy to come up with a notation of shapes and arrows to represent any type of process, that’s an easy spec to write. You can also use Visio.
But what if you want to generate a process and use software to manage it? You need to map the visual diagram into a language that software understands. For example, BPEL. But as we discovered, there’s a gap between BPMN and BPEL.
There are several reasons for this gap:
- BPMN is graphs, BPEL is structure.
- BPMN is abstracting the wrong thing.
- BPMN is abstracting the flow.
The first point is easy to understand, and also easy to solve. Think SVG. It’s vector, but you render it as raster on the screen. The difference is there for a reason. It’s easier to create graphics if you can edit them in different scales. It’s easier to render them on the screen if you output pixels. But you can easily move from one to the other.
The second point is a result of the process that created BPMN. It’s not a feature, it’s a bug. BPMN is said to be a generic process modeling language, but that’s an illusion. Generic modeling languages are descriptive but not prescriptive, they need shapes, not constraints. BPMN is first and foremost a visual notation for some machine processable language. It’s a hypothetical execution language that’s attempting to abstract BPEL and closely match it. Just not close enough.
There’s no reason why you can’t have an underlying model that maps well to BPEL. But practically, too many things are lost in translation:
1. BPMN is loosely based on BPEL 1.1. BPEL 1.1 has a lot of incompatible interpretations, in fact, no two compatible interpretations exist. BPMN is one of those interpretations, which makes it non-portable across tools.
2. And it interprets BPEL 1.1 different than we do.
3. And interprets BPEL 2.0 different from the WS-BPEL TC. Over several years a lot of features were added, a lot of understandings made, but BPMN was never updated to match.
4. And while trying to abstract BPEL 1.1, it also slipped in some wishful thinking interpretations. They may be good ideas, but they never made it past the WS-BPEL TC.
All of these are a result of the BPMN process, specifically the decree to abstract the executable language and focus on one spec as if the other never existed. Followed by failure to catch up with any progress made since BPMN 1.0 was finalized a long time ago. None of these are a feature customers actually asked for, or benefit from.
But most important, BPMN has no reference implementation. A reference implementation is how you catch bugs in the spec. In fact you need at least two of them if you hope to interoperate. A spec that doesn’t have at least two reference implementations is going to have bugs and incompatible implementations. The only exception to the rule is… I can’t think of a single one.
WS-I is a good example. It came to fix all the problems discovered while implementing WSDL 1.1. WSDL 1.1 was a great start, but without any implementations (it was “spec then code”), it had more holes than swiss cheese.
The third point is best illustrated by looking at E-R diagrams. E-R diagrams are a great way to model database schemas, but they’re not interoperable. The reason is not lack of standards, but completeness.
A real database schema requires information that’s not in the diagram itself. For Oracle that would be the ever important tablespace, for MySQL the engine type. If you designed for InnoDB (transactional) but got a schema for MyISAM (not), your application will not behave the way you expect it. And while both Oracle and MySQL do stored procedures, they don’t use the same syntax of features.
Which doesn’t matter since E-R diagrams don’t model stored procedures.
If you use one tool to create a database schema for Oracle, and then import the E-R diagram into another tool, you get: a diagram. All the important information is lost, and the second diagram will not produce the same SQL statements.
If you use one tool to create a database schema, and then export it as SQL statements, you can take the result into any other E-R modeling tool, and into administration and deployment and O/R tools. And all of them will have the same understanding of the database schema, because all the information is contained in the SQL. Agreeing on notation is nice, but insufficient.
The same problem happened to BPMN. If you use one BPMN-based tool to model your process, you can feed it to an process manager. But if you export the BPMN diagram and import it in another modeling tool, you get boxes and arrows. Nothing you can execute. There are no services to connect to, no variables to hold data, no expressions for conditional flows. There’s even no way to tell the difference between while and until. The visual diagram itself adds no value to what you can already do with SVG or Visio.
So how do we fix all of that?
To solve the import/export problem you need a spec that captures all the relevant information, adding variables, partner links, scopes, expressions, etc. Or you can add position and color information to BPEL. Guess which one is easier?
Which assumes you’re doing BPEL. We can argue which language it should be, but by trying to please everyone you get to serve no one. So BPMN needs to slice the cheese and pick a machine processable model.
Once you do that, you can easily tell the difference between features and bugs, and fix the bugs. Drawing a sequence of activities as a graph of boxes is a feature you want to keep. Inconsistent support for scopes and compensations is a bug you want to fix.
There’s no shame in admitting to these bugs, they all exist for a good reason. When you solve a complex problem like business processes, you end up having to choose between a lot of different options. BPMN made some decisions. BPEL made other decisions, and through the life of the WS-BPEL TC, changed a lot of these decisions, sometimes back and forth. But eventually BPEL reached some decisions that are based on more feedback and most important, on working implementations.
So there’s a gap between BPMN/BPEL. Vendors address it in one of two ways:
- Diagram BPEL processes and call it BPMN.
- Use one tool to diagram BPMN, another to diagram the BPEL process.
There’s a third way. To build an implementation that captures all the semantics of BPEL, draws on the experience of the WS-BPEL TC, but remains loyal to the spirit of BPMN. An implementation that imagines what BPMN 2.0 would look like if it abstracted the right thing, and fixed all the bugs.
There’s also a fourth way, but asking customers to wait the years it takes a committee to propose a spec is doing a dis-service, and by then we’ll be having the same discussion over the BPMN/BPEL 3.0 gap.
BPMN is also intended to do high level modeling that is not married to any execution language. That’s an illusion. BPMN is far too constrained for that, and all of these constraints are derived from the fact that some piece of software will manage the process.
There are good reasons to relax those restrictions and keep the visual representation of boxes and arrows. To model the descriptive instead of the prescriptive. But that’s not BPMN itself, but a smaller, lighter spec. And we already have interoperable ways to exchange such diagrams: SVG, EMF, XMI to name but a few.
Given that BPMN is now at the OMG, which is also doing BPDM, here’s a few predictions about the future:
1. BPMN will keep abstracting BPEL flows. That will keep the OMG happy, on reliance on XML Schema, XPath or other NIH specs. But it will also keep BPMN from ever being interoperable.
2. Matching up to BPEL 2.0 will take at least two years, by which we’ll need to start over with BPEL 3.0.
3. BPDM will attempt to abstract BPEL 2.0 as well, but in doing all the layers of abstraction (down to modeling a generic container for generic objects, and then typing them all) will take even longer to complete.
Lacking a reference implementation, neither will actually work.
Or someone will develop a code base and contribute it to Eclipse, and in those two years it will become the reference implementation. BPMN 2.0 and BPDM will then argue whether to use that code base, argue against decisions made in the code base, and eventually rewrite the entire specification.
What I’d like to see? I’d like to see us slicing the cheese on this one.
Make the decision to marry BPMN and BPEL to get the semantic gap solved for at least one language. One is better than none. Fix all the bugs in the spec. Concrete features are more interesting than abstract possibilities. And donate the code base to Eclipse. Open source has a great way of solving interoperability problems, and allowing more vendors to participate.
What do you think? What would you like to see?
Entry filed under: BPM 2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Given you provide many good arguments as of why BPMN doesn’t work with BPEL 2.0, why would Intalio pursue an architecture (BPMN+BPEL from the website) that is essentially broken in the short to medium term?
[…] Assaf Arkin guest-posts an impassioned love-hate note to BPMN on IT|Redux. I admit I only understood about half of it, and I think you’d need to have stayed awake through many a BPEL TC conference call to get most of the references. His first core assumption - that BPMN’s deeper purpose is to provide process design portability, not just a drawing - is one I agree with (e.g. here and here and also here)… but it’s hard for me to accept his second one, which is that a BPMN independent of BPEL makes portability impossible, and we should just accept that BPMN’s purpose is to be the notation for BPEL. […]
I think you’re right to say that BPMN is too constrained in relation to higher-level modeling. It does a very credible job as it is, though naturally it should track BPEL to ensure that it makes best use of that language.
My wish would be to see a new graphical language that could be used at that higher level, that would represent business processes whether they be automated or manual, and would incorporate business objects and indicate what systems are needed. This would allow a complete description of a process, and the interfaces and handoffs between people and systems.
I guess there are tools that do this, some better than others. But it would be nice to see a standard emerge. Then the icons that represent the automated processes can just be transformed to BPMN so that the more constraining features can be added.
Bob,
One possibility is to create a high-level notation out of BPMN that relaxes the restrictions, adds new high level constructs, but remains similar in spirit to BPMN. Then you have a standard way to represent ideas and exchange them with people, at different levels of detail.
John,
I really like the BPMN visual notation, of all the alternatives I’ve seen, it’s the one I will pick. That’s the starting point, but it needs to be modified to fix the bugs. And that’s exactly what we’re doing. We’re using the notation as much as we can, with changes where they are necessary, and underneath relying on the more proven process model.
Your argument rests on the premise that BPMN has been created for the purpose of translating into an executable language and that the language should be BPEL. However, while the first is often true, and BPM platforms have proprietary ways of translating that diagram into an executable process application, the latter assumption is questionable.
BPEL does not address manual steps, non-WS integration, and a variety of use cases that are fundamental to many real world processes. So BPMN and BPEL cannot be bridged in the near term, because BPEL only addresses a subset of process problems.
I think what companies want is a common language for describing and collaborating on all processes (BPMN) and a platform for managing and orchestrating those processes (BPM suites). Ideally that executable would be standards-based and portable, but just having an effective BPM environment that allows you to build processes quickly and enable a whole new level of visibility across the enterprise is much better than what they have now. BPDM will address the portability issues when it is locked down.
Trevor,
You are right, we need to add more layers on top of BPEL. For example, to better handle human workflow. Start with implementations, derive a common specification, and you get something that’s proven to work and is portable. Why? Because you have working code to prove both points.
So imagine a BPMN notation for BPEL + BPEL4People extensions. And a server that executes both. Wouldn’t that cover the range of use cases?
As for non-WS, we use BPEL to invoke Java code, talk to SAP R/3 over its proprietary RFC, queue messages using JMS, and much more. WSDL has a nice way of abstracting all the different protocols, and we’ve yet to run into a message exchange that can’t be described using a one-way or request-response operation.
For BPDM to address an executable that is standard and portable, it needs to at the least re-invent BPEL. Our customers don’t have time for that. They want something that works today.
It is interesting to see that IT practitioners keep on re-inventing the wheel. One thing that we do know through experience is that there is no such thing as a universal modeling language. UML is a case in point. It was created to model objects and their behaviours, it then proliferated to cover everything under the sun! States, activities, and the likes were after thoughts that were incorporated into the UML grammar and syntax over time.
I believe that BPMN and BPEL are undergoing the same process as UML. It should be noted that we need a minimum of two models to represent the real world phenomena; one to represent “structure” and one to represent “transformation” or “interaction” of entities (or objects). The purpose of E/R has always been to represent the structure of a database, not for manipulating or enforcing the integrity of a database. The “Class” diagrams in UML are now used to replace E/R and the “method” compartment is used for specifying ‘’store procedures” and ‘triggers”.
In my article on “Business User Perspective on UML, BPMN, and BPMS”, I urged for the unification of BPMN and UML. I believe that Ismael has come to similar conclusions in one of his recent articles. Let us not re-invent the wheel, and let us concentrate on simplifying rather than enriching modeling symbols unnecessarily!
Assaf,
A lot of interesting points…
I just have a couple of comments at this point.
I would not agree that BPMN is too constrained for higher-level process modeling. I don’t think there is a lack of capability in BPMN in as much as there is a lack of appropriate “how to” books or articles. Not all the elements and properties that are necessary for BPEL are required for generic modeling. By looking at the BPMN 1.0 spec, a reader would naturally see all the complexity, but not necessarily the simplicity (we tried to incorporate).
Higher-level (e.g., executive level) process concepts are often shown in diagrams that include processes, but are not strictly process flows, which is the focus of BPMN. These other views could be standardized and should trace down to BPMN, but they have a different purpose and shouldn’t be confused with current BPMN diagrams. However, libraries of BPMN artifacts could be defined and standardized to represent higher-level concepts within a process. These artifacts could be included in a process diagram, but they would not affect the semantics of a process flow.
The mapping to BPEL is another topic under much discussion at this point…
Assaf has several fallacies in his thesis. The goal of BPMN is not to enable business people to become better programmers, i.e., to make it easier to write a computer language. It is to give business people a well-defined language to express how they want the business to operate. The transformation to BPEL, if necessary, is to provide a language to describe some of the BPMN processes to a computer. BPEL wasn’t designed for people, it was designed for computers. BPMN is not an abstraction of BPEL, but a representation of real world business process concepts.
Computer people have always wished the world were different and have tried to force business solutions to fit nicely into computer programs. Efforts to provide languages to better describe real world problems started with COBOL and Fortran and have been going on ever since.
Viewing BPMN as just a bunch of boxes and lines completely overlooks the value. The standard defines semantics that allow people to communicate with other people. If a tool implements the semantics correctly, then, at a minimum the diagrams will be syntactically correct and provide a process definition at an appropraite level of abstraction for the purposes of many business people. Just because you can’t design an engine, doesn’t mean you can’t drive a car.
Yes there is more to defining robust business processes. There is also more than is addressed by BPEL (like human participation and choreography). BPDM will add the modeling elements and constructs to BPMN that will enable the abstract descriptions of some business people to be more fully developed by others. This does not mean that BPDM will include all of the design considerations of an automated business process.
When COBOL and Fortran emerged, there were assembly language programmers who complained about how they could no longer do everything they could do in assembler. It turned out that for the development of most (all?) applications, the power of assembly language was not necessary, but the productivity of higher-level languages was essential. The same is happening with business process modeling — it’s most important to make it easy to express the business ideas — the computer can do the work of making it efficient to execute.
BPEL is an assembly language for a virtual machine; BPMN is a language for business people. If you want BPEL from BPMN (and BPDM), we can make the computer generate it and optimize it — we don’t need to try to make business people think like computers.
Fred,
I’m not sure I understand your point. I’d be happy for a COBOL-like solution, moving away from assembly language is a good thing.
If your analogy is that BPMN is like COBOL, then I’m missing the how. COBOL is an execution language, it’s a What You Write Is What Executes kind of language.
When you say “If a tool implements the semantics correctly”, the question is which semantics? The BPMN semantics? Then I don’t know of a single BPMN implementation in the market place.
The BPEL or XPDL semantics? As long as those semantics differ — the whole point of this post — what you model is not what you execute.
And that’s a problem. How do we get one set of semantics, so what you model is in fact what you execute?
[…] Assaf Arkin’s recent article on BPMN generated some of the most interesting discussions I have seen on this blog in a long time, so I asked him to write a follow-up to his controversial piece. He came up with a superb article only he could write. Enjoy! […]
Assaf,
I think you need to think outside the box — the box being the design of business process languages for programmers. We are talking about business process languages for business with the semantics of business. The evolution of computer-based languages has been driven by efforts to better represent the concepts of the problem domain, letting the computer do the work to translate this to executable form. You have particular execution semantics in mind, and, yes, BPMN does not fit those semantics.
BPMN is intended to represent the concepts of business processes in a way that is easily understood by people who understand the domain. The graphical notation provides a way for them to communicate with each other. The semantics of BPMN are the concepts they communicate, and the concepts that ultimately should be reflected in the execution of those BPMN processes that are to be automated, as well as those processes that are not automated. The semantics specification can be improved, and there is a task force making improvements to be available in the next few months.
Just as a COBOL compiler translates COBOL expressions to very different assembler or machine language expressions, so BPMN models must be translated to the language of a business process execution engine — BPEL being one of those, although in fact BPEL is a family of languges when we consider the proprietary extensions.
BPMN has limitations, primarily because it provides only a two-dimensional view of processes. In addition, we understand the needs of business process modeling better today than we did when most of BPMN (and BPEL) was developed. BPDM (Business Process Definition Metamodel) will provide a more robust definition of modeling concepts, and will be the basis for further improvements of BPMN.
For example, BPDM addresses both private processes in a web services or SOA context, and the collaborative process or choreography that defines how autonomous processes or collaboration participants interact. It does this in a way that allows a tool to ensure that a participant’s processes are aligned to the shared choreography. BPMN provides a weak representation of choreography. BPEL provides none. Choreography is important to business people because it defines how businesses will interact to achieve mutual benefits from automated business transactions.
So the semantics are there and will get more robust. They are not the semantics of how the computer executes the processes, but rather the business semantics of what the business processes are intended to do and that the execution engines should support.
Fred,
I fully agree, we need something that works at the level of the business process, which is why I’m such a big proponent of BPMN. I’d like to design business processes with BPMN, not with BPEL.
But I’m also a big proponent of the COBOL argument. COBOL is executable code. I can write a business logic that takes data, manipulates it, calls other services, takes decisions.
I can’t think of many business processes that can do without this basic requirements.
I probably need to read the BPMN specification again, because I’m missing how BPMN does that. How do you call a service? Store data? Calculate? Make decisions?
Where in the language can you put all that information necessary to make BPMN the equivalent of COBOL?
It’s incomplete. And the question is: do we reinvent the wheel to complete BPMN, or just reuse something that exists?
I happen to like reuse.
[…] Recent discussions on BPMN and BPEL have created a level of activity rarely seen on these pages. The quality of interactions is outstanding, and I strongly encourage you to read through the comments of this post, and this older one contributed by Francis Ip. In the meantime, let me use the opportunity of our weekly BPM 2.0 post to shed some light on why BPMN and BPEL nicely complement each other, and who really needs a complete BPMS. Disclaimer: this post might upset some readers, especially towards the end, but I need to get some messages across, loud and clear. Consider yourself warned! […]
Assaf,
BPMN is not intended to be a computational langauge. Activities within BPMN may call for the execution of computational expressions that might be written in COBOL or Java. Activities might also call for somebody to weld a joint, or paint a door, or wire a lighting fixture, but we aren’t going to provide the detailed instructions for that either.
BPMN is incomplete, but so are the other BPM languages. It is a viewpoint for definition of business processes — a two-dimensional representation. BPDM will provide additional capabilities that will involve extensions to BPMN and additional viewpoints (possibly another form of extension to BPMN, i.e., graphics for different kinds of diagrams). But these extensions wil focus on a more robust repesentation of the business, not the computations that may be necessary to perform specific activities.
That’s the old workflow view of processes: you sequence activities together, then write code to implement those activities, often writing code just to call some other piece of code.
Say you have a service that handles orders. You can’t just use it from the workflow, but you can write some code that gets data from the workflow engine, and passes it to the service. That’s a lot of coding just to get simple things done. I don’t know a single customer who’s asking for this feature (more on this here).
The market, as far as I can tell, is always looking for better ways to do more with less. That’s why SOA works so well with BPM. You either have or build new services so you can reuse them from different places. And you make reuse easy (i.e. no coding), so you can quickly implement processes, and easily change them.
It’s simpler, it’s cheaper, but most of all it’s agile, which means you can improve your business (not the code) faster and respond to customers better!
[…] After I posted re the piling-on Assaf Arkin was taking over his assertion on IT|Redux that BPMN belonged to BPEL, the man himself posted a thoughtful response on BPMS Watch. I was waist-deep in real work at the time and didn’t have time to think through an appropriate continuation of the thread, so I dashed off a quick response by email. That began an interesting out-of-band conversation that, if nothing else, narrowed the gap between the BPEL-si and BPEL-no positions, and shifted the debate from lofty analogies and name-calling to things reasonable people can at least discuss. […]
Buon luogo, congratulazioni, il mio amico!
Trackback this post | Subscribe to the comments via RSS Feed
Leave a Comment