The Next Step in Process Modeling
Friday, February 3rd 2006 | Bruce Silver
This is my latest BPMS Watch column on BPM Institute.
One of the fundamental promises of BPMS was supposed to be improved business-IT alignment through model-driven implementation. We’re headed in the right direction but the tools and standards don’t completely support it yet. In preparation for the upcoming 2006 BPM Think Tank, a gathering of BPM poo-bahs contemplating the next round of process standards, I have a modest proposal for review.
Here’s how BPMS was supposed to work: After your BPM team hashes out the basic workings of the as-is and to-be processes using yellow stickies on the conference room wall, they are supposed to record that hard-won understanding not in a simple Visio flowchart but in a process modeling tool. That tool should be based on a standard diagramming notation and semantics, and allow optimization of process performance metrics through simulation analysis. The notation should be independent of implementation architecture (e.g. workflow vs service orchestration), intuitive to a “business user,” and â”abstract” in the sense that executable design detail is optional. Yet even with all these restrictions, the model should be able to generate some kind of artifact — a skeleton design — that can be imported into the implementation design environment and which encapsulates the key features of the modeled process flow, resources, and performance metrics.
In that vision, the Business Process Modeling Notation (BPMN from OMG) is the process modeling notation standard and BPEL from OASIS is the executable process design standard, at least for those BPM suites based on service orchestration architecture. A number of BPMN tools now can also export to XPDL, the standard for executable designs based on workflow architecture. So far so good, in theory at least.
But the need to export a diagram drawn by “business users” to a valid (though not yet executable) BPEL program creates some obvious problems. The first one is that not every flow you can draw in BPMN can be exported to BPEL — at least unless you draw it the “right way.” I found that out the hard way after transcribing a client’s hand-drawn flowchart into a BPMN tool. Everything was great until I switched on BPEL validation, when whole blocks of process turned red with BPEL error messages. BPEL doesn’t like “interleaved” process segments, looping back to previous steps, and other common features of user-drawn flowcharts. Of course, you can get around all of these problems by redrawing the flow using structured loops, control variables, events and other things that programmers take for granted. But not only does this turn the business user into a programmer, it creates a diagram that may be no longer intuitive to anyone on the BPM team. In the end, the effort to get the BPMN to be “BPEL-valid” was too great and I just built the skeleton design directly in BPEL.
Now you might say that’s not a BPMN problem but a problem of my modeling tool. Also, since the BPEL language isn’t functionally complete — it omits really fundamental things like human tasks, subprocesses, and data transformation — each BPEL-based BPMS is going to add those features using its own design constructs. There’s no way the modeling vendor could get the export right for every BPEL tool. What should happen instead is the modeling tool should just save the BPMN and let the design environment generate the BPEL on import. That might also eliminate the need to torture the BPMN into unintuitive patterns just to get valid BPEL.
That would be logical, except that the creators of BPMN standardized just the diagram shapes and lines, not how the diagram could be saved as an XML document! So there’s no standard way to share models with implementation tools other than by exporting them to BPEL (or to XPDL). That hole was widely discussed at last year’s Think Tank, and OMG is now addressing it with its Business Process Definition Metamodel. Let’s hope this effort comes to fruition quickly.
But now we come to the second problem, the heart of my modest proposal. The issue is how to keep the model in sync with the evolving BPEL design. BPM is based on the notion of a cycle of continuous process improvement. Remember, modeling and analysis aren’t just the first step of the BPM project, but are repeated periodically to optimize performance and cope with shifting requirements. Actual performance metrics continually refine modeling assumptions and simulation parameters, and the flow itself can be tweaked easily, relying on the inherent agility of service orchestration. But all of that implies that the process as originally modeled — the BPMN diagram, annotated with performance metrics and simulation parameters — remains accurate after the BPEL designer has finished the implementation. The model and the executable design have to be mutually interchangeable. It can’t be a one-way trip.
So the next challenge is re-creating BPMN from BPEL. That doesn’t mean all the executable detail needs to be surfaced in the diagram — but it shouldn’t be lost, either. You should be able to go from BPEL to BPMN and back to BPEL without losing anything you started with! In a sense, BPMN becomes a business-oriented diagramming notation, supporting both simulation parameters and optional executable detail, which remains fully interchangeable with and functionally equivalent to executable BPEL. Changes to the process can be added in either environment.
I have a hunch that creating the BPMN diagram from BPEL should be easier than going the other way, and that implementing such a bidirectional interchange would probably provide valuable lessons for tool vendors on both the BPMN and BPEL sides.
Entry filed under: BPM 2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
I’m new to the BPM field. When I read you paper, I understand that there is no real exemple that works using BPMN and BPEL in a straight line. The complete translation from BPMN to BPEL and the reverse are not yet available. But what do you think about tools like JaWE and Shark, which use XPDL langage? It seems to solve translation’s problem. Doesn’t it?
Isn’t the interchangeability mentioned above between BPMN and BPEL the subject of BPDM. The Business Process Definition Metamodel is an upcoming OMG standard that exactly tries to cover this.
I don’t think I meant to say there is no real example of going from BPMN to BPEL. I’m sure it can be done — for example, Intalio claims to use BPMN as their BPEL design tool — but with most tools it’s not easy. Because it forces the business analyst to supply implementation detail, it is somewhat orthogonal to what the analyst is trying to achieve, which is process documentation and optimization through simulation analysis.
My real point is that if BPMN is ever to become a bridge between business and IT by providing a skeleton BPEL design, the model (BPMN) has to stay in sync with changes made in the BPEL, which implies an unambiguous back-translation (i.e. BPEL-to-BPMN). It would be an interesting exercise and perhaps a good masters thesis topic for a student. Once OMG gives BPMN a schema you might even reduce the translation (in either direction) to XSLT. I certainly don’t think the issue has anything to do with BPEL vs XPDL as an execution language.
I wonder how much of the two-way transfer difficulty between BPMN and BPEL is based on the fact that business processes as modeled by the business analyst cannot possibly encapsulate all functionality required to drive system development.
I base this not on experience with using such tools but on working with live business and IT clients. No one understands 100% of the picture of how a system should operate in terms of both automated and manual processes. One can argue that IT has the most comprehensive view — they have to — simply because they have to marshall all system resouces in support of business requirements. This may involve system functionality that is (appropriately) invisible to the business person.
If this is the case, translating a model back to a form that is readable by the business analyst might have to include boxes labeled “and then a a miracle occurs†simply because IT has had to add processess unknown to the business person.
With a BPMS, no miracle has to occur. This is called Design Design Architecture (DDA). In a DDA, based on processes, processes a new executable abstract entity. There is no translation between BPMN and BPEL, becasuse they are just views onto the same executable entity - the PROCESS. I use capitals to indicate that PROCESSES are something really new. They come with their own vm for example. What we need are standards to define what a BPMS is, not a new meta-model for processes to link two languages that should have been aligned from the outset, BPMN and BPEL. The BPMN working group, which began in BPMI.org, went off the rails somewhat when it lost the link to execution. That was part of the original charter for BPMN at BPMI.org - but it was ackward for some modeling vendors, so they broke the link. And with BPML losing to BPEL, with BPEL out in OASIS-land, it was inevitable the two languages don’t join up properly. But this does not matter. Because on a BPMS, a BPM 2.0 BPMS, BPMN and BPEL are just views onto the real thing, the PROCESS.
I agree with the academic world that the problem with BPMN-to-BPEL translations comes from fundamental differences. BPMN is graph-oriented, while BPEL is mainly block-structured. What is more, my experience shows for the time being that in order to manage such a transformation, I would have to model in BPMN (using a BPM tool) almost all data, interaction elements, exception handling, etc. in order to generate an XML file. I am asking myself: where is the added value?
Thomas,
The added value is in getting an executable process implemented for a fraction of the cost of alternative approaches, such as writing J2EE or .Net code manually. Our experience shows us that in average, behind each BPMN graphical element, about ten lines of complex BPEL code get generated, while writing equivalent logic in Java would require about 100 lines of Java code. If you give me a tool that replaces 100 lines of code with a single box or arrow, I think I’ll go for it.
I had similar experiences trying to generate BPEL from BPMN using a BPMN tool. There are basically two problems: first, much implemention details need to be added before hitting the “export/convert” button, thus raising the usual question of “do you expect to turn your business analyst into a programmer or vice-versa?”; second, sometimes the export is not possible or leads to invalid BPEL code, especially if you go too freely about connecting your nodes arbitrarily in the BPMN diagram.
The first question is one of methodology, and is more general to the whole MDA approach (not just BPMN-to-BPEL). High-level models need to be properly synchronized with lower-level views, and unless you have one-to-one mappings (which would be surprising), this is not easy to support. Two-way translations that perfectly preserve the structure of the higher-level model (in this case BPMN) are just one element of the solution. But even with tools implementing such translations, stakeholders would need to follow strict guidelines to ensure that they do not break what has been done at a higher level (BPMN) when they do modifications at the lower level (BPEL), and vice-versa. I think it will take a lot of time to get there since it entails getting people used to work in certain ways (not just technological issues, but methodological ones).
The second problem is probably solvable, but not as easy as it seems. If we restrict the structure of BPMN processes to e.g. structured ones, it is not hard to define translations that preserve the structure, so that a reverse translation that renders the original BPMN process is possible. If we lift these restrictions, things become more complex due to fundamental differences, as Tomasz points out. We have recently defined a BPMN-to-BPEL translation that can deal with BPMN graphs with arbitrary topology. In this translation, if the original graph is well-structured, the resulting BPEL code follows the structure of the original graph and everything works well. But if the BPMN graph is not structured, the translation generates rather convoluted BPEL code and the way back to BPMN would be far from easy. For those interested, see this technical report.
An implementation of an earlier version of this translation is also available at the BABEL tools page (it’s called SPM2BPEL). An improved version of the implementation incorporating the whole translation described in the technical report is underway and should be posted on the same page soon.
Ismael,
Frankly, the problem I experienced is that behind the BPMN graphical elements, I cannot still find the correct XML document. Having tested some tools and trying to export and import the XML files, I encountered the crap files and not really the document that I can use. What about the interoperability promised by the OMG’s MDA?
Marlon,
I was previously unaware of your academic work on this topic but I think it is absolutely crucial and want to encourage you in it. I went to the paper you cite in your post, and despite the fact that it is largely written in some form of indecipherable runic script, the gist of it seems that you have discovered an iterative algorithm for turning BPMN into “readable” BPEL by analyzing basic patterns, with the caveat that there are some non-standard patterns that perhaps don’t work. The current version of your tool does not work with BPMN — alas it has no schema — but a simple XML-based list of activities and connectors. Since we mortals live in a world of tools, I might suggest as an alternative something like XPDL 2.0, which some BPMN tools can create and which provide the information your algorithm needs.
I am also interested in a class of tools, such as I imagine the Intalio BPMN Designer to be when it is finally released, where BPMN is a real-time graphical facade for BPEL code, such that if it violates BPEL it probably can’t be drawn. You might say that’s a much easier problem since it excludes lots of possible BPMN topologies, but to me it is the only one of interest since if you can’t execute it, what’s the point? Do you think for such a class of models, the reverse translation problem is significantly easier, or not?
Hi Bruce,
I am sorry for the “runic script” used in the technical report describing the BPMN-to-BPEL translation. This is a notation used by computer science academics, which is our initial target audience, as we want to get feedback regarding the correctness of the translation. But in the interest of bridging academia with industry, we are working on implementing the proposed translation as an open-source tool, so that tool vendors or open-source projects can pick things from there. You’re right about the lack of a standard XML serialisation of BPMN and your XPDL suggestion is a very good one. We have already started the work using an XML serialisation of the ILOG’s BPMN serialisation for which the documentation (in latin script) is publicly available. You will be right to say that this is a proprietary notation, but I can bet my dearest fifty cents that the final XML serialisation of BPMN, whether based on MOF/XMI or not, will be “isomorphic” to ILOG’s BPMN serialisation (and to other serialisations available out there). So it will be possible to define simple transformations between the standard and the “proprietary” serialisations. After all, they will hopefully be just two serialisations of the same language. The real problem is the BPMN-to-BPEL part.
There are classes of models for which a reversible BPMN-to-BPEL-and-back translation is possible. We can define three such classes of models:
1) The class of “BPMN models without livelocks”: If a BPMN model (with any topology) is not “divergent”, we can translate it to BPEL using BPEL’s “event handler” feature. You don’t want to read the resulting code though, because it’s just a bunch of event handlers throwing message events at each other. This is pretty much what we show in this paper (also in “runic script”) where we explain what is meant by a model without livelocks and we give an example of a livelock.
2) The class of “Synchronising BPMN models”: We are currently working on another paper in runic script that contains an algorithm which can determine whether a BPMN model can be translated into BPEL structured activities and control links (without using event handlers, which is what causes the mess). This algorithm could be used for the purpose that you describe: we could allow modellers to write their models as they wish, but on the background we run the algorithm and the algorithm will come back and tell you: Yes, this BPMN model can be translated to “readable BPEL”, or no it can’t.
3) The class of “Structured BPMN models”. Structured BPMN models (where every “splitting” gateway has a corresponding “joining/merging” gateway) can be easily translated into BPEL process definitions containing only BPEL structured activities. An algorithm for determining whether a BPMN process model is structured or not is not difficult to write. Some academic folks from Austria and Denmark have written such an algorithm in another paper written in “runic script”. They do not consider the full spectrum of BPMN constructs, but it could be a point of departure for tool implementors.
There are some “small” glitches though when translating BPMN to BPEL: they have to do with the fact that BPMN has a notion of sub-process and BPEL does not, but this is not difficult to circunvent and it is discussed in the BPMN specification. Also, BPMN distinguishes a lot of types of events, while BPEL doesn’t, so some nuances in the BPMN model may be lost on the way down to BPEL. But we can say that these are details and they can probably be resolved.
Sorry, my previous posting went out by mistake before I could conclude it. My conclusion was going to be that for the second and third classes of BPMN models mentioned above, reversible translations are possible, while for the first class of BPMN models, a reversible translation would be very difficult. Note that the first class of models is larger than the second, and the second is larger than the third.
The second class of models is fairly large, and my hunch is that it includes almost any meaningful BPMN diagram that people will be drawing out there. So in summary, there are good hopes that in a not-too-distant future, there will be some reversible BPMN-to-BPEL translations that generate readable BPEL code and cover fairly large classes of models. But there is a bumpy road ahead.
Tomasz,
I am not surprised. I have never been a big fan of MDA…
Marlon,
Once again, I think you’re absolutely on the right track. Class 1 is probably of little practical interest for process designers who want to import into a BPEL design tool. Solving Class 2 would be a giant step forward. Please keep us posted on your progress.
Ismael,
I think that the basic premises of MDA are directly link with the efforts of transformation BPMN to BPEL. We need though the methodology that describes how the business logic should be maintained in a technology neutral model that is understandable and potentially controllable by the business users (BPMN). We need the framework that separates business functionality concerns from implementation time technical concerns, such as scalability, security, and granularity, quality of services. We need to define the process how business users could model their business requirements in a fairly straightforward set of semantics dedicated to business requirements. Those requirements finally could then be instantiated by IT people using BPEL on IT infrastructure appropriate to the needs of the business community.
[…] This is a copy of a very interesting answer… […]
Bruce and Marlon,
In the first release of eClarus Process Modeler we have solved the Class 2 practically, even though the translation can be fine tuned.
Class 1 can be solved using simple pattern matching.
With synchronous links, simple pattern match is not enough. We use a flow analysis technique (we call it “Static Flow Token Analysis”) to identify those links. And our tool can translate them to “readable BPEL” if it can be. If you are interested, we can discuss more offline.
[…] If you follow that topic, you probably know that Yi Gao of Seattle startup eClarus and Marlon Dumas of Queensland University of Technology are basically the two guys in the world who know what they’re talking about. Way back in February, when I was still blogging on Ismael’s site, I posted on the roundtripping problem. You can find that thread here on BPMS Watch or here at IT|Redux. […]
Two-way translations that perfectly preserve the structure of the higher-level model (in this case BPMN) are just one element of the solution. But even with tools implementing such translations, stakeholders would need to follow strict guidelines to ensure that they do not break what has been done at a higher level (BPMN) when they do modifications at the lower level (BPEL), and vice-versa.
[…]cheap apartments, apartment cheap rent, low cost rental and student housing[…]
Trackback this post | Subscribe to the comments via RSS Feed
Leave a Comment