Where BPM and ECM Intersect
Friday, January 5th 2007 | Ismael Ghalimi
Following my recent post on the excellent Koral, several astute readers have asked what the integration points between Business Process Management (BPM) and Enterprise Content Management (ECM) could or should be, and whether some industry standards were available in this intersection area. Here is a first shot at answering the question.
If you take a very good Business Process Management System (BPMS) and a very good Content Management System (CMS), I could think of at least three ways to get 1 + 1 equal more than 2. Two take a process centric approach, while the third one takes a document centric one.
First, you can use the CMS to manage the lifecycle of documents that are attached to process instances. Say you’re building a procurement process, and you need to capture the multiple revisions that will be made by members of your procurement team to a contract that was drafted by your supplier. You could try to model such a process in BPMN, but it would be cumbersome and overkill. Instead, you’ll be much better off using a separate CMS to manage your contract, let people interact through the CMS when they need to create revisions to the document or comment on it, capture these orthogonal interactions through the Business Activity Monitoring (BAM) component of your BPMS, and only store a reference to the document within the process instance that is managed by the BPMS. This is what Intalio is doing with Alfresco for a couple of customers, one of them being the government of a country which GDP puts it in the top 15 worldwide. For this first scenario, there is no industry standard I could think of, especially if you want to cover integration with both the process engine and the monitoring component. That’s the bad news. The good news is that the former is fairly trivial, and there is no standard for BAM anyway, so the problem is not specific to this BPM+ECM discussion.
Second, you can use the CMS to manage all the artifacts related to a process model, instead of using a source control system such as CVS or PVCS. There, you’ll find two schools of thought that rarely agree with each others. Business-minded people view process models as documents rather than pieces of code, while technical-oriented folks have the reverse outlook on them. Both approaches have their respective merits, and I won’t try to list them out now. Instead, I will focus on the first category of users. For these, all artifacts related to a process model, including business requirements documents, spreadsheets, process diagrams, services definitions, user interface mockups, and the like, should be managed by a full fledged CMS. Even better, this CMS should be coupled to a blog and a wiki that would allow discussions to be carried on, in a semi-structured manner, both at development time and at runtime, and you’re trying to optimize some of its parameters. Again, Intalio is using Alfresco to come up with such an architecture, working for another set of customers. For this second scenario, I must believe that the good old WebDAV standard, or the slightly more powerful but less platform-neutral JSR-170, are pretty close to what you would want.
Third, you can use the BPMS to manage some advanced review and approval processes for documents that are managed by a CMS. Such an option makes sense when your processes become too complex, or too volatile, or both, to be hard coded within the CMS, or you want users of the CMS to be able to modify them themselves, without having to learn complex APIs. In this case, what you’re looking for is an embeddable workflow solution that you could integrate within your CMS. For this purpose, you would not need the power and complexity of a full fledged BPMS, but rather take some of its pieces: a simplified version of a process design tool that would use pre-packaged services only, a streamlined BPEL process execution engine that would provide only one connector — the one for the CMS native API, and some primitive management APIs that you could invoke directly from the CMSs’ management console. This is exactly what a couple of CMS vendors are doing today, either starting from scratch, or replacing alternative workflow solutions that do not natively support BPEL, nor provide the right development tool that could be used by pure business analysts. For this second scenario, the only standard I can think of is Wf-XML from the WfMC. It would not give you everything you need, but it would be a good starting point.
So here we are, BPM and ECM intersect in at least three distinct ways, and each of them has a set of business cases that would make it a good thing to have. You should expect Intalio to support all three in some fashion, sometime this year, and you’re more than welcome to fund one of our Demand Driven Development projects if you’d like it sooner, or according to your set of specific requirements. We already have three customers doing exactly that. And if you can think of additional ways of extending the intersection area across BPM and ECM, feel free to post your thoughts on this forum.
On this, I wish you all a very good week end. See you Monday!
Entry filed under: BPM 2.0, Standardization
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Your posting is interesting, but you missed several things.
First, in the integration you describe between BPM and ECM, the security models are disconnected. Maybe you could describe how to enforce a consistent security model in the future?
Second, you did a good job of outlining where standards do not exist, but didn’t mention whether you and Alfresco where going to step up and actually start creating some.
Roger,
Good points. On the security front, I will write a follow-up post that will outline our model for it, covering both ECM and ESB, as suggested in your other comment. On the standard development front, this is a tricky one. Intalio has done quite a bit of work there in the past, having founded the Business Process Management Initiative (BPMI.org), and this required very significant investments, which we might not be prepared to dedicate today. We will have to consider this. If we decide to go down that path, what would you say should be our priorities?
Best regards
-Ismael
Ismael,
I believe that there is a standard for ECM from AIIM.
This article reminds me of an ex-client who coined the term Information Resource Management (IRM), which is equivalent to what is called ECM today. In the early 80s, many organizations created VP positions for IRM. I believe that the equivalent for today should be CKO (Chief Knowledge Officer). Unfortunately, North American enterprises hand the responsibilites ot knowledge management to the CIO or equivalent! In the U.K., a CKO is either a seasoned librarian, or an experienced professional.
A good ECM system must be able to facilitate the management of digitized documents and records, but also those in other media such as microfilm, microfiche, paper, sound tape or record, and film. Moreover, a good ECM also supports the management of structured data and information in databases and knowledge bases. In essence, it must provide the capability of populating a suite of built-in enterprise taxonomy and ontology. These form the foundation of an enterprise’s knowledge base.
The way I see it is that the intersection of current BPM and ECM provide only a partial solution. There is still a large gap for BPM to reach at the EPM (Enterpise Processes Management) level. We have to wait until an EPM concept, standard, and system emerges. ISO and NIST are currently working together on a standard called PSL (Process Specification Language) that would facilitate the interchange of enterprise process descriptions between an enterprise and its suppliers, customers, and/or partners. Currently, the PSL standard covers primarily the manufacturing or production activities of an enterprise. The current BPMS cannot address the manufacturing or production activities as they are too complex for BPMN to handle! It would not be too difficult to extend it to cover other enterprise functional areas and activities though.
In terms of security, I have mentioned in several comments within this blog that many enterprise-class software products should be C2 certified. This is to ensure that only authorized personnel can acces documents or records, based on their levels of security clearance. BPM and ECM are such classes of software that should be C2 certified. The reason is that, other than national security, a secured system should not be vulnerable to industrial espionage in order to protect an enterprise’s trade secrets and comply with many privacy acts.
Best regards,
-Francis
Ismael,
I believe that this article is an extension of the ECM for the Masses article. Why do you need to upload documents to an ECM that was built on top of SOA and AJAX? Shouldn’t that ECM be capable of virtually locating and organizing your documents such as rich text documents, spreadsheets, presentations, project plans, simple text files, graphics, audios, e-mails, etc. across all hosting sites that a user subscribes to?
Security would not be an issue for personal use. When it comes to business use, it is a very different story. When a regulatory agency comes to perform an audit, or a litigation is filed against a business, all documents and records must be available for discovery. When documents, records, and e-mails are scattered all over the place on the Internet, it is not a pretty scenario, is it?
A small business may be able to manage it, because it usually has low volumes of documents and records, relatively speaking. But do you think that a large enterprise will go for an unsecured subscription service for ECM? And don’t forget, not everyone should have access to every available document or record! As an example, would you allow everybody to look into your personal medical history if all hospitals publish medical records on the Internet? Another example: would you allow all financial institutions to expose all your financial records over the Internet?
One thing I am sure about is that the Bush Administration would love to gain access to all medical and financial records without warrants in the name of national security. With unsecured subscription services, the Bush Administration doesn’t need a warrant, does it? The Bush Administration even made a claim recently that it doesn’t need warrants to open everyone’s mails!
Best regards,
-Francis
Francis,
I do not see why BPMN could not model the processes you referred to.
Can you further explain?
-Ismael
Francis,
You seem to imply that data stored onto online services cannot be secured and is automatically available to anyone with an Internet connection. I disagree with such an assumption, as many Software as a Service (SaaS) providers such as Salesforce.com have demonstrated that large organizations, like Merrill Lynch in the case of Salesforce.com, can rely on their services for business critical applications.
Also, I would prefer that we keep the blog free of politics if possible.
Best regards
-Ismael
Ismael,
PSL is a set of definitions, axioms, theories, and grammars based on KIF (Knowledge Interchange Format from MIT) that defines process, activity, instance, and timepoint to model manufacturing processes with their associated inputs, outputs, and controls such as plans, schedules, persons assigned, actions, quality inspections, materials consumed, etc. Perhaps, you can apply BPMN to construct models that can really emulate all of the above easily, as you were one of the architects of BPMN. Personally, I don’t know where to begin.
It should also be noted that in manufacturing processes, there are many real-time control processes. Some require human interventions, and others adjust themselves automatically based on prescribed tolerance limits. The display of process statuses is usually done in a graphical form, with something like the Executive Dashboard. Colors change based on pre-set limits, on a real-time basis. Red is for over or under safety limit, Yellow for conditions getting close to a limit, and Green for normal operation in an analog fashion. This is also augmented by audio and visual alarms for each control point. I need an explanation from you as how the application of BPMN can emulate all the above! My combined experience in the petroleum (refinery), telecommunication (network management), healthcare (intensive & critical cares), public utilities (adjust inputs to meet forecast demands based on prevalent conditions), and traffic management (traffic flow control in an area) industry sectors tells me that using BPMN to emulate real-time process control and management is extremely difficult, if not totally impossible! Unless I overlooked something when reading the BPMN specification that is.
My understanding of BPMN is that it primarily addresses transactional types of business activity. Operational activitie were outside the scope of BPMN! Moreover, BPMN is not an ISO standard, but only an OMG (consortium) standard, driven primarily by vendors. In BPMN, there is no such thing as activity, it only has task and sub-process. PSL follows the time-honored convention of process, which consists of activities, and an activity entails tasks (or actions). If ISO and NIST want to use BPMN to replace PSL, if at all feasible, BPMN will be elevated to an ISO standard like UML! The way I see it is that SysML will have a better chance to be elevated to an ISO standard when it matures to the point of supporting executable models. As DoD and ISO AP233 (part of ISO 10303) are the driving forces behind it, it could be fast-tracked to become an ISO standard.
CRM on the web is a web-based replacement of desktop Sales Force Management, that covers sales leads, reminders to follow up with customers, customer orders, answers to customer queries, etc. Of course, CRM is a critical component in the Value Chain, but security requirements for it are less stringent than other operations. There is a good incentive to outsource it. That is, there is no need to upgrade software sitting in the notebooks of the mobile workforce. Before Internet was released to the general public, each salesperon kept a copy of the data on his/her laptop’s local (or client) database, dialed in at the end of the day, and then updated & synchronized with the central database. To minimize telephone expenditures, many organizations had regional dial-in centers for uploading and downloading data using http://FTP. Scripts were then run for database updates and synchronizations. This is because the replication of databases failed all the time over the dial-up network, unless you subscribed to the more expensive X.25 dial-up service! For cellular phone service providers, CRM (previously called Customer Care) goes at least one step further. That is, to activate/deactivate subscribers and update services for subscribers on the switches in real-time.
My point about security was that SOA and AJAX, in their current forms, are vulnerable to many forms of attack, as warned by several security watches.
One thing about standards is that JCP is not an accredited standard body. Its standards are only applicable within the Java community. Since Sun withdrew from ISO for Java standardization, most, if not all JSRs are practically ignored by accredited standard bodies like ISO, NIST, ECMA, IEEE, EIA, W3C, IETF, etc. NIST, on behalf of DoD, would only pay attention to embedded Java, as many device vendors use it to drive firmwares in cell phones, real-time process control devices, and signal processing devices (e.g. analog to digital signal converter), etc. This is because many military system components are built from COTS (Commercial Off The Self) software and hardware. On the contrary, most JSRs comply with de jure standards!
As an Open Source vendor, JSRs are standards for you. As an end-user of software products, they mean nothing to me! I only care about what functionality a software product can offer, and its performance. For practical reasons, I would only use software that can also run on my machine when I am not connected to a network, be it intranet or extranet. The key reason is that the backbone of the Canadian segments of the Internet has not been upgraded yet to accommodate IPv6, as far as I know. Services from ISPs, telcos, or cable companies are unrealiable. From time to time, service could be out for days at a time. It could be as long as 4 consecutive days, or even a week! Another example: around 10 pm Eastern Time (7 pm Pacific Time), it is very difficult to reach web sites hosted on the West Coast — too many timeouts! Should I put all my eggs in one basket? That is, does it make sense to rely on hosted software over the Internet only?
OK, I will refrain myself from any political comments.
Best regards,
-Francis
Francis,
That’s a lot to go through. I’ll answer step by step.
-Ismael
An interesting divergent thread worthy of a separate posting!
I would be hesitant in suggesting significant changes to BPMN. I would tend to propose a collaborative approach similar in philosophy to BPMN, but that could also invoke PSL sub-processes.
Consider the advantages of integrating business and manufacturing in terms of activity monitoring. You start a manufacturing process from a customer order. You obtain a planned delivery date from the manufacturing process. Any change to the manufacturing schedule is fed to the business process to keep management and customer informed of changes to delivery dates and schedules. If changes to manufacturing activities changes the production duration, the business process needs not store this information, it asks for the duration of the manufacturing process each time.
This way the business process works its own way, and manufacturing its own way as well. I can see that one way of viewing the differences is that the transactional nature of BPM does not lend itself to the needs of scheduling for manufacturing processes. But I don’t see that each could not benefit from the information that each can provide through a suitable messaging format.
Cheers
-Bob
Bob,
I believe that you have started thinking more in terms of an enterprise. A significant change to the BPMN will be required in order to collaborate with PSL. PSL is an extension of IDEF0, which is being replaced by SysML. The current BPMN does not support layering. Meaning, one can drill down from process to activity, and then further down from activity to tasks in PSL or SysML, but not in BPMN. PSL also covers business transactions that can extend backward to suppliers and forward to customers. When coupled with ISO AP233 (Product Data Interchange), this enhances EDI significantly.
Don’t forget, an order for a complex product from a customer also comes with specifications and design drawings (Mass Customization). I don’t know how BPMN, in its current form, would handle speciifications and design drawings when they arrive in XML based on ISO AP233! In terms of activity monitoring, I haven’t seen a BAM tool in action yet. I would imagine that it can do far less then a Supervisory Control And Data Acquisition (SCADA) system. With a SCADA system, one can capture volumes of inputs (i.e. materials, labors, utilities, etc.), outputs (i.e. product and its components and parts), and wastes including costs. SCADA is one of the basic components for Actiivity Based Costing, Management, and Budgeting (ABC/M/B). They form part of the feeders into the BSC (Balanced Score Card), when an enterpirse uses it for measurements.
In its current form, BPMS only does a small fraction of the job of automating the management of an enterprise. It has a long long way to go for it to be able to manage more than business activities.
Cheers
-Francis
Good post, and interesting discussion.
My experience with such an intersection shown on this slide.
This experience is partially based on the automation of the standards production chain at the ISO Central Secretariat in Geneva. At that time I had just a workflow engine for the orchestration of services. With a product like Intalio, my job would have been much simpler.
Thanks
-Alexander
Alexander,
You have a very impressive set of experiences. A presentation was made to IEE (i.e. Institute of Electrical Engineer of the U.K., a counterpart of IEEE in the U.S.) on SysML and BPMN: When is a Process Model Not a Process Model? The presenter highlighted the shortcomings of BPMN. As I indicated in my earlier comments, BPMN would need a lot of modifications to enable mapping to PSL, if at all feasible! I am still waiting for Ismael to show me how he could use BPMN, in its current form, to replace PSL and SysML.
Cheers,
-Francis
Francis,
You could model in BPMN all the transactional parts of PSL. Additional dimensions (such as business metrics) can be handled by your BAM framework, but this is not standardized. Now, the benefit of doing it with BPMN is that it would give you the implementation for free…
Can PSL do that?
-Ismael
Ismael,
PSL covers more than transactional activities. It includes processes & activities of manufacturing and control. A partial solution is not a real solution. Moreover, I don’t see how BPMN coupled with BAM can replace SCADA. As an example, a Trust Officer or Security Trader in financial services would watch the price fluctuation of stocks. For some portfolios, the share price at a given time would trigger a put or a call automatically. For a portfolio that consists of shares from diferent stock exchanges, which span across all over the globe, the rise and fall of stock prices can trigger different actions, as the same stock can be priced differently in each exchange. A security trader may sell shares in one stock exchange and buy them back from another if this results in a net gain in the value of that stock.
Moreover, I did not ask for a partial solution. I was waiting for a complete solution on how BPMN supplemented by BAM can replace SysML, PSL, and SCADA. Does it make sense to implement transactional processes with BPMN when it is incompatible with PSL in manufacturing processes? Another question is whether all BPMS offerings currently available on the market can interoperate with one another by simple plug and play? It would be interesting to see how BPMN would handle manufacturing activity costing or pricing based on a set of customized specifications and design drawings from a customer! Don’t forget, process consists of activities, and an activity entails tasks in PSL, while activity is either a task or sub-process in BPMN. When we follow the time-honored convention, the first level IDEF0 diagram depicts processes. For each process, activities are defined and expanded as second level diagram. By the same token, tasks are defined and expanded as the third level diagram for each activity. For complex processes, it may be prudent to insert one level of diagrams between process and activities as sub-process diagrams. Again, another level can be inserted between activity and tasks as sub-activity. It seems to me that BPMN mixes all those up as one level diagram, with lots of redundant symbols.
Intalio’s BPMS may be free, and possibly the best of the breed. It is still a partial solution as I mentioned in previous comments.
Cheers!
-Francis
Francis,
I do not have a complete solution yet, but you should keep an eye on Intalio’s website. We will announce some interesting partnerships in the manufacturing space very soon, and I would not be surprised if they help us increase the scope of the solution we’re talking about.
Best regards
-Ismael
Ismael,
I would like to see Intalio’s BPMS evolving into an EPMS (Enterprise Process Managment System), by leaps and bounds, with support for all activities in the entire Value Chain. Particularly, it must be capable of supporting an enterprise’s activities in Strategic Management, Resources Management, and Operations Management.
I will follow Intalio and its strategic partners’ collaborative activities closely.
Cheer!
-Francis
[…] An example of such a stack is the one currently developed by Intalio around our Business Process Management System (BPMS). Over time, we realized that our customers needed more than just a process design tool, a process execution engine, and a workflow framework. Some wanted integration with a modern Enterprise Content Management (ECM) system in order to support scenarios where BPM and ECM intersect, others wanted to get the functionality offered by a complete Enterprise Service Bus (ESB) as a way to build their own SOA puzzle. […]
I too am looking forward to your post regarding where ECM, BPM, and security intersect.
Milly,
I’m gathering the necessary information for it.
Best regards
-Ismael
Milly,
What I have witnessed so far is that many open-source or Web-based software products are void of access control security; basically, you get all or nothing. We are repeating the same chaos that we had back in the 60’s and 70’s, many independent (silo or stovepipe) applications need a lot of work to get them to play in harmony.
There are a lot competing standards, each of which works in isolation to address a problem from a unique perspective. In essence, there is no architecture to unify the standards, de jure as well as de facto.
Most of the ECMs are dependent on the operating system’s security capabilities. Only C2 certified OSes provide access control for hardware, documents, files, and software. In terms of BPMS, only one (i.e. Microsoft’s BizTalk) provides C2 equivalent security, because it only runs on NT-based OSes. Open Source BPMSes provide none!
Many organizations are now working on enterprise architecture to align IT to business operations, but there is no universal architecture yet to guide the unification of diverging standards, of which some are contradictory. For example, BPMN is incompatible with SysML and PSL. Nevertheless, SysML and PSL are ISO standards, while BPMN is only a vendor-driven consortium standard. When DoDAF (Department of Defense Architecture Framework—US) and MODAF (Ministry of Defence Architecture Framework—UK) become ISO standards, there may be a chance that a Universal Architecture (i.e. Enterprise Architecture for everyone) emerges! In essence, many ECMs and BPMSes need to be re-written to gain C2-equivalent security!
Best regards,
-Francis
[…] What makes enterprise architecture both difficult and fascinating is that it’s all about dealing with a multi-dimensional problem. Focus on one or two dimensions, and the others quickly become orthogonal considerations, usually relegated to a later time, actually never really implemented. More often than not, security is one of these dimensions that does not get the attention it deserves. Dealing with security is a little bit like cleaning your house: when its clean, nobody can really tell how much work had to be done for getting there, and only when things get dirty do people start noticing. This post from security architect James McGovern is a good summary of the problem at hand, and gives me an opportunity to answer a question that was asked following the publishing of this post on the intersection of BPM and ECM: what about security? […]
[…] It seems the smartest guys in the room […]
I came a little late, but It seems that this paper is related to discussion.
What’s your opinion?
S. Nasser,
Pretty interesting, but very workflow centric.
Best regards
-Ismael
Trackback this post | Subscribe to the comments via RSS Feed
Leave a Comment