Where BPM and BI Intersect
Friday, February 2nd 2007 | Ismael Ghalimi
The BPM acronym means a lot of different things to a lot of different people. If you’re into techno music, it’s all about beats per minute. If you’re a fan of Business Intelligence (BI), you know all about Business Performance Management. And if you’re a regular reader of this blog, you must have a strong interest for Business Process Management. Today, we will focus on the later topic, and try to outline where it intersects with BI.
BI for going beyond Business Activity Monitoring
One of the first areas that people look at when trying to figure out how to make better use of their BPM and BI investments is Business Activity Monitoring (BAM). Your BPMS might generate a lot of events that are captured by your BAM infrastructure in order to monitor specific Key Performance Indicators (KPIs). That’s all fine and dandy, but what do you do with the massive amount of data afterward? Quite frankly, it would be a shame not to put it to good use, and this is where BI can help. Take this data feed from the BPMS, archive it into some kind of datamart, and use your BI tool of choice to slice and dice it every way you want. If you know what you’re looking for, this should give you some useful hint as to how you could improve your business processes at the first place.
BPM for feeding BI in a smarter way
If you adopt a process centric view of the world, data will look to you as nothing more than the audit trail of processes that have been executed. It’s actually not a bad way to look at things, especially if you’re trying to be smarter about the way you’re using BI for helping you make business decisions. One of the main challenges with any BI project is in finding the right information, and getting it aggregated in the right fashion. Usually, the data you need to make your decisions is scattered across many different systems, and simply getting access to it can be difficult. Using a BPM system to automate some processes and collect valuable information along the way can be one of the easiest ways of getting the right data all in one place. And because business processes tend to provide a fairly high level of abstraction — as opposed to data schemas or web services — the data they collect tends to be easier to consume for decision making purposes.
BPM for automating ETL
Extraction, Transformation and Loading (ETL) is one of the most complex and expensive aspects of BI, but one you cannot live without. As mentioned before, the data you need is rarely available all in one place, and usually not in the right format. Getting it to fit into your datamart or datawarehouse is the job of ETL tools, but these tend to work best when extracting data from one place, applying any number of transformations to it, then loading it into your BI repository. Unfortunately, this kind of straight-through processing does not always work, and sometimes you will need to lookup multiple systems for getting the data you need, or call for human intervention when automated rules for data cleansing are not enough. This is where a BPMS with a good collection of connectors to legacy systems, and the ability to process a large number of transient process instances in memory, can be helpful.
BPM for handling actions following BI-driven decisions
Once the slicing and dicing has been done, and the decisions made, it’s time for action. But in a corporate environment, action usually requires the participation of multiple contributors, and this is where a good workflow tool can be put to use. But because some of the tasks that are to be completed might require integration with different systems, using a complete BPMS for handling the downstream process that follows BI-driven decisions would make a lot of sense. This type of application has generated quite a bit of interest in the industry, but it will require a lot on the part of BI vendors, for they will need to get into transaction processing, an area they have traditionally shied away.
As we have seen, there are many scenarios where BPM and BI complement each others nicely. Recently, many traditional BI vendors have expressed strong interest for BPM technologies, and some have pulled the trigger with various levels of partnerships with BPM vendors. It will be interesting to see what comes out of that, and which BPM vendor goes out first in embedding a complete BI solution into its own product offering.
Editor’s note: A friend of mine launched the Pascal on BPMS blog. Check it out!
Entry filed under: BPM 2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Ismael,
Business Intelligence (BI) is more than slicing and dicing data collected from internal systems. The most important BI data are external environmental data such as economic climate, geo-political events, socio-economic conditions, shifts in demographics, and industry activities in the market segments within which a business entity competes. Without external data, one will never know how a business stacks up against competitive forces! In other words, one needs to measure performance against competitive forces, not just the KPI’s set to improve business operations. This is the essence of Strategic Management. A business needs both the inward and outward outlooks to sustain its competitive advantage.
The oil embargo by the OPEC in the 1970’s was a good lesson for the petroleum and transportation industries. Operations Research was applied successfully in optimizing internal operations, but failed in forecasting the shortage of crude oil supply in the international markets! The key reason is that there was plenty of internal information, to the point of information pollution, but environmental data were in short supply, or virtually non-existent! The shortage in crude oil supply triggered the Ontario Government to lower the Freeway speed limits from 100 km/h to 70 km/h. That was also the time during which sales of small size cars began to take off in North America. The Royal Dutch Shell also came up with the Scenario Analysis for Strategic Planning around that time.
It looks like IT practitioners are at work again in misusing a business term—business intelligence. Moreover, there is a misconception by IT people that decision making is based on data. Hardly, as one of my ex-bosses said, “I’ve already made up my mind, don’t confuse me with facts and statistics.” Most major decisions are made based on gut-feels, not rational analyses! Many studies in Decision Making and Analyses have proved over and over again how decisions are made, but IT types never learn!
Best regards
-Francis
P.S. The most important part in data warehouses or data marts is the Star Schema in organizing transaction data based on time, location, and product type dimensions. ETL becomes easy once you nail down the data model. This is based on my past experience in Very Large Data Banks (VLDB) back in the 70’s, before RDBMS was on the market. In the Transportation Planning field, a series of data banks in land-use, demographics, socio-economic, and trip generations are required to project future growth or decline in traffic. E. F. Codd (author of the relational theory), C. J. Date (guru of IBM’s System R), and Jim Gray (architect of Tandem’s Enscribe, and now working at Microsoft Research) were active members of ACM’s VLDB SIG. They were the pioneers in data warehouses and data marts. I was a voting member of ACM at that time, but I can no longer afford to pay the annual membership, journal, and transaction subscription fees!
Hi Francis,
A very good insight, something that people do forget. I have to say I agree that terms are reused to mean different things in different domains of usage, when perhaps a little thought could come up with something better. This reminds me of my father’s objection to the use of the term “renewable engergy”! How is it renewable? Replaceable perhaps. I have to agree with his view, but then what chance do you have to change it?
I do think that you underestimate the value of a good BI tool that looks at the appropriate information (including external information) and overestimate the effectiveness of those gut feelings (at least a little). I have read several articles that indicate good BI can supplement and re-enforce the business person’s understanding of a situation, and the numbers (dollars or efficiency gains) are there to show this. I suppose that implies this explanation re-enforces your take in that dollar and efficiency gains are very inward looking.
My personal take on gut feelings is that they are an ability to absorb the kind of Business Intelligence that you are referring to internally, find a pattern, and come to a conclusion based on inference. I guess the architects of BI software are trying to do something similar. BI is just one more evolutionary step to the future state of computing systems.
Cheers
-Bob
Hello Ismael,
Great to see the BI topic being picked up from the BPMS supplier side. Indeed, delivering BI in the context of business processes promises to drive better alignment of strategy and execution, and to accelerate BI adoption within companies’ information worker communities. See for example Gartner’s recent CIO survey results, published last week.
From a technology perspective, embedding BI into processes means 3 things:
(1) Provide analytics in the context of a process step. An example would be a purchasing manager, selecting a vendor for an IT project. Providing the right insights (transaction history, price/performance, etc.) at the point of decision making (creation of purchase requisition) certainly helps make better decisions, aligned with corporate goals.
(2) Take actions based on insights. Similar to (1) above, however this time it’s not the process that incorporates analytics, but its analytics that trigger a process. An example would be a sales analyst, identifying a spike in a competitor’s sales, and instigating an ad-hoc (workflow-driven) process to investigate and react.
(3) Measure efficiency of business process execution. And example would be a marketing manager, analyzing the execution of marketing campaigns, based on geographies.
All three above could involve a BPMS. While optional, the growth in adoption of BPM technologies certainly will foster and accelerate the adoption of process embedded analytics, also enabling more focused technical usage scenarios, such as BAM (Business Activity Monitoring) or IPA (Intelligent Process Automation).
I believe that we are in sync on the abovementioned scenarios. I am not sure though about the suggested role in BPM for automating ETL… No question that BPM does play a role in data quality management (MDM), however I don’t see the direct correlation to classical data migration and data loading (although I have some ideas). I would be interested to learn more about what you had in mind though.
Regards
-Lothar
Lothar,
Take a complex ETL process, and identify the areas where you’re spending much of your time configuring your ETL tool. Furthermore, identify these areas where the tool breaks, and human beings need to be involved in order to manually do some data cleansing or reconciliation. Now, identify these cases where help from more than one person is needed, and data lookups from more than one system are required. These are good scenarios for the use of a BPMS in the context of an ETL process.
Best regards
-Ismael
Bob,
The current BI tools are far from being intelligent. Most of them are just statistical packages with a new label. I’ve used statistical packages such as Bimed, SPSS, and SAS in the transportation planning field to do cross-tabulation, stratification, cluster analysis, regression analysis, variance analysis, chi-square test, curve fitting, forecating, etc.
Slicing and dicing transaction data to extract patterns would only help marketing to target existing customer groups to promote new products and services. It is one way to retain customers, but it does not create more customers. You need external data to help targeting potential customers!
A true BI tool is one that can capitalize on business environmental data and leverage on internal product and service data to generate scenarios and establish market trends using expert systems. I don’t think there is any commercial BI tool comes close to that.
Most IT people keep on promulgating the myth that data warehouses and data marts supplemented by BI tools are the basis for business decision making. This is not true. Data warehouses, data marts, and BI tools are just laying the ground work for decision making. There are many more steps, depending on the idustry, before business executives can make final decisions. They never look at the data extracted from data warehouses and data marts and manipulated by BI tools. Moreover, I have never seen a business executive drilling down on data to look for an answer himself or herself. It is always his or her staff members who doing the work! In essence, no amount of analysis can replace the gut-feel in decision making.
Cheers
-Francis
Trackback this post | Subscribe to the comments via RSS Feed
Leave a Comment