The State of Workflow by Tom Baeyens

 

The State of Workflow by Tom Baeyens

If database systems are like a respected, wise man telling a clear story, then workflow must be a bunch of spoiled brats shouting their own truth at each other. With this statement I would like to point out that workflow management systems are at the very initial phase of the technology hype curve. We are heading for some exciting times in this area. To illustrate this, we can make the comparison with relational database management systems (RDBMS). When talking about an RDBMS in a software development team most people will get the picture and shake their heads slightly up and down confirming they understand what you're saying. When using workflow terminology, the same crowds will shake their heads similarly but this time, every person will have a very different picture.

 

One of the observations that form the basis of this statement is the overwhelming number of concepts used. None of the numerous specs and tools in this area is similar. Of course, they tend to overlap but it usually takes a dictionary and an extended concept mapper to compare any 2 of those.

A topic that certainly deserves to be included in an introduction about workflow is the relation between workflow and business process management (BPM). The term workflow is used to denote a series of related interactions between people and a computer system. Workflow is more often heard in developer communities. Sometimes, the meaning of workflow is being restricted to the different screens of a user interface. Business process management (BPM) has a broader scope. Where workflow is more often used in a technical context, BPM has a broader scope. It implies also the non-technical issues like analysis, organisational impact, from the viewpoint of a manager.

First I start with explaining what workflow management systems are. Then, the advantages of business process management are covered. Next is an explanation of why the workflow market looks confusing at first sight. The main conclusion will be that selecting a workflow management system is the hardest task companies have to face. To give you a solid basis for this task, the core section of this article defines the four layers of a process definition. It describes the common workflow & BPM concepts in a neutral language. At the end, a guided tour along the specs and tools is added.

What is a WFMS

Definitions

A workflow management system (WFMS) is a software component that takes as input a formal description of business processes and maintains the state of processes executions, thereby delegating activities amongst people and applications.

To get started, lets define the basic terminology: a process definition and a process instance. A process definition is a formal description of a business process or procedure. A process instance is one execution of a process definition. So far so good. But when going one level deeper, we need to be very careful of the language we use. The way to express the steps within a process still hasn't found a common ground yet. That is the major difference between the specs and tools for workflow and BPM.

Why the term 'activity' should be banned... Process definitions are usually expressed in terms of activities. I think *that* is the main reason of all the confusion in the workflow space and therefore a bad idea. I'll tell you why: the term activity blurs the distinction between a state and an action. A state (aka wait-state) in a process specifies a dependency upon an external actor. At process execution time, this means that the workflow engine has to wait until the external actor notifies the WFMS that the state is finished. E.g. waiting for an approval. An action is a piece of programming logic to be executed by the WFMS upon a specified event that occurs during process execution. The WFMS initiates the execution of the action on a specified event during process execution. E.g. sending an email when a state is assigned to an actor. As you can see, state and actions are really different so using the same name for these concepts is a bad habit. My proposal is to avoid the term 'activity' and replace it by either 'state' or 'action'.

Another important responsibility of a WFMS is maintaining context information for each process execution. A process context variable -or variable for short- is a variable associated with a process instance. E.g. the start-date of a holiday request, the key of a record in the database, a reference to a document in the document management system, ... Usually the process definition declares the process context variables. Then, at process instance creation time, the variables are instantiated. All mature WFMSs have customisable variable types.

Target usage

One option is to use a WFMS as an enterprise application integration (EAI) platform. Currently, in most enterprise environments, the IT infrastructure has various heterogeneous applications and databases running in an intranet. Typically, these applications have a clear purpose when they are introduced in the organisation. Examples are customer management, document management, supply chain, order entry, billing, resource planning... Let's call them dedicated applications. Each of these dedicated applications contain knowledge about the business processes they have to support. The enterprise fits these automated processes of the software components into their overall non-automated business processes. Once such dedicated applications are installed and running, opportunities arise for new feature requirements that span over multiple applications. Enterprise application integration (EAI) is the discipline of implementing software requirements that involve multiple dedicated applications. Sometimes this is only data exchange that requires the implementation of a communication channel between two applications. Dedicated applications incorporate a number of business processes hard coded in the software. You could say that you buy a set of fixed, automated business processes. A WFMS on the other hand has got no prior knowledge of the domain. A WFMS takes a description of a business process as input and manages the executions of the process instances. That makes it much more flexible then dedicated applications but you have to do the effort to create a formal description of the business process. That is why WFMS and dedicated applications are complementary. A WFMS can be used to manage the overall process. As a rule of thumb, you could say use a dedicated application if it supports the processes you need. The first way to use a WFMS that was discussed here is to tie together all dedicated applications and create an EAI-platform with it.

A second option where a WFMS delivers high added value is for the development of workflow software that has a lot of people-related tasks. For this purpose, most WFMSs have a very convenient mechanism of creating forms for tasks. This target usage is especially productive in organisations focussing on ISO or CMM certification. Instead of documenting the procedures in a text format, a WFMS allows you to create automated support (e.g. by means of a web-application) of the procedures that are modelled as process definitions.

The third option is to embed a workflow engine inside another application. Remember from the first option that dedicated application incorporate a set of fixed business processes for a specific domain. The companies that develop the dedicated applications, can embed workflow engine software inside of their software. The workflow engine is in this case only used as an application component, hiding it from the application users. The main reason for embedding a workflow engine inside an application is for reuse (not reinventing wheels & hot water) and maintainability of the application software.

The case for workflow

Introducing workflow in an organisation delivers benefits on the software development level as well as the business level.

Easy development

A workflow management system will ease the development and even more the maintenance of standard enterprise software.

• Reduced development risk - the business analyst will talk the same language as the developer (in terms of states and actions). That implies that the developer will not have to make a translation from user requirements to a software design.
• Centralized implementation - its the business processes that change so the biggest advantage of using a workflow system is that the implementation is not a fuzzy combination of software-pieces scattered over various systems.
• Rapid application development - your software is freed from the task of keeping track of the participants in a process leading to faster development and code that is better maintainable.

Business process management (BPM)

Before you can automate your business processes, the toughest but mostly rewarding task is analysing them and creating a formal description. e-workflow.org has a nice description about the business benefits gained by this analysis.

• Improved efficiency - automation of many business processes results in the elimination of many unnecessary steps
• Better process control - improved management of business processes achieved through standardizing working methods and the availability of audit trails
• Improved customer service - consistency in the processes leads to greater predictability in levels of response to customers
• Flexibility - software control over processes enables their re-design in line with changing business needs
• Business process improvement - focus on business processes leads to their streamlining and simplification

I think they even missed the most important reason to start using a WFMS: improved support for iterative development. If the business process part of a software system is not easy to change, organisations tend to spend a lot of effort in business process analysis and hope they'll get it right first time. Well sadly enough that is seldom the case as in any software development project. A WFMS makes it easy to deploy new versions of business processes. As a consequence the use of a WFMS enables more efficient and risk-reduced development because the business process software can be developed in an iterative way.

Missing link

I really believe a WFMS is the missing link in enterprise development. First of all I want to state that the default way of incorporating business process logic into enterprise software is scattered approach. This means that the logic of the business process is spread over various systems like the EJB-container, database-triggers, message broker and the likes. No need to mention that this leads to software that is not maintainable. As a consequence, these organisations will think of changing their business process software only as a last resort. They often prefer to change their processes to the software. This argument also applies to a larger extend ERP software packages. In a second attempt, suppose we are aware of this problem and really want to centralize all code related to one process. This is fine for one business process, but if you implement multiple processes, you see duplication of code that manages state and process variables. Then in a third approach we could extract the duplicated code and put it in a central library. Well, ... that is actually a WFMS so don't bother implementing it yourself and choose one from the list below.

A closer look

WFMS interfaces

A WFMS takes process definitions as input. For the moment, think of a process definition as a UML activity diagram, a UML state diagram or a finite state machine. Examples are submitting an expense note, asking a holiday, requesting a quote, ... Then, the WFMS maintains the state and context of the executions of these process definitions. For this, it needs to be informed of state-changes. The state changes of the process executions can be logged for monitoring purposes.