23

I'm new to Workflow developement, and I don't think I'm really getting the "big picture". Or perhaps to put it differently, these tools don't currently "click" in my head.

So it seems that companies like to create business drawings to describe processes, and at some point someone decided that they could use a state machine like program to actually control processes from a line and boxes like diagram. Ten years later, these tools are huge, extremely complicated (my company is currently playing around with WebSphere, and I've attended some of the training, its a monster, even the so called "minimalist" versions of these workflow tools like Activiti are huge and complicated although not nearly as complicated as the beast that is WebSphere afaict).

What is the great benefit in doing it this way? I can kind of understand the simple lines and boxes diagrams being useful, but these things, as far as I can tell, are visual programming languages at this point, complete with conditionals and loops. Programmers here appear to be doing a significant amount of work in the lines and boxes layer, which to me just looks like a really crappy, really basic visual programming language.

If you're going to go that far, why not just use some sort of scripting language? Have people thrown the baby out with the bathwater on this? Has the lines and boxes thing been taken to an absurd level, or am I just not understanding the value in all this?

I'd really like to see arguments in defense of this by people that have worked with this technology and understand why its useful. I don't see the value in it, but I recognize that I'm new to this as well and may not quite get it yet.

3
  • 1
    "Workflow tools" are nothing but "visual programming tools", and I think this blog post tells enough: blog.davor.se/blog/2012/09/09/Visual-programming
    – Doc Brown
    Dec 16, 2014 at 17:27
  • 1
    Nope workflow tools, are tools to replace paper and the way how people work in standardized ways. Think of an hospital, wouldnt it be great if all offical documents passed the equal routes, without some people prefering document route X, or speaking / phoning directly ita about standardizing work, often a legal requirement.
    – user613326
    Dec 19, 2014 at 22:08
  • @user613326: honestly, you should read the question again. It deals exactly with what I wrote - workflow tools to control and execute workflows, not just for modeling them. I do not deny the benefits of modeling workflows (especially in electronic form instead of using pencil and paper), or standardizing them, but when starting to use the tools for "visual programming", I do not expect better results as described in the above Blog.
    – Doc Brown
    Dec 23, 2014 at 7:09

7 Answers 7

8

From a developer's point of view, you are right in saying that these "visual" environments are really hard to work with. SharePoint 2010 Workflows, which I use, throws out every best practice around creating good enterprise software - no automated testing, no code reuse, unreadable software... Anything more complex than an out-of-the-box template can be painful to maintain, as you are experiencing.

But from the business' point of view, workflows have massive benefits - at least, in theory. To quote from this white paper, the Efficiency, Accountability, Control and Ease of use of an automated workflow provide huge productivity gains. Imagine how much more inefficient a simple approval or on-boarding process would be without this automation. Also, the very act of defining a workflow is valuable to an organization that is trying to get control over their business processes.

The current state of workflow software is not the business' fault. They just want to make their lives easier, and workflows are great for that. I would mostly blame us, the IT department:

  1. For not being more transparent with the business about how complex and fragile the system really is. We hide all the complexity.
  2. For not being able to "scratch our itch" with intuitive, simple workflow solutions. This is probably more of a rant against large enterprise packages like SharePoint and SAP, but they are better than the custom solutions out there
1
  • 2
    The link is still dead - there is no chance to see on which real-world experience the white-paper is based when the resource is missing.
    – Doc Brown
    Dec 23, 2014 at 7:12
8
+25

There is only one real benefit, yet its huge: Separation of Concerns.

So, instead of process orchestration logic being embedded in our system, it becomes and external configuration. A map, basicly. You can change it (much more) independently, you can have multiple processes, multiple versions of processes, multiple versions of multiple processes running at the same time, and that's all out of the box in any decent solution.


Historically, the concept of SoC have won many times - starting from the Unix principle "do one thing, but do it good", and being applied again and again - like having dedicated server components like ESB, different persistance systems, caching, load balancing, monitoring, like splitting CSS from HTML, etc.

Your business process and its flow rules are often orthogonal to your data, UI "screens" or users "hierarchy". So, it makes perfect sense to develop and change it separately from the other aspects of the system. That was the premise on which BPM has appeared in early 1990s.

Since then, many of tools and languages were created to support this concept, with most well known being BPMN - a graphical language for creating "flowcharts" that directly map to processes. While people complain that its large and unwieldy (having over 100 symbols in vocabulary), and advocating modern approaches like S-BPM (has only 5 base symbols), current industry practice is to stick to BPMN or its derivatives, subsets or siblings.

You do not look pleased with BPMN:

Programmers here appear to be doing a significant amount of work in the lines and boxes layer, which to me just looks like a really crappy, really basic visual programming language.

But its not that bad ) There is theory behind it. And version 2.0 took some good insight from 1.0 shortcomings.

If you're going to go that far, why not just use some sort of scripting language?

Imperative paradigm and scripting languages are not always the best answer. As you've probably seen in declarative languages (like HTML, CSS, SQL, Drools or internal ones of Nginx, Graddle and Maven, Puppet etc) the resulting code can be much smaller and cleaner, than a version written in "decent language, like Java or C++".

As for your other point:

as far as I can tell, are visual programming languages at this point, complete with conditionals and loops.

have you looked into the Events and Triggers ? BPMN is a language and you have to learn it before using, or at least get familiar with it.

Under the hood, BPMN is XML, so you can edit it by hand, or generate. And you can version control them, because XML is text-based. However, just having an XML that can be translated into flowcharts, does not sound like its goona help you, and that's correct - writing your own parser or editor for it is a tough and expensive task with questionable benefits.

Luckily, there are already tools at the market which do exaclty that.


Activiti is free, and quite popular among both developers and busines owners, because of its initial price (zero), availability of information and humbleness. The last point is really unique, as Activi only focuses on managing your business processes, without trying to tie you in with whole-package solutions. Also, its open - so you only need to know Java and REST to get it up and running. The drawback is that client side, integration and application/businesss logic and the whole architecture are left to developer, so if your develpment team is weak - prepare for the fail. Total cost of ownership can be surprisingly high for a free tool ;)

On the other side of spectrum is Pega (Pega PRPC), the reigning king of BPM systems (according to Gartner and Forester), which looks surprisingly good for its age. This and-a-kitchen-sink behemoth is even capable of CRM, OCR and (if i'm not mistaken) speech recognition capabilites, predictive analytics, business rules management and WYSIWYG UI editor. It comes with a serious a price tag, though. Not only does it cost a fortune, but all the development is being done within a web-app, which means you have to use browser, which is IE8 (plus some plugins, plus ugly hacks, like using Excel for editing data tables). Also, Java, Javascript or HTML/CSS editing is also being done withing web browser - so say goodbye to unit tests, IDE code highlighting, refactoring and all your programming toys you've leaned to love.

Good side of it ? you can implement complex system WITHIN WEEKS [PDF, see page 22]. And yes, result is not guaranteed.

IBM have somewhat recently (accoring to the enterprise time pace) have bought Lombardi, and is now offering very competitive solution (but then you will have to buy everything ibm, you'know). Appian is young vendor who has interesting insights and positive feedback, but the way they are written (two extra DSL languages in addition to visual one) just doesn't appeal to me.

There are other players, and their solutions. Most of them are plain horrible. Like - your eyes, brain and heart would literally bleed, when you simply look at them. So, trust your guts and don't make your developers and user hate you.


Closing note:

BPM system is the same for for processes, what Photoshop is for images. Don't be afraid that its visual. Don't make it do the job not suited for it (remember web sites created entirely in Photoshop, which were next to impossible to edit?). Keep it simple and don't make bugs ;)

3

Years ago, before most of us were born, software developers had to write their own code to store data. If you needed to save program state, well, that was seen as part of the function of the code, so many software developers ended up writing code to organize data and save and read it and so forth.

And then someone realized that this was something that happened a lot -- that, the logic to save, organize, read and search data was actually a component that was so commonly used that it should be it's own component. And we got databases.

In the last 10 to 15 years, IT departments have been realizing that the logic to choreograph and/or orchestrate business processes is so common that it should also be it's own component, which has lead to all sorts of different workflow tools.

There are 3 primary benefits of a workflow tool:

  1. Time needed to make and deploy changes: you can develop and change the logic of a workflow without the same technical risks that you have with changing a piece of code.
  2. Transparency: the business logic in a BPM based system is immediately available to and readable by the business analyst, whereas only developers will be able to read the business logic in "code-based" systems.
  3. Reuse of technical components: BPM tools are often used in conjunction with systems that have a Service Oriented Architecture. By separating the business logic from the technical components -- especially for enterprise systems which must implement hundreds or thousands of different business processes -- you're able to reuse the technical components while spending relatively little time on developing the business logic (with a workflow tool).

However, one of the most common problem that I run into with workflow and BPM tool usage, is that developers still try to "bury" business logic in non-transparent code.

What I see, all the time, is developers still trying to add the business logic in the most technical ways possible, instead of the most transparent way possible. This is natural, because developers are, by their very nature, much more comfortable with code than with a workflow tool. Furthermore, the more logic you keep in a technical way, the more you'll be needed as a developer. Unfortunately, this is the worst thing a developer can do to a BPM system because he or she is undercutting the primary benefits of using BPM.

Lastly, most BPM tools aren't far enough that business analysts can develop the workflows themselves: however, that's what the goal is. Ideally, business analysts would develop the workflows/business process diagrams and developers would only work on the technical components that are called by the workflow engine.

6
  • 1
    Thank you for your reply. So, afaik, there are basic workflow tools based around directly graphs, and there are complex workflow tools, that are essentially visual representations of Turing complete programming languages. What I don't understand, is if you need a Turing complete programming language... why not do it with a real general purpose programming language? If you're using loops and conditionals, why aren't you doing it in say... Python?
    – user16549
    Nov 18, 2013 at 18:31
  • 2
    Because visual representations of Turing complete programming languages are accessible to a vastly larger audience than developers, which means that companies using these tools only have to hire developers for technical components and can let domain experts do the rest (with the workflow tool). Also, visual representations are immediately transparent, unlike code of any kind.
    – Marco
    Nov 18, 2013 at 19:56
  • Did you consider that the real reason for developers implementing business logic in code instead into "lines and boxes" is not because "the developers feel more comfortable in code", but that most of the existing graphical workflow tools are simply unsuitable for describing real world workflows in an executable manner (that means, with all exceptions, failure handling, partly failure handling, state visualization, non-functional requirements, and so on)?
    – Doc Brown
    Dec 16, 2014 at 17:12
  • @DocBrown The whole point of workflow tools is to avoid situations in which developers implement business logic. Ideally, deverlopers only spend their time implementing technial components and let the business analysts (with help from the workflow tools) develop and maintain the business logic components.
    – Marco
    Dec 17, 2014 at 10:15
  • @Marco: the conclusion I draw from what you wrote is that the tools are far from meeting the expectations, otherwise developers would not even be tasked to develop workflow logic.
    – Doc Brown
    Dec 17, 2014 at 10:26
1

Below statements are my personal experience using workflow tools, specifically Oracle BPM Suite(10.3G & 11G). First to specify, your question is focusing on workflow tools which are enabling modeling executable process models, these tools are part of Business Process Management Systems(BPMS). This specific process modeling is definitely developing and you can refer to it as a visual programming language.

The main benefit is the agility of understanding and changing process logic

With business process models you cover a visual explaination of the process logic and at the same time an executable integrative component. This allows faster on-and offboardings of programmers, because less documentation on the transitions, conditionals(Gateways or Business Rules) and process flow in general has to be documented, since its part of the development.

Additionally you have attached reporting/monitoring capabilities, what you would have to develop individually per each application, which is covered by most BPMS.

Moreover in most BPM development environments service adapters are prebuilt(e.g. JMS, Web Services, JDBCs etc.), allowing to develop middleware solutions faster in a step by step interactive integration, reducing human errors in coding.

Following workflow platform does achieve a lot of benefits mentioned above - Platform based approach for automation of workflows

0

The Value

The value proposition is that workflows can be created or altered quickly and easily to suit the changing nature of a business. An important part of realising this value proposition, is that business processes themselves are units of resource in the system.

Workflows as units of resource, means that a business process is defined as a single 'unit'. To understand what I mean by this, consider any computer program written for a business. It implements a business process for sure, but the logic for that process is likely to be spread around the source code to a certain extent. A workflow tool should allow the process to be defined in one well contained place. It could be in one single config file, or extracted from one visual diagram, or it could mean that the workflow is in a single module of code that can be plugged in, perhaps even swapped or configured on the fly.

Why the Value may not be realised

This value proposition can be undermined by the difficulties of covering non-vanilla cases, integrating with UI technology which changes very rapidly, bad practices such as using the workflow tool as a wrapper only and putting the real logic in code anyway.

Poor design of the tools themselves may be a limiting factor. An example would be a tool that requires all parameters passed between processes to be in a Java Map, with restrictions on what values you can actually put on the map, instead of just plain old method parameters (I'm thinking of one of the more popular tools in particular that does this).

I think it is probably fair to say that IBM as a big player with a large technology eco-system, fairs better than the others. If they also control the UI technology, and database and SOA technology that is used in conjunction with the workflow tool, they stand a better chance of coming up with an eco-system that integrates together nicely, and creates an opportunity to really make use of this idea.

It is definitely true that the effort of writing the interfacing between a workflow tool and the other parts of a system can completely negate the whole value proposition. It is always worth considering if there is a better way of doing things.

Programming is Workflow

The truth is that programming (in imperative languages at least) IS WORKFLOW already. You may have code that implements workflow which is to do with handling the system technology; accessing files and running SQL queries and so on. You may have code that implements business workflow; setting the state of a document and passing it to a reviewer for example.

Recognizing this and designing your code to cleanly split these separate concerns is hard to pull off completely, but you can certainly get a long way with doing just that.

I agree with you, sometimes we end up using these tools because someone else decided it was a good idea, and they are just too complex and making our job harder. I don't think that is always the case, it needs some careful consideration to decide if its worth it or not.

4
  • 1
    " I don't think that is always the case" - can you backup this by some real-world experience? That would be interesting.
    – Doc Brown
    Dec 16, 2014 at 17:32
  • @DocBrown Unfortunately not. I have heard others claim successes with these tools, and rapid turnarounds of new processes. The only direct experiences I have had of them have been of huge, unwieldy and enormously time consuming efforts that lead me to seriously doubt their value. I'll resist naming the tool vendor since I used to work for them, but my feeling was that a lot of the fault was with the tool itself and the way it made programming more awkward. Dec 17, 2014 at 9:00
  • @DocBrown I should add, it has been suggested that we use such a tool on my current work project. I am therefore currently trying to consider if it is worth it vs rolling our own code. Something more light weight than the big tools might be worth exploring, so far I don't know what that would be. Dec 17, 2014 at 9:03
  • @DocBrown worth noting that the question currently has a bounty stating "Looking for an answer drawing from credible and/or official sources." In the light of this, purely speculative answer doesn't look particularly useful (wonder what could be the reason to vote it up)
    – gnat
    Dec 18, 2014 at 13:46
-2

It is not very clear which tool you are using. I guess you may be referring to the general set of tools called Business Process Modeling tools. There is a good reason for using such tools. Any quality business define its functions in terms of processes and analyst as well as business experts can draw such processes comfortably (until you tie them with standards...). You can create such processes at the conceptual level without any knowledge of web programming and if you have the right tool, you may be able to convert the process into a working application too (experienced people must come on board for this magic to go into production of course). So the idea is good.

The objective of the visual tools is not just documentation of the processes. The use of the tools is meant to help professional re-engineer processes and, in times, run simulations to discover points where the processes are less efficient than desired so that plans can be put in place to remove the bottlenecks.

There is a standard way that many companies use today called BPMN 2.0 (Business Process Modeling Notation). I do recommend that you get into understanding this notation if your tool is using it.

The Business Process Management Body of Knowledge is a famous resource, that you may want to consider as well.

The above covers the business side. The technical side requires SOA and BPEL, I am not sure I can provide advice on those here and now though.

3
  • 2
    OP is mentioning which tools he has in mind, and those are not modeling or simulation tools. In fact, BPM tools are mainly for "creating business drawings to describe processes", and the OP sees the value in them. The question is about tools for controlling those processes.
    – Doc Brown
    Dec 16, 2014 at 16:28
  • @DocBrown, clarification appreciated.
    – NoChance
    Dec 16, 2014 at 18:42
  • 1
    @doc Brown -- the tools in question control component execution using the the various Models and diagrams as "code"! (And it works about as well as you would expect -- hence the tearing of hair and gnashing of teeth from the OP). Dec 17, 2014 at 16:43
-2

In simple example by history.

Stone age

At first you had small menhir companies where people just where told what to do, and they did it. Sometimes things wend wrong, and Person X or Y got blamed (never sure who really did it).

Next internet and Email was invented.

People now wrote others what to do, and those people often had problems with their Email, read it wrongly, or just simply deleted an Email without ever reading it; so often things where not don blaming it on bad Email

Workflow evolved out of administration By standardizing the actions, finally people could see at what phase who halted a process, and at the same time, got a digital overview of what actually had been done. This worked nice till people wanted to change standard processes, or till some unknown X Y person caused an improper database request resulting in database corruption, sending production back to stone age..

2
  • 1
    is this merely your opinion, or you can back it up somehow? Note the question currently has a bounty stating "Looking for an answer drawing from credible and/or official sources."
    – gnat
    Dec 18, 2014 at 13:44
  • its based on history, it was ment to be a hilarious example, but not everyone understands work-flow and humor together i see.
    – user613326
    Dec 19, 2014 at 22:05

Not the answer you're looking for? Browse other questions tagged or ask your own question.