The BPM in the Microservice Environment

If you are thinking about implementing a system in a web development project where functionality can be changed in runtime mode, then you may need to add a workflow engine that can execute business processes. This feature allows you to:

– Make the system functionality more flexible;

– Speak the business language;

–  And even change the rules of game!

In an ideal scenario, businesspeople would be able to not only see the description of their service’s work in a recognizable notation, such as BPMN 2.0 but also influence the description on their own.

Everything seems quite straightforward when it refers to a single monolithic application, near or inside of which, a workflow engine is installed [Pic. 1]. However, what’s to be done if you want to get a modern system, build a microservice architecture, or take advantage of all its benefits?

When you have dozens of microservices, where each is performing a separate function, and dozens of instances of some of them in order to cope with the load, is it worth to being frightened and refusing to use workflow engines? The answer is not! We know for sure that even in these cases you can take advantage of all the benefits of BPM. We at K&C have done it.

Indeed, the K&C agile web development team has experience in creating microservice systems that include a workflow engine. Today, we want to share some tips.

Our goal: to create the system, the functionality of which will be divided between several microservices, each of which can be run multiple times and where at least the part of functionality will be represented in the form of a workflow, described in the BPMN 2.0 notation.

To do this

1) We take each of the microservices we want to improve, carefully extract the logic from them, and then present it in the form of a workflow.

It is assumed that the functionality of each microservice is sufficiently isolated, and after turning hardcoded logic into a flexible workflow, it will still be an isolated part of the service and will only interact with the workflow.

2) We also need an engine that will execute our workflows. There are many such engines, so there is a great deal of choice.

3) We add a database to the engine. Although a microservice may already have its database, it is still usually not worth using the same database for both the microservice and its processes – it is better to have separate storages. When transferring the logic to the process, you should adequately divide the stored data into those that the process will take and those that will remain in the microservice.

4) Next, you need to determine how many instances of the engine there will be and how the engine will interact with the microservices. It would seem that you could use one separate engine for all instances of the microservice [Pic. 2]. Probably, in this case, they will communicate via messages, or even by REST API.

However, under this approach, the engine risks becoming dependent on other services – which can “bury” all our microservice architecture. In addition, the possibility of horizontal scaling is lost, for the sake of which we have chosen a modern micro-service architecture.

Of course, you can think about clustering the engine, but there is another solution: we take a lightweight engine and build it right into the microservices! If the engine is well chosen, then there is nothing complicated about this; and you’ll still have as many instances of logic as there was before.

Moreover, the interaction is simplified: it is assumed that the engine is designed for such an embedding and allows direct calls right in the code of the microservice without the need to use any channels! In this case, microservices are simultaneously connected to several bases: its own base and the engine’s base [Pic. 3].

5)  If you want your application to have an Administrator UI and/or a Cockpit, – which is quite likely, then you should also know how to create it. Since we assume you have a significant number of microservice instances, you’ll have to run a separate workflow microservice, which includes whatever is required for an administrator control panel and is connected only with the engine’s base [Pic. 4].

That’s about it. The dedicated K&C team hopes it managed to show that the use of the workflow engine in a microservice environment is possible, as well as identify at least some of the potential solutions. In this way, we achieve the fantastic power that the microservice architecture can bring, as well as the transparency and flexibility provided by the process approach. Of course, there are always hundreds of questions to be answered in each specific case. However, who said you won’t be able to answer them?