Bonitasoft development: monolithic vs micro services

We at Bonitasoft are open to sharing the lessons we've learned while creating the BPM application development platform that is now Bonita BPM 7. Here’s what we learned when we faced the challenge of deciding which approach to take for our process-based application development platform: do we go with monolithic, or with micro services?

Our company, Bonitasoft, was established in June 2009. We released the first version of Bonita Open Solution 5.0 under a very tight schedule - in less than 6 months, a huge amount of work was done on the Bonita 4.0 initial foundation. We developed a brand-new studio to model business processes, and a brand-new portal to manage the runtime in a browser. The third part of Bonita Open Solution was the runtime engine, a Java backend to execute processes and manage the interaction with the database.

Bonitasoft onboarded a lot of Bonita Open Solution customers very quickly and the project become more and more complicated. We worked to continuously extend the capabilities of the engine to better address new use cases. This is when we really encountered the challenge of the “monolithic vs micro services” approach.

The monolithic era

By the time we were at the end of the v5 series of Bonita Open Solution, the team had learned a lot. Bonitasoft was growing very fast, with a wider and wider range of customers and projects. Some of them were quite big, with many processes, huge volumes of throughput and big integration challenges. We started to face issues around robustness and performance. Long story short: we needed to evolve the engine quickly to be more robust, faster, more scalable and with more new features. We carefully considered the options before us, which were basically two: improve the existing code base or build a new engine from scratch.

The bet

This was a tough decision. But, we decided to go for a new engine. Something fast, clean, modular, maintainable, scalable, reusable (for a future new product, who knows?) and replaceable in case we face issues with it later. How did that feel? Pretty scary. Overwhelming. But… innovate or die. Being the technical guys we are, we focused on the positive: what a fantastic challenge, to completely rebuild an 8-year old BPM engine, in less than a year, with a brand new team approach. Let’s go!

The micro services development era

We were pretty proud of our genius idea :). The team started defining the architecture in detail. At that time, I was spending nearly half my time traveling among Beijing, Grenoble, and San Francisco. I spent a lot of time in planes, airports, and in transit coding and defining new services. With a small team, we started to refine the new architecture and develop the first key services. The basic concept was to keep each service as small as possible, the most maintainable approach. This was true, for a while.

At the beginning, with no specific customer pressure, we approached the development much like a proof-of-concept. We were handling nominal use cases. We started with a few services, and it really looked great and powerful. With some of those earlier issues still in mind, we decided to provide several implementations for some services, like the one in charge of persistence (a single service which ended up in 6 Maven modules). At the time, the advantages were clear and we did not see any downside.

At this point the whole team joined the project, moving from the v5.x series to the brand new work on v6. It seemed logical to continue to build all other services following the same rules and source code split.

Time flew. After a while the first signs appeared...this systematic approach did not, in fact, seem to be the most efficient for our developers. Not actually wrong, in general, but definitely not optimal. As the release date loomed, we knew we would not be able to change our approach right away. We released 6.0 with about 50 services sliced into about 200 Maven modules, all in the same code base.

Findings

6.0 was released. With the release pressure off, we had more time to re-think the architecture.

Looking in depth to the services and their dependencies, we saw that it was a bit more complicated than we had expected. Perhaps we didn’t need so many small services, but we are quite sure that it is a drastically better story now, as we can merge selected modules together rather than than splitting them apart. Of course we will change things, improve, split again or merge - but we have all hands on the code base, we can explain our choices and we know both advantages and drawbacks of our decisions.

Internally, the new architecture has also brought a lot of advantages in terms of unit testing. Developers can now produce unit tests easily and we have a clear idea of the quality of the software.

Externally, the great thing is that now, this architecture can scale. The 6.x product series and the recently released Bonita BPM 7.0 will keep moving forward and, release by release, continue to add value to this architecture.

We can also say that the code base is easier to extend and to maintain. Our support team is happy; we can now fix bugs and provide patches way faster than before. We also have developers in a much more comfortable seat. They know where to look for a bug fix and are confident they can make fixes without breaking something else. The roadmap team is also happy to see our efficiency growing!

Post mortem

The question I asked myself is, “what would I do if I had the chance to do it again?”

The answer is not straightforward because it is not all about theory. Software is built by humans and every single developer plays a key role. In addition, software development occurs with the context of a company or a community culture (both in our case), and it is of course subject to various constraints.

We learned dozens of things building those services. The team had a lot of deep discussion and made good moves in the product. We were able to widely explore various libraries for each service implementation and had the chance to go deeper into multiple technologies. I’m quite sure the team matured fast, due at least in part to this. Our experience bears out Martin Fowler’s point : "By starting with microservices you get everyone used to developing in separate small teams from the beginning, and having teams separated by service boundaries makes it much easier to scale up the development effort when you need to."

In the end, the product is less and less buggy. The team boosted unit testing. We leveraged some of those services to build specific features. This is definitely positive!

The team sees clear opportunities for improvements. We want to better manage dependencies, reduce dozens of interfaces to manage less code, handle versioning better, re-organise the code to build reusable artifacts to match the services. And of course, we are looking at those services one by one to merge them when appropriate.

We have mastered the code, and with every release cycle we are making it better.